Sunday, September 25, 2016

Using datashader to visualize billions of data points

Visualization is the best way to explore and communicate insights about data. But many times, we find it is difficult to visualize if we have a lot of data points, the traditional visualization techniques break down. datashader is a tool developed by Continuum Analytics come to rescue. It is extremly easy and fast to use it to plot even billions of data points (see the examples or a video from Scipy 2016). 
This week, we will use datashader to plot the global seismicity for earthquakes larger than M2.0. To speed up the process, I downloaded the earthquake locations/times from the ANSS Catalog from 1898 to 2016, and save them as a csv file in the data folder (You can also grab the data directly using the APIs from my previous blog post). All together, we only have 1120244 earthquakes
And with a couple of lines, you can see we get a very nice looking map with the global seismicity. You can find the script and data at Qingkai's Github.
import pandas as pd
import datashader as ds
import datashader.transfer_functions as tf
from functools import partial
from datashader.utils import export_image
from datashader.colors import colormap_select, Greys9, Hot, viridis, inferno
# read in the earthquake data from file, note that I only read
# in 5 columns with Origin time, Latitude, Longitude, Depth, and
# Magnitude of the earthquakes
df_eq = pd.read_csv('./data/catalog.csv', usecols=[0,1,2,3,4])
# now let's plot them using a black background
background = "black"
export = partial(export_image, background = background)
cm = partial(colormap_select, reverse=(background!="black"))

cvs = ds.Canvas(plot_width=1600, plot_height=1000)
agg = cvs.points(df_eq, 'Longitude', 'Latitude')
export(tf.interpolate(agg, cmap = cm(Hot,0.2), how='eq_hist'),"global_seismicity")

5 comments:

  1. I was scrolling the internet like every day, there I found this article which is related to my interest. The way you covered the knowledge about the subject and the 4 BHK Duplex in hoshangabad Road was worth to read, it undoubtedly cleared my vision and thoughts towards B 5 BHK Duplex in hoshangabad Road. Your writing skills and the way you portrayed the examples are very impressive. The knowledge about 5 BHK Duplex in hoshangabad Road is well covered. Thank you for putting this highly informative article on the internet which is clearing the vision about top builders in Bhopal and who are making an impact in the real estate sector by building such amazing townships.

    ReplyDelete
  2. To get proper information of construction is very necessary hence this blog is very useful for me. I want to know more about flats in Bhopal because i am planning to start construction in Bhopal.

    ReplyDelete
  3. Five weeks ago my boyfriend broke up with me. It all started when i went to summer camp i was trying to contact him but it was not going through. So when I came back from camp I saw him with a young lady kissing in his bed room, I was frustrated and it gave me a sleepless night. I thought he will come back to apologies but he didn't come for almost three week i was really hurt but i thank Dr.Azuka for all he did i met Dr.Azuka during my search at the internet i decided to contact him on his email dr.azukasolutionhome@gmail.com he brought my boyfriend back to me just within 48 hours i am really happy. What’s app contact : +44 7520 636249‬

    ReplyDelete