Category Archives: Data Visualization

Spatial Debris visualization in Scientific American

Some time ago Scientific American approached me to commission me for the creation of a visualization of spatial debris. And there is a lot of spatial debris floating around the earth. Think of abandoned rockets or broken satellites. The purpose of the visualization I created was to illustratie that there is a huge amount of  spatial debris, and not necessarily to provide an exact representation of it. In fact, not all the required data in order to determine exact position of debris and satellites was available in the dataset I received: right ascension and argument of perigee were missing, so I have used random numbers for those. But all the other data to calculate the orbit was there. In order to calculate the orbits of satellites and debris I had to apply Kepler’s Laws of Planetary Motion.

The graphic has been created in Processing. With all the orbits calculated and the satellites and debris positioned randomly on those orbits, the next thing was to get the color and positioning right. Positioning was rather easy, since it’s just applying some transformations to the image, which resulted in a nice perspective (circles nearby are larger than the ones further away). Coloring was the final step. Initially the idea was to color by country (US, USSR, China and Others), but this resulted in an image with colored dots all over. So to communicate a more focussed message, we decided to show the difference between active satellites (magenta) and spatial debris (black). As a nice extra, the ISS space station is also marked to get an even better sense of the amount of debris.

After completing the project, I played around with the data a little more, just to see if an animated version would have an even greater impact communicating the message. Well, judge for yourself…

Global Agenda Survey 2011


I’ve had the honor to collaborate with the fabulous Moritz Stefaner on a project for the World Economic Forum. We have created several interactive visualizations that allow users to explore the Global Agenda Survey 2011, a survey held each year by the World Economic Forum to discover the most important trends in the world according to its council members.

Except for the network graph, the visualization is compatible down to IE6. We achieved this by using raphael.js, underscore.js. The network has been created in d3.js.

Interview on Fell In Love With Data

Enrico Bertini from Fell In Love With Data has been so kind to post an interview with me about Protovis and D3.

You can read the full interview here.

Thanks Enrico!

Nerds Unite recap

Wednesday I had the honor to give a small presentation about data visualization at Nerds Unite in Utrecht (NL). It was a small group of people who are interested in data, open data, government and of course visualization. After a short introduction I showed some of my work, which was well received. After that Eugene Tjoa gave a talk about creating visualizations for the Central Bureau of Statistics (CBS) in The Netherlands.

It was a great evening with many interesting people. Thanks!

Photo by Sebastiaan Terburg

Guest posts about Eyeo

I am writing 3 daily guest posts about Eyeo Festival on Infosthetics.com and 1 summary guest post on Visualizing.org. I’ll update this post with links to each of the individual posts.

Visualizing Europe

Today was one of those days I won’t forget very soon: today was Visualizing Europe day, and many enthusiasts, practitioners, researchers and users of data visualization gathered in Brussels for an inspiring day of talks and meeting interesting and kind people from the data visualization community. The day was divided into 3 sessions:

  1. the power and potential of data visualization
  2. a vision for Europe
  3. where do we go from here?

The first sessions showed some of the best works currently created in data visualization: Santiago Ortiz from Bestiaro showed the power of the visual programming paradigm of Impure can be used to create sophisticated data visualizations in minutes (did I say minutes? seconds!)

Next, Moritz Stefaner showed two of his recent and impressive projects: the Better Life Index project that was recently launched by the OECD. And his previous famous project: Notabilia, which shows deletions on Wikipedia.

 

Enrico Bertini gave a fantastic talk from a research perspective and explained different approaches of making a data visualization for the public and for the tiny group of people who are actually solving real world problems with data visualization. A quote that was tweeted numerous times immediately: “data visualization is useless, it is indispensable”. He also highly recommends the book: “How Maps Work” (which of course is on my wishlist now!).

Last but not least of this first sessions was Dave McCandless from Information is Beautiful. Dave showed some of his work, and a remarkable quote was: “I disagree with Moritz, I’m not looking for 1000 stories, I’m looking for 1 story that’s interesting”.

After a short coffee break, session 2 started with Gregor Aisch showing how he creates data visualizations as the Open Spending project for the Open Knowledge Foundation. He proposed a new approach for data visualization, namely ‘open data visualization’, which is open source + open data + open to community. An fascinating idea I’d like to learn more about.

Assaf Biderman from the MIT Senseable City Lab impressed us with some of their cutting edge projects they do together with governments and cities, like the Trash Tag project which tracks and visualizes where trash is being transported all over the USA after people have emptied their trash bin. Another project that keeps impressing me is the Copenhagen Wheel, an augmentation to bicycles that allows bikers to track their own performance, and at the same time measures various air conditions of the city. This data is collected and visualized to understand more about the city’s air pollution.

Salvatore Iaconesi from Art is Open Source elaborated on how the artistic world uses data and visualization to change paradigms, for example in supermarkets: while in the supermarket data is visualized on your iPhone and shows the geographic origins of the chemical compounds of your products.

Last but not least, Peter Miller from ITO World had to rush through his slides where he showed some very compelling and sometimes fine-grained user contributions to the Open Streetmap project. It’s impressive to see how user contributions can lead to sometimes more correct maps than non-crowd-sourced maps.

The final session was a discussion between Franco Accordino and Jean-Claude Burgelman from the Europen Commission and Toby Green from the OECD. The main subject was: what did they take from today’s sessions, and what will they do with it. It was very good to see that the value of data visualization was recognized, and that the EU sees data visualization as one possible and valuable way to create new knowledge, which is very important.

Finally, the day was finished by meeting so many people from the data visualization community. It was amazing to meet so many people whom I’ve been in contact with for quite some time now. Thanks Visualizing.org for organizing this wonderful day, and everybody who has contributed. It was a memorable experience!

Gost Counties visualization wins DataViz challenge

Just a short note that I am honored, thrilled and excited to announce that my Ghost Counties data visualization has won the Visualizing.org ‘Visualizing the 2010 US Census data visualization challenge‘. A great thank you to the jury from Visualizing.org and Eyeo Festival and all the kind messages I received!

Eyeo Data Visualization Challenge: Ghost Counties

About a day before the deadline I have submitted my entry for the Eyeo Data Visualization Challenge by Visualizing.org where the grand prize is a ticket to the brilliant Eyeo Festival.

You can see the final result here: http://www.janwillemtulp.com/eyeo.

The visualization depicts the number of homes and vacant homes for all the counties for each state. The size of the outer bubble represents the total number of homes, the size of the inner bubble represents the number of vacant homes. The y-axis shows the population size (on a logarithmic scale) and the x-axis of the bubbels shows the number of vacant homes per population. Each bubble is also connected with a line to another axis: the population / home ratio. On the top right you can see some exact numbers for this data.

This time I built the visualization in Processing, mainly because I expected to work with large datasets from the US Census Bureau and I might had to use some OpenGL for better performance. Eventually I didn’t use OpenGL. Building the visualization in Processing was lots of fun. To get sense of the data I tried as many as 5 completely different approaches. Here are some of the sketches that eventually led to this visualization (view this selection on my Flickr stream).

The data itself was not very complex, but rather big, and the biggest challenge was to find a creative approach to visualize this data, but without using a map (which would be rather obvious since it’s about locations).

SEE#6 conference

This weekend I have visited SEE#6 in Wiesbaden, Germany. Although this was the sixth SEE conference, it was my first visit. And I must say, it was overwhelming! Here’s a brief overview of my experience.

Day 1 – conference

The conference was situated in the beautiful Lutherkirche, which was one of the most beautiful and atmostpheric conference locations I’ve ever been to. After a friendly welcome from Micheal Volkmer from the hosting organization Scholz & Volkmer, prof. dr. Harald Welzer. He gave an inspirational talk about sustainability, the main subject of the conference, and he gave us his view on how to fight climate change and change human behavior to improve a sustainable society. 

After the keynote Carlo Ratti of the MIT Senseable City Lab in Boston showed us some of his recent projects on how ubiquitous computing is entering our society more and more, and how sensors can help cities and citizens to be more aware of the environment, and improve sustainability. On this image you see a visualization by the Lab that shows real-time whether and rain situation in combination with taxi locations: are taxi’s on places where they are needed the most?

After a break, the young and talented Alexander Lehman took over. He showed us some of his animated infographics, and how he uses satire to tell a story. He elaborated on his most successful project: “du bist terrorist” where he uses satire in an attempt to make people more aware about the increasing danger of the government collecting all kinds of data about its citizens.

Brendan Dawes had a very humorous and inspirational talk about how he is using his creativity to do very inspiring projects, both for customers as personal research projects. He had many great examples of his projects and one of them was a homebuilt digital/wooden weather indicator: not because it really solves a problem, but just because it’s fun and cool:

Wesley Grubbs from Pitch Interactive showed us some of his great projects. Projects where he has visualized many rows of data that resulted in high resolution (and long rendering time) images. One of his most compelling examples was a visualization where he shows some insight in how the US Defense is spending its money.

After the beautiful images from Wesley, Joshua-Prince Rasmus from REX architects gave his presentation. Before the conference I didn’t really know how architecture would fit in a conference about information visualization, but it turned out that this was one of the talks that impressed me most. At REX they’ve defined a very strong process of doing work for clients. Joshua talked us through it. He showed the beautiful images he uses in presentations to show his ideas, and at REX they’re brilliant in working within constraints, creating flexible buildings and, perhaps most importantly, really understand their clients, so that they eventually build something clients really understand and agree with. Great talk!

Final talk was by the talented Justin Manor from Sosolimited. Justin inspired us by showing a few of his great installations. One that he gave special attention was real-time analysis of political debates, where a very large number of different approaches were shown how to interpret, categorize and analyze the words and sentences politicians are saying. His visualization is built in Processing. This concluded the first day.

Day 2 – discussion and workshop

The second day was an extra day for data visualization die hards, to have some good discussions about data visualization. The discussion was led by 3 prominent people from the data visualization community: Moritz Stefaner, Andrew vandeMoere and Benjamin Wiederkehr. The day was basically split into 2 halves where the first part was a discussion, and the second part some of the presenters of the conference gave some insight in how they do their work.

The discussion took of immediately after the main topics had been presented:

  • how to engage people through visualization?
  • how to use visualization to change people?
  • “facts are useless, stories are everything!” (quote by prof. dr. Harald Welzer made during the keynote on the first day)
  • what is the impact of data.gov shutting down?
  • how can design critique push the community forward?

Although all of these topics were a used as a starting point for the discussion, the main topic of the discussion evolved around questions like:

  • as a (visualization) designer, should you help the user make his conclusion?
  • is linear storytelling, like the satire movies from Alexander Lehman, the right way of passing information to the user?
  • can you really be objective if you visualize data?
  • is showing war casualties, for example like the one from Stamen design, a good way to really engage people?

The discussion was really interesting, and people had some very good points. But at the same time I found it somewhat hard to really take a stand in this, because I really think it depends. Anyway, after a short break, some of the presenters from day 1 gave some insight into their work process, things they run into, etc.

  • Wesley Grubbs showed us some more details on how he approaches visualization work for both clients and as personal research projects, like handling large amounts of data.
  • Alexander Lehman showed us how he uses 3DS Max to create his infographic movies, and how the overkill on functionality of 3DS Max actually helps him to be very productive.
  • Moritz Stefaner gave a very short but great introduction to Protovis, and how you could use Google Docs cleverly to use it as a real-time data source for your visualization
  • Justin Manor showed some more of his great installations with water, pneumatics, LED, Processing, Arduino, etc., and various ways of how they were made.

That wraps it up.

The conference was a blast! It was so inspiring, a great location, fantastic speakers, and very good talks! I am looking forward to SEE#7 conference.

If you want to see the full talks of SEE#6, go to http://www.see-conference.org/video-stream/. On the conference website you can also see video registrations of previous talks, which are highly recommended to watch!

Tutorial: Line chart in D3

One of the most common visualizations is a line chart. D3 is not a charting framework, but instead allows you to manipulate the document based on data. That’s what you’re actually doing with D3: adding elements to a document, removing them, updating them, etc. The advantage is that you are much more flexible in creating the visualization that you want. Some may consider it a slight disadvantage that you may do have to do some extra work to get things done.

For this tutorial we’re going to create a basic line chart with an x-axis and y-axis, tickmarks and labels.

First, we defined some variables:

var data = [3, 6, 2, 7, 5, 2, 1, 3, 8, 9, 2, 5, 7],
w = 400,
h = 200,
margin = 20,
y = d3.scale.linear().domain([0, d3.max(data)]).range([0 + margin, h - margin]),
x = d3.scale.linear().domain([0, data.length]).range([0 + margin, w - margin])

The data variable contains our dataset we want to display as a line chart. The w and h variables are used for the width and height of our chart and the margin will be used to create some room to display our labels. The y and x variables are our linear scale functions, and remember that you can use x and y as functions. The scales are used to convert our data values (which are defined in the domain of the scale) to x- and y-positions on the screen (which are defined in the range of the scale). Note the use of margin in the range.

Next we append an svg element to our document with the proper width and height, and then we append a g element to this svg element so that all the elements that will be appended to this g element will be grouped together. The transformation that we apply on the g element makes sure that our coordinate grid moves down 200 pixels.

var vis = d3.select("body")
    .append("svg:svg")
    .attr("width", w)
    .attr("height", h)

var g = vis.append("svg:g")
    .attr("transform", "translate(0, 200)");

In order to create a line chart, we’re going to add an path element to our visualization. This path element needs a d attribute that contains the data for the path. Now, you could write that yourself, but that may be a little hard to do, so we use the helper function d3.svg.line to do that for us (see Tutorial: Line interpolations in D3 for more details on this helper function). Now we add the line variable which is, just like the scales earlier, a function that you can call. So when we bind our data to the path, this d3.svg.line function will be called so that the value of the d attribute will be created:

var line = d3.svg.line()
    .x(function(d,i) { return x(i); })
    .y(function(d) { return -1 * y(d); })

The i parameter is the index of the current item, and the d is the current item itself. Note the usage of the scale functions (x() and y() to convert from i and d to x- and y-coordinates (note the -1. This is needed because right now the coordinate have a positive y-axis down, and since we have translated our g down 200 pixels, we need to use the negative values of the y-axis now). To use our line function we add a path element to our g element like this:

g.append("svg:path").attr("d", line(data));

Next up is adding the x-axis and y-axis. The following snippet adds them by appending a line to our g element.

g.append("svg:line")
    .attr("x1", x(0))
    .attr("y1", -1 * y(0))
    .attr("x2", x(w))
    .attr("y2", -1 * y(0))

g.append("svg:line")
    .attr("x1", x(0))
    .attr("y1", -1 * y(0))
    .attr("x2", x(0))
    .attr("y2", -1 * y(d3.max(data)))

Next are the labels. We can use a convenient function of the scales here: x.ticks() and y.ticks(). These functions will return the proper tickmarks, and you can provide how many you want. D3 does return nicely rounded numbers though. Also note the .text(String) usage for obtaining the string value of the current data item. The labels are aligned in the center by using .attr("text-anchor", "middle"). And for the y-labels I’ve used dy in order to vertically position the labels correctly.

g.selectAll(".xLabel")
	.data(x.ticks(5))
	.enter().append("svg:text")
	.attr("class", "xLabel")
	.text(String)
	.attr("x", function(d) { return x(d) })
	.attr("y", 0)
	.attr("text-anchor", "middle")

g.selectAll(".yLabel")
	.data(y.ticks(4))
	.enter().append("svg:text")
	.attr("class", "yLabel")
	.text(String)
	.attr("x", 0)
	.attr("y", function(d) { return -1 * y(d) })
	.attr("text-anchor", "right")
	.attr("dy", 4)

The last thing to do is to add the ticks themselves. These are actually just very short lines, so all you have to do is to provide the start x- and y position (x1 and y1) and the end x- and y-position (x2 and y2).

g.selectAll(".xTicks")
	.data(x.ticks(5))
	.enter().append("svg:line")
	.attr("class", "xTicks")
	.attr("x1", function(d) { return x(d); })
	.attr("y1", -1 * y(0))
	.attr("x2", function(d) { return x(d); })
	.attr("y2", -1 * y(-0.3))

g.selectAll(".yTicks")
	.data(y.ticks(4))
	.enter().append("svg:line")
	.attr("class", "yTicks")
	.attr("y1", function(d) { return -1 * y(d); })
	.attr("x1", x(-0.3))
	.attr("y2", function(d) { return -1 * y(d); })
	.attr("x2", x(0))

And that’s it. Finally, some of the styling of the objects was moved to a style section at the top of the page in order to get the final result:

path {
    stroke: steelblue;
    stroke-width: 2;
    fill: none;
}

line {
    stroke: black;
}

text {
    font-family: Arial;
    font-size: 9pt;
}

View a live version here.

Page 1 of 212