Gost Counties visualization wins DataViz challenge

Just a short note that I am honored, thrilled and excited to announce that my Ghost Counties data visualization has won the Visualizing.org ‘Visualizing the 2010 US Census data visualization challenge‘. A great thank you to the jury from Visualizing.org and Eyeo Festival and all the kind messages I received!

Eyeo Data Visualization Challenge: Ghost Counties

About a day before the deadline I have submitted my entry for the Eyeo Data Visualization Challenge by Visualizing.org where the grand prize is a ticket to the brilliant Eyeo Festival.

You can see the final result here: http://www.janwillemtulp.com/eyeo.

The visualization depicts the number of homes and vacant homes for all the counties for each state. The size of the outer bubble represents the total number of homes, the size of the inner bubble represents the number of vacant homes. The y-axis shows the population size (on a logarithmic scale) and the x-axis of the bubbels shows the number of vacant homes per population. Each bubble is also connected with a line to another axis: the population / home ratio. On the top right you can see some exact numbers for this data.

This time I built the visualization in Processing, mainly because I expected to work with large datasets from the US Census Bureau and I might had to use some OpenGL for better performance. Eventually I didn’t use OpenGL. Building the visualization in Processing was lots of fun. To get sense of the data I tried as many as 5 completely different approaches. Here are some of the sketches that eventually led to this visualization (view this selection on my Flickr stream).

The data itself was not very complex, but rather big, and the biggest challenge was to find a creative approach to visualize this data, but without using a map (which would be rather obvious since it’s about locations).

SEE#6 conference

This weekend I have visited SEE#6 in Wiesbaden, Germany. Although this was the sixth SEE conference, it was my first visit. And I must say, it was overwhelming! Here’s a brief overview of my experience.

Day 1 – conference

The conference was situated in the beautiful Lutherkirche, which was one of the most beautiful and atmostpheric conference locations I’ve ever been to. After a friendly welcome from Micheal Volkmer from the hosting organization Scholz & Volkmer, prof. dr. Harald Welzer. He gave an inspirational talk about sustainability, the main subject of the conference, and he gave us his view on how to fight climate change and change human behavior to improve a sustainable society. 

After the keynote Carlo Ratti of the MIT Senseable City Lab in Boston showed us some of his recent projects on how ubiquitous computing is entering our society more and more, and how sensors can help cities and citizens to be more aware of the environment, and improve sustainability. On this image you see a visualization by the Lab that shows real-time whether and rain situation in combination with taxi locations: are taxi’s on places where they are needed the most?

After a break, the young and talented Alexander Lehman took over. He showed us some of his animated infographics, and how he uses satire to tell a story. He elaborated on his most successful project: “du bist terrorist” where he uses satire in an attempt to make people more aware about the increasing danger of the government collecting all kinds of data about its citizens.

Brendan Dawes had a very humorous and inspirational talk about how he is using his creativity to do very inspiring projects, both for customers as personal research projects. He had many great examples of his projects and one of them was a homebuilt digital/wooden weather indicator: not because it really solves a problem, but just because it’s fun and cool:

Wesley Grubbs from Pitch Interactive showed us some of his great projects. Projects where he has visualized many rows of data that resulted in high resolution (and long rendering time) images. One of his most compelling examples was a visualization where he shows some insight in how the US Defense is spending its money.

After the beautiful images from Wesley, Joshua-Prince Rasmus from REX architects gave his presentation. Before the conference I didn’t really know how architecture would fit in a conference about information visualization, but it turned out that this was one of the talks that impressed me most. At REX they’ve defined a very strong process of doing work for clients. Joshua talked us through it. He showed the beautiful images he uses in presentations to show his ideas, and at REX they’re brilliant in working within constraints, creating flexible buildings and, perhaps most importantly, really understand their clients, so that they eventually build something clients really understand and agree with. Great talk!

Final talk was by the talented Justin Manor from Sosolimited. Justin inspired us by showing a few of his great installations. One that he gave special attention was real-time analysis of political debates, where a very large number of different approaches were shown how to interpret, categorize and analyze the words and sentences politicians are saying. His visualization is built in Processing. This concluded the first day.

Day 2 – discussion and workshop

The second day was an extra day for data visualization die hards, to have some good discussions about data visualization. The discussion was led by 3 prominent people from the data visualization community: Moritz Stefaner, Andrew vandeMoere and Benjamin Wiederkehr. The day was basically split into 2 halves where the first part was a discussion, and the second part some of the presenters of the conference gave some insight in how they do their work.

The discussion took of immediately after the main topics had been presented:

  • how to engage people through visualization?
  • how to use visualization to change people?
  • “facts are useless, stories are everything!” (quote by prof. dr. Harald Welzer made during the keynote on the first day)
  • what is the impact of data.gov shutting down?
  • how can design critique push the community forward?

Although all of these topics were a used as a starting point for the discussion, the main topic of the discussion evolved around questions like:

  • as a (visualization) designer, should you help the user make his conclusion?
  • is linear storytelling, like the satire movies from Alexander Lehman, the right way of passing information to the user?
  • can you really be objective if you visualize data?
  • is showing war casualties, for example like the one from Stamen design, a good way to really engage people?

The discussion was really interesting, and people had some very good points. But at the same time I found it somewhat hard to really take a stand in this, because I really think it depends. Anyway, after a short break, some of the presenters from day 1 gave some insight into their work process, things they run into, etc.

  • Wesley Grubbs showed us some more details on how he approaches visualization work for both clients and as personal research projects, like handling large amounts of data.
  • Alexander Lehman showed us how he uses 3DS Max to create his infographic movies, and how the overkill on functionality of 3DS Max actually helps him to be very productive.
  • Moritz Stefaner gave a very short but great introduction to Protovis, and how you could use Google Docs cleverly to use it as a real-time data source for your visualization
  • Justin Manor showed some more of his great installations with water, pneumatics, LED, Processing, Arduino, etc., and various ways of how they were made.

That wraps it up.

The conference was a blast! It was so inspiring, a great location, fantastic speakers, and very good talks! I am looking forward to SEE#7 conference.

If you want to see the full talks of SEE#6, go to http://www.see-conference.org/video-stream/. On the conference website you can also see video registrations of previous talks, which are highly recommended to watch!

Tutorial: Line chart in D3

One of the most common visualizations is a line chart. D3 is not a charting framework, but instead allows you to manipulate the document based on data. That’s what you’re actually doing with D3: adding elements to a document, removing them, updating them, etc. The advantage is that you are much more flexible in creating the visualization that you want. Some may consider it a slight disadvantage that you may do have to do some extra work to get things done.

For this tutorial we’re going to create a basic line chart with an x-axis and y-axis, tickmarks and labels.

First, we defined some variables:

var data = [3, 6, 2, 7, 5, 2, 1, 3, 8, 9, 2, 5, 7],
w = 400,
h = 200,
margin = 20,
y = d3.scale.linear().domain([0, d3.max(data)]).range([0 + margin, h - margin]),
x = d3.scale.linear().domain([0, data.length]).range([0 + margin, w - margin])

The data variable contains our dataset we want to display as a line chart. The w and h variables are used for the width and height of our chart and the margin will be used to create some room to display our labels. The y and x variables are our linear scale functions, and remember that you can use x and y as functions. The scales are used to convert our data values (which are defined in the domain of the scale) to x- and y-positions on the screen (which are defined in the range of the scale). Note the use of margin in the range.

Next we append an svg element to our document with the proper width and height, and then we append a g element to this svg element so that all the elements that will be appended to this g element will be grouped together. The transformation that we apply on the g element makes sure that our coordinate grid moves down 200 pixels.

var vis = d3.select("body")
    .append("svg:svg")
    .attr("width", w)
    .attr("height", h)

var g = vis.append("svg:g")
    .attr("transform", "translate(0, 200)");

In order to create a line chart, we’re going to add an path element to our visualization. This path element needs a d attribute that contains the data for the path. Now, you could write that yourself, but that may be a little hard to do, so we use the helper function d3.svg.line to do that for us (see Tutorial: Line interpolations in D3 for more details on this helper function). Now we add the line variable which is, just like the scales earlier, a function that you can call. So when we bind our data to the path, this d3.svg.line function will be called so that the value of the d attribute will be created:

var line = d3.svg.line()
    .x(function(d,i) { return x(i); })
    .y(function(d) { return -1 * y(d); })

The i parameter is the index of the current item, and the d is the current item itself. Note the usage of the scale functions (x() and y() to convert from i and d to x- and y-coordinates (note the -1. This is needed because right now the coordinate have a positive y-axis down, and since we have translated our g down 200 pixels, we need to use the negative values of the y-axis now). To use our line function we add a path element to our g element like this:

g.append("svg:path").attr("d", line(data));

Next up is adding the x-axis and y-axis. The following snippet adds them by appending a line to our g element.

g.append("svg:line")
    .attr("x1", x(0))
    .attr("y1", -1 * y(0))
    .attr("x2", x(w))
    .attr("y2", -1 * y(0))

g.append("svg:line")
    .attr("x1", x(0))
    .attr("y1", -1 * y(0))
    .attr("x2", x(0))
    .attr("y2", -1 * y(d3.max(data)))

Next are the labels. We can use a convenient function of the scales here: x.ticks() and y.ticks(). These functions will return the proper tickmarks, and you can provide how many you want. D3 does return nicely rounded numbers though. Also note the .text(String) usage for obtaining the string value of the current data item. The labels are aligned in the center by using .attr("text-anchor", "middle"). And for the y-labels I’ve used dy in order to vertically position the labels correctly.

g.selectAll(".xLabel")
	.data(x.ticks(5))
	.enter().append("svg:text")
	.attr("class", "xLabel")
	.text(String)
	.attr("x", function(d) { return x(d) })
	.attr("y", 0)
	.attr("text-anchor", "middle")

g.selectAll(".yLabel")
	.data(y.ticks(4))
	.enter().append("svg:text")
	.attr("class", "yLabel")
	.text(String)
	.attr("x", 0)
	.attr("y", function(d) { return -1 * y(d) })
	.attr("text-anchor", "right")
	.attr("dy", 4)

The last thing to do is to add the ticks themselves. These are actually just very short lines, so all you have to do is to provide the start x- and y position (x1 and y1) and the end x- and y-position (x2 and y2).

g.selectAll(".xTicks")
	.data(x.ticks(5))
	.enter().append("svg:line")
	.attr("class", "xTicks")
	.attr("x1", function(d) { return x(d); })
	.attr("y1", -1 * y(0))
	.attr("x2", function(d) { return x(d); })
	.attr("y2", -1 * y(-0.3))

g.selectAll(".yTicks")
	.data(y.ticks(4))
	.enter().append("svg:line")
	.attr("class", "yTicks")
	.attr("y1", function(d) { return -1 * y(d); })
	.attr("x1", x(-0.3))
	.attr("y2", function(d) { return -1 * y(d); })
	.attr("x2", x(0))

And that’s it. Finally, some of the styling of the objects was moved to a style section at the top of the page in order to get the final result:

path {
    stroke: steelblue;
    stroke-width: 2;
    fill: none;
}

line {
    stroke: black;
}

text {
    font-family: Arial;
    font-size: 9pt;
}

View a live version here.

Tutorial: The basics: working with arrays in D3

In this tutorial we’re going to cover some basic stuff in D3: ranges, min and max and merge. These are all pretty straightforward, but nevertheless very useful to have in your D3 toolbox!

d3.range()

Ranges are actually a pretty convenient way to generate a list of numbers. It is modeled after the built-in range method from Python. The full signature of the function looks like this:

d3.range(start, stop, step)

But the stop and step parameters are optional. So if you provide just the start parameter, you would do something like this:

d3.range(5);

The result of this function call is an array of numbers, starting with 0, and the last number in the array is 4, so 5 is not included:

[0, 1, 2, 3, 4]

Now, if you provide a second argument, it will be interpreted as the stop parameter. Note that the order of the parameters is: start first, and then the stop parameter. So, the result of this:

d3.range(3, 5);

is this:

[3, 4]

Note that again 5 is exclusive. At the moment it is not possible to create descending arrays directly with d3.range(), so if the start value is larger than the stop value, you get an empty array. In order to create a descending array of numbers, you can just call the standard reverse() Javascript function. So, to create the following result:

[4, 3, 2, 1]

you would write this:

d3.range(1, 5).reverse();

The last parameter you can use is the step parameter. If you don’t provide a step argument, it will default to 1. But if you do, then the result of this:

d3.range(5, 10, 2);

is this:

[5, 7, 9]

d3.min() and d3.max()

So, now you know how to generate a list of numbers in an easy way. Next up is the usage of min and max. It is really straightforward: to find the max number of an array of numbers, you use:

d3.max([4, 2, 6, 3, 8, 2]);

which will return 8 of course. Likewise, if you want to find the min number of an array, you use:

d3.min([4, 2, 6, 3, 8, 2]);

which will return 2 in this case.

d3.merge()

The d3.merge() can be used to merge an array of arrays so that it becomes one array. It’ll become clear when you see an example. If you have the following 2-dimensional array:

[[1, 2, 3], [4, 5, 6]]

you can turn this into a 1-dimensional array by calling the d3.merge() function:

d3.merge([[1, 2, 3], [4, 5, 6]]);

This will result in the following 1-dimensional array:

[1, 2, 3, 4, 5, 6]

The d3.merge() function will only merge one level. For example, the following 3-dimensional array will not result in a single 1-dimensional array:

d3.merge([[[1, 2], [3]], [[4, 5], [6]]]);

It will however, result in a 2-dimensional array:

[[1, 2], [3], [4, 5], [6]]

Tutorial: Line interpolations in D3

In this tutorial we’re going to explore line interpolations in D3.

First we start with 2 scales that we will use to convert values to x- and y-coordinates on the screen:

var x = d3.scale.linear().domain([0,10]).range([0,400]),
y = d3.scale.linear().domain([0,1]).range([0,50]),
groupHeight = 60,
topMargin = 100

Next we generate some random data:

var data = []
d3.range(10).forEach(function(d) { data.push(Math.random()) })

We’re also creating an array which contains all the possible interpolations D3 supports. We’ll see the effects of every interpolation in a moment:

var interpolations = [
	"linear",
	"step-before",
	"step-after",
	"basis",
	"basis-closed",
	"cardinal",
	"cardinal-closed"]

In SVG there is a difference between a line and a path. A line is a straight line where you define the start and end position of the line: , whereas with a path you draw the outline of any arbitrary shape by specifying a series of connected lines, arcs, and curves. You do this by specifying the d attribute of the path. Every path must begin with a moveto command. The command letter is a capital M followed by an x- and y-coordinate, separated by commas or whitespace. This command sets the current location of the “pen” that’s drawing the outline. This is followed by one or more lineto commands, denoted by a capital L, also followed by x- and y-coordinates, and separated by commas or whitespace. You can see more of this specification here.

In our example we’re not actually creating an SVG line, but an SVG path, so we need to set the d attribute of the paht. Luckily D3 has a helper function to ease the burden to create this data: d3.svg.line. For this helper function you can set:

  • an accessor function for obtaining x values
  • an accessor function for obtaining y values
  • an interpolation type, which defaults to linear
  • a tension value which affects the cardinal interpolations only

In our example, we want to show the different kinds of interpolations for the same data, so we create a function that takes the name of an interpolation as an argument, and then returns the d3.svg.line function as a result. This is the code that does that (you can play with the out-commented tension property to see the effect):

function getLine(interpolation) {
    return d3.svg.line().x(function(d,i) {
        return x(i)
    }).y(function(d) {
        return y(d)
    }).interpolate(interpolation)
//.tension(0)
}

Note the following: the function for x has 2 arguments: d and i. The d is the current item in the dataset (which we will provide later), and i is the index of the current item in the dataset. Also note that we’re using the x-scale to convert i to an x-coordinate, and the y-scale to convert the data value to a y-coordinate.

Now we initialize the visualization:

var vis = d3.select("body")
	.append("svg:svg")
	.attr("class", "vis")
	.attr("width", window.width)
	.attr("height", window.height)

Next up is greating a group for each of the lines we want to show:

var lg = vis.selectAll(".lineGroup")
	.data(interpolations)
	.enter().append("svg:g")
	.attr("class", "lineGroup")
	.attr("transform", function(d,i) {
	return "translate(100," + (topMargin + i * groupHeight) + ")"
}).each(drawLine)

We set the interpolations array as data for this group, so that svg:g elements for each of the interpolations will be added to the visualization. The svg:g element can be used to group other elements together, so that if you apply a transformation to the group for instance, it will be applied to all of its members. Note that we add the class lineGroup in our selection to select all these elements. Next we set the transform attribute, and we use the index to position the groups based on their position in the interpolation array. For each of the group, we want to draw a line. We do that by calling the drawLine function in the .each(drawLine) statement. The drawLine function itself looks like this:

function drawLine(p,j) {
	d3.select(this)
		.selectAll(".lineGroup")
		.data(data)
		.enter().append("svg:path")
		.attr("d", getLine(p)(data))
		.attr("fill", "none")
		.attr("stroke", "steelblue")
		.attr("stroke-width", 3)
		//.attr("stroke-dasharray", "15 5")
}

The drawLine function itself has to parameters: p and j where p is the parent data item (the current interpolation name), and j is the parent index. First we select the current element with d3.select(this), and next we select all the .lineGroup elements. We assign the line data to the data property, and append a path to each lineGroup element. The d attribute calls the getLine function and provides the current interpolation name as an argument. The result of that is the d3.svg.line function with the that uses the interpolation we just provided. Next we assign the line data to this function so that D3 will calculate the data string that will be used by the d attribute of the svg:path element. Finally we set some basic properties. The final out-commented is one of the stroke properties you can set, where 15 is the dash length and 5 is the gap length. Just play around with those properties to see what else is possible.

This concludes this tutorial. All the lines you see are using the same data, but they use different interpolations.

Tutorial: Conway’s Game of Life in D3


See the final version here.

This is an example of Conway’s Game of Life, built in D3. According to Wikipedia:

The Game of Life, also known simply as Life, is a cellular automaton devised by the British mathematician John Horton Conway in 1970.[1]The “game” is a zero-player game, meaning that its evolution is determined by its initial state, requiring no further input. One interacts with the Game of Life by creating an initial configuration and observing how it evolves.

There are only 4 rules in the Game of Life:

The universe of the Game of Life is an infinite two-dimensional orthogonal grid of square cells, each of which is in one of two possible states, live or dead. Every cell interacts with its eight neighbours, which are the cells that are horizontally, vertically, or diagonally adjacent. At each step in time, the following transitions occur:

  1. Any live cell with fewer than two live neighbours dies, as if caused by under-population.
  2. Any live cell with two or three live neighbours lives on to the next generation.
  3. Any live cell with more than three live neighbours dies, as if by overcrowding.
  4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

    The initial pattern constitutes the seed of the system. The first generation is created by applying the above rules simultaneously to every cell in the seed—births and deaths occur simultaneously, and the discrete moment at which this happens is sometimes called a tick (in other words, each generation is a pure function of the preceding one). The rules continue to be applied repeatedly to create further generations.

    I’m sure there are multiple ways of implementing Conway’s Game of Life, but this is just one of them. We start by declaring a few variables:

    			var ccx = 120, // cell count x
    				ccy = 30, // cell count y
    				cw = 5, // cellWidth
    				ch = 5,  // cellHeight
    				del = 100, // delay
    				xs = d3.scale.linear().domain([0,ccx]).range([0,ccx * cw]),
    				ys = d3.scale.linear().domain([0,ccy]).range([0,ccy * ch]),
    				states = new Array()
    

    The states variable will be used to hold the states of each cell: true for on and false for off. Next up, we’re going to fill the states array with data:

    d3.range(ccx).forEach(function(x) {
    	states[x] = new Array()
    	d3.range(ccy).forEach(function(y) {
    		states[x][y] = Math.random() > .8 ? true : false
    	})
    })
    

    This is a good example of the use of d3.range([start], stop, [step]) function which returns a range of number. We’re using only the stop argument, so in our case the first use of the range() function will generate an array of 0 (which is the default start) to 120, and uses the default step of 1. As you can see we’re building a 2-dimensional array here, so that we can easily access each state, for example states[0][0] to access the first state. We randomly set the state to either true or false.

    I have created the toGrid() function so that the 2-dimensional array is turned into an array of Objects, so that D3 can easily bind all the values to an SVG element (binding to the 2-dimensional array directly should also be possible, which I leave as an exercise for you at the moment…):

    			function toGrid(states) {
    				var g = []
    				for (x = 0; x < ccx; x++) {
    					for (y = 0; y < ccy; y++) {
    						g.push({"x": x, "y": y, "state": states[x][y]})
    					}
    				}
    				return g
    			}
    

    Now we initialize the visualization:

    			var vis = d3.select("body")
    				.append("svg:svg")
    				.attr("class", "vis")
    				.attr("width", window.width)
    				.attr("height", window.height)
    

    After that we create the initial state of the grid:

    vis.selectAll("rect")
    	.data(function() { return toGrid(states) })
      .enter().append("svg:rect")
    	.attr("stroke", "none")
    	.attr("fill", function(d) { return d.state ? "green" : "white" })
    	.attr("x", function(d) { return xs(d.x) })
    	.attr("y", function(d) { return ys(d.y) })
    	.attr("width", cw)
    	.attr("height", ch)
    

    Note that the data that is bound to svg:rect elements is the result from the toGrid() function. Also, there are multiple ways to show the on and off state of a cell, for instance using the visibility property. I chose to use the fill property in this case. It is either colored green or white (which is the background color) based on the state. This is all that’s needed to create the initial grid. Now comes the fun part: creating the new generations:

    			function createNewGeneration() {
    				var nextGen = new Array()
    
    				for (x = 0; x < ccx; x++) {
    					nextGen[x] = new Array()
    					for (y = 0; y < ccy; y++) {
    						var ti = y - 1 < 0 ? ccy - 1 : y - 1 // top index
    						var ri = x + 1 == ccx ? 0 : x + 1 // right index
    						var bi = y + 1 == ccy ? 0 : y + 1 // bottom index
    						var li = x - 1 < 0 ? ccx - 1 : x - 1 // left index
    
    						var thisState = states[x][y]
    						var liveNeighbours = 0
    						liveNeighbours += states[li][ti] ? 1 : 0
    						liveNeighbours += states[x][ti] ? 1 : 0
    						liveNeighbours += states[ri][ti] ? 1 : 0
    						liveNeighbours += states[li][y] ? 1 : 0
    						liveNeighbours += states[ri][y] ? 1 : 0
    						liveNeighbours += states[li][bi] ? 1 : 0
    						liveNeighbours += states[x][bi] ? 1 : 0
    						liveNeighbours += states[ri][bi] ? 1 : 0
    
    						var newState = false
    
    						if (thisState) {
    							newState = liveNeighbours == 2 || liveNeighbours == 3 ? true : false
    						} else {
    							newState = liveNeighbours == 3 ? true : false
    						}
    
    						nextGen[x][y] = newState
    					}
    				}
    
    				return nextGen
    			}
    

    This function implements the Game of Life rules mentioned earlier. We’re building a new 2-dimensional array here that will eventually be used to replace the value of the states variable. We’re determining the top, right, bottom and left index to use for each cell. If one of the index numbers would fall out of the range (greater than the length, or smaller than 0), then the index of the opposite side is used. For example, if we are currently at cell(0,5) (which means x = 0 and y = 5), then to calculate li (left index) we end up with index -1. This of course does not exist, so we use ccx.length - 1 instead, which is 199. This way all the cells will have a top, right bottom and left to work with. Next the number of liveNeighbours is calculated by summing up the number of true states of 8 neighbour cells. Finally the new state for this cell is calculate by actually applying the Game of Life rules. The new state is stored in the temporary array, which is being returned as the result of the function.

    The last part we need to do is to create new generations repeatedly and animate the grid accordingly:

    			function animate() {
    				states = createNewGeneration()
    				d3.selectAll("rect")
    					.data(toGrid(states))
    				  .transition()
    					.attr("fill", function(d) { return d.state ? "green" : "white" })
    					.delay(del)
    					.duration(0)
    			}
    
    			setInterval("animate()", del)
    

    This is done by the setInterval() Javascript function. We call the animate function with a delay of 100 milliseconds (the value of the del variable). The animate function itself is pretty straightforward. The value of the states variable is replaced with a new generation. Then all the rect elements are selected and the new generation is bound to these rectangles (note that again the 2-dimensional array is converted with the toGrid() function. After that we define the transition() we want to apply, and all we do is modify the fill property of each cell. Setting the delay to the del value as well seems to be working quite well. I guess this helps the browser to have enough time to calculate the new generation. We explicitly set the duration to 0 to override the default.

    That’s all there’s to it. I thought it would be more complex to build the Game of Life in D3, but it appears to be fairly straightforward. This code of course does require some calculation power from your browser, so just play around with the delay or grid size to get an optimal setting that works for you. Also, you can play with various other attributes to create interesting effects, for instance, using this for creating the grid gives a nice blurry effect:

    			vis.selectAll("rect")
    				.data(function() { return toGrid(states) })
    			  .enter().append("svg:rect")
    				.attr("stroke", "none")
    				.attr("fill", function(d) { return d.state ? "green" : "white" })
    				.attr("fill-opacity", .3)
    				.attr("x", function(d) { return xs(d.x) })
    				.attr("y", function(d) { return ys(d.y) })
    				.attr("width", function() { return 2 * cw })
    				.attr("height", function() { return 2 * ch })
    

    Enjoy!

    Tutorial: Introduction to D3

    Check out the final result here!

    D3 is a brand new visualization framework created by Mike Bostock. It is the successor of the successful great visualization framework Protovis. There are a few differences and similarities with Protovis. One of the most important differences is that in D3 you work more directly with SVG which gives you much greater flexibility than Protovis. Also, the performance of D3 is much better than Protovis, especially with animation, because in D3 only the properties that are changing are updated, instead of re-rendering the entire visualization. And once you dive into D3, it’s really easy to pick up, so let’s get started!

    The example we’re going to work on is really simple and will show you some of the basic concepts of D3. Basically we just plot hidden circles randomly on the screen, and then transition them to a portion of the screen. Then we add some interaction to it so that the circles will move once you move your mouse over them.

    The first thing you need to do is to make a reference to D3 from your HTML page. You can download D3, or make a link to a stable version on GitHub. The reference I used for this tutorial is: https://github.com/mbostock/d3/raw/v1.8.2/d3.js

    To get started we generate some data that we want to bind the circles to. All we do here is just generate a bunch of x and y values and put them in an array:

    var data = []
    for (i=0; i &lt; 1000; i++) {
        data.push({"x": Math.random(), "y": Math.random()})
    }
    

    Now we need to add an SVG element to the body of the page. This is how you do that:

    var h = 1000
    var vis = d3.select("body")
        .append("svg:svg")
        .attr("width", screen.width)
        .attr("height", screen.innerHeight)
    

    First the body is selected using d3.select("body") which is a similar selection you would do with jQuery selectors. The h variable is the height I want to use. I refer to it later in the code. Next we’re going to plot the invisible circles:

    var x = d3.scale.linear().domain([0,1]).range([screen.width / 2 - 400,screen.width / 2 + 400]),
    y = d3.scale.linear().domain([0,1]).range([0,h]),
    r = d3.scale.linear().domain([0,1]).range([5,10]),
    c = d3.scale.linear().domain([0,1]).range(["hsl(250, 50%, 50%)", "hsl(350, 100%, 50%)"]).interpolate(d3.interpolateHsl)
    
    vis.selectAll("circle")
    	.data(data)
    	.enter().append("svg:circle")
    	.attr("cx", function(d) { return x(d.x) })
    	.attr("cy", function(d) { return y(d.y) })
    	.attr("stroke-width", "none")
    	.attr("fill", function() { return c(Math.random()) })
    	.attr("fill-opacity", .5)
    	.attr("visibility", "hidden")
    	.attr("r", function() { return r(Math.random()) })
    

    First we create some d3.scale variables. Since we’re working with random data, we need to convert the output of Math.random() outputs to screen positions, radii and colors. We use linear scales, and you see that the domain is [0,1] for all the scales. That’s because the output of Math.random() yields a number between 0 and 1. The range tells us to which range the domain should be converted. Each of these variables is actually a function, so you can use r for example as a function to convert a number of the domain to a number of the range, so for example: r(.5) will result in 7.5. The c variable will turn any value between 0 and 1 to a color.

    Next we select all the circle elements, which is an empty collection at this time. We bind our data to this collection with the .data(data) statement, and then use the .enter().append("svg:circle") statement so that each element in our data array will be bound to a new svg:circle element. Here we set various properties: cx for the x-position of the circle (note that in SVG position 0,0 is the top left corner). The cy is used for the y-position of the circle, the r is used for the radius of the circle. The other properties speak for themselves. Note that a difference with Protovis is that you cannot use the shorthand way for using a function: function(d) x(d.x), but in D3 you have to write out the return keyword and the curly braces: function(d) { return x(d.x) }.

    Now, when you view this page, you’ll see nothing, because we’ve set the .attr("visibility", "hidden"). If you want to see the results so far, just remove this line. Next we want to move all those hidden circles to their new random position, so that together they will form the colorful bar. This is the code that does just that:

    var y2 = d3.scale.linear().domain([0,1]).range([h/2 - 20, h/2 + 20])
    var del = d3.scale.linear().domain([0,1]).range([0,1])
    
    d3.selectAll("circle").transition()
    	.attr("cx", function() { return x(Math.random()) })
    	.attr("cy", function() { return y2(Math.random()) })
    	.attr("visibility", "visible")
    	.delay(function(d,i) { return i * del(Math.random()) })
    	.duration(1000)
    	.ease("elastic", 10, .45)
    

    First we need 2 extra scales to convert random numbers: y2 which is the scale that converts a random number to a new range. The del scale will be used for the delay of the transition. We now first select all the circle elements and we start a transition(). We set a few properties of the transition itself: delay, duration and ease, and also the properties of the selected circles elements that we want to animate. The cx, visibility and cy properties of the circle will all be animated. All you have to do is provide the end state for the properties you want to animate. The animation will last 1000 milliseconds, there will be some delay for each circle before starting the transition, and we apply an elastic easing function to create the nice elastic bouncing effect. Go see what you’ve got so far! Fun, isn’t it?

    The last part will be a little extension of the piece of code we used to add the circles initially, so change your code so that it looks like this:

    vis.selectAll("circle")
    	.data(data)
    	.enter().append("svg:circle")
    	.attr("cx", function(d) { return x(d.x) })
    	.attr("cy", function(d) { return y(d.y) })
    	.attr("stroke-width", "none")
    	.attr("fill", function() { return c(Math.random()) })
    	.attr("fill-opacity", .5)
    	.attr("visibility", "hidden")
    	.attr("r", function() { return r(Math.random()) })
    	.on("mouseover", function() {
    		d3.select(this).transition()
    		.attr("cy", function() { return y2(Math.random()) })
    		.delay(0)
    		.duration(2000)
    		.ease("elastic", 10, .3)
    	})
    

    A mouseover event has been added to each of the circles. And basically we’re doing something similar again: just add a transition, set some transition properties and the properties of the selected elements. In this case d3.select(this) selects the current circle so that you apply the transition to the current selected circle.

    That’s it!

    A few highlights of the InfoGraphics Conference 2011

    Yesterday I visited the Dutch InfoGraphics Conference in Zeist. It was a great day, met some nice people and saw some good talks.

    The keynote was presented by Gert K. Nielsen, the founder of VisualJournalism.com. Though he is a great speaker and had some valid points, I also disagreed with many of his views. His opinion was that most infographics and data visualizations actually tell you nothing, give no information and remove the truth and emotion from reality (He illustrated this with ‘bad’ examples on visualcomplexity.com, including my World Economic Forum visualization :) ) He wanted to make his point by showing movies of people committing suicide, people being blown up by bombs etc., and then showing an infographic that only shows an explosion symbol on a map as a representation of these events. I agree that over simplification may not be right, but it really depends on what you are trying to communicate, to what audience, and with what purpose. So, although interesting, I also thought his views were actually to simplistic.

    A very animated talk was given by John Grimwade. He showed us how the tablet will be the platform infographic designers will eventually be dealing with. His talk was fun with lots of jokes and very animated, but he didn’t have a very strong message in my opinion.

    Bas Broekhuizen gave an update on his scientific research on interactive infographics. Too bad the talk was rather short, because this may be quite interesting, especially since the definition of an interactive infographic may be somewhat fuzzy. Wouter Kroese showed some very nice animations he does for the NCRV. The animations are very clean and Wouter Kroese won one of the Infographics Awards for 2010. The last talk by the Gert Kuiper of the VPRO was very interesting: a preview of the upcoming Dutch documentary ‘Holland from above’ was shown. Quite impressive, and similar to the UK version: Britain from above.

    There  were more speakers but the ones mentioned above moved me most in some way. I’m looking forward to next years conference!

    Lorem ipsum dolor sit amet consectetur

    Etiam malesuada fringilla est a varius. Praesent quis dolor quis orci venenatis placerat. Maecenas facilisis tristique ipsum, at ultricies leo iaculis eget. Nunc ac tincidunt felis. Phasellus ut dui nisl, in tempus est. Mauris porttitor cursus eros sed luctus. Integer gravida congue quam, eu tempor nulla consectetur vel.

    Phasellus bibendum ultricies mi, sit amet sagittis sapien sagittis in. Suspendisse massa enim, sagittis eget pulvinar sit amet, cursus molestie lacus. Nam malesuada accumsan venenatis. Mauris eget urna odio, ac scelerisque mi. Aliquam fermentum velit quis est dignissim placerat. Quisque a tellus eu turpis laoreet ornare.

    Phasellus bibendum ultricies mi, sit amet sagittis sapien sagittis in. Suspendisse massa enim, sagittis eget pulvinar sit amet, cursus molestie lacus. Nam malesuada accumsan venenatis. Mauris eget urna odio, ac scelerisque mi. Aliquam fermentum velit quis est dignissim placerat. Quisque a tellus eu turpis laoreet ornare.

    Phasellus bibendum ultricies mi, sit amet sagittis sapien sagittis in. Suspendisse massa enim, sagittis eget pulvinar sit amet, cursus molestie lacus. Nam malesuada accumsan venenatis. Mauris eget urna odio, ac scelerisque mi. Aliquam fermentum velit quis est dignissim placerat. Quisque a tellus eu turpis laoreet ornare.