What Google Autocomplete tells us about countries

Together with my collaborators in the “Information Geographies” project at the Oxford Internet Institute, I have published a blogpost that analyses Google Autocomplete. This is seemingly a popular pastime these days, but unlike the maps I have seen so far, we don’t just map the most prominent term for each country but actually visualise multiple categories, often in one map.

Why does it matter?

Autocomplete is Google’s ‘type-ahead’ suggestion algorithm: As soon as you enter a word or two into the Google Search field, the algorithm will try to guess the completion of your query and offers you a list of likely queries. This functionality is baked into Google’s interface and cannot be turned off by the user.

It’s unclear if and how much such algorithms affect our perception of the subjects that we are querying for. But we can certainly say that they reduce serendipity and can help reinforcing filter bubbles.

 

Google-Autocomplete-countries-Oxford-Internet-Institute-Ralph-Straumann

 

Data acquisition and cartographic technique

The data that went into these maps Continue reading “What Google Autocomplete tells us about countries”

Happy 2014!

2013 has been a very interesting year for me: I rebuilt my website, started to self-host this blog and successfully applied for it to be included in the Planet Geospatial aggregator. I also revamped the Ernst Basler + Partner GIS blog. From March to May I had an unpaid leave from Ernst Basler + Partner (EBP) and worked in the … Continue reading Happy 2014!

On the Beyoncé tweet map

Kenneth Field, Senior Cartographic Product Engineer with Esri and controversial blogger (to some, at least), felt compelled to write a critique of Simon Rogers‘ Beyoncé twitter map (on occasion of an album release) that was hyped by Time Magazine (in a punny way) as being “flawless” (to be very clear: the “flawless” attribute does not originate from the map author and … Continue reading On the Beyoncé tweet map

Ubiquity of Python

I have written about the ubiquity of Python before and I’ve also given a talk in that direction at this year’s FOSSGIS conference (“Python as ‘glue’ in the GIS software domain: Sun glare analysis of road traffic accidents”). There is also a video of my talk here (but beware, it’s in German). So, this post by Tal … Continue reading Ubiquity of Python

The online perspective: Africa on Wikipedia

My friends and colleagues at the Oxford Internet Institute, Mark Graham and Stefano de Sabbata, are compiling a collection of maps and visualizations at geography.oii.ox.ac.uk that give insight into the contemporary geographies of knowledge:

  • Which places are connected with the online world? Which places are participants in the so-called participatory Web?
  • Which places are represented in online gazetteers, in online photo repositories, in social networks?
  • What places are reported on in traditional newspapers, in online events databases?
  • And, most importantly and in completion to all above questions: Which places aren’t?

Continue reading “The online perspective: Africa on Wikipedia”

Hexagons, quasi-maps and cartograms

Some months ago I mapped Switzerland’s administrative regions using a hexagonal cartogram. That idea was inspired by pioneering work of the Leicestershire County Council. After consulting with experts in the field – Leicestershire’s Alex Lea, Danny Dorling of Worldmapper (currently at the University of Oxford) and Adrian Herzog of MAPresso fame –  I came up with my own workflow for the creation of such a cartogram, documented here and here. (Though beware: Mike Bostock liked the results but found it relatively laborious when he made one).

Ever since I published that cartogram, I have the feeling to see hexagonal maps everywhere. Most recently, my friends and colleagues at Oxford Internet Institute, Stefano de Sabbata and Mark Graham, have published the following cartogram along with their now famous map “Ages of Internet Empires”:

Internet Empires by OII
Internet Empires by OII

Another nice example, though not a cartogram, is this visualization of New York transit times (let’s not talk of the rainbow colour scale for a moment): Continue reading “Hexagons, quasi-maps and cartograms”

Keynote about trends in GIS and visualization

People at Zurich electricity provider ewz have recently kindly invited me to give a keynote presentation at one of their telematics events.

The task was to sketch emerging trends in GIS and visualisation from three perspectives: as GIS consultant at Ernst Basler + Partner, as a researcher at Oxford Internet Institute and as an individual that sometimes undertakes GIS and geovisualisation projects. I ended up highlighting four big trends that I think will continue to control developments in GIS/visualisation (although I will concede that certain aspects of these trends are over-hyped here and there). The actual keynote consisted of significant interactive parts, but I embed the slides here in case you want to check out trends and examples: Continue reading “Keynote about trends in GIS and visualization”

Reworked versions of my hexagonal population cartogram

In March, I have published a linked view display with a population cartogram of Switzerland (in German, in French). The occasion was a federal poll that convinced the majority of the voting population but didn’t gain support in enough many cantons. The cartogram has sparked quite some interest and I have covered its conceptualisation as … Continue reading Reworked versions of my hexagonal population cartogram

Zurich Sky

… is the title of my most recent project. It’s a bit artsy, but I think some of the concepts behind it may also have practical applications in this world of ever more abundant webcam footage (maybe need to think a bit more on this point later).

In Zurich Sky, I destilled yearly aggregates of the sky over Zurich Switzerland. I did this by first scraping tons of images from the website of the Swiss domain registrar SWITCH. They have two webcams, one in Zurich one in the Alps, whose images are publicly accessible in their archive (thanks!).

Zurich Sky: Web-scraped sky colour over Zurich, Switzerland
Zurich Sky: Web-scraped sky colour over Zurich, Switzerland

The SWITCH archive features one image every hour. Luckily for my project, the URLs of the individual pictures adhere to a nice structured format which makes automatic downloading of several thousand images rather easy. An example is here:

http://cam.switch.ch/pano/2013/0621/zueri_n201306211300.jpg
(as can be seen from this webpage) Continue reading “Zurich Sky”

Creating a hexagonal cartogram

Some weeks ago I visualised the Swiss cantons (states) and their population numbers using what information visualization scientists call a linked view. You can click through to the actual, interactive visualization: here in German or here in French. In what follows I’ll describe the steps of data preparation for this visualization. I decided to keep the specifics on the implementation in D3.js for a third post in order to spare your scroll-wheel and -finger (so stay tuned for that one). 

Intro

Welcome to the second part of this series in which I describe the production of this linked view with a population cartogram (top right):

Image

In case you missed it: in the first post of this series, you can read about the conceptual thinking that went into this visualization. But now let’s dive into some geodata-crunching:

Technically

GIS pre-processing

In what follows, I’ll try to give you a thorough description of my approach at data processing. I’ll include some screenshots of intermediate results. Obviously, I don’t know how familiar you are with GIS and spatial analysis terminology, so please bear with me if my description is too exhaustive. Conversely, speak up in the comments section, if I have forgotten something or something is not clear. I did all of the GIS analysis in Esri ArcGIS, however, any GIS that can handle vector data will do.

I started off with the following input data:

  • Outlines of administrative units (cantons and cities)
  • Spatially distributed population data from Swiss census

The preparation of the administrative units was quite straightforward: I applied a Union operation in GIS (ArcGIS Help Topic here). Then I did some tidying of the attributes and applied a set of geometric simplifications (polygon outline generalisations). The purpose of these is basically weeding out vertices from the geometries while preserving shape as well as possible. The bigger goal being, of course, simplifying the geometries enough for a fluid web experience down the line.

Swiss census data comes as a point grid at 100 meters resolution. Precise data characteristics don’t matter too much. And one could also use a thematic variable that comes at the same resolution as the display units – cantons and cities in this case. While the handling of canton/city level thematic data would be much easier, the spatially distributed thematic variable in this case allows for a more representative cartogram. If you wonder why, consider, for example, a US setting: Salt Lake City would cause a big local distortion in a cartogram using spatially distributed data, whereas its population would be spread out uniformly throughout all of Utah, if you use state-level data. This effect causes visible differences in the cartogram in regions where population distribution is not spatially uniform.

The GIS processing chain starts with these steps:

  • Generation of a grid (in my case at 5 km resolution, but that number is a bit dependent on the resolution of your input data, your area of interest and maybe your application; as a rule of thumb, I’d suggest a grid resolution that is similar to the size of your hexagons). Any regular tesselation other than a rectangular grid will also do.
  • Union operation on the grid cells and the administrative units. This yields smaller spatial analysis units, that follow the boundaries between administrative units.
  • Spatial join of thematic variable to the new spatial units. A spatial join is a GIS operation where the spatial relationship of entities in two different datasets is evaluated. If a specified relationship is fulfilled, the characteristics of the features in the join dataset are joined to the features in the target dataset. The spatial relationship for this operation was containment (i.e. the criterion was: is a given census data point within the spatial unit at hand?). The join operation encompassed summing up the values. The overall process yields the sum of the population at all census data points which fall within a given spatial analysis unit – or, without the GIS lingo: the total population per unit).

 

For distortions you need a Scape… toad

The resulting data in Shapefile format was then transferred to the cartogram software Scapetoad. Scapetoad is a freely available Java software developed in the Choros Laboratory at EPFL in Lausanne. It employs the diffusion-based cartogram algorithm by Gastner–Newman. I did several model runs and iteratively tuned the algorithm parameters. That encompassed mainly striking an acceptable balance between subjective quality of the result and cartogram computation time. Unfortunately, I cannot give heuristics for this, you’ll really simply have to try with your data.

When I was happy with the result, I re-imported the cartogram dataset from Scapetoad into the GIS and used a Dissolve operation to aggregate the units back into regions (again, any GIS will do, but the precise name for the operation may vary).

Cartogram production part 1: (1) Preparation of cantons and cities dataset (2) Union of dataset with grid (3) Import into Scapetoad and distorting (4)
Cartogram production part 1: (1) Preparation of cantons and cities dataset (2) Union of dataset with grid (3) Import into Scapetoad and distorting (4) Re-import into GIS and dissolving the geometries

Continue reading “Creating a hexagonal cartogram”

Conceptualisation of a D3 linked view with a hexagonal cartogram

Some weeks ago I visualised the Swiss cantons (states) and their population numbers using what information visualization scientists call a linked view. You can click through to the actual, interactive visualization: here in German or here in French. In what follows I want to give a bit more detail about what led to this visualization and what conceptual thinking went into the design.

In a subsequent post I will also describe the toolset I used to produce this visualization, so that you can build your own. If you’re not interested in the Background, you can skip to the Conceptually section. If that’s neither your cup of tea and you’re here primarily because you want to know how to produce such a visualization yourself, you’ll unfortunately have to patience yourself and wait for the second part of this series (it’s here!).

Image

Background

Why population sizes matter – in such a small country

Why is the particular piece of information that is visualised here important or interesting? Well, in the Swiss political system cantons are represented at the federal level, whereas cities aren’t. However, some of the big cities represent a considerably larger number of people than quite some of the smaller cantons. There have been many debates if and how cities ought to be represented in the political system, about the specificity of urban issues and how those are dealt with or ignored in Swiss politics and if weighting of the cantons should be adapted to better match their population size. The issue crops up both in relation to elections and polls (Switzerland having a direct democracy there are really many of the latter).

Cartogram on Swiss TV
Cartogram on Swiss TV

When I published the visualization Switzerland has just held such a poll. The poll did not pass, it achieved only 54.3% of “yes” votes.

– Wait, what? Yep, the vote won a solid majority of the people, but too many cantons said “nay” and thus, by the rules, it was a “nay”.  Now, one can argue that this is not sensible or that it is perfectly sensible, I’m not going to do this here. But this background means, to my pleasure, that the visualisation was able to spark and inform many discussions (and met quite an audience). To my big surprise, it was even briefly featured in nation-wide primetime news, in a slightly reworked version. Continue reading “Conceptualisation of a D3 linked view with a hexagonal cartogram”