{"id":35527,"date":"2017-08-03T17:38:15","date_gmt":"2017-08-03T15:38:15","guid":{"rendered":"https:\/\/www.hiig.de\/?p=35527\/"},"modified":"2023-03-28T17:21:12","modified_gmt":"2023-03-28T15:21:12","slug":"personalized-search-results-elections","status":"publish","type":"post","link":"https:\/\/www.hiig.de\/en\/personalized-search-results-elections\/","title":{"rendered":"How significant is algorithmic personalization in searches for political parties and candidates?"},"content":{"rendered":"

Of the many things we do online, search is probably the most common. Beyond messaging, entertainment, shopping, and a myriad of other activities, utilizing search engines, such as Google, to seek information about politics is now considered commonplace. The fact that search engines can be used to find virtually any type of information becomes especially relevant in elections, when voters are likely to rely on Google to inform themselves about and connect with political parties and candidates.<\/em><\/p>\n

Katharina Zweig: Analytical results (in German)<\/a><\/p>\n

The issue<\/h3>\n

As part of the postdoctoral research network \u201cAlgorithmed Public Spheres\u201d<\/a> at the Hans Bredow Institute for Media Research<\/a>, we study the outcomes of algorithmic personalization. Ideas such the filter bubble<\/a> \u2013 where we are constantly fed information from websites that we have clicked on in past \u2013 are widely discussed. These ideas are also the object of intense academic debate, as research by our fellows demonstrates<\/a>. I was particularly interested in how these issues play out in political searches: How similar are the results of Google searches for the names of political parties and candidates? What role could personalization play in these differences?<\/p>\n

The data<\/h3>\n

Beginning to answer these questions is normally quite difficult, because we lack access to other individual\u2019s search results. Though we often assume that they are similar to our own, this is anything but certain. And while search engines can be automatically queried, it is difficult to get results that are similar to those obtained by actual users with real browsing histories \u2013 in other words, data that is ecologically valid(*). Lucky for us, the lovely people at Algorithm Watch<\/a> are currently running a crowdsourcing project on this exact issue, called Datenspende BTW17<\/a>. The project is supported by six \u201cLandesmedienanstalten\u201d, the regulatory authorities for new media in the German federal states of Bavaria (BLM), Berlin-Brandenburg (mabb), Hesse (LPR Hessen), Rhineland-Palatinate (LMK), Saarland (LMS), Saxony (SLM), as well as the University of Kaiserslautern. Through a browser plugin, users donate their search results to the project. The plugin does not collect actual<\/em> searches of users, but automatically conducts searches for a fixed list of 16 terms. The results are then sent back to AW\u2019s servers. This approach creates a high degree of comparability. If two users with similar browser settings and location search for the same phrase (say \u201cAngela Merkel\u201d) at the same time, shouldn\u2019t they receive identical (or at least very similar) results? Thankfully, Algorithm Watch makes public the anonymized search results that users collect<\/a> to answer questions like this one.<\/p>\n

So far, close to 4,000 contributors to the experiment have donated the results of over 2 million queries. The number of individual search results is considerably higher, since each query returns 12 (search) and 20 (news) results; the equivalent of the first page of results. Each result contains thirteen variables (see below).<\/p>\n

\"Data<\/a><\/p>\n

To answer these questions, we examined a fairly restrictive subset of the overall data. We included only results returned by Google Search, rather than Google News, published between July 5th and July 20th. This still includes a considerable number of news sources, as these also shows up in Google Search as \u2018Top Stories\u2019 (see the screenshot below). We only included results with the language setting \u201cde\u201d or \u201cde-DE\u201d, ignoring other languages (including Swiss German). We also only included results where the location was given as \u201cDE\u201d, in some (but not all) cases followed by the exact postal code. Furthermore, we removed some results that we believe to be duplicates, which were likely generated through the data collection process. Finally, for the analysis below we only included the first three results by rank for each individual query. This may seem like a strange choice, but we wanted to stress that even when only looking at this small subset of results, the degree of variation is quite interesting.<\/p>\n

In a next step, we split the data into searches by users who were logged into a Google account and those who were not. Having a Google account means that, at least in theory, your results could be more thoroughly personalized, since more information is available to Google when you conduct a search. In practice, many other things can (and are) used to personalize results, including your language, location, type of browser and (most importantly) browser history as stored in various tracking cookies. While the degree to which personalisation is possible is likely to vary a lot, it is safe to assume that Google has some indicators of your interests when you make a search.<\/p>\n

How personalized are results for users who are logged into a Google account?<\/strong><\/h3>\n

To assess the results, we calculated the share of each website (e.g. \u201cwikipedia.org\u201d) for each search query (\u201cAngela Merkel\u201d), differentiating between whether users were logged in or not. Below are the top six results for Angela Merkel, with the total share of that site among the top three results for users who were logged in and the difference for users who were not logged in.<\/p>\n

    \n
  1. bundeskanzlerin.de (logged in: 14%, not logged in: -1%)<\/li>\n
  2. spiegel.de (logged in: 13%, not logged in: +2%)<\/li>\n
  3. wikipedia.org (logged in: 11%)<\/li>\n
  4. facebook.com (logged in: 10%)<\/li>\n
  5. focus.de (logged in: 8%, not logged in: +2%)<\/li>\n
  6. instagram.com (logged in: 7%, not logged in: +1%)<\/li>\n<\/ol>\n

    Unsurprisingly, Angela Merkel\u2019s official page as head of the German government comes in at first place. By contrast, her page as a parliamentarian and head of the Christian Democratic Union (CDU), angela-merkel.de, comes in at rank # 17 (logged in: 2%, not logged in: 1%). Der<\/em> Spiegel<\/em> and Focus<\/em> are news sources, Facebook and Instagram obviously represent her social media accounts and Wikipedia her bio, though we did not differentiate between German-language Wikipedia and other national Wikipedias here. The picture looks quite similar for her main challenger, Martin Schulz, the main difference being that Instagram is replaced by Twitter and Focus<\/em> by Frankfurter Allgemeine Zeitung (FAZ)<\/em>, another news site.<\/p>\n

      \n
    1. spiegel.de (logged in: 15%, not logged in: +2%)<\/li>\n
    2. welt.de (logged in: 10%, not logged in: +1%)<\/li>\n
    3. facebook.com (logged in: 9%)<\/li>\n
    4. martinschulz.de (logged in: 9%)<\/li>\n
    5. twitter.com (logged in: 8%)<\/li>\n
    6. faz.net (logged in: 8%, not logged in: -1%)<\/li>\n<\/ol>\n

      However, Schulz\u2019 own page comes in lower than Merkel\u2019s, in large part because her page as head of the government is so prominent \u2013 this popularity is hard to match for an opposition leader. If you are wondering why Der Spiegel<\/em> is so prominent, this is likely to be in part the result of selection bias. Algorithm Watch recruited the participants in Datenspende BTW17 through spiegel.de, so an outsized share of users in the sample are likely to be spiegel.de readers, to which Google recommends more stories from that source.<\/p>\n

      What about the difference between users who are logged in and users who are not? The difference does not appear very important if you look only at the results above, but the distinction becomes clearer once you look at the data in more detail. Big sites like Der<\/em> Spiegel<\/em> and Focus<\/em> \u2013 but also the candidate\u2019s pages and social media presences \u2013 consistently have a larger share of the overall results for users who aren\u2019t logged into Google. This can be explained by a long tail of sites which are only<\/em> part of the search results of users who are logged in, e.g. users that Google knows more about and can accordingly offer more tailored results.<\/p>\n

      Here are all the sites for Angela Merkel present in the \u201clogged in sample\u201d but not in the \u201cnot logged in\u201d sample:<\/p>\n

      20minutos.es
      \nabendzeitung-muenchen.de
      \nbernerzeitung.ch
      \nbloomberg.com
      \nbuzzfeed.com
      \nbz-berlin.de
      \ncidob.org
      \ndailystar.co.uk
      \ndeutsche-wirtschafts-nachrichten.de
      \ndeutschlandfunk.de
      \ndw.com
      \nelmundo.es
      \nfirstpost.com
      \nfortune.com
      \nft.com
      \ngoogle(**)
      \ngoogle.com(**)
      \nhuffingtonpost.com
      \nindependent.co.uk
      \nksta.de
      \nlatimes.com
      \nln-online.de
      \nmsn.com
      \nndtv.com
      \nnine.com.au
      \nperfil.com
      \nrt.com
      \nrundschau-online.de
      \nslate.com
      \ntelegraph.co.uk
      \nthedailybeast.com
      \nthehill.com
      \nthelocal.de
      \nthetimes.co.uk
      \ntichyseinblick.de
      \nvox.com
      \nwashingtonexaminer.com
      \nwashingtonpost.com
      \nzdf.de<\/p><\/blockquote>\n

      And here is the same for Martin Schulz:<\/p>\n

      deutsche-wirtschafts-nachrichten.de
      \ngolem.de
      \ngoogle(**)
      \ngoogle.com(**)
      \nthelocal.de
      \nwaz.de<\/p><\/blockquote>\n

      Why the difference between the two? This is likely to be conditioned both by who searches for Angela Merkel vs. Martin Schulz, and by the availability of content for both candidates that Google can index. Merkel, being the head of the German government, presumably gets a higher volume of international searches, as well as more international coverage. It is important to note that these are users with the language set to German and who, according to their geo-data, are in Germany. Google must know that they still prefer English- (or Spanish-, Polish-\u2026.) language content. Whether users searching for Martin Schulz are (on average) less likely to be polyglots, or there is a lower volume of international coverage is hard to tell, though a combination of both may be the case. Other opposition candidates show similar patterns. Another aspect is the share of specialized news sources. I won\u2019t go into detail, but Golem and DWN for Schulz are unusual (for different reasons), as are RT and Tichys Einblick for Merkel. While the percentages for these are small, one should remember that N is equivalent to the portion of the German population using Google to search for political issues, which is potentially quite large.<\/p>\n

      You can download an Excel spreadsheet with the data here<\/a>.<\/p><\/blockquote>\n

      How similar are results for the same query in general?<\/strong><\/h3>\n

      While this is all well and good, how about a more systematic comparison of search results? How similar are they, and what could their similarity (or difference) be based on? To investigate this, we introduce a simple measure of similarity (S)<\/em> based on the Jaccard similarity coefficient of two search results<\/em>. S has a value of between 0 (results are completely different) and 1 (results are completely identical) for any given two searches. We can calculate S for a random sample of results and compare their similarity(***). When doing this, it makes sense to compare results where a number of variables are controlled for, e.g. where<\/p>\n