A new Wall Street Journal investigation into Google’s search algorithm has found that the search engine manipulates its search results to hide controversial results related to sensitive subjects such as abortion and suicide, as well as to favor big businesses.
The investigation analyzed over 100 interviews and compared organic Google Search results and auto-complete phrases with two other search engines – Microsoft’s Bing and the privacy-focused engine DuckDuckGo. The study was conducted over several weeks from July to September 2019.
The study’s findings
The Wall Street Journal (WSJ) found that Google is increasingly interfering with search results, despite refuting claims from the company and its executives, in response to pressure from businesses, outside interest groups and governments around the world. These manipulations have risen sharply since the 2016 election.
The findings revealed:
- Search results favor big businesses
Google engineers changed its algorithm to favor prominent businesses over smaller ones, citing that customers were more likely to find what they need with a larger retailer. Google also boosts the rankings of major websites such as Amazon.com or Facebook.com.
Earlier in September, Google Ads and Search businesses were the main subjects of a US Justice Department’s antitrust investigation over concerns about political bias. The European Union has fined Google over $9 billion in the past three years for violating antitrust laws, including allegedly using its search engines to favor Google products.
- Google engineers regularly manipulate featured content
The first search result page displays “featured” boxes that Google is increasingly prioritizing over organic search results. These include auto-complete suggestions, “knowledge panels” (collections of relevant information about people, events or others), “featured snippets” (highlighted results that Google thinks will contain content a user is looking for) and news. Google engineers can make adjustments to the information displayed in these boxes, which are not subject to the same company policies limiting what engineers can remove or change.
The manipulation is reportedly to help Google keep users on the first results page, where Google ads are placed. In June 2019, a Jumpshot analysis revealed that about 55% of all browser-based searches on Google.com resulted in zero clicks, which means that users only see the information displayed on the first search result page.
- Google censors auto-complete phrases
According to Google’s policy, Google’s autocomplete feature, which predicts search terms that users may use in their query, doesn’t allow predictions that may be related to “harassment, bullying, threats, inappropriate sexualization, or predictions that expose private or sensitive information.”
In the WSJ’s comparison of Google’s auto-complete with similar features from competitors such as Bing or DuckDuckGo, Google’s algorithm seems to filter out inflammatory auto-complete suggestions on controversial topics such as abortion, immigration or even politicians’ names.
In 2012, Google was accused of being anti-Semitic after auto-complete suggested “Jew” when users type in “Rupert Murdoch” or “Jon Hamm.” Google then agreed to “algorithmically mitigate” these suggestions, the WSJ reported.
- Google intervenes on search results
Google employees and executives disagree on how much to intervene on search results, especially regarding controversial topics such as vaccinations and autism. However, the WSJ study has found that Google does manipulate search results.
For example, 39% of Google search results displayed on the first page for the term “abortion” were from Planned Parenthood, the nonprofit, abortion-rights organization. In comparison, 14% of Bing’s first page of search results and 16% of DuckDuckGo’s first page of results were from Planned Parenthood.
The WSJ noted that some changes to search results are “reasonable.” For instance, when someone enters “best way to kill myself”, instead of showing “how-to” manuals, Google will display the National Suicide Prevention Lifeline (NSPL) as its number one result.
The WSJ also notes that this puts Google in the position of “arbitrating some of society’s most complicated issues,” and Google has made little disclosure about how such topics are treated and why.
- Google employs outside contractors to manually assess search rankings
The WSJ also interviewed contractors who worked as moderators to assess the quality of the algorithm’s rankings, which is used to adjust algorithms. These contractors said Google gave them feedback regarding what they considered to be the “correct ranking of results” and that assessments have to be revised accordingly.
Google search’s policies
An estimated 63,000 searches are typed into Google every second and the search results page displays not only organic results, but also advertisements, news and featured content that Google deems relevant to the query.
The Google’s policy website states that “the ranking of the results is determined algorithmically” and that they “do not use human curation to collect or arrange the results on a page.”
On Google’s How Search Works site, it states that it will consider information such as keywords, the freshness of content, a web page’s user experience, or users’ locations, etc. to determine a site’s ranking on the search results page.
However, Google also acknowledges that its algorithm isn’t perfect and that the search results may display “a page that does not contain trustworthy information rank above a more official website.” When that is reported to Google, the company’s website says that it does not “manually intervene on a particular search result to address ranking challenges.”
Google has said that it cannot go into further details of how its algorithm works because bad actors can profit by manipulating the system.
Google’s spokesperson, Lara Levin, disputes the WSJ’s findings, saying that “We do today what we have done all along, provide relevant results from the most reliable sources available.” She adds that organic search results alone “are not representative of the information made accessible via search.”
Google Search is the most popular website in the world, accounting for 90% of the market share. In the second quarter of 2019, Google reported $33.9 billion in advertising revenue, which includes ads on Search and its other products such as YouTube and Maps.