10.6 C
New York
Sunday, December 10, 2023

7 must-see Google Search ranking documents in antitrust trial exhibits

The U.S. Department of Justice has released several new trial exhibits – including internal Google presentations, documents and emails related to ranking.

Here are seven that specifically discuss elements of Google Search ranking:

1. Life of a Click (user-interaction)

This is a heavily redacted PowerPoint presentation put together by Google’s Eric Lehman – and like most of the other documents, it lacks the full context accompanying it.

However, what’s here is interesting for all SEOs.

Google Ranking PillarsGoogle Ranking Pillars

In “The 3 Pillars of Ranking,” slide, Google highlights three areas:

  • Body: What the document says about itself.
  • Anchors: What the Web says about the document.
  • User-interactions: What users say about the document.

Google added a note about user interactions:

  • “we may use ‘clicks’ as a stand-in for ‘user-interactions’ in some places. User-interactions include clicks, attention on a result, swipes on carousels and entering a new query.

If this sounds familiar to you, it should. Mike Grehan has written and spoken extensively about this for 20 years – including his Search Engine Land article The origins of E-A-T: Page content, hyperlink analysis and usage data.

Google User Interaction SignalsGoogle User Interaction Signals

In this slide, titled “User interaction signals,” Google illustrates the relationships of queries, interactions and Search results, alongside results for the query [why is the ocean salty]. Specific interactions mentioned by Google:

  • Read
  • Clicks
  • Scrolls
  • Mouse hovers

In September, Lehman testified during the antitrust trial that Google uses clicks in rankings. However, once again, it’s important to make clear that individual clicks alone are a noisy signal for ranking (more on that in Ranking for Research). Google has publicly said it uses click data for training, evaluation, controlled experiments and personalization.

What is redacted:

  • A slide titled “Metrics” – all that is visible is one line: “Web Ranking Components.”
  • Seven additional slides, including two titled “Outline” and “Summary.”

Link: Google presentation: Life of a Click (user-interaction) (May 15, 2017) (PDF)

2. Ranking

These seven slides were part of a larger Q4 2016 Search All Hands presentation, prepared by Lehman.

Google Fake UnderstandingGoogle Fake Understanding

In this slide, Google says “We do not understand documents. We fake it.”

  • “Today, our ability to understand documents directly is minimal.
  • So we watch how people react to documents and memorize their responses.”

And the source of Google’s “magic” is revealed:

So how does this work?

Google Searcher BenefitsGoogle Searcher Benefits

In this slide, Google explains how “each searcher benefits from the responses of past users … and contributes responses that benefit future users”:

Google Look Stupid 800x585Google Look Stupid 800x585

And in the final slide, Google sums up with this statement:

  • “When fake understanding fails, we look stupid.”

The other four slides are entirely skippable, unless you’re interested in knowing that “Search is a great place to start understanding language. Success has implications far beyond Search.”

Link: Google presentation: Q4 Search All Hands (Dec. 8, 2016) (PDF)

So when you see Google claiming links aren’t a top 3 ranking factor, now you can hopefully start to better understand why. That isn’t to say links are unimportant or that user data is the entire reason – machine learning and natural language processing are other huge pieces, more on that in Bullet points for presentation to Sundar.

Google is looking at end users – how people interact with Search results. Not as individuals – but as a collective.

3. Ranking for Research

It’s unclear who created this presentation, but there are some very interesting findings in here.

Google Search QualityGoogle Search Quality

In this slide, Google talks about 18 aspects of search quality:

  • Relevance
  • Page quality
  • Popularity
  • Freshness
  • Localization
  • Language
  • Centrality
  • Topical diversity
  • Personalization
  • Web ecosystem
  • Mobile friendly
  • Social fairness
  • Optionalization
  • Porn demotion
  • Spam
  • Authority
  • Privacy
  • User control of spell correction
Google Live Traffic ShortcomingsGoogle Live Traffic Shortcomings

This slide discusses the shortcomings of live traffic evaluations. Yes, essentially Google is talking about clicks not being a good signal because they are hard to interpret.

  • “The association between observed user behavior and search result quality is tenuous. We need lots of traffic to draw conclusions, and individual examples are difficult to interpret.”
Google Search Results RankingGoogle Search Results Ranking

Finally, this slide provides a different illustration of how Google Search result ranking works:

There are some other interesting tidbits in this presentation, though not necessarily tied to ranking. Of note:

  • “Attempts to manipulate search results are continuous, sophisticated, and well-funded. Information about how search works should remain need-to-know.” (Slide 5)
  • “Keep talk about how search works on a need-to-know basis. Everything we leak will be used against us by SEOs, patent trolls, competitors, etc.” (Slide 10)
  • “Do not discuss the use of clicks in search, except on a need-to-know basis with people who understand not to talk about this topic externally. Google has a public position. It is debatable. But please don’t craft your own.” (Slide 11)

Link: Google presentation: Ranking for Research (November 16, 2018) (PDF)

4. Google is magical.

In this presentation, we learn how search really works.

Google Not How Search WorksGoogle Not How Search Works

This slide explains how search does not work. From the notes:

Google How Search Does WorkGoogle How Search Does Work

In this slide, we learn how search does work:

Google DialogueGoogle Dialogue

Next, we learn the source of Google’s “magic.” From the notes:

Google Learn From Users 800x457Google Learn From Users 800x457

So how does Google learn more from users? From the notes:

Google 10 Blue LinksGoogle 10 Blue Links

This slide looks at the 10 blue links.

Google Image SearchGoogle Image Search

This slide is on Image Search:

Google Knowledge Cards 800x458Google Knowledge Cards 800x458

Finally, knowledge cards:

Link: Google presentation: Google is magical. (October 30, 2017) (PDF)

5. Logging & Ranking

This presentation discusses the “critical role that logging plays” in ranking and search.

Google Dialogue MagicGoogle Dialogue Magic

This familiar-looking slide revisits the two-way dialogue being the source of Google’s magic. As explained in the notes:

Google Extracting Value JudgmentsGoogle Extracting Value Judgments

In this slide, Google discusses translating user behaviors. From the slide notes:

Google Logs Ranking MoneyGoogle Logs Ranking Money

Finally, this slide discusses how logging supports ranking and Search. From the notes:

Link: Google presentation: Logging & Ranking (May 8, 2020) (PDF)

6. Mobile vs. desktop ranking

This newsletter dove into the differences between desktop and mobile search ranking, user intents and user satisfaction – at a time when mobile traffic was starting to surpass desktop traffic on some days.

Google did a comparison of metrics, including:

  • CTR
  • Manual refinement
  • Queries per task
  • Query length (in char)
  • Query lengths (in word)
  • Abandonment
  • Average Click Position
  • Duplicates

Based on the findings, one of the recommendations was:

  • “Separate mobile ranking signals or evaluation reflecting different intents. Mobile queries often have different intents, and we may need to incorporate additional or supplementary signals reflecting these intents into our ranking framework. As discussed earlier, it is desirable that these signals handle local-level breakdowns properly.

Link: Email from Google’s Web Ranking Team to Pandu Nayak – Subject: [Web Ranking Team] Aug 11 –Aug 15, 2014 was updated — Ranking Newsletter (August 16, 2014) (PDF)

7. Bullet points for presentation to Sundar

Nothing surprising in this document (it’s unclear who wrote it), but one interesting bullet on BERT and Search ranking:

  • “Early experiments with BERT applied to several other areas in Search, including Web Ranking, suggest very significant improvements in understanding queries, documents and intents.”
  • “While BERT is revolutionary, it is merely the beginning of a leap in Natural Language Understanding technologies.”

Link: Google document: Bullet points for presentation to Sundar (Sept. 17, 2019) (PDF)

Related Articles

Latest Articles