How does Google know if its search results are improving?
As Google rolls out algorithm updates and claims to reduce “unhelpful” content, many wonder about the true impact of these changes.
In an episode of Google’s Search Off The Record podcast, Google Search Directer, Product Management, Elizabeth Tucker discusses how Google measures search quality.
This article explores Tucker’s key revelations, the implications for marketers, and how you can adapt to stay ahead.
Tucker, who transitioned to product management after 15 years as a data scientist at Google, says it’s difficult to determine whether search quality is improving.
“It’s really hard,” she admitted, describing a comprehensive strategy that includes user surveys, human evaluators, and behavioral analysis.
Tucker explained
“We use a lot of metrics where we sample queries and have human evaluators go through and evaluate the results for things like relevance.”
She also noted that Google analyzes user behavior patterns to infer whether people successfully find the information they seek.
Tucker revealed that users make more complex queries as search quality improves.
This creates a constantly shifting landscape for Google’s teams to navigate.
Tucker observed:
“The better we’re able to do this, the more interesting and difficult searches people will do.”
Tucker shared that in the short term, poor search performance might lead to increased search activity as users struggle to find information.
However, this trend reverses long-term, with sustained poor performance resulting in decreased usage.
Tucker cautioned:
“A measurement that can be good in the long term can be misleading in the short term.”
To tackle the challenge of quantifying search quality, Google relies on an expansive (and expanding) set of metrics that gauge factors like relevance, accuracy, trustworthiness, and “freshness.”
But numbers don’t always tell the full story, Tucker cautioned:
“I think one important thing that we all have to acknowledge is that not everything important is measurable, and not everything that is measurable is important.”
For relatively straightforward queries, like a search for “Facebook,” delivering relevant results is a comparatively simple task for modern search engines.
However, more niche or complex searches demand rigorous analysis and attention, especially concerning critical health information.
Google aims to surface the most helpful information for searchers’ needs, which are as diverse as they are difficult to pin down at the scales Google operates at.
Tucker says:
“Understanding if we’re getting it right, where we’re getting it right, where needs focus out of those billions of queries – man, is that a hard problem.”
As developments in AI and machine learning push the boundaries of what’s possible in search, Tucker sees the “human element” as a key piece of the puzzle.
From the search quality raters who assess real-world results to the engineers and product managers, Google’s approach to quantifying search improvements blends big data with human insight.
As long as the web continues to evolve, Google’s work to refine its search quality measurements will be ongoing, Tucker says:
“Technology is constantly changing, websites are constantly changing. If we just stood still, search would get worse.”
Google’s insights can help align your strategies with Google’s evolving standards.
Key takeaways include:
As Tucker’s insights show, this user-first approach is at the heart of Google’s efforts to improve search quality – and it should be at the center of every marketer’s strategy as well.
Listen to the discussion on measuring search quality in the video below, starting at the 17:39 mark:
Featured Image: Screenshot from YouTube.com/GoogleSearchCentral, June 2024
WordPress announced over the weekend that they were pausing plugin updates and initiating a force…
Here is a recap of what happened in the search forums today, through the eyes…
Bing added a new guideline to its Bing Webmaster Guidelines named Prompt Injection. A prompt…
A Google community manager, Alistair D., posted a Reviews troubleshooting and FAQs document in the…
Gary Illyes from Google explained two reasons why soft 404 errors are bad. Soft 404s…
There are reports that some sites hit by the Google September 2023 Helpful Content Update…
This website uses cookies.
Leave a Comment