Google will deduplicate a link from its main web results if that same link appears in the first Top Stories slot, so long as the Top Stories section appears before the main web results, according to Danny Sullivan, public search liaison at Google.
However, if this is not the case (e.g., the main web results appear above the Top Stories section), Google will not deduplicate the link (as shown above, to the right). “And again, it’s something we’re reviewing,” Sullivan noted.
What Google said. “Just to cap off with the further clarification I promised, we deduplicate a link from web results if a link appears as the first link in Top Stories and if the Top Stories box appears before web results,” Sullivan said on Twitter, “If it comes after, we don’t.”
This explanation was provided after Dieter Bohn, executive editor at The Verge, called out Google over search listings that were “stealing [The Verge’s] content.”
The query can influence deduplication. As Sullivan explained, searching by exact headline may not reflect how most people seek out information. Bohn’s example query, “trials and tribulations turning a real camera,” was a partial match for The Verge’s article — Sullivan’s example of what a typical user might search for was “turning camera into webcam.”
For users searching for solutions, “Our systems also are going to generally seek to show the most helpful, reliable info they can,” Sullivan wrote, “That’s why you don’t see a lot of duplicates of your article showing.”