“First Step of Compost” via Wikipedia

Google is starting to lose market share of an esoteric, yet important, sort. For early adopter web sites including the leading tech blogs, Google is becoming a little less powerful as measured by the number of visitors it drives to a web site. If the trend continues and goes mainstream, it’s a big deal. Billions of dollars are spent on Search Engine Optimization (SEO) each year. SEO is a series of practices and tools through which web sites on seek to drive clickthroughs from Google’s organic (i.e. free) search results. According to SEMPO, Google’s advertisers alone spent $1.4 billion on SEO over and above their payments to Google in 2008. Millions of sites that don’t buy AdWord also practice SEO and the entire market for SEO has to be at least twice the SEMPO figure.

When a user clicks on one of Google’s ten blue links, the web address of that particular Google Search Engine Return Page (SERP) is written into the records of the destination site that to which the user clicked through. The same is true of any other link from any other page on the web. Those records are called Referrer Logs. They are the lifeblood of scientific web traffic management and online marketing. For the in-crowd blogs and a number of other leading sites, Search Referrers are dropping as a percent of total referrers in favor of so-called Social Referrers from Twitter, Bit.ly, Facebook, and others.

Semantic Referrers
I say “so-called Social Referrers” as I don’t believe social is the key property of these referrering URLs. The key property is that these referrers are generated by data systems more heavily structured, i.e. semantic, than the huge majority of Google’s index. Facebook’s heavily detailed social graph and Application Platform are the clearest example. It would be simple for them to let web site owners know how the the link came to be clicked on, by a person of what anonymous profile, etc. Facebook is starting to provide such tools to its platform and would hurt Google significantly if they provide the information to outside web publishers via the Referrer URL. Twitter is not formally structured like Facebook but its abundance of links, #hashtags, and limited message length make structure far easier to infer than in freeform web content. In aggregate, the rise of this Semantic Referrer traffic means that we all finally need to take the Semantic Web seriously.

Accidentally Semantic, i.e. Semantic Search Provides no Clicks
Taking the Semantic Web seriously doesn’t mean relying on magic or Google to get us there. In very few cases can semantics be reliably derived out of the unstructured web. That’s why Semantic Web Search will remain a pipe dream for years to come.

Machine approximation of structure does work in a few cases, but Google is pursuing those cases with Google Janitors and will dominate them. Per this post in June, Googling “Rocco Mediate age” provides the answer of his birthdate displayed in a differentiated fashion above the standard search results. Similarly, Googling “Maverick 17th Street” provides the specific address, phone number, map, etc. for the restaurant. Notice that the user gets a self-contained answer to their query, that doesn’t depend on a link to the answer. This sort of search with no click is particularly useful on my Blackberry.

Maverick probably doesn’t care if you click through to their web site as long as you actually eat there, but the ESPN Golf with the great Rocco Mediate profile [NB: what terrible SEO on that link] certainly does as do the Wikipedia page editors. With little or no traffic to gain, why would ESPN Golf or Wikipedia optimize their content for semantic search à la SEO or even allow the semantic search engine to gather its content?

Semantic and Networked Publishing
Taking the Semantic Web seriously doesn’t mean relying on Twitter and Facebook to get us there either. Hopefully, Google’s behavior has taught the big publishers the dangers of dependence on a monolithic third party.

Because semantic data and Semantic Web Search can’t generally be derived ex poste facto, almost the entire web needs to be re-recorded in a structured form. That sounds like an insane amount of work, and it is, but it’s already happening. Facebook, Twitter, Tumblr, Bit.ly, Awe.sm, FourSquare, Zemanta, and their various peers are getting all of us to type everything on the web back in, this time using more defined forms that they store and publish more robustly. It’s called User-Generated Content and Social Media.

The best way to think about the overall trend is that the mainstream web audience learned how to use unstructured web content during the DotCom boom; learned how to publish and search it during Web2; are beginning to learn how use structured web content now; and will learn how to semantically publish to and search the web starting in 7 or 8 years. Following the same patterns as the DotCom and Web2 eras, the professional publishers who start publishing semantic content early and as their mainstream effort stand a far better chance of becoming the dominant media property in their sector. Those who wait will devolve into unstructured compost.

Posted 6 August 2009 at 11h45 |  3 notes and  Comments
• Tags:
  • Search Engine Optimization
  • Facebook
  • twitter
  • zemanta
  • bitly
  • tumblr
  • semantic web