David Amerland

Semantic Image Search: How Computers Learn to See

Computer Vision - Google Learns to See Pictures

When it comes to quality in semantic search results the defining parameter between a person and a machine always comes down to a picture. Usually the former can work out what it is without even thinking about it much while the latter can’t, even after thinking about it a lot.

All of this is now about to change and semantic technology plays a huge part in it. Employing a method it calls computer vision Google’s search has now become capable of looking at photographs and recognising the objects within them, similar to how a person might. 


This is a breakthrough in itself. Though far from complete, the achievement is only made possible by training a computer to recognise Entities in a photograph. Entities are key to semantic search and they were the topic I introduced in SMX East 2013 in NYC at the The Coming Entity Search Revolution.

The fact that Google has begun to get good at recognising images like this is important because it signals an increase in the data represented in the Knowledge Graph and a significant increase in the Entities it has now indexed (and these two are intrinsically linked).

Why is this important? Well, because it signals a shift in how search works and how search results are delivered. Google’s Hummingbird update, for instance, is a direct result of the growth in Entities in Google’s semantic index and the data in its Knowledge Graph.

From an SEO point of view photographs that have not been labelled can still be recognised, though, right now, there will still be a margin of error, experienced. This is important because it squares the circle with that other drive by Google: personalization and recognition. At the SMX East 2013 conference Google’s Webmaster Trends Analyst, Pierre Farr announced how Google now also delivers Google Authorship in Google Image Search results, like the one below where an article I wrote for Realtors and how they can benefit from Google+ posted by Bill Gassett on his blog, shows authorship to the picture.

Google Authroship now on Google Image Search

Semantics Change the SEO Game

The moment you apply semantic indexing to the web the entire search engine optimisation game begins to change. Authorship is not a ranking signal, yet, but it is an important part of the signals Google collects to verify the authenticity and importance of the content it finds on the web.

With the ability to better recognise photographs, Google’s search now enters a new phase, with a much deeper understanding of search terms and a deeper recognition of content. The SMX East conference had a very clear message across its three day program: Traditional SEO with its checkbox optimisation is dead. Search engine optimisers now need to adapt.


What You Missed

Google’s Hummingbird Deepens Semantic Search Results
Semantic Search and What it Means to Businesses
How Semantic Search is Changing End-User Behavior
Most Common Questions On Semantic Search Answered

External Links

Image Search Finds Your Photographs Easier
Semantic Programming Comes to Images
Introduction to Entities in Semantic Search

© 2019 David Amerland. All rights reserved