Anthony Contoleon sheds some light on how search engines decipher what results to present each time we type a query into that blank box

What do you want to find when you type the word ‘pizza’ into a search engine? Did you want a recipe, a definition, the Wikipedia entry, or the closest Domino’s? What was the rest of the question; what didn’t get typed? Search engines are going beyond just looking at the short strings of text people type. More and more different cues are being considered from both the searcher and the sites they index to judge what is the best answer to the user’s question.

Both Bing and Google (and until recently, Yahoo) are always innovating, and both use a range of factors such as location, social connections, user behaviour individually and as a group, and the semantics of language, both on the sites and as queries, in order to return the pizza result most suited to you.

Creating the best possible result that satisfies the user requires a combination of factors. The search engine needs to identify the user’s intent and return relevant, authoritative and trustworthy information that would be seen as such by the user. Finding the balance between what matches the query, what matches the user’s assumed intent and what source would satisfy the searcher in an ecosystem where information is constantly being optimised to maximise visibility is a challenge, which search engines address in a number of different ways with ongoing algorithm changes, such as the recent Panda updates.

To find the best result for a query like ‘pizza’, search engines use a lot of information, including data such as your location, which Google domain you are searching from, what Google sees others in your social network do, what you personally have looked at and previous trends experienced by the data centre. The sites returned and the order in which they are displayed when you search for ‘pizza’ when logged into your Google account can be different from what you see when you are not, or when you search from another country. The location set in your profile is used on searches when you’re logged-in on devices without GPS or other location sensing features, confining your pizza options to the city your profile says you live in. On devices that have access to other location data, either through GPS, or mobile and wi-fi networks, the search results will be tailored using that information, making it easier to find pizza even if you just stepped off the plane in another state or country.

To put search in context, Google goes beyond location and also collects data on user behaviour. The activity of the user as an individual (making it easier to order the same pizza you clicked on last time), the actions of their social network (so you can order the pizza they +1’ed), and how users behave as a group (favouring the pizza result everyone else also clicked on) all provide information that helps Google tailor the result. A few significant ways that Google uses user data to affect the results page were introduced with Google Instant last year. A few of these include keeping track of the results pages already shown to a given user and basing predictions on user behaviour as a group by regions. Which is why you need to type ‘piz’ before ‘pizza’ becomes visible as a suggestion, because ‘Pirate Bay’ is a more popular query than ‘pizza’.

As important as location and behaviour are to creating a personalised search experience, content still counts. Even as true semantic search remains in the future, search engines are heavily invested in understanding websites and content semantically. Each search engine holds patents for the various techniques it uses, such as phrase based indexing or query and document classification. Just recently Google, Bing and Yahoo launched Schema.org, a collection of tags to be added to websites to classify content such as products, articles and contact information, to create context in a machine readable way.

While the methods they use change all the time, the goals that Bing and Google are pursuing remain the same. Bing seeks to be the ‘Decision Engine’ and Google aims to organise the world’s information and make it universally accessible and useful. Understanding what their users actually mean and expect when they perform a search is critical, and the only certainty is going to be change in pursuit of this goal.