Web Searching

Web searching is big business. Google has built its empire around Web searches. What is Web searching? How does it work? How can money be made from it?

Web searching is the activity of looking for information of interest to the searcher in the trillions of pages available on the Web. Some searches are for nefarious purposes. Others are benign. We will look at the more benign searches done by companies like Google, Yahoo, and Microsoft.

Searches

Companies, such as Google, who provide users with an interface that allows users to enter search terms and and get a list of links back with content hints from which to choose Web pages to visit those that appear to have the desired information. It used to be relatively easy. Now it has become more difficult.

In the Beginning

When the Web was in its infancy, it was really not available for use by the general population. It was used to allow researchers in a Swiss facility known as CERN to share documents on a local network.

Documents were created in an early form of HTML
Document contents were identified by keywords placed a <meta>...</meta> tag pair (e.g., <meta> temperature, plutonium, fission </meta> that was placed inside the <head>...</head> tags of a document.
A primitive browser would allow a user to type in one or more keywords, such as plutonium and light, and have a list of links to articles that had that keyword in the <meta> tag pair returned.

This was a very simple and effective method for document searches. It was up to the writers of an article to create the <meta> section with proper keywords so that the browser would return relevant articles.

There were two problems with this approach

The browser had no way to determine which of the many articles that might be returned were the most relevant or important to the searcher.
The simple <meta> tags didn't work once the Web went public.

In the Middle--Spoofing Searches and Ranking Pages

putting bogus terms in the <meta> tags
Finding new ways to search
ranking became became the holy grail of searching

Search Engines

web crawling
- how it is done
- how it works (how can it be so fast?)
- local caching
- bring up portfolio web page and notice the various links and how they differ form anchor tags
secrets
- one of Google's early successes was to rank pages by how many links were pointing to them.
- other things are definitely not made public
- Microsoft's Bing search engine was discovered to be running searches people typed in the through Google search to better tune their results
What Google Says: https://www.google.com/insidesearch/howsearchworks/thestory/
Summary of what Google says.

Making Money from Advertising