Google helps us think, Facebook finds us friends and Spotify plays our own personalised soundtrack. It’s hard to say whether the computer algorithms that these services use to anticipate our needs and wants are turning us into puppets or geniuses. But algorithms have a huge impact on our tastes, buying habits and decisions about our digital lives. Yet algorithms could trap us in a world where advertisers and government agencies couple behavioural data with computer formulae to predict and manipulate what we do or buy next. The trend toward ever-more-sophisticated algorithms isn’t limited to situations where consumers seek information or products. Private companies and government agencies are also harnessing the power of algorithms: to boost their efficiency in dealing with inventory control, and effectiveness in monitoring behaviour and predicting a cybercriminal’s next move. The internet is a gold mine of data with which businesses can predict behaviour. Tracking IP addresses across the net, knowing what websites people visit and when they visit them, counting banner ad clicks, and harvesting data from social networks are all much easier than following someone around with a clipboard. Here, we look at some of the algorithms that rule the web – and the companies that use them – and assess whether their increasing use signals the death of serendipity.

Many people credit Google’s search algorithm as the source of its $193bn fortune and tight grip on the search-engine market. After all, it’s the only company whose name is synonymous with the verb ‘search’. According to research conducted by Columbia University, Google’s algorithm could also be changing the way we think. Assistant professor Betsy Sparrow says search engines are altering human thought patterns, causing people to remember less on their own and instead rely on their ability to find the answer online. Many companies have developed business models around displaying ads on pages with low-quality content that’s been customised to rank high in Google’s search results. In February, the effectiveness of these ‘content farms’ caused Google to adjust its algorithm and reduce the standing of low-quality sites within its search results. Recently, the New York Times exposed jewellery store JC Penny’s efforts to inflate its Google page rank by creating thousands of third-party links and sites. JC Penny denied any knowledge of the shenanigans, but Google penalised it by reducing the company’s prominence within its search results. Smaller businesses may succeed or fail depending on the whims of Google’s search algorithm. In 2006, KinderStart.com sued Google, claiming that it had sustained significant financial harm when Google changed its algorithm and subsequently ranked the site low in its search results. Ultimately, KinderStart.com lost its court battle – as have other companies that filed similar suits against Google.

Algorithms that rule the web: News

Google constantly updates its news algorithm, which it uses to power Google and Yahoo News. Either service is able to tell you the most popular story of the hour, which Google bases on attributes such as keywords, originality, freshness, quality and expertise of source. But Jim Barnett, who writes for Harvard’s Nieman Journalism Lab, has questioned how financial pressures might shape Google’s news and search algorithms, and what this might mean for explanatory and investigative journalism. He warned that “what’s good for the bottom line doesn’t always jibe with what’s best for consumers,” and that “it isn’t always possible to divine what people will want in the future based on a profile of what they (or people like them) have wanted in the past”. “When we don’t know what we want, sometimes what we really need is to figure it out for ourselves,” added Barnett.

Algorithms that rule the web: Social

Facebook’s social algorithm, EdgeRank, can help you find old school friends and colleagues. It can also determine whose updates appear in your Top News feed, using a combination of factors such as your affinity with someone, the type of message and when it was written. Facebook recently upped its algorithm ante by tying it to facial-recognition software and analysing every photo uploaded to the service – with 750 million users, that’s around 20 billion images. When you upload a photo, facial-recognition software is coupled with your social circles to identify who is pictured; Facebook then asks you whether you want to tag them. You don’t have to tag a friend, but this fact hasn’t quelled privacy activists’ concern over the feature.

Algorithms that rule the web: Love

Chemistry may be the decisive factor in the human phenomenon of falling in love, but algorithms provide the matchmaking spark for many who use online dating services such as eHarmony and Match. Finding the perfect je ne sais quoi for potential love birds requires such sites to look beyond the barebone facts that someone is a non-smoker who likes swing dancing, for example, and number-crunch users’ personal-attraction tests. eHarmony puts your answers to its 258-question personality test into its trade secret: its love algorithm. In 2008, 19 million people had taken this test, and a study commissioned by eHarmony concluded that it was responsible for 2 percent of all US marriages in 2007.

Algorithms that rule the web: Recommendations

Algorithms that rule the web: Advertising

Online advertising sits at the crossroads of commerce and algorithm deployment. Its objective is to display the right ad to the right person at the right time. An advertising algorithm that succeeds in this mission can mean the difference between a sale and no sale. To better their odds, advertisers use algorithms to slice and dice a mix of data. In essence, sophisticated online advertisers pair offline demographic data about you with your web-surfing habits, then entice you with targeted online ads. Some observers argue that presenting relevant ads helps website owners stay in business and deliver high-quality content. Others say that trusting private companies with huge databases of user profiles is like putting foxes in charge of henhouse security.

Algorithms that rule the web: The death of serendipity

Do algorithms signal the death of serendipity – if not free will and privacy – online? This debate will rage for years. But as we quickly morph into the supercomputer age of gargantuan databases, algorithms need to be protected from exploitation by Orwellian governments, sociopathic hackers and intrusive companies, privacy experts warn. Unfortunately, few laws have caught up with technology in this arena. Current do-not-track legislation is working its way through government in the UK, US and Europe. Other initiatives put forth on the state level in the US, such as in California, tangentially address how companies gather and use data. But so far the lords of the algorithms have the upper hand.