29 January 2009 | Author: R. Falconer SEO Consultant

Blue search terms still giving algorithms headaches

It's the world's worst kept secret that sex rules the internet - or, more precisely, porn rules the internet. It's really difficult to put even a vague figure on it, but it's likely that somewhere between 20 and 60 percent of internet use (depending on who you listen to) is directly or indirectly related to adult content. What's more, this sector is arguably the most spam-filled area of the web. This, among other related issues, creates a huge problem for search engines and even bigger problem for sites with non-porn, sexually related content.

People expect search engines to give them relevant results based on the queries they supply. Search engines do their best to supply those results. But due to certain filtering and censoring issues, when it comes to saucy queries - or terms that could be construed as risqué terms but aren't - the rules are different.

Type any blue query into Google with SafeSearch off and you'll more than likely see a load of pornographic content. Type pretty much any girl's name into Image Search with SafeSearch off and you know you're not going to see the "take home to mother" type. That's probably okay if your query is "Cindi" but perhaps not if your six-year old types in her first name.

It's easy to point out the flaws in Google's SafeSearch censorship but when it comes to racy terms, adult sites are often targeting the same terms as educational and medical sites, bloggers writing about sex or online stores selling erotic toys. There is certainly at least one case of a multinational company that has had problems with their brand name being filtered. It's difficult for humans to agree on what's offensive and not, so how do you build an algorithm to do it?

It's no wonder search engines err on the side of caution when it comes to potentially offensive results. Unfortunately, that caution can sometimes result in genuine sites losing out on traffic because they use certain censored terms. In some ways, this is a great argument for the internet needing a stronger competitor to challenge Google. The fact that 85% of UK searches are done on Google means a simple change in their algorithm can have a devastating effect on search traffic to a site.

In 2007, Google made some changes to their ranking algorithm that saw a number of popular erotic blogs and sex sites outside the mainstream porn industry lose their rankings in Google. Google admitted it was a mistake and fixed the problem promptly but by this time the debate had opened up about the way search engines treat sexual content.

It's clear that there is an ongoing issue here, so what possible steps can we take to combat it? As previously mentioned, the main problem is that people can't agree on what's obscene so Google really has to try to reflect a consensus of opinion that doesn't actually exist.

Personalised search helps individual users who are logged in and their results could be harnessed to reflect a general public opinion on what should and shouldn't be shown in their results. Were Google's SearchWiki to take off in a big way, Google could use the user indications to bury obscene content. Imagine a site has been flagged up as borderline between legitimate and adult content but 95% of Google Searchwiki users bury it - you'd have a good indication that it could be removed from results without much opposition. Perhaps an "obscene" option could be added to Google Searchwiki. This could be done by users highlighting content they felt was obscene to Google; the search giant could then build up a picture of what the user doesn't want to see and stop providing it in their results.

Right now, Google is smarter than ever before but still relies on blacklisting certain words to decide whether a site should appear in results. One thing is clear: controversy surrounding blue results is likely to be with us for a long time to come.

Home | Careers | RSS | Contact Us | Newsletter

International sites: