Over the last days it's been reported that some of the most important online properties in China have decided to block Baidu spiders from indexing their websites and making them accessible through the search results.Influential Chinese portal Sohu.com, the leading C2C platform Taobao.com and several social networking sites - such as 51.com and Xiaonei - have added exclusion rules against Baidu, and partially against other search engines like Yahoo! and Google, alleging that it will benefit the prvacy and safety of their users. According to Brand Republic Asia, some analysts say that much of their motivation comes from dissatisfaction at Baidu's domineering web presence.
Chinese social network Xiaonei.com has created a robots.txt rule to exclude Baidu spiders from accessing their site. If user privacy is the main reason, why not exclude other spiders like those from Google and Yahoo! too? Or why not provide members of the social network members with the option to set their own privacy level?
Due to the nature of social networks and the often personal content they contain, the ability to search the pages may not be as important as for informational or commercial websites. However, some social networks, like Facebook, have chosen to make the most of potential visibility on search engines, and have designed specific versions of the social profiles only for search engine spiders. Visitors coming from search engines will still be required to register or login to gain access to their friends' detailed profiles.
Taobao.com, which belongs to the Alibaba Group, owner of Yahoo! China, may have slightly different reasons behind the Baidu exclusion. Earlier this week Baidu revealed its plans to release an e-commerce site before the end of 2008. The new site, which would aim to pilfer market share from Taobao, will soon be put on a beta stage. Although Taobao has declared that Google and Yahoo! have also been partially blocked, its current robots.txt file only includes rules for Baiduspider.
Finally, Sohu.com, the most popular portal in China, has also included specific rules against Baidu in their Blogs subdomain. Sohu has decided to exclude Baidu and any other spider that supports the Robots Protocol but surprisingly it allows Googlebot to keep crawling and indexing its blog content.
Even with Baidu's leading position in China's search market, the alignment of some of the most powerful forces in the online may accelerate changes in the current situation. Maybe Baidu's stronghold is set to weaken as the winds of change blow - surely music to the ears of its many competitors.
