On Wednesday, Opera Software announced MAMA: Metadata Analysis and Mining Application". It's a search engine that indexes structural information about Web pages, so Web developers and standards bodies can get a look in to what technologies are currently being used to create and build websites and also how they are being used.This search engine came about through tests Opera routinely does to make sure its own browser software products are compatible with already existing Web pages that make use of the most commonly used technology.
"We realized internally that we needed to be able to find lots of live sites out there that used certain technologies in certain combinations so we could test our browser on them," said Snorre Grimsby, vice president of quality assurance at Opera in Oslo, Norway.
The current result is a search engine that crawls the Web, discards the content of the Web pages and indexes the types of technologies being used on the sites. This will allow you to find out information such as how many sites are using CSS, the average size of a webpage and whether the site passes mark-up validation.
Grimsby also went on to say that in Opera's own use of MAMA, they found that the average Web page has 47 discrepancies in how the site renders W3C-maintained technologies and the W3C (World Wide Web Consortium) specifications themselves.
Over the next 8 weeks, Opera will be publishing a series of articles on its developer website about their own internal use of MAMA where they will be noting their key findings, statistics and trends that this new search engine will discover.
The MAMA database currently has 100 million total records in 22 tables and 21GB of data representing the analysis of just over 3.5 million URLs, according to Opera's Brian Wilson.
In his main post about MAMA on Dev.Opera he said:
"When the seeds of this project were sown in early 2004, there was little in the way of effective data about the state of the Web... I talked with a variety of my co-workers at Opera regarding what they wanted to know about the Web - program managers, core developers, QA, those involved with writing specs at the W3C, marketing people - anyone I could think of that might provide unique perspectives and questions. Almost every person involved with creating Opera had different questions about 'what is out there' on the Web; thus began the genesis of MAMA."
MAMA is currently being tested by the company and should be released via invitation-only beta by the end of this year with the aim being to release it publicly to developers sometime in the first or second quarter of 2009.
