RE: Clustering search engines http://www.economist.com/daily/columns/techview/displaystory.cfm?story_id=13761953&fsrc=nwl
[...] In a bid to distinguish itself, Microsoft’s new search engine (it was code-named Kumo, but debuted formally as Bing) organises its results in terms of relevant groups rather than a series of links. That way, it hopes to anticipate a person’s actual interests. Thus, a search for “cheap air fares to London” would also return hotels, restaurants, shops and theatre tickets in broadly the same price bracket. People can refine their queries using a table of contents with multiple options.
Underneath the bonnet, Bing is believed to use semantic technology from a search firm in San Francisco called Powerset, which was acquired by Microsoft last year. Semantic search engines like Powerset or Hakia of New York look at the meaning of the phrase being searched, and try also to distinguish between words with the same spellings (such as the bark of a tree versus the bark of a dog) by taking their context into account. That alone cuts out a lot of stupid answers.
Whether Bing also includes the natural-language processing module that was part of Powerset’s original engine has not yet been revealed. To date, natural-language processing has tended to be used only in specialist fields, such as medicine or law, where the terminology is limited. The computational burden of performing natural-language processing would slow things down too much if applied to an all-purpose search engine that has to index the entire web continuously rather than a mere “vertical” slice of it.
To speed things up, Hakia has invented a new way of analysing web pages and storing their contents. First, a query detection and extraction algorithm (QDEX for short) crawls the vertical segment in question (say, medicine) and extracts all the possible queries that can be asked of each page’s content. It then uses the queries as “gateways” to the document’s paragraphs, sentences, phrases and facts during the retrieval process.
Unlike the “inverted index” (a method for mapping words and numbers in a web page to locations in a database) used by conventional search engines, the QDEX algorithm can handle vast amounts of semantically rich data on the fly. If the Google algorithm tried to do the same, the computational burden would increase exponentially and bring the search engine to its knees. By contrast, the Powerset engine uses a traditional inverted index like Google, but overlays it with natural-language processing plus huge amounts of computational horsepower.
Though still under development, Hakia’s semantic engine is currently “QDEX-ing” such vertical segments as finance, law, travel, arts and history as well as science and medicine. When it is ready, it will be a powerful tool for delivering contextually relevant answers to specific questions.
But your correspondent is still not satisfied. He wants a machine that can ask questions as well as answer them. In short, he wants to see search engines evolve into invention engines.
Imagine you are a chemist at Pfizer with a process for making alpha blockers for treating enlarged prostate glands. With the drug’s patents about to expire, how do you replace the billion-dollar-a-year revenue stream and find a new role for the equipment?
Plug the problem into a semantic search engine that has read all the literature on alpha blockers, interrogated the patent libraries, medical records, catalogues of competitors’ products, and studied the chemical theory. Hey presto! A range of pharmaceutical opportunities is spat out.
Science fiction? At the moment, yes. Within a decade, however, invention engines could be commonplace.
What makes your correspondent confident is the progress computer scientists have made in getting machines to understand the complete and unambiguous meaning of human sentences. The approach goes way beyond the realm of semantic engines based on linguistic analysis. It is what researchers are beginning to call “deep content analysis”.
A few weeks ago, your correspondent witnessed a demonstration of a medical diagnosis and treatment model built by NetBase in Mountain View, California. It was far more impressive than Wolfram Alpha. The machine parsed a statement such as “Magnesium is known to help with high blood pressure often due to stress” and extracted not only the keywords (“magnesium”, “known”, “help”, “high”, “blood”, “pressure”, “stress&rdquo —as Google and any other keyword search engine would do. But it also recognised that “magnesium” was a chemical entity and “high blood pressure” and “stress” were medical conditions—in much the way a semantic search engine might. Then it went on to define “magnesium” as a possible treatment, and the phrase “is known to help with” as a problem-solution relationship and “often due to” as a causal relationship.
The important thing about such relationships is that they are the “connective tissue” between problems and answers—and the key to a whole new approach to asking questions and getting meaningful answers. NetBase calls such relationships “semantic lenses”.
Apart from actually understanding statements like the one above, the NetBase engine retrieved all the benefits and problems associated with magnesium as well as products containing the element, and organisations selling it. It even identified the various causes, drugs, complications, treatments and useful foods for dealing with hypertension. In one instance, the NetBase model found, in minutes, the best drug for treating a rare disease that had taken a skilled researcher months to identify.
The health model was just one example. NetBase has built a library of semantic lenses that can be applied to practically any topic. It has already delivered a research-and-development engine (called Illumin8) to Elsevier, an Anglo-Dutch technical publisher. It is now building a market-research engine for a large household-goods company capable of surveying up to a billion people at a time.
It cannot be long before NetBase (or one of its fledgling rivals) creates an engine capable of inventing things people never realised they needed. Just imagine the productivity gains such an innovation would unleash.
[Edited by Jacek K. on May 31, 2009 3:11 PM]
| 
Translation Profession 
TC Mobile |
Ρυθμίσεις Φόρουμ |
Σύνδεση
Reply
Quote
Edit
Delete