swMATH ID: 2620
Software Authors: Cohen, W.W.
Description: The integration of distributed, heterogeneous databases, such as those available on the World Wide Web, poses many problems. Here we consider the problem of integrating data from sources that lack common object identifiers. A solution to this problem is proposed for databases that contain informal, natural-language “names” for objects; most Web-based databases satisfy this requirement, since they usually present their information to the end-user through a veneer of text. We describe WHIRL, a “soft” database management system which supports “similarity joins,” based on certain robust, general-purpose similarity metrics for text. This enables fragments of text (e.g., informal names of objects) to be used as keys. WHIRL includes textual objects as a built-in type, similarity reasoning as a built-in predicate, and answers every query with a list of answer substitutions that are ranked according to an overall score. Experiments show that WHIRL is much faster than naive inference methods, even for short queries, and efficient on typical queries to real-world databases with tens of thousands of tuples. Inferences made by WHIRL are also surprisingly accurate, equaling the accuracy of hand-coded normalization routines on one benchmark problem, and outperforming exact matching with a plausible global domain on a second.
Homepage: http://www.cs.cmu.edu/~wcohen/whirl/
Keywords: knowledge representation; information retrieval; textual similarity; heterogeneous databases; information integration; text categorization; information extraction
Related Software: C4.5; XPath; SSE; Dyna; Figaro; PRISM; MCINTYRE; BLOG; Church; IBAL; PITA; CP-logic; ProbLog; XIRQL; See5; XQuery; WordNet; Python; ADMIT; Odyssee
Cited in: 11 Documents

Standard Articles

1 Publication describing the Software, including 1 Publication in zbMATH Year
WHIRL: A word-based information representation language. Zbl 0938.68841
Cohen, W. W.

Citations by Year