Retriever: Improving Web Search Engine Results Using Clustering

Abstract

Web search engine have become increasingly ineffective as the number of documents on the web have proliferated. Typical queries retrieve hundreds of documents, most of which have no relation with what the use was looking for. The objective of this work is to propose new techniques to cluster the results of a query from a search engine into groups. These groups and their associated keywords are presented to the user, who can then look into the URLs for the group(s) that s/he find interesting. N-gram and vector space methods are used to create the dissimilarity matrix for clustering. We compare these distance metrics by clustering the data using a robust fuzzy algorithm and evaluating the results.