Semi-Automated Identification of Faceted Categories from Large Corpora‪.‬

INTRODUCTION This paper describes FFID (Fast Facet Identifier), a system that can be used to compute facets from a corpus of documents. FFID uses a fast simplified clustering algorithm that allows the identification of hundreds of facet clusters from a corpus of hundreds of thousands of sentences in a very short time (seconds). The automatic identification of facets may be a very powerful tool to design better information retrieval systems. The goal of information retrieval is to support people in searching for the information they need. Given an information problem, finding relevant (let alone high quality) documents is difficult. The sheer amount of information available on line makes this a difficult problem. The size of the web is debatable (Markoff, 2005) but it must be by now at least 12,000 million pages. If each one of these web pages were printed on a standard A4 sheet of paper (21-cm wide), and put side to side on a straight line, it would take about 60 earth circumferences to lay them all down. This is a lot of information. People learn about their information problem and about the information resource they are using through interaction with the resource. Human computer interaction is the crucial phenomenon of the information retrieval process. Fast algorithms, hardware for storage and processing, data and knowledge structures are important but useless if we do not understand how humans interact with machines when looking for information. All the techniques we use must first take into account what we are doing this for: the user. Users encounter several problems when they approach an information resource: