Retrieving Information in Croatian : Building a Simple and Efficient Rule-Based Stemmer

Ljubešić, Nikola and Boras, Damir and Kubelka, Ozren. (2007). Retrieving Information in Croatian : Building a Simple and Efficient Rule-Based Stemmer. In: 1. međunarodna znanstvena konferencija "The Future of Information Sciences: INFuture2007 – Digital Information and Heritage", 7.-9. studenoga 2007., Zagreb.

Preview

PDF (English) Download (302kB) | Preview

Abstract

Since Croatian is a highly flective language there is a need for morphological normalization of natural language information so that information could become retrievable in a more efficient way. Although this topic has been researched for more than two decades in Croatia, the vast majority of information systems that store information written in Croatian still do not have this problem solved. The primary cause for this situation is the high price of existing systems. The aim of this paper is to analyze the current situation in the industry regarding this problem and to build a rule-based stemmer which would consist of a minimal set of rules for expanding queries to the whole possible paradigm. Such a system could make expensive morphological databases in information retrieval obsolete. We used a corpus sample, a morphological lexicon and a query sample of 1.000 most frequent nouns in base form to build a rule-based stemmer optimized through the steepest ascent hill climbing algorithm. Using this method we built a stemmer which performs almost equally good as the noun lexicon with F1 measures of 97.82% without the rules for adjectives and 97.64% with them.

Actions (login required)

View Item