Statistical Class-Based Language Modelling

Scientific Report, University of Cambridge, 1997. — 71 p. In this report, an introduction to natural language modelling is given in the context of speech recognition. Various techniques for formulating stochastic language models are discussed, focusing particularly on N-gram models based on classes of words. A presentation of a number of statistical techniques for the automatic classification of words is given. Results for two automatic clustering techniques are presented along with notes on their implementation in class-level language models. These results are compared with a number of word-level models. Finally, a section on the direction in which subsequent research will develop is included. Introduction Language Modelling Automatic Classification Techniques Results Plans for Further investigation A N-gram Statistics for three sizes of Wall-Street Journal Corpora B Algorithm for word rearrangements C Update equations implemented for hill-climbing algorithm D Update equations implemented for multiple word-to-cluster rearrangements E Experimental Method F Russian language text sources