Discriminative Machine Learning Methods for Natural Language Processing

Natural language processing offers a rich problem domain for machine learning approaches. Many NLP problems require the induction of a mapping that involves complex, discrete structures such as strings, labeled sequences, or trees. "Generative" statistical models (such as hidden Markov models, or probabilistic context-free grammars) are a very common approach for this kind of problem.

This talk will focus on discriminative methods as an alternative to generative models. In the first part of the talk I'll describe how "large margin" methods in machine learning -- for example, support vector machines, the perceptron algorithm, or boosting algorithms -- can be generalized to structured problems found in NLP.

The second part of the talk will describe how discriminative methods can be extended to the problem of mapping sentences to lambda-calculus encodings of their semantics. A key challenge in this problem is that the derivations from sentences to their logical forms, and the lexical entries used within these derivations, are not observed in the training data. I'll describe a learning algorithm that takes as input a training set of sentences labeled with expressions in the lambda calculus. The algorithm induces a combinatory categorial grammar (CCG), along with a probabilistic model that represents a distribution over syntactic and semantic analyses for a given input sentence.

The talk includes joint work with Nigel Duffy, Mark Johnson, Terry Koo, Brian Roark, Murat Saraclar, and Luke Zettlemoyer.