Exploring Computational Semantics: Syntax, Meaning, and Inference in Natural Language Processing

1. Computational Semanticshttp://www.coli.uni-sb.de/cl/projects/milca/esslli Aljoscha Burchardt, Alexander Koller, Stephan Walter, Universität des Saarlandes, Saarbrücken, Germany ESSLLI 2004, Nancy, France

2. Computational Semantics • How can we compute the meaning of e.g. an English sentence? • What do we mean by “meaning“? • What format should the result have? • What can we do with the result?

3. The Big Picture • Sentence: “John smokes”. • Syntactic Analyses: S NPVP John smokes • Semantics Construction: smoke(j) • Inference:x.smoke(x)snore(x),smoke(j) => snore(j)

4. Course Schedule • Monday - Thursday: Semantics Construction • Mon.+Tue.: Lambda-Calculus • Wed.+Thu.: Underspecification • Friday: Inference • (Semantic) Tableaux

5. The Book • If you want to read more about computational semantics, see the forthcoming book: Blackburn & Bos, Representation and Inference: A first course in computational semantics. CSLI Press.

6. Today (Monday) • Meaning Representation in FOL • Basic Semantics Construction • -Calculus • Semantics Construction with Prolog

7. Meaning Representations • Meaning representations of NL sentences • First Order Logic (FOL) as formal language • “John smokes.“ => smoke(j) • “Sylvester loves Tweety.” => love(s,t) • “Sylvester loves every bird.” => x.(bird(x)  love(s,x))

8. In the Background: Model Theory • x.(bird(x)  love(s,x)) is a string again! • Mathematically precise model representation, e.g.: {cat(s), bird(t), love(s,t), granny(g), own(g,s), own(g,t)} • Inspect formula w.r.t. to the model: Is it true? • Inferences can extract information: Is anyone not owned by Granny?

9. FOL Syntax (very briefly) FOL Formulae, e.g. x.(bird(x)  love(s,x)) FOL Language • Vocabulary (constant symbols and predicate/relation symbols) • Variables • Logical Connectives • Quantifiers • Brackets, dots.

10. What we have done so far • Meaning Representation in FOL  • Basic Semantics Construction • -Calculus • Semantics Construction with Prolog

11. Syntactic Analyses Basis: Context Free Grammar (CFG) Grammar Rules: S  NP VP VP  TV NP TV  love NP  john Lexical Rules / Lexicon NP  mary ...

12. Example: Syntactic Analyses

13. Example: Semantic Lexicon

14. Example: Semantics Construction

15. Example: Semantics Construction

16. Compositionality The meaning of the sentence is constructed from: • The meaning of the words: john, mary, love(?,?) (lexicon) • Paralleling the syntactic construction (“semantic rules”)

17. Systematicity • How do we know that e.g. the meaning of the VP “loves Mary” is constructed as love(?,mary) and not as love(mary,?) ? • Better: How can we specify in which way the bits and pieces combine?

18. Systematicity (ctd.) • Parts of formulae (and terms), e.g. for the VP “love Mary”? • love(?,mary) bad: not FOL • love(x,mary) bad: no control over free variable • Familiar well-formed formulae (sentences): • x.love(x,mary) “Everyone loves Mary.” • x.love(mary,x) “Mary loves someone.”

19. Using Lambdas (Abstraction) • Add a new operator to bind free variables: x.love(x,mary) “to love Mary” • The new meta-logical symbol  marks missing information in the object language (-)FOL • We abstract over x. • How do we combine these new formulae and terms?

20. Super Glue • Glueing together formulae/terms with a special symbol @: x.love(x,mary) john x.love(x,mary)@john • Often written as x.love(x,mary)(john) • How do we get back to the familiar love(john,mary)?

21. Functional Application • “Glueing” is known as Functional Application • FA has the Form: Functor@Argument x.love(x,mary)@john • FA triggers a very simple operation: Replace the -bound variable by the argument. • x.love(x,mary)@john => love(john,mary)

22. -Reduction/Conversion • Strip off the -prefix, • Remove the argument (and the @), • Replace all occurences of the -bound variable by the argument. x.love(x,mary)@john • love(x,mary)@john • love(x,mary) • love(john,mary)

23. Semantics Construction with Lambdas S: John loves Mary (yx.love(x,y)@mary)@john NP: John john VP: loves Mary yx.love(x,y)@mary TV: loves yx.love(x,y) NP: Mary mary

24. Example: Beta-Reduction (yx.love(x,y)@mary)@john => (x.love(x,mary))@john => love(john,mary)

25. In the Background • -Calculus • A logical standard technique offering more than -abstraction, functional @pplication and β-reduction. • Other Logics • Higher Order Logics • Intensional Logics • ... • For linguistics: Richard Montague (early seventies)

26. What we have done so far • Meaning Representation in FOL  • Basic Semantics Construction  • -Calculus  • Semantics Construction with Prolog

27. Plan Next, we • Introduce a Prolog represenation. • Specify a syntax fragment with DCG. • Add semantic information to the DCG. • (Implement β-reduction.)

28. Prolog Representation: Terms and Formulae (man(john)&(~(love(mary,john)))) (happy(john)>(happy(mary)v(happy(fido))) forall(x,happy(x)) exists(y,(man(y)&(~(happy(y)))) lambda(x,…)

29. Prolog Representation:Operator Definitions Binding Strength :- op(950,yfx,@). % application :- op(900,yfx,>). % implication :- op(850,yfx,v).   % disjunction :- op(800,yfx,&).   % conjunction :- op(750, fy,~).    % negation forall(x,man(x)&~love(x,mary)>hate(mary,x)) („Mary hates every man that doesn‘t love her.“)

30. Definite Clause Grammar • Prolog‘s built-in grammar formalism • Example grammar: s --> np,vp. vp --> iv. vp --> tv,np. ... np --> [john]. iv --> [smokes]. • Call: s([john,smokes],[]).

31. Adding Semantics to DCG • Adding an argument to each DCG rule to collect semantic information. • Phrase rules of our first semantic DCG: s(VP@NP) --> np(NP),vp(VP). vp(IV) --> iv(IV). vp(TV@NP) --> tv(TV),np(NP).

32. Lexicon Of Our First Semantic DCG np(john) --> [john]. np(mary) --> [mary]. iv(lambda(X,smoke(X))) --> [smokes],{vars2atoms(X)}. tv(lambda(X,lambda(Y,love(Y,X)))) --> [loves],{vars2atoms(X), vars2atoms(Y)}.

33. Running Our First Semantics Constrution ?- s(Sem,[mary,smokes],[]). Sem = lambda(v1, smoke(v1))@mary ?- …, betaConvert(Sem,Result). Result = smoke(mary) Note that we use some special predicates of freely available SWI Prolog (http://www.swi-prolog.org/).

34. betaConvert(Formula,Result) 1/2 betaConvert(Functor@Arg,Result):-betaConvert(Functor,lambda(X,Formula)),!,substitute(Arg,X,Formula,Substituted),betaConvert(Substituted,Result). • The input expression is of the form Functor@Arg. • The functor has (recursively) been reduced to lambda(X,Formula). Note that the code displayed in the reader is wrong. Corrected pages can be downloaded.

35. betaConvert(Formula,Result) 2/2 betaConvert(Formula,Result):- compose(Formula,Functor,Formulas), betaConvertList(Formulas,Converted), compose(Result,Functor,Converted). Formula = exists(x,man(x)&(lambda(z),walk(z)@x)) Functor = exists Formulas = [x,man(x)&(lambda(z),walk(z)@x))] Converted = [x,man(x)&walk(x)] Result = exists(x,man(x)&walk(x))

36. Helper Predicates betaConvertList([],[]). betaConvertList([F|R],[F_Res|R_Res]):-        betaConvert(F,F_Res),        betaConvertList(R,R_Res). compose(Term,Symbol,Args):- Term =.. [Symbol|Args]. substitute(…) (Too much for a slide.)

37. Wrapping It Up go :- resetVars, readLine(Sentence), s(Formula,Sentence,[]), nl, print(Formula), betaConvert(Formula,Converted), nl, print(Converted).

38. Adding More Complex NPs NP: A man ~> x.man(x) S: A man loves Mary Let‘s try it in a system demo!

39. Tomorrow S: A man loves Mary ~> *love(x.man(x),mary) • How to fix this. • A DCG for a less trivial fragment of English. • Real lexicon. • Nice system architecture.