Logic Programming for Linguist(ic)s + Logic Grammars with Constraints ≈ an easy way to Semantics

1. Logic Programming for Linguist(ic)s+Logic Grammars with Constraints≈ an easy way to Semantics Tutorial at GRLMC, Tarragona, 20 November 2009 Henning Christiansen professor of Computer Science Research group PLIS: Programming, Logic and Intelligent Systems Department of Communication, Business and Information Technologies Roskilde University P.O.Box 260, DK-4000 Roskilde, DENMARK http://www.ruc.dk/~henning

2. Speaker’s background • Computer Science; 1980ies and onwards programming languages, compilers and syntax (“extensible syntax”) • Current interests: Logic Programming, Databases, AI, Implementation of advanced reasoning, applications for NLP, Bioinformatics, Probabilistic Models, etc. • Background in linguistics = Ø, but inspiration from • working with people who know more about language than I do • the reception of my LP work by such people • Projects • The CONTROL research project (2004–8) • “CONstrainTs for RObust Language processing” • The LoSt research project (2007-2012) • "Logic-Statistical analysis of biological sequence data" • 3 PhD students, 2 postdocs, 2 companies, partners in Denmark, Tokyo, ... contacts to GB, Norway...

3. Motivation & intuition Logic programming in linguistics / for linguistic students: • You can program without being a programmer  • Logic programming in Prolog is simple, intuitive and well-suited for phrase-structure grammars Involving constraints in logic grammars: • Provides an easy way to semantics/pragmatics • while keeping the qualities of Prolog programming

4. Outline of this tutorial • Crash course on Prolog programming • with references if you want to continue yourselves • Definite Clause Grammars, DCGs(simple add-on to Prolog) • Introducing Constraint Handling Rules, CHR (another add-on to Prolog) • Relation to Abductive Logic Programming • Using CHR for to add meanings to DCGs • A practical application • Conclusion • Approach won’t solve all problems, but good for illustrative purposes • and in teaching • (prototype) applications • and (not least) fun to play with!

5. 1. A crash course on Prolog Prolog differs from Java, C, etc.: • based on logic • focuses on "what" rather that "how" • programs are concise and compact • programs can be tested and built incrementally • rule-based • symbols and structure (rather that numbers) • and very easy to learn! History • A.Colmerauer & co. (Marseille), ca. 1970: "Prolog" • D.H.D. Warren: Efficient compiler, 1975, • Language made known by R.Kowalski "Logic for Problem solving", 1979, .... • Since then: several available systems...

6. Program is a description of data parent( pam, bob). % Pam is a parent of Bob parent( tom, bob). parent( tom, liz). parent( bob, ann). parent( bob, pat). parent( pat, jim). pam tom bob liz ann pat jim

7. Basic notions: • predicates: parent • describes a relation • defined by facts, rules, collectively called clauses • constants: tom, bob, x, y • variables: X, Y, Tom • simple goals: parent(A,a) • Queries....

8. Queries Atomic queries ?- parent(X,Y). ... give me values of X and Y so parent(X,Y) logically follows from program Compound query ?- parent(pam, X), parent(X, Y). ... give me X and Y, so that...

9. Procedural semantics X=bob • Unification term=term? • from left to right • from start to end • backtracking • ≈ undo and try new choices parent( pam, bob). parent( tom, bob). parent( tom, liz). parent( bob, ann). parent( bob, pat). parent( pat, jim). ?- parent(pam, X), parent(X, Y). ?- parent(bob, Y). Y=ann Success! Other solutions? Success! Other Solutions? Y=pat No more possible solutions at all :( No more possible solutions here :(

10. Rules Procedural semantics as before + rewrite subgoal using rules Declarative semantics ≈ logical consequence in 1st order logic The nice property: procedural ≈ declarative grandparent(X,Z):- parent(X, Y), parent(Y, Z). • Other features • structures and lists (will see later) • control mechanisms (advanced; not needed for today)

11. 2. Definite Clause Grammars Syntactic sugar for Prolog programs that perform top-down, backtracking parsing. In Prolog since 1970ies [Colmerauer; Pereira, Warren; ...] Features ≈ arguments and terms as in Prolog predicates Example: Popular, easy to work with, fun to play with, etc. ?- [simpleDCG].

12. Adding contexts ≈ knowledge bases Usual approach: more and complex features. Discourse analysis with semantics story(C0,M1+M2) --> [] ; s(C0,C1,M1), ['.'], story(C1,M2). We try another something else: use a global resource to store context and meanings. Thus stay with the rule story --> [] ; s, ['.'], story. But “global resource” in Prolog?? = constraint store and meanings etc. as constraints...

13. 3. Constraint Handling Rules, CHR A recent addition to Prolog; available in several major Prologs [Frühwirth, 1992, 1998] A “glass box” approach to constraint solving: • defining constraints and their behaviour in a declarative language Prolog: Backward chaining CHR: Forward chaining Prolog+CHR by an example that anticipates applications for language analysis... ?- [happy1]. ?- [happy2]. ?- [happy3].

14. 3. Theoretical interlude: You have just seen an implementation of Abductive Logic Programming using Prolog+CHR (Peirce, 18-something); (Conan Doyle, 19-something - Sherlock Holmes); Kakas, Kowalski, etc. 1980ies, ......) Def: Given Rules and Observations, abducible predicates and Integrity Constraints, a set of abducible facts A is an abductive answer whenever Rules U A |= Observations Rules U A |= Integrity Constraints ?- [RulesAbdPredsICs]. ?- Observations. abductive-answer ? ; Relationship ALP <-> CHR: [S.Abdennadher, HC, 2000; HC, V.Dahl, 2004, etc.] H. Christiansen. Executable specifications for hypothesis-based reasoning with Prolog and Constraint Handling Rules, Journal of Applied Logic, vol 7, 2009. pp. 341-362


15. 3. Back to CHR Three kinds of rules: Propagate c, c, ..., c ==> Guard | ... c ... adds constraints Simplify c, c, ..., c <=> Guard | ... c ... replaces constraints Simpagate c, ... \ c, ... <=> Guard | ... c ... remove some, keep some and add others ——————————————————————— Declarative semantics: as indicated by the arrow shapes Obs: Prolog+CHR sum up to a language with declarative semantics Theorem: Implementation of abduction is correct :)

16. 5. Back to discourse analysis ?- [discourse1]. First version, just collecting facts Second version, adding rules about semantic/pragmatic universe ?- [discourse2]. • ————————————————————————————— • Reflections: • An instance of “Interpretation of abduction”, [Hobbs & al, 1993] • Notice that distinction semantics/pragmatics disappears, i.e., • i.e., no “objective and context independent” meaning before mapping to “real world objects”, • [Christiansen, Dahl, Context05] formalize “Meaning in Context” with possible-worlds semantics and relates to abduction and CHR

17. A taste of “Meaning in Context”[Christiansen, Dahl, Context05] Meaning of The tall, red-haired man carrying a laptop won a brand new Ferrari. is won(X,Z) in the context tall(X), red_haired(X), carries(X,Y), laptop(Y), won(X,Z), ferrari(Z), brand_new(Z) Might be the same as the meaning of He won it

18. Context in disambiguation ?- [contextLex1]. • version 1, static context • version 2, non-monotonic change of context ?- [contextLex2]. • Theoretical interlude • Version 2 goes beyond the indicated declarative semantics • A programmer’s mind uses procedural knowledge to implement a sort of non-monotonic logic. • [Betz, Frühwirth, CP05] has defined a linear-logic semantics for CHR which fits such usages

19. Assumptions in logic grammars • [Dahl & al., 1997; Christiansen, Dahl, 2004, ...] • Similar to abduction but with explicit creation and application + simple scoping • Can be implemented in CHR more or less the same way • Included in the Hyprolog system Examples...

20. Assumptions in Hyprolog An example: Pronoun resolution assumptions acting/1. abducibles fact/1. sentence --> np(A,_), verb(V), np(B,_), {fact(A,V,B)}. sentences --> [] ; sentence(S1),sentences(S2). np(X,Gender) --> name(X,Gender), {*acting(X,Gender)}. name(peter,masc) --> [peter]. ... np(X,Gender) --> {-acting(X,Gender)}, pronoun(Gender). pronoun(fem) --> [her]. ... verb(like) --> [likes]. “Peter likes Mary. She likes him” *acting(peter,masc) *acting(mary,fem) -acting(X,fem)  X=mary -acting(X, masc)  X=peter fact(peter,like,mary) fact(mary, like,peter)

21. Assumptions in Hyprolog Another example: Coordination assumptions ..., verb_used/1. sentence --> np(A,_), verb(V), np(B,_), {*verb_used(V), fact(A,V,B)}. ... verb(like) --> [likes]. verb(V) --> [],{-verb_used(V)}. “Peter likes Mary, and she him” ... *verb_used(like) -verb_used(X)  X=like fact(peter,like,mary) fact(mary,like,peter)

22. Side remark What we have shown until now, may be seen as logic programming dressed up by a nice syntax. It has been described in a formal way: Henning Christiansen, Verónica Dahl Abductive Logic Grammars. Logic, Language, Information and Computation. 16th International Workshop, WoLLIC 2009. Tokyo, Japan, June 21-24, 2009. Lecture Notes in Computer Science 5514, pp. 170-181, Springer 2009. 


23. 6. An practical application in more detail Mapping use case text (a la OOP/OOD) into UML diagrams Example of discourse analysis, building a knowledge base = context Features: • non-trivial properties of semantic/pragmatic universe • resolution of pronouns based on well-defined heuristics [Christiansen, Have, Tveitane, RANLP2007, CSLP2007]

24. The setting... • Use Cases • Textual description of what a “system” does without specifying how. • Used in the inception phase of the object oriented software development process. • Reasoning about Use Cases • Parsing use cases to uncover programmatic semantics • Result: Diagrammatic representation (UML)

25. Language for Use Cases • Restricted natural language (English) • Natural flow, style and expressiveness • Unambiguous parsing • Why avoid ambiguity

26. Programmatic Semantics • Classes and objects • Methods • Properties • Inheritance • Instantiation • Temporality (not shown here)

27. Programmatic semantics, methods • The professor teaches. Generates one class: With one method

28. Programmatic semantics, methods • The professor teaches. • A student reads, writes projects and takes exams.

29. Programmatic Semantics, properties • A professor has an office.

30. Programmatic Semantics, properties • A professor has an office. • The university has five study lines.

31. Programmatic Semantics, inheritance • Students and professors are persons.

32. System Architecture • Architecture = Prolog + DCG + CHR • Diagrams are generated by GraphViz

33. sentence --> fc_noun_phrase(Number,_ , subj, IdSub), subord_verb(Number,_ ), fc_noun_phrase(_ , Number, obj, IdObj), {extends(IdSub,IdObj)}. Input grammar fc_ “cats and dogs” cat+dog indiv_ “her, Peter and Paul” mary+peter+paul rc_ “Mary and the boys” woman+boy q_ “a tail and some legs” tail:1+legs:n The use case text is parsed using a Definite Clause Grammar (DCG) 'extends' is not a usual Prolog predicate, but declared as a constraint • There are sentence grammar rules like this for different sentence types • Each rule captures the semantic features of a particular type of sentence

34. Reasoning with Constraints, I Example: Professors and students are persons. Example: A professor is a teacher and a researcher. Constraint Handling rule: Constraint Handling rule: extends(A+B, C) <=> extends(A,C), extends(B,C). extends(A, B+C) <=> extends(A, B), extends(A, C). Constraint store: Constraint store: extends(professor+student, person) extends(professor, teacher+researcher) extends(professor, person) extends(student, person) extends(professor, teacher) extends(professor, researcher)

35. Reasoning with Constraints, II Example: Paul has a dog and Peter has five dogs. (Peter and Paul are men). Constraint Handling rule: property(C,P:N), property(C,P:M) <=> q_count(N), q_count(M), q_less_eq(N,M) | property(C,P:(N..M)). ... property(man, dog:1) property(man, dog:5) property(man, dog:(1..5))

36. Reasoning with Constraints, III Ranges can be merged: property(man, dog:(0..2)) property(man, dog:(1..n)) property(man, dog:(0..n)) Constraint Handling rule: property(C,P:(N1..M1)),property(C,P:(N2..M2)) <=> q_min(N1,N2,N), q_max(M1,M2,M), property(C,P:(N..M)).

37. Reasoning with Constraints, IV Anaphora resolution - Individuals as prototypes. Example: John is a student. John studies. class(student) object(john, student) method(john, study) class(student) object(john, student) method(student, study) Constraint Handling Rule: object(Id, Cl) \ method(Id, M) <=> method(Cl, M)

38. Reasoning with Constraints, V Anaphora resolution - Pronouns. Example: Jack and John are teachers. Jack teaches music. John teaches computer science. Mary is a student. He has many students. He refers to John. Constraint store: ... referent(sing,masc,1,jack) referent(sing,masc,1,john) referent(sing,masc,2,jack) referent(sing,masc,3,john) referent(sing,fem,4,mary) expect_referent(sing,masc,X) Referents are labelled with the sentence number in which they occur. Heuristic method resolves the entity matching number and gender in the most recent sentence. Ambiguous pronouns are rejected. Constraint Handling Rule: sentence_no(Now), referent(No,G,Id,T) \ expect_referent(No,G,X) <=> T < Now, there is no other relevant referent with Timestamp > T | ifthere is another relevant referent with Timestamp = T then X = errorcode(ambiguous) else X = Id.

39. Conclusion of the example Shows an application of discourse analysis with • restricted language • “concise” semantics • a desire of an unambiguous interpretation Demonstrates that Prolog+DCG+CHR applied for “interpretation as abduction” in this case provides • flexibility and ease of modeling • concise, executable specifications • logic embedded in a full programming language, i.e., you can hack the logic if you feel like it

40. Other approaches to languages analysis using CHR CHR grammars [Christiansen, 2002-3, TPLP2005] Grammar notation for bottom-up parsers entirely with CHR; features • Context-sensitive rules a -\ b,c /- ..., d ::> ecompiled into a(_,N1), b(N1,N2), c(N2,N3), d(N4,_) ==> N3=<N4 | e(N1,N3) • Ambiguity and partial parsing handled for free • Includes abduction and assumption • Disadvantage: Alternative interpretations all mixed up • a version exists, which handles this correctly; never documented or released (awkward implementation in CHR; low-level impl. considered) Applications • Blache & Dahl: Property grammars (repr. phrase as set of sat/unsat constraints) • Dahl and her students: Biomedical text; sequence analysis; etc. • My students: Hieroglyph inscriptions; use cases (a la OOP); sign language (strong potential; but, sic!, we did not get the grant)

41. Conclusions • Constraints adds a state component to logic grammars • The state can be first-order formulas or other other kinds of knowledge representation • Interpretation-as-abduction for discourse analysis fits in naturally • Nonmonotonic updates can be modelled, e.g., reflecting a time dimension over a changing world • Prolog + CHR provides • a flexibility for a programmer’s mind to hack the logic (and perhaps formalize afterwards;-) • an effective environment for prototyping / executable specifications / implementation

42. Recent results Probabilistic & prioritized abduction in CHR, best first search • H.Christiansen. Implementing Probabilistic Abductive Logic Programming with Constraint Handling Rules, 35 pp., Constraint Handling Rules, Current Research Topics. T. Schrijvers, T. Frühwirth (eds.). Lecture Notes in Artificial Intelligence 5388, Springer 2008. pp. 85-118 • H. Christiansen. Prioritized Abduction with CHR, Proc. Fifth Workshop on Constraint Handling Rules, Hagenberg, Austria, July 14, 2008. Probabilistic DCGs • C.T.Have. Stochastic Definite Clause Grammars. Proc. RANLP 2009, Recent Advances in Natural Language Processing, Bulgaria, September 14-17, 2009. • C.T.Have., Stochastic Definite Clause Grammars; Master thesis, IT-University of Copenhagen, 2008 Work in progress • General probabilistic-logic models for bioinformatics, e.g. • H. Christiansen. C.T. Have, O.T. Lassen, M. Petit. A Constraint Model for Constrained Hidden Markov Models: a first Biological Application WCB09, Workshop on Constraint Based Methods for Bioinformatics, Associated to CP 2009. Lisbon, Portugal, September 20, 2009.

43. References Texts • Intro to Prolog with CHR and AI applications H.Christiansen: Introduction to Prolog as a framework for Knowledge Representation and Artificial Intelligence, (2006) http://www.ruc.dk/~henning/KIIS08/courseMaterial/CourseNote.pdf • Intro to Prolog, focus on DCG P.Blackburn, J.Bos, and K.Striegnitz: Learn Prolog Now! (2001) http://www.learnprolognow.org • Overview of Prolog+CHR for logical reasoning Henning Christiansen. Executable specifications for hypothesis-based reasoning with Prolog and Constraint Handling Rules, Journal of Applied Logic, vol 7, 2009. pp. 341-362 Preliminary version: http://www.ruc.dk/~henning/publications/HypoReason2008_HC.pdf Good Prolog systems with CHR • SICStus Prolog (costs money, institution license?, 30 days test version) http://www.sics.se/isl/sicstuswww/site/index.html • SWI Prolog, free http://www.swi-prolog.org/ Software by the speaker: • HYPROLOG: http://www.ruc.dk/~henning/hyprolog/ • CHR Grammars: http://www.ruc.dk/~henning/chrg/

44. The end