Practical Analysis of Non-Termination in Large Logic Programs

1. Practical Analysis of Non-Termination in Large Logic Programs Senlin Liang and Michael Kifer

2. Motivation • High-level LP languages, e.g., • SILK http://silk.semwebcentral.org • Flora-2 http://flora.sourceforge.net are designed to be suitable for knowledge engineers, who are notprogrammers • KBs created by engineers typically • are complex, large • stress the capabilities of the underlying engine =>Non-termination happens often – very hard to debug • To address this, we developed • Terminyzer – a non-Termination analyzer (this extends our previous work in PADL-13)

3. Outline • Preliminaries: causes of non-termination, tabling, and forest logging • Adding ids to rules • Terminyzer: analyses causes, repairs problems • Experiments • Conclusion and future work

4. Causes of Non-Termination • Cause 1: loops in SLD-resolution • Example p(X) :- p(X). ?- p(a). • Solution: tabling[SW12] • Caches calls to subgoals, which cuts recursive loops • If enough predicates are tabled then • Each subgoal is tabled once • Each answer is tabled once • Evaluation terminates if there are finitely many subgoals and answers [SW12] Terry Swift and David S. Warren. XSB: Extending prolog with tabled logic programming. TPLP’12.

5. Causes of Non-Termination (cont’d) • Cause 2: • Engine supports tabling • But the program generates infinitely many tabled subgoals • Example p(X) :- p(f(X)). ?- p(a). Subgoals to be tabled: p(a), p(f(a)), p(f(f(a))), ... • Solution: subgoalabstraction [RS] • Abstracts subgoals that are deeper than a threshold • Assuming threshold = 2, p(f(f(f(a)))) would be abstracted to p(f(f(X))), X = f(a) • Guarantees that there will be only a finite number of tabled subgoals [RS] F. Riguzzi and T. Swift. Terminating evaluation of logic programs with finite three-valued models. ACM on Computational Logic. To appear.

6. Causes of Non-Termination (cont’d) • Cause 3: • Engine supports both tabling and subgoal abstraction • But infinitely many answers • Example p(a). p(f(X)) :- p(X). ?- p(X). Answers to be derived: p(a), p(f(a)), p(f(f(a))), … • Solution: does not exist • Halting problem is undecidable • Whether a program has a finite number of answers is undecidable • We can only try to help the user to deal with the issue • If user really intended the program to be the way it is • We need a way to limit output. E.g., bounded rationality[BS13] • Our focus: Unexpected non-termination (ie, when it’s a bug) [BS13] B. Grosof and T. Swift. Radial restraint: A semantically clean approach to bounded rationality for logic programs. AAAI’13

7. Tabling and Forest Logging • Tabling needs no introduction • Forest logging is a new tracing facility in XSB where • Timestamp preserves the order of events • status = new, complete, incomplete

8. Adding IDs to Rules – Key Insight • Add unique ids to rules s.t. tabled subgoals remember their host rules: • Each tabling declaration :- table p/nis changed to :- table p/(n+1) • For a query ?- p(x1, …, xn) • If p/n is tabled, then • Change it to ?- p(x1, …, xn, Newvar) • Chop off the last arguments of returned answers • Otherwise, the query stays the same

9. Terminyzer Overview • Two versions • Version 1 (most precise) requires: • Tabling • Forest logging • Subgoal abstraction • Version 2 requires only • Tabling • Forest logging • Currently only XSB has all three features, but: • Several systems have tabling • Forest logging info exists internally in all of them – just needs to be exposed to the user • => at least Version 2 is easily portable

10. Terminyzer Overview • Suppose a query does not terminate, Terminyzer then • Analyzes an execution forest log • Determines the causes of non-termination • The exact sequence of unfinished tabled subgoals, and the host rule id of each subgoal • Subgoals forming recursive cycles • Rectifies some causes of misbehavior (heuristically)

11. Call Sequence Analysis • Identifies the exact sequence of unfinished calls and their host rule ids that lead to a non-termination • Find unfinished subgoals– whose answers have not been completely derived – by unfinished(Child,Parent,Timestamp) :- tabled_call(Child,Parent,new,Timestamp), not_exists(completed(Child,SCCNum,Timestamp1)). • unfinished(child,parent,timestamp) says that • Subgoal parent calls subgoal child, and it happened at timestamp • Neither child nor parent have been completely evaluated • Sort unfinished calls by their timestamps • Host rule ids are kept in the last arguments of child-subgoals not_exists is the XSB well-founded negation operator; existentially quantifies SCCNum and Timestamp1.

12. Call Sequence Analysis (cont’d) • Example 1 @!r1 p(a). @!r5 r(X) :- r(X). @!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X). @!r3 q(b). @!r7 s(f(b)). @!r4 q(g(X)) :- p(X). ?- r(X). where @!ruleidis the syntax to assign rule ids • Its unfinished calls – the red ones form a non-terminating loop unfinished(r(_h9900,_h9908), root, 0) – root is for intial query unfinished(r(_h9870,r5), r(_h9870,_h9889), 8) unfinished(r(_h9840,r5), r(_h9840,r5), 11) unfinished(p(_h9810,r6), r(_h9810,r5), 12) unfinished(q(_h9780,r2), p(_h9780,r6), 16) unfinished(p(_h9750,r4), q(_h9750,r2), 20) unfinished(q(_h9720,r2), p(_h9720,r4), 24) • Next, we will find recursive cycles

13. Call Sequence Analysis (cont’d) • Unfinished calls can be represented using an unfinished-call graph UCG = (N,E) • N: the set of unfinished subgoals • E: {(parent,child) |unfinished(child,parent,ts) is true} • For the above example unfinished(r(_h9900,_h9908), root, 0) unfinished(r(_h9870,r5), r(_h9870,_h9889), 8) unfinished(r(_h9840,r5), r(_h9840,r5), 11) unfinished(p(_h9810,r6), r(_h9810,r5), 12) unfinished(q(_h9780,r2), p(_h9780,r6), 16) unfinished(p(_h9750,r4), q(_h9750,r2), 20) unfinished(q(_h9720,r2), p(_h9720,r4), 24) where: • Each node is represented by the timestamp when it is first called • -1 represents root • Edges are labeled with timestamps of calls • Loops in UCG represent recursive cycles • However, not all cycles are causing non-termination • E.g. [8, 8] Can be represented as thousands of subgoals

14. Call Sequence Analysis (cont’d) • Assume all predicates are tabled + subgoal abstraction • Theorem(Soundness of the call sequence analysis) If there are unfinished calls in a query’s complete trace, then • Call sequence analysis finds the exact sequence of unfinished calls that caused non-termination, and • The ids of the rules that issued these calls • Theorem (Completenessof the call sequence analysis) If the evaluation of a query does not terminate, then • There is at least one loop in the UCG for its complete trace, and the loop’s subgoals are responsible for generating infinite number of answers, and • The last argument of each of these subgoalsspecifies the rule ids from whose bodies these subgoals were called. • The complete trace is infinite due to non-termination so, practically speaking • We work with only a prefix of the trace by limiting term depth/size or execution time/space • It may produce false negatives, but they are also useful for identifying computational bottlenecks.

15. Answer Flow Analysis • Recall that not all cycles in UCG are causing non-termination, so we need to refine call sequence analysis • Answer flow analysis does precisely that: it identifies the cycles that actually cause non-termination • Non-termination happens if and only ifa subset of subgoalskeeps: • Receiving answers from producers, • Deriving new answers, and • Returning answers to callers • Answer flow analysis looks for repeated patterns of answer returns

16. Answer Flow Analysis (cont’d) • Compute answer-flow patterns (AFP) • Answer-return sequence (ARS): the sequence of (child,parent) pairs where child returns answers to parent • Candidate AFP: a sequence cafps.t.cafp2+is a suffix of ARS • AFP: the shortest candidate AFP cafps.t. its repetition forms the maximal suffix of ARS among all candidate AFP’s • In previous Example 1 • ARS = [ (p(_h599,r4),q(_h599,r2)), (q(_h619,r2),p(_h619,r4)), (p(_h639,r4),q(_h639,r2)), (q(_h659,r2),p(_h659,r4)), (p(_h679,r4),q(_h679,r2)), (q(_h699,r2),p(_h699,r4)), (p(_h719,r4),q(_h719,r2)), (q(_h739,r2),p(_h739,r4)), (p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))]. where (child,parent) indicates child returns answers to parent • Candidate AFPs are: • cafp1 = [(p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))] • cafp2 = cafp1• cafp1 • AFP is cafp1 • AFP captures information flow pattern without redundancy @!r2 p(f(X)) :- q(X). @!r4 q(g(X)) :- p(X).

17. Answer Flow Analysis (cont’d) • An AFP can be represented as an answer-flow graph AFG = (N,E), where • N: the set of subgoals in afp • E: {(child,parent) | (child,parent) ∈ afp} • Loops in AFG represent cycles that cause non-termination

18. Answer Flow Analysis (cont’d) • As before, we assume all predicates are tabled + subgoal abstraction • Theorem (Soundness of the answer flow analysis) If the complete trace of a query has an AFP then the query does not terminate. • Theorem (Completenessof the answer flow analysis) If the query evaluation does not terminate, then: • There is an AFP in its complete trace, • AFG = (N, E) contains at least one loop, • Every sub ∈ N appears in at least one loop, and • Each edge (sub1,sub2)∈E, where sub1=pred(..., ruleid), tells us that sub2 calls sub1 from the body of a rule whose id is ruleid.

19. More on UCG and AFG • Theorem (Relationship between UCG and AFG) Consider the UCG and AFG for a non-terminating forest log, we have: • nodes(AFG) ⊂ nodes(UCG) • edges(AFG) ⊂ reverse-edges(UCG) • loops(AFG) ⊆ loops(UCG) • Theorem (No false results for finite traces) If the evaluation of a query, Q, terminates, then both the UCG and the AFG for Q’s trace are empty.

20. Auto-Repair of Rules • A query does not terminate if • It has infinitely many answers, or • It has a finite number of answers, but one of its subqueries has an infinite number of them • In this case: a different evaluation order may terminate the query • This case is targeted by our auto-repair heuristic • For each unfinished(child,parent,timestamp) • We know • the host rule for this call, and • the common set of the unbound arguments of parent and child – the arguments whose bindings are to be derived • Thus, to reduce the possibility that parent receives infinite number of bindings from child, one can delay issuing a child-call from its host rule until these arguments are bound

21. Auto-Repair of Rules (cont’d) • Example @!r1 p(a). @!r5 r(X) :- r(X). @!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X). @!r3 q(b). @!r7 s(f(b)). @!r4 q(g(X)) :- p(X). ?- r(X). Its unfinished calls are: unfinished(r(_h9900,_h9908), root, 0) unfinished(r(_h9870,r5), r(_h9870,_h9889), 8) unfinished(r(_h9840,r5), r(_h9840,r5), 11) unfinished(p(_h9810,r6), r(_h9810,r5), 12) unfinished(q(_h9780,r2), p(_h9780,r6), 16) unfinished(p(_h9750,r4), q(_h9750,r2), 20) unfinished(q(_h9720,r2), p(_h9720,r4), 24) • Applying auto-repair @!r1 p(a). @!r5 r(X) :- wish(ground(X))^r(X). @!r2 p(f(X)) :- wish(ground(X))^q(X). @!r6 r(X) :- wish(ground(X))^p(X), s(X). @!r3 q(b). @!r7 s(f(b)). @!r4 q(g(X)) :- wish(ground(X))^p(X). ?- wish(ground(X))^r(X). • Then the query will terminate with X = f(b)

22. Tabled Engines without Subgoal Abstraction • Additional cause of non-termination: infinite number of subgoals • Steps • Compute the sequence of unfinished subgoals • Compute simplified subgoal sequence (SSS) out of unfinished subgoal sequence • Each unfinished subgoal, predicate(…, ruleid), is simplified to predicate(ruleid) • Find the SSS pattern, as in the case of answer flow pattern • SSS pattern contains the predicates and their rule ids that recursively call one another to form increasingly deep subgoals

23. Tabled Engines without Subgoal Abstraction (cont’d) • Example @!r1 p(a). @!r4 r(X) :- r(X). @!r2 p(X) :- q(f1(X)). @!r5 r(X) :- p(X), s(X). @!r3 q(X) :- p(f2(X)). @!r6 s(a). ?- r(a). Its unfinished calls are: unfinished(r(a,_h46), root, 0). unfinished(r(a,r4), r(a,_h27), 8). unfinished(r(a,r4), r(a,r4), 11). unfinished(p(a,r5), r(a,r4), 12). unfinished(q(f1(a),r2), p(a,r5), 16). unfinished(p(f2(f1(a)),r3), q(f1(a),r2), 19). unfinished(q(f1(f2(f1(a))),r2), p(f2(f1(a)),r3), 22). unfinished(p(f2(f1(f2(f1(a)))),r3), q(f1(f2(f1(a))),r2), 25). unfinished(q(f1(f2(f1(f2(f1(a))))),r2), p(f2(f1(f2(f1(a)))),r3), 28). unfinished(p(f2(f1(f2(f1(f2(f1(a)))))),r3), q(f1(f2(f1(f2(f1(a))))),r2), 31). unfinished(q(f1(f2(f1(f2(f1(f2(f1(a)))))),r2), p(f2(f1(f2(f1(f2(f1(a)))))),r3), 34). …… • SSS = [root, r(_), r(r4), r(r4), p(r5), q(r2), p(r3), q(r2), p(r3), q(r2), p(r3), q(r2)] • SSS pattern = [p(r3), q(r2)] – says that it is predicate p of r3 and predicate q of r2 that recursively call each other, thus forming increasingly deep nested subgoals

24. Status • Unfinished-call/answer flow implemented in SILK and Flora-2 • SILK has a GUI, Flora-2’s underway • Rule Ids are crucial for practicality • Auto-repair: not implemented yet

25. Experiments • System • Dual core 2.4GHz Lenovo X200 with 3GB RAM • Ubuntu 11.04 with Linux kernel 2.6.38 • Small programs • They took a tiny fraction of a second to analyze • Correctness of analysesis manually verified • Large programs: one biology ontology from SILK • KB size: • Flora-2 program with 4,774 rules and 919 facts • Compiled into XSB’s 5,500+ rules and 1,000+ facts • Logs produced until evaluation consumed all memory • Size: ~2GB • Number of records: ~14M • Took 170 seconds

26. Conclusions • Terminyzer – a tool for analyzing non-termination • Future work: • Implement auto-repair • better auto-repair algorithms • Comparison with others: • All other work deals with underpowered logic engines that are so last Century (Prolog) • Or with trying to find sufficient conditions for termination (different focus)

27. Thank you!

28. Forest Logging – Example :- table path/2. edge(1,2). edge(1,3). edge(2,1). path(X,Y) :- edge(X,Y). path(X,Y) :- edge(X,Z), path(Z,Y). ?- path(1,Y).