Cellular Automata Generalized To An Inferential System

2. Cellular Automata Generalized To An Inferential System 27th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Saratoga Springs, New York, 11 July 2007 David J. Blower Cogon Systems Pensacola FL

3. Motivation Why is it impossible to predict the behaviour of a cellular automaton?

4. The Motivating Question But isn’t the quintessential feature of probability theory and inferential systems the ability to predict future events?

5. Proposed Solution Jaynes used probability theory to generalize classical logic functions. Treat CA from an inferential and informational point of view.

6. Why Examine Cellular Automata? Because Cellular Automata are a stand-in for any sufficiently detailed complicated ontological explanation for how the world works.

7. Issues in Order Addressed Boolean Algebra Logic Functions Cellular Automata

8. Why Start with Boolean Algebra? The following few slides on Boolean Algebra are solely to set the stage for analogous operations with classical logic functions (and cellular automata).

9. Some Basic Boolean Questions • How are Boolean functions defined? • How are syntactically correct Boolean formulas produced? • What is a good canonical expression for Boolean functions?

10. Why Is It Helpful? • Useful for both logic functions and cellular automata • Axioms of Boolean Algebra used in Bayes’s Theorem • Canonical expressions substituted for complicated logic and CA rules

11. Formal Rules, Boolean Algebra, and Probability Theory Boolean Algebra on a finite carrier set is a “closed” system, that is, there is always an answer. Moreover, neither numbers nor arithmetic operations are required to find that answer. Perfect choice for discussing formal manipulation rules of probability theory where the actual numerical assignments are not the issue.

12. Boolean Algebra The carrier set Special Elements Characterized by the quintuple Binary operators Function definition: A mapping from the set of ordered pairs of the carrier to an element in the carrier set.

13. Boolean Algebra An example of a carrier set with four elements All 16 ordered pairs from the carrier set A mapping from an element of B x B into an element of B

14. Example of Boolean Function Function Table Boolean Formula Substitute specific arguments

15. Boole’s Expansion Theorem Any Boolean function can be expanded in the following manner. Applying this theorem in a recursive manner yields the disjunctive normal form (DNF).

16. Disjunctive Normal Form For example, here is the expansion of any Boolean functionf(x,y)withn = 2arguments. These are called the discriminants.

17. Disjunctive Normal Form Repeating generic expansion from previous slide Calculate discriminants and then substitute

18. Issues in Order Addressed Boolean Algebra Logic Functions Cellular Automata

19. Logic Functions A special case of Boolean Algebra

20. Different Notation Functions with two arguments written in generic Boolean Algebra notation and then in Classical Logic notation.

21. Boole’s Expansion Theorem Using Boole’s expansion theorem, the DNF for any logic function now looks like,

22. The DNF for Logic Functions Since the discriminants always take a functional assignment of either TRUE or FALSE, a typical expansion might look something like this, The logic function that returns B when arguments A and B given. f10 (A,B) on next slide.

23. All 16 Logic Functions

24. Classical Syllogism Modus Ponens A is TRUE A implies B Therefore, B is TRUE

25. Classical Syllogism Recapitulate Jaynes’s demonstration generalizing classical logic with probability theory (but here I emphasize the Boolean Algebra aspects).* * Chapter 2, pp. 35--36

26. Generalizing Classical Logic Bayes’s Theorem Substitute DNF expansion of Implication function A  B Boolean operations reduce above to

27. Modus Ponens Substitute the shortened DNF expansion for the implication function in Bayes’s Theorem Bayes’s Theorem now looks like this

28. Modus Ponens Boolean operations on denominator Boolean operations on numerator Bayes’s Theorem

29. Different Approach Now solve modus ponens using a joint probability table. The answer should be the same.

30. Joint Probability Table Two statements A and B each take on only two values. There are four cells in the joint probability table. The model assigning these numerical values is the implication function AB.

31. Joint Probability Table Cell 1 Cell 1 Cell 3 The same answer as before. The model Mk assigns legitimate numerical values to the joint statements in the four cells of the jpt. The model is the implication function.

32. Probability Theory Generalizes Classical Logic Here is an “invalid’’ logical argument, but one that is easily solved using probability theory in exactly the same manner as before. Assume the same logic function, but now B is TRUE.What is the impact on A?

33. The “Invalid” Logical Argument Solved by Probability Theory Use the numerical assignments from the joint probability table. Probability theory as a generalization of logic returns an answer, while classical logic refuses to address the issue calling it “undecidable.”

34. Placement of 0s in JPT Cell 3 indexes joint statement: A is TRUE and B is FALSE. f13 (A, B) has functional assignment of FALSE if A is TRUE and B is FALSE by very definition of  operator. Therefore, cell 3 MUST HAVE a numerical assignment of 0 under this model.

35. Issues in Order Addressed Boolean Algebra Logic Functions Cellular Automata

36. Elementary One-Dimensional Cellular Automata Wolfram’s famous example of a CA proven to be a Universal Turing Machine

37. Elementary One-Dimensional Cellular Automata First few steps of a CA following Rule 110

38. Many Steps of CA Following Rule 110 Interacting Localized Structures. They compute anything that can be computed!! Our stand-in for a complicated detailed ontological model of the world.

39. Logic Functions with Three Variables But Rule 110 is simply a logic function with three arguments instead of the two arguments as we have examined previously in discussing classical logic. There are a total of 256 possible logic functions with three variables and Rule 110 is one of those 256 functions. Here is Rule 110’s function table for all eight possible settings of the three variables. Black  T White  F

40. The DNF for Rule 110 And, like any logic function, Rule 110 can be expanded via the DNF. The DNF expansion of a three variable logic function, Rule 110.

41. The DNF and 0s in the JPT And, just as we did when examining modus ponens, we can use the DNF expansion of a logic function to locate the 0s in a joint probability table. We will employ the jpt as an easier alternative to the formal Boolean operations in solving Bayes’s Theorem applied to CA.

42. Joint Probability Table with Numerical Assignments Following Rule 110 Placement of 0s dictated by model following logic function f110 (A,B,C)

43. Joint Probability Table with Numerical Assignments Following Rule 110 BN+1 cannot be TRUE(black) if AN, BN and CN are also TRUE(black)

44. Joint Probability Table with Numerical Assignments Following Rule 110 The DNF expansion of Rule 110 also tells us where the non-0s must be placed. If functional assignment BN+1 is TRUE at these five terms, non-zero probability is assigned. Or, 0s are placed where functional assignment isFALSE.

45. Bayes’s Theorem for Rule 110 Write out the generic template for updating a state of knowledge (Bayes’s Theorem, of course) about the cell to be updated given the colors of three cells at the previous time step and the numerical assignment following Rule 110.

46. Updating Color of Cell in CA Using Bayes’s Theorem Bayes’s Theorem with denominator expanded. Numerical assignment follows from model implementing Rule 110. Locate and insert values from jpt. Insert the numerical assignment from Cell 12 in the numerator and the numerical assignment from Cells 12 and 4 in the denominator.

47. What is the Point? Jaynes generalized logic functions (Boolean functions) by treating them from a probabilistic and inferential standpoint. CA are composed of Boolean functions, so think of them not from the deductive viewpoint, but rather from an informational standpoint. For example, let a wide range of models insert legitimate numerical values into the cells of a joint probability table for a CA. Suppose there is a lack of information about which model correctly governs the evolution of the CA. Then, as a consequence ensuing for all informational systems, prediction at least becomes a feasible concept to explore.

48. Different Models Assign Different Numerical Values “Almost” the  model.

49. How Do We Predict Future Events in an Inferential System? We use the same formal manipulation rules we always use in probability theory. For example, to update a state of knowledge about some future event FE conditioned on some observed data Dand involving models assigning numerical values Mk

50. Predicting Future Events in CA The inferential approach provides a quantitative way for the information processor to update its state of knowledge about the color of any cell in the cellular automaton. Here, we have lost information about which model (rule) is governing the evolution of the CA. We must average over the predictions made by 256 models