Theory of computing, part 1

1. Theory of computing, part 1

2. Course outline

3. Old computers

4. Abacus

5. Abacus

6. The difference engine (1832) Born December 26, 1791 in Teignmouth, Devonshire UK, Died 1871, London; Known to some as the Father of Computing for his contributions to the basic design of the computer through his Analytical machine..

7. ENIAC (1946) The ENIAC machine occupied a room 30x50 feet. The controls are at the left, and a small part of the output device is seen at the right.

8. ENIAC (1946) The IBM 360 was a revolutionary advance in computer system architecture, enabling a family of computers covering a wide range of price and performance.

9. Books

10. Books

11. Introduction

12. Models of computing • Finite state machines (automata) Pattern recognition Simple circuits (e.g. elevators, sliding doors) • Automata with stack memory (pushdown autom.) Parsing computer languages • Automata with limited tape memory • Automata with infinite tape memory Called `Turing machines’, Most powerful model possible Capable of solving anything that is solvable

13. Chomsky hierarchy of grammars • Regular grammars • Context free grammars • Context sensitive grammars • Unrestricted grammars

14. Computers can recognise languages Computers can be made to recognize, or accept, the strings of a language. There is a correspondence between the power of the computing model and the complexity of languages that it can recognize! • Finite automata only accept regular grammars. • Push down automata can also accept context free grammars. • Turing machines can accept all grammars.

15. Languages

16. Sets • A set is a collection of things called its elements. If x is an element of set S, we can write this: x  S • A set can be represented by naming all its elements, for example: S = {x, y, z} • There is no particular order or arrangement of the elements, and it doesn’t matter if some appear more than once. These are all the same set: {x, y, z}={y, x, z}={y, y, x, z, z}

17. Combining sets • A set with no elements is called the empty set, or a null set. It is denoted by  ={}. • If every element of set A is also an element of set B, then A is called a subsetof B : A  B • The union of two sets is the set with all elements which appear in either set: C = A  B • The intersection of two sets is the set with all the elements which appear in both sets: C = A  B

18. Alphabet and strings • A string a sequence of symbols that are placed next to each other in juxtaposition • The set of symbols which make up a string are taken from a finite set called an alphabet E.g.{a, b, c} is the alphabet for the string abbacb. • A string with no elements is called an empty string and is denoted . • If Σ is an alphabet, the infinite set of all strings made up from Σ is denoted Σ*. E.g., if Σ={a}, then Σ*={, a, aa, aaa, …}

19. Languages • A language is a set of strings. • If Σis an alphabet, then a language over Σ is a collection of strings whose components come from Σ. • So Σ* is the biggest possible language over Σ, and every other language over Σ is a subset of Σ*.

20. Examples of languages • Four simple examples of languages over an alphabet Σ are the sets , {}, Σ, and Σ*. • For example, if Σ={a} then these four simple languages over Σare , {}, {a}, and {, a, aa, aaa, …}. • Recall {} is the empty string while  is the empty set. Σ* is an infinite set.

21. Example: English • The alphabet isΣ = {a,b,c,d,e…x,y,z} • The English language is made of strings formed from Σ: e.g.fun, excitement. • We could define the English Language as the set of strings over Σ which appear in the Oxford English dictionary (but it is clearly not a unique definition).

22. Concatenation • The natural operation of concatenation of strings places two strings in juxtaposition. • For example, if then the concatenation of the two strings aab and ba is the string aabba. • Use the name catto denote this operation. cat(aab, ba) = aabba.

23. Combining languages • Languages are sets of strings, so they can be combined by the usual set operations of union, intersection, difference, and complement. • Also we can combine two languages L and M by forming the set of all concatenations of strings in L with strings in M.

24. Products of languages • This new language is called the product of L and M and is denoted by L  M. • A formal definition can be given as follows: L  M = {cat(s, t) | s  L and t  M} • For example, if L = {ab, ac} and M = {a, bc, abc}, then the product L•M is the language L  M = {aba, abbc, ababc, aca, acbc, acabc}

25. Properties of products • The following simple properties hold for any language L: L  {} = {} L = L L= L =  • The product is not commutative. In other words, we can find two languages L and M such that L  M  M  L • The product is associative. In other words, if L, M, and N are languages, then L  (M  N) = (L  M)  N

26. Powers of languages • If L is a language, then the product L  L is denoted by L2. • The language product Ln for every n{0, 1, 2, …} is as follows: L0 = {} Ln = L  Ln-1 if n > 0

27. Example For example, if L = {a, bb} then the first few powers of L are L0 = {} L1 = L = {a, bb} L2 = L  L = {aa, abb, bba, bbbb} L3 = L  L2 = {aaa, aabb, abba, abbbb, bbaa, bbabb, bbbba, bbbbbb}

28. Closure of a language • If L is a language over Σ (i.e.L Σ*) then the closure of L is the language denoted by L* and is defined as follows: L* = L0L1L2… • The positive closure of L is the language denoted by L+ and defined as follows:  L+ = L1L2L3 …

29. L* vs. L+ • It follows that L* = L+ {}. But it’s not necessarily true that L+= L* - {} • For example, if we let our alphabet be Σ = {a} and our language be L = {, a}, then L+= L* • Can you find a condition on a language L such that L+ = L* - {}?

30. Closure of an alphabet The closure of Σ coincides with our definition of Σ* as the set of all strings over Σ. In other words, we have a nice representation of Σ* as follows: Σ* = Σ0Σ1Σ2….

31. Properties of closure Let L and M be languages over the alphabet Σ. Then:  a) {}* = * = {} b)L* = L*  L* = (L*)* c)L if and only if L+ = L* d)(L*  M*)* = (L*  M*)* = (L  M)* e)L  (M  L)* = (L  M)*  L

32. Grammars

33. Grammars • A grammar is a set of rules used to define the structure of the strings in a language. • If L is a language over an alphabet Σ, then a grammar for L consists of a set of grammar rules of the following form:    • where  and  denote strings of symbols taken from Σ and from a set of grammar symbols (non-terminals) that is disjoint from Σ

34. Productions A grammar rule  is often called a production, and it can be read in any of several ways as follows: replace  by   produces   rewrites to   reduces to 

35. Where to begin …… • Every grammar has a special grammar symbol called a start symbol, and there must be at least one production with left side consisting of only the start symbol. • For example, if S is the start symbol for a grammar, then there must be at least one production of the form S

36. The 4 parts of a grammar • An alphabet N of grammar symbols called non-terminals. (Usually upper case letters.) 2. An alphabet T of symbols called terminals. (Identical to the alphabet of the resulting language.) 3. A specific non-terminal called the start symbol. (Usually S. ) 4. A finite set of productions of the form , where  and  are strings over the alphabet N  T

37. Example Let Σ = {a, b, c}. Then a grammar for the language Σ* can be described by the following four productions: S   S  aS S  bS S  cS Or in shorthand: S | aS | bS | cS S can be replaced by either , or aS, or bS, or cS.

38. Grammar specifies the language If G is a grammar, then the language of G is the set of language strings derived from the start symbol of G. The language of G is denoted by: L(G)

39. Grammar specifies the language If G is a grammar with start symbol S and set of language strings T, then the language of G is the following set: L(G) = {s | s  T* and S + s}

40. Finite languages If the language is finite, then a grammar can consist of all productions of the form S  w for each string w in the language. For example, the language {a, ab} can be described by the grammar S  a | ab.

41. Infinite languages • If the language is infinite, then some production or sequence of productions must be used repeatedly to construct the derivations. • Notice that there is no bound on the length of strings in an infinite language. • Therefore there is no bound on the number of derivation steps used to derive the strings. • If the grammar has n productions, then any derivation consisting of n + 1 steps must use some production twice

42. Infinite languages • For example, the infinite language {anb | n 0 } can be described by the grammar, S  b | aS • To derive the string anb, use the production S  aS • repeatedly --n times to be exact-- and then stop the derivation by using the production S  b • The production S  aS allows us to say If S derives w, then it also derives aw

43. Recursion • A production is called recursive if its left side occurs on its right side. • For example, the production S  aS is recursive. • A production S  is indirectly recursive if S derives (in two or more steps) a sentential form that contains S.

44. Indirect recursion • For example, suppose we have the following grammar: S  b | aA A  c | bS • The productions S  aA and A  bS are both indirectly recursive   S  aA  abS  A  bS  baA

45. Recursion • A grammar is recursive if it contains either a recursive production or an indirectly recursive production. • A grammar for an infinite language must be recursive! • However, a given language can have many grammars which could produce it.

46. Combining grammars • Suppose M and N are languages. We can describe them with grammars have disjoint sets of nonterminals. • Assign the start symbols for the grammars of M and N to be A and B, respectively: M : A  N : B  … … • Then we have the following rules for creating new languages and grammars:

47. Combining grammars, union rule The union of the two languages, M  N, starts with the two productions S  A | B followed by the grammars of M and N A  … B  …

48. Combining grammars, product rule Similarly, the language M  N starts with the production S  AB followed by, as above, A  … B  …

49. Combining grammars, closure rule Finally, the grammar for the closure of a language, M*, starts with the production S  AS |  followed by the grammar of M A  …

50. Example, union Suppose we want to write a grammar for the following language:  L = {, a, b, aa, bb, ..., an, bn, ..} L is the union of the two languages M = { an| n N} N = { bn| nN}