Formal Languages, Automata and Models of Computation CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 1 - Intro Mälardalen University 2007

Content Adminstrivia Mathematical Preliminaries Countable Sets (Uppräkneliga mängder) Uncountable sets (Överuppräkneliga mängder)

Lecturer & Examiner Gordana Dodig-Crnkovic

Visit home page regularly! Course Home Page http://www.idt.mdh.se/kurser/cd5560/07_04 Visit home page regularly!

20 hours a week for this type of course (norm) How Much Work? 20 hours a week for this type of course (norm) 4 hours lectures 2 hours exercises 14 hours own work a week!

Why Theory of Computation? A real computer can be modelled by a mathematical object: a theoretical computer. Robustness of a computational model. A formal language is a set of strings, and can be used for representation of a computational problem. Simulation: the relative power of computing models can be measured by the ease with which one model can simulate another. 4. The Church-Turing thesis: anything that can be computed can be computed by a Turing machine. 7. Solvability/Decidability: for some computational problems there is no corresponding algorithm that will solve them.

Practical Applications Efficient compilation of computer languages String searching Identifying the limits of computational method; recognizing difficult problems Applications to other areas: circuit verification economics and game theory (finite automata as strategy models in decision-making); theoretical biology (L-systems as models of organism growth) computer graphics (L-systems) linguistics (modeling by grammars)

Reasoning - Logics and Mathematics Logics and Mathematics will generally accept only those results for which there are demonstrative proofs. Codifying the rules of proof is therefore an important part of the foundations for computing. If you change the rules of reasoning you change the mathematical (computational) results you can derive.

Ontology Prior to providing basic concepts such as that of a function, set theory provides a complex universe of sets (many of which are then used as representatives of functions). Changes to the ontology will affect the mathematics which is subsequently derived. In set theory, logic and ontology appear to be separated, but in other systems this isn't so clear.

Absoluteness and Universality Logic is often said to be general and topic neutral. The first attempts, by Frege and Russell, to provide logical foundations to mathematics were based on the idea that the whole of mathematics could be developed from a neutral logic by defining mathematical concepts and then deriving mathematical theories. Since then many doubts have been cast upon this vision. There remain philosophical, technical and pragmatic reasons for the disbelief that underlying the syntactic confusion there can be found an absolute logical truth and absolute foundation system which can provide a basis for the development of any mathematical subject domain.

History Euclid's attempt to axiomatize geometry (Archimedes realized, during his own efforts to define the area of a planar figure, that Euclid's attempt had failed and that additional postulates were needed. ) Leibniz's dream of a symbolic logic de Morgan, Boole, Frege, Russell, Whitehead: Mathematics as branch of symbolic logic!

History 1900 Hilberts program 1880 -1936 first programming languages 1931 Gödels incompleteness theorem 1936 Turing maschine (showed to be equivalent with recursive functions). Commonly accepted: TM as ultimate computer 1950 automata 1956 language/automata hierarchy

Every mathematical truth expressed in a formal language is consisting of a fixed alphabet of admissible symbols, and explicit rules of syntax for combining those symbols into meaningful words and sentences

Turing used a Universal Turing machine (UTM) to prove an even more powerful incompleteness theorem because it destroyed not one but two of Hilbert's dreams: Finding a finite list of axioms from which all mathematical truths can be deduced Solving the entscheidungsproblem, ("decision problem“) by producing a "fully automatic procedure" for deciding whether a given proposition (sentence) is true or false.

Mathematical Preliminaries Sets Functions Relations Proof Techniques Languages, Alphabets and Strings Strings & String Operations Languages & Language Operations

SETS A set is a collection of elements We write

Set Representations C = { a, b, c, d, e, f, g, h, i, j, k } C = { a, b, …, k } S = { 2, 4, 6, … } (finite set) S = { j : j > 0, and j = 2k for some k>0 } (infinite set) S = { j : j is nonnegative and even }

A = { 1, 2, 3, 4, 5 } Universal Set: All possible elements U = { 1 , … , 10 } 1 2 3 4 5 A U 6 7 8 9 10

Set Operations A = { 1, 2, 3 } B = { 2, 3, 4, 5} Union A U B = { 1, 2, 3, 4, 5 } Intersection A B = { 2, 3 } Difference A - B = { 1 } B - A = { 4, 5 } A B U A-B

A A Complement Universal set = {1, …, 7} 4 A A 6 3 1 2 5 7 A = A

{ even integers } = { odd integers } 2 4 6 1 3 5 7 even odd Integers

DeMorgan’s Laws A U B = A B U A B = A U B

Empty, Null Set: = { } S U = S S = S - = S - S = U = Universal Set

Subset A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 } A B U Proper Subset: A B

Disjoint Sets A = { 1, 2, 3 } B = { 5, 6} A B = U A B

Set Cardinality For finite sets A = { 2, 5, 7 } |A| = 3

Powersets A powerset is a set of sets S = { a, b, c } Powerset of S = the set of all the subsets of S S = { a, b, c } 2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} } Observation: | 2S | = 2|S| ( 8 = 23 )

Generalizes to more than two sets Cartesian Product A = { 2, 4 } B = { 2, 3, 5 } A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) } |A X B| = |A| |B| Generalizes to more than two sets A X B X … X Z

PROOF TECHNIQUES Proof by construction Proof by induction Proof by contradiction

Construction We define a graph to be k-regular if every node in the graph has degree k. Theorem. For each even number n > 2 there exists 3-regular graph with n nodes. 2 3 2 1 1 4 5 3 n = 6 n = 4

Construct a graph G = (V, E) with n > 2 nodes. V= { 0, 1, …, n-1 } E = { {i, i+1}  for 0  i  n-2}  {{n-1,0}} (*)  {{i, i+n/2  for 0  i  n/2 –1} (**) The nodes of this graph can be written consecutively around the circle. (*) edges between adjacent pairs of nodes (**) edges between nodes on opposite sides Proof by Construction END OF PROOF

Induction We have statements P1, P2, P3, … If we know for some k that P1, P2, …, Pk are true for any n  k that P1, P2, …, Pn imply Pn+1 Then Every Pi is true

Proof by Induction Inductive basis Find P1, P2, …, Pk which are true Inductive hypothesis Let’s assume P1, P2, …, Pn are true, for any n  k Inductive step Show that Pn+1 is true

Example Theorem A binary tree of height n has at most 2n leaves. Proof let L(i) be the number of leaves at level i L(0) = 1 L(3) = 8

We want to show: L(i)  2i Inductive basis L(0) = 1 (the root node) Inductive hypothesis Let’s assume L(i)  2i for all i = 0, 1, …, n Induction step we need to show that L(n + 1)  2n+1

Induction Step hypothesis: L(n)  2n Level n n+1

Induction Step Level n hypothesis: L(n)  2n n+1 L(n+1)  2 * L(n)  2 * 2n = 2n+1 Induction Step END OF PROOF

Inductionsbevis: Potensmängdens kardinalitet Påstående: En mängd med n element har 2n delmängder Kontroll Tomma mängden {} (med noll element) har bara en delmängd: {}. Mängden {a} (med ett element) har två delmängder: {} och {a}

Påstående: En mängd med n element har 2n delmängder Kontroll (forts.) Mängden {a, b} (med två element) har fyra delmängder: {}, {a}, {b} och {a,b} Mängden {a, b, c} (med tre element) har åtta delmängder: {}, {a}, {b}, {c} och {a,b}, {a,c}, {b,c}, {a,b,c} Påstående stämmer så här långt.

Bassteg Enklaste fallet är en mängd med noll element (det finns bara en sådan), som har 20 = 1 delmängder.

Induktionssteg Antag att påståendet gäller för alla mängder med k element, dvs antag att varje mängd med k element har 2k delmängder. Visa att påståendet i så fall också gäller för alla mängder med k+1 element, dvs visa att varje mängd med k+1 element har 2k+1 delmängder.

Vi betraktar en godtycklig mängd med k+1 element Vi betraktar en godtycklig mängd med k+1 element. Delmängderna till mängden kan delas upp i två sorter: Delmängder som inte innehåller element nr k+1: En sådan delmängd är en delmängd till mängden med de k första elementen, och delmängder till en mängd med k element finns det (enligt antagandet) 2k stycken.

Delmängder som innehåller element nr k+1: En sådan delmängd kan man skapa genom att ta en delmängd som inte innehåller element nr k+1 och lägga till detta element. Eftersom det finns 2k delmängder utan element nr k+1 kan man även skapa 2k delmängder med detta element. Totalt har man 2k + 2k = 2. 2k= 2k+1 delmängder till den betraktade mängden. END OF PROOF (Exempel från boken: Diskret matematik och diskreta modeller, K Eriksson, H. Gavel)

Proof by Contradiction We want to prove that a statement P is true we assume that P is false then we arrive at a conclusion that contradicts our assumptions therefore, statement P must be true

Example Theorem is not rational Proof Assume by contradiction that it is rational = n/m n and m have no common factors We will show that this is impossible

Thus, m and n have common factor 2 Contradiction! = n/m 2 m2 = n2 Therefore, n2 is even n is even n = 2 k 2 m2 = 4k2 m2 = 2k2 m is even m = 2 p Thus, m and n have common factor 2 Contradiction! = n/m 2 m2 = n2 END OF PROOF

Infinite sets are either Countable or Uncountable

Countable Sets Definition A set is countable if there is a one to one correspondence between elements of the set and natural numbers.

We started with the natural numbers, then add infinitely many negative whole numbers to get the integers, then add infinitely many rational fractions to get the rationals, then added infinitely many irrational fractions to get the reals. Each infinite addition seem to increase cardinality: |N| < |Z| < |Q| < |R| But is this true? NO!

Example The set of integers is countable Integers: Correspondence: Natural numbers:

Example The set of rational numbers is countable Positive Rational numbers:

Naive Idea Rational numbers: Correspondence: Natural numbers: Doesn’t work! we will never count numbers with nominator 2:

Better Approach ... ... ... ... Rows: constant numerator (täljare) Columns: constant denominator ...

... ... ... ...

We proved: the set of rational numbers is countable by describing an enumeration procedure (a constructive proof)

Definition An enumeration procedure for is an algorithm that generates all strings of one by one Let be a set of strings

Observation A set is countable if there is an enumeration procedure for it

Example The set of all finite strings is countable Proof We will describe the enumeration procedure

Naive procedure: Produce the strings in lexicographic order: Doesn’t work! Strings starting with will never be produced

Better procedure: Proper Order 1. Produce all strings of length 1 2. Produce all strings of length 2 3. Produce all strings of length 3 4. Produce all strings of length 4 ..........

length 1 Produce strings in Proper Order length 2 length 3

Theorem The set of all finite strings is countable Proof Find an enumeration procedure for the set of finite strings Any finite string can be encoded with a binary string of 0’s and 1’s

Produce strings in Proper Order length 2 length 3 length 1 1 00 01 10 11 000 001 …. 2 3 4 5 6 7 String = program Natural number

PROGRAM = STRING (syntactic way) PROGRAM = FUNCTION  (semantic way) PROGRAM string PROGRAM natural number n

A set is uncountable if it is not countable. Uncountable Sets Definition A set is uncountable if it is not countable.

Theorem The set of all infinite strings is uncountable. Proof (by contradiction) We assume we have an enumeration procedure for the set of infinite strings.

Cantor’s Diagonal Argument Encoding Infinite string = ... = = ... ... ... ... ...

Cantor’s Diagonal Argument We can construct a string that is missing in our enumeration! Conclusion The set of all infinite strings is uncountable!

An infinite string can be seen as FUNCTION  (n:th output is n:th bit in the string) Conclusion There are some integer functions that cannot be described by programs/algorithms (finite strings ).

Example of uncountable infinite sets: Theorem Let be an infinite countable set. The powerset of is uncountable.

Proof Since is countable, we can write

Elements of the powerset have the form: ……

We encode each element of the power set with a binary string of 0’s and 1’s Encoding Powerset element ... ... ... ...

Let’s assume (by contradiction) that the powerset is countable. Then: we can enumerate the elements of the powerset.

Powerset element Encoding ... ... ... ... ...

Take the powerset element whose bits are the complements in the diagonal.

... ... ... ... New element: (binary complement of diagonal)

The new element must be some of the powerset However, that’s impossible: from definition of the i-th bit of must be the complement of itself Contradiction!

Since we have a contradiction: The powerset of is uncountable END OF PROOF

uncountable infinite An Application: Languages Example Alphabet : The set of all finite strings: infinite and countable The powerset of contains all languages: uncountable infinite

Finite strings (algorithms): countable Languages (power set of strings): uncountable There are infinitely many more languages than finite strings.

Conclusion There are some languages that cannot be described by finite strings (algorithms).

Kardinaltal Kardinaltal är mått på storleken av mängder. Kardinaltalet för en ändlig mängd är helt enkelt antalet element i mängden. Två mängder är lika mäktiga om man kan para ihop elementen i den ena mängden med elementen i den andra på ett uttömmande sätt, dvs det finns en bijektion mellan dem. Detta mäktighetstänkande kan utvidgas till oändliga mängder. Till exempel är mängden av positiva heltal och mängden av heltal lika mäktiga.

Kardinaltal Däremot kan man inte para ihop alla reella tal med heltalen på detta sätt. Mängden av reella tal har större mäktighet än mängden av heltal. Man kan införa kardinaltal på ett sådant sätt att två mängder har samma kardinaltal om och endast om de har samma mäktighet. T ex kallas kardinaltalet som hör till de hela talen för 0 (alef 0, alef är den första bokstaven i det hebreiska alfabetet). Dessa oändliga kardinaltal kallas transfinita kardinaltal.

Mer om oändligheter… Georg Cantor utvecklade i slutet av 1800-talet matematikens logiska grund, mängdläran. Cantor införde begreppet transfinita kardinaltal. Den enklaste, "minsta", oändligheten kallade han 0. Det är den uppräkningsbara oändliga mängdens (exempelvis mängden av alla heltal) kardinaltalet. Kardinaltalet av mängden punkter på en linje, och även punkterna på ett plan och i en kropp, kallade Cantor 1. Fanns det större oändligheter?

Mer om oändligheter… Ja! Cantor kunde visa att antalet funktioner på en linje var ”ännu oändligare” än punkterna på linjen, och han kallade den mängden 2. Cantor fann att det gick att räkna med kardinaltalen precis som med vanliga tal, men räknereglerna blev något enahanda.. 0 + 1= 0 0 + 0 = 0 0 · 0 = 0.

Mer om oändligheter… Men vid exponering hände det något: 0 0 (0 upphöjt till 0) = 1. Mer generellt visade det sig att 2 n (2 upphöjt till n) = n+1 Det innebar att det fanns oändligt många oändligheter, den ena mäktigare än den andra!

Men var det verkligen säkert att det inte fanns någon oändlighet mellan den uppräkningsbara och punkterna på linjen? Cantor försökte bevisa den så kallade kontinuumhypotesen. Cantor: two different infinities 0 and 1 http://www.ii.com/math/ch/#cardinals Continuum Hypothesis: 0 < 1 = 2 0 Se även: http://www.nyteknik.se/pub/ipsart.asp?art_id=26484