Telecom ParisTech
Dep. Informatique & Réseaux

J-L. Dessalles Home page

April 2019


From Complexity to Intelligence
Lecturers:
    Jean-Louis Dessalles
    Pierre-Alexandre Murena

                                other AI courses



Objectives

The mathematical notion of complexity has been invented 50 years ago to solve issues related to machine learning, randomness and proof theory. Complexity corresponds to the size of algorithms (and not to their speed; see caveat below). Complex objects cannot be described by short algorithms. The notion led to the development of Algorithmic Information Theory (AIT). Complexity and AIT have more recently been shown essential to address aspects of human intelligence, such as perception, relevance, decision making and emotional intensity. These aspects of cognition were sometimes considered mysterious and unpredictable. They can be regarded now as resulting in part from computations based on complexity and its converse, simplicity. For instance, abnormally simple situations such as a coincidence (two colleagues having dressed in purple independently) or a remarkable lottery draw (e.g. 1-2-3-4-5-6) are systematically perceived as unexpected and interesting. The design of intelligent systems must take advantage of this sensitivity of the human mind to complexity and to simplicity.

Caveats:

Content

This course begins with an introduction to the mathematical notion of complexity (also known as Kolmogorov complexity). The notion will be shown to be useful for the study of reasoning, for the definition of relevance (interestingness, unexpectedness), and for machine learning. We will also explore applications to the study of perception (hidden shapes, pattern recognition), of decision making (subjective probability), of responsibility and of emotional intensity. All these aspects will be studied using concrete examples. Half of the time will be devoted to personal work in lab sessions.

Topics

Monday:     1. Description complexity
Tuesday:     2. Information as compression
Wednesday:     3. Algorithmic information applied to mathematics
4. Machine learning as compression
Thursday:     5. Subjective information

Slides


Read
    JL. Dessalles - Complexity    

Read
    PA. Murena - Compression    

Read
    PA. Murena - Complexity & maths    

Read
    PA. Murena - Randomness    

Read
    PA. Murena - Analogy    

Read
    PA. Murena - Leaning    

Read
    PA. Murena - Learning (AIXI)    

Read
    PA. Murena - Complexity & cognition    

Validation

Students will also be asked to make a small original contribution and to present it orally.
They will also have to answer a short quiz on the last day.
Answers to the 2019 quiz.

You are expected to choose a topic of study, and to do something for this project (typically write a small program). The topic of the project must be related to complexity.

Please indicate your choice using the link above, not later than Thursday 15:00pm of the Athens week.

On Thursday night of the Athens week, we expect:

DON’T SEND ANYTHING THROUGH E-MAIL Use the upload link.

Before the end of the Athens week, you’ll upload a small text describing your project and what you found (typically: two pages, more if there are images) This text should present:

So don’t forget to upload: We hope you’ll enjoy doing this. We also hope you’ll enjoy others’ presentations.

Short bibliography

In French: