# Subject description - A0B01PSI

Summary of Study |
Summary of Branches |
All Subject Groups |
All Subjects |
List of Roles |
Explanatory Notes
Instructions

Basic types of distributions.

Interval estimates of mean and variance.

A0B01PSI | Probability, Statistics, and Theory of Information | ||
---|---|---|---|

Roles: | P, V | Extent of teaching: | 4+2 |

Department: | 13101 | Language of teaching: | CS |

Guarantors: | Completion: | Z,ZK | |

Lecturers: | Credits: | 6 | |

Tutors: | Semester: | Z |

**Anotation:**

**Study targets:**

**Course outlines:**

1. | Basic notions of probability theory. Kolmogorov model of probability. Independence, conditional probability, Bayes formula. | |

2. | Random variables and their description. Random vector. Probability distribution function. | |

3. | Quantile function. Mixture of random variables. | |

4. | Characteristics of random variables and their properties. Operations with random variables. |

5. | Characteristics of random vectors. Covariance, correlation. Chebyshev inequality. Law of large numbers. Central limit theorem. | |

6. | Basic notions of statistics. Sample mean, sample variance. |

7. | Method of moments, method of maximum likelihood. EM algorithm. | |

8. | Hypotheses testing. Goodness-of-fit tests, tests of correlation, non-parametic tests. | |

9. | Discrete random processes. Stationary processes. Markov chains. | |

10. | Classification of states of Markov chains. | |

11. | Asymptotic properties of Markov chains. Overview of applications. | |

12. | Shannon entropy. Entropy rate of a stationary information source. | |

13. | Fundamentals of coding. Kraft inequality. Huffman coding. | |

14. | Mutual information, capacity of an information channel. |

**Exercises outline:**

1. | Elementary probability. | |

2. | Kolmogorov model of probability. Independence, conditional probability, Bayes formula. | |

3. | Mixture of random variables. Mean. Unary operations with random variables. | |

4. | Dispersion (variance). Random vector, joint distribution. Binary operations with random variables. | |

5. | Sample mean, sample variance. Chebyshev inequality. Central limit theorem. | |

6. | Interval estimates of mean and variance. | |

7. | Method of moments, method of maximum likelihood. | |

8. | Hypotheses testing. Goodness-of-fit tests, tests of correlation, non-parametic tests. | |

9. | Discrete random processes. Stationary processes. Markov chains. | |

10. | Classification of states of Markov chains. | |

11. | Asymptotic properties of Markov chains. | |

12. | Shannon entropy. Entropy rate of a stationary information source. | |

13. | Fundamentals of coding. Kraft inequality. Huffman coding. | |

14. | Mutual information, capacity of an information channel. |

**Literature:**

[1] | Papoulis, A.: Probability and Statistics, Prentice-Hall, 1990. | |

[2] | Stewart W.J.: Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling. Princeton University Press 2009. | |

[3] | David J.C. MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003. |

**Requirements:**

**Note:**

A necessary condition for the assignment is active participation at seminars, successful test, and one homework. More info: http://cmp.felk.cvut.cz/~navara/psi/ |

**Webpage:**

**Keywords:**

**Subject is included into these academic programs:**

Page updated 14.6.2021 19:52:31, semester: L/2021-2, L/2020-1, Z,L/2022-3, Z/2021-2, Send comments about the content to the Administrators of the Academic Programs | Proposal and Realization: I. Halaška (K336), J. Novák (K336) |