INFORMATION THEORY
for Mathematics (MAT.460UF + MAT.461UF + MAT.409UF)
and Computer Science (503.737 + 503.738)

SS 2023 Mathematics: LV-Nr. MAT.460UF (3 VO) LV-Nr. MAT.461UF (1 UE) LV-Nr. MAT.409UF (1 KV) Computer Science: LV-Nr. 503.737 (3 VO) LV-Nr. 503.738 (1 UE) Contents News Lectures Contents of the course Exams Exercises Groups and dates Grading of the exercises Guidelines for uploading Weekly exercises Online checking system

Lecturer: Joshua Erde E-mail: erde@math.tugraz.at Office: Steyrergasse 30/II

News

Please read modalities.pdf carefully.

As of February 12, 2024, this page is provisionial and may be subject to small modifications!

Lectures

Contents of the course

The lectures will take place Mondays from 12:15 to 14:00 and Tuesdays from 14:15 to 15:45 in BE01 (Steyrergasse 30, ground floor). For details see TUGonline.

• Basic probability theory (reminders for Comp. Sci.)
• Entropy, conditional entropy, Kullback-Leibler divergence, related properties and formulas
• Entropy rate, rate of stationary processes and Markov chains
• Asymptotic equipartition property, AEP for iid random variables and ergodic Markov chains, typical sets and data compression
• Codes, prefix codes and entropy, proof of the optimality of Huffman codes
• Noisy channels, channel capacity, Shannon's channel coding theorem
• Entropy of continuous random variables, Shearer's inequality, Combinatorial applications (Math only)

These are partial skeleton notes for the course, which will be updated as the course progresses.

Exams

The exam of the lecture (VO) will be held oral. Dates for the exam should be fixed with Joshua Erde by sending an email to erde@math.tugraz.at.

Exercises

The exercises will take place Thursdays between 14:15 to 15:15 in AE06 (Steyrergasse 30, ground floor). For details see TUGonline. There will be 11 sessions, each of 75 minutes.

Registration for the exercises via TUG online: links for Mathematics and Computer Science.

In case of de-registration: observe deadlines!

Grading of the exercises

Any solved exercise should be marked in the Online checking system. Please do note that the grade listed on the checking system will likely be incorrect, and the actual grade is calculated as listed below.

For each exercise, one of those participants who have marked it is called to share and explain the solution at the blackboard. For this, up to one "bonus point" is awarded. If it becomes apparent that the exercise was not carefully prepared or understood, there may be point deductions.

The following table gives an overview:

Note that the above rules imply that attendance is mandatory, if exercises were marked in the online system. Only in exceptional cases of illness, participants - after notifying the exercise tutor by email before the exercise class - may send their solution by mail and mark the corresponding exercises. In this case, a medical certificate has to be delivered, and the careful presentation of some of the marked exercises at a later date will be required.

The success quotient is given by:
(number of marked exercises + achieved blackboard points)/ total number of exercises

Weekly exercises

Below you will find links to the exercises to be prepared for the respective class. Each week an exercise sheet (pdf) is made available online. The solved exercises have to be marked at the latest in the morning before the exercise classes by Thursday 10:00 a.m. in the Online checking system. Participaton is required, later marking is NOT possible!

Online checking system

Link to online checking system

Deadline: Thursday, 10:00 a.m. (before each session)

At the first login, a password has to be required, which is sent to the email address resulting from TUGonline. The solved exercises of the current week have to be marked online until the morning on the day of the respective exercise class (Thursday) before 10:00 a.m.