INFORMATION THEORY

for Mathematics (MAT.460UF + MAT.4461UF + MAT.409UF)

and Computer Science (503.737 + 503.738)

SS 2018

Course:
Wolfgang Woess
E-mail:
woess "at" tugraz.at
Office:
Steyrergasse 30/III, Room ST03198
Organisation of the exercise classes:
Adrian Fuchs, Gundelinde Wiegel
E-mail:
adrian.fuchs "at" student.tugraz.at,
wiegel "at" math.tugraz.at
Office:
Steyrergasse 30/III, Rooms ST03210 and ST03240


As of February 28, 2018, this page is provisorial and may be subject to small modifications !

TUGonline links for Mathematics: VO and UE and KV
Mathematic Students: please register also for the KV !

TUGonline links for Computer Science: VO and UE

Contents of the course

Entropy, conditional entropy, Kullback-Leibler divergence, related properties and formulas

Entropy rate, rate of stationary processes and Markov chains

Asymptotic equipartition property, AEP for iid random variables and ergodic Markov chains, typical sets and data compression

Codes, prefix codes and entropy, proof of the optimality of Huffman codes

Noisy channels, channel capacity, Shannon's channel coding theorem

Entropy of continuous random variables and related issues (Math only)

The exam for the course is oral; date to be fixed in contact with W. Woess. Course language: English, exam language: English or German according to preference of candidate.

Modalities of the course (IMPORTANT)

Please read modalities.pdf carefully.

Exercise classes

Registration for the exercises via TUG online: links for Mathematics and Computer Science.
In case of de-registration: observe deadlines!

Weekly exercises

Below you find links to the exercises to be prepared for the respective class. Every week an exercise sheet (pdf) is made available online. The solved current exercises have to be marked at the latest on the evening before the exercise classes by Tuesday 11 pm. in the online checking system. Participaton is required, later marking is NOT possible!

Day Exercises
14.03.2018 Exercise sheet 1
21.03.2018 Exercise sheet 2
11.04.2018 Exercise sheet 3
18.04.2018 Exercise sheet 4
25.04.2018 Exercise sheet 5
Some solutions
Day Exercises

Grading of the exercises

The Online checking system is applied. For each exercise, one of those participants who have marked it are called to explain the solution at the blackboard. For this, "blackboard points" are awarded, maximum number of points according to the exercise sheet. If it becomes apparent that the exercise was not prepared, there are point detractions.

The following table gives a overview:

blackboard presentation blackboard points
extraordinary or very good presentation of the solution all points of the presented example
correct solution between 0 and the maximal number of possible points
person is not able to explain the example or makes mistakes minus points: up to the number of made crosses of the current week
absence of the called person (first time) minus doubled number of made crosses of the current week
absence of the called person (second time) negative evaluation
Only in exceptional cases of illness, participants - after notification of the group leader by email before the exercise class - can submit written versions of the solved exercises and set the corresponding crosses within a week after the exercise class during the office hours of G. Wiegel, resp. A. Fuchs. In this case, a medical certificate has to be delivered, and the careful presentation of some of the marked exercises at the office blackboard will be required.

The exercise modalities do not support mass desertion caused by other exams or similar causes coming from other courses or seminars. In this case, participants are asked to inform the person responsible for the other event about the problem and to ask her or him to adapt the schedule accordingly. In any case, the available points are sufficient for obtaining even the best mark while missing an exercise class.

The success quotient is given by:
100 * (number of made ticks + achieved blackboard points)/ possible number of ticks

The ticks are not weighted by the possible black board points or anything else.
final mark success quotient
1 ≥90%
2 ≥80%
3 ≥70%
4 ≥60%

Online checking system

The website of the online checking system can be found here:

Link to online checking system

At the first login, a password has to be required, which is sent to the email address resulting from TUGonline. The solved exercises of the current week have to be marked online until one day before the respective exercise class (Tuesday) before 11:00 p.m. The list of participants is generated automatically from the TUGonline inscriptions.



Erstellt am 2.3.2017, modifiziert Feb. 2018.

Impressum:
Für den Inhalt verantwortlich: Wolfgang Woess, Gundelinde Wiegel
Institut für Diskrete Mathematik
TU Graz, Steyrergasse 30, A-8010 Graz