Time Schedule:
Jeffrey A. Bilmes
E E 595
Seattle Campus
Extension of 507, 508, 518, 519, 520. Material differs each year, covering such topics as: detection theory, decision theory, game theory, adaptive communication systems, nonlinear random processes. Prerequisite: permission of instructor.
Class description
This course will be an thorough introduction to information theory. Topics will include entropy, mutual information, asymptotic equipartition properties, data compression to the entropy limit (source coding), communication at the channel capacity limit (channel coding theorem), coding theory (including ECC and other modern codes), method of types, differential entropy, maximum entropy, rate-distortion theory, alternating minimization, and information geometry in general. Additional topics will include information theory as it is applicable to pattern recognition, natural language processing, computer science and complexity, biological science, and communications.
We will use two texts including: 1) the classic text "Elements of Information Theory" by Thomas Cover and Joy Thomas, 1991 edition and, 2) a new modern text by David Mackay from Cambridge University.
Prerequisites: basic probability, statistics, and random processes. Basic knowledge of matlab. The course is open to students in all UW departments.
Student learning goals
General method of instruction
Recommended preparation
Class assignments and grading
