Overview - ECE 6605, Information Theory

Topics Covered

1. Basic Concepts related to Information Theory
   1. Entropy - Shannon and Renyi
   2. Information
   3. Fano's Inequaluty
   4. Jensen's Inequality
   5. Markov Chains
   6. Log-Sum inequality
   7. f-Divergence
2. Typical Set theorem
3. Asymptomatic Equipartition Property
4. Data Compression - Source Coding
5. Channle Coding
   1. Binary Switching Channel
   2. Binary Erasure Channel
   3. With Side Information
6. Coding for secrecy -
   1. Soft-Covering: As close to random distribution
   2. WireTap Channel: Zero information leakage to eavesdropper
7. Privacy Amplification
8. Secrecy

Other Topics

• Bit commitment