**A Short Course in Information Theory**

by David J. C. MacKay

**Publisher**: University of Cambridge 1995

**Description**:

Is it possible to communicate reliably from one point to another if we only have a noisy communication channel? How can the information content of a random variable be measured? This course will discuss the remarkable theorems of Claude Shannon, starting from the source coding theorem, which motivates the entropy as the measure of information, and culminating in the noisy channel coding theorem. Along the way we will study simple examples of codes for data compression and error correction.

Download or read it online for free here:

**Download link**

(multiple PDF,PS files)

## Similar books

**Algorithmic Information Theory**

by

**Peter D. Gruenwald, Paul M.B. Vitanyi**-

**CWI**

We introduce algorithmic information theory, also known as the theory of Kolmogorov complexity. We explain this quantitative approach to defining information and discuss the extent to which Kolmogorov's and Shannon's theory have a common purpose.

(

**10749**views)

**A primer on information theory, with applications to neuroscience**

by

**Felix Effenberger**-

**arXiv**

This chapter is supposed to give a short introduction to the fundamentals of information theory, especially suited for people having a less firm background in mathematics and probability theory. The focus will be on neuroscientific topics.

(

**9162**views)

**Data Compression Explained**

by

**Matt Mahoney**-

**mattmahoney.net**

This book is for the reader who wants to understand how data compression works, or who wants to write data compression software. Prior programming ability and some math skills will be needed. This book is intended to be self contained.

(

**10762**views)

**Information, Entropy and Their Geometric Structures**

by

**Frederic Barbaresco, Ali Mohammad-Djafari**-

**MDPI AG**

The aim of this book is to provide an overview of current work addressing topics of research that explore the geometric structures of information and entropy. This survey will motivate readers to explore the emerging domain of Science of Information.

(

**7910**views)