Complexity Explorer Santa Fe Institute

Your progress is not being saved! Enroll now or log in to track your progress or submit homework.

Introduction to Information Theory

Lead instructor:

Video Zip Files:

 

Further Reading

Books and articles

  1. Cover and Thomas, Elements of information theory, Wiley, 2006.
    The “standard” textbook.  It’s well-written, fairly accessible, and provides a very rigorous foundation.
     

  2. Pierce, An Introduction to Information Theory: Symbols, Signals and Noise, Dover, 1980.

    A classic introduction to information theory and its application. Well-written and less technical than Cover & Thomas, while also covering a broader range of applications. Keep in mind that the "Information theory and physics" chapter is somewhat dated.

    .

  3. Gleick, The Information: A History, A Theory, A Flood, Vintage, 2012
    A popular science book that discusses about the historical development of the idea of information.
     

Frigg and Werndl, Entropy - A guide for the Perplexed, 2010. http://charlottewerndl.net/Entropy_Guide.pdf
A great, slightly technical overview of entropy as it is used in information theory, statistical physics, and other scientific fields.