# Complexity Explorer Santa Fe Institute ## Introduction to Renormalization

• Introduction
• Coarse graining Alice and Dinah
• Coarse graining part I - Clustering algorithms
• Coarse graining part II - Entropy
• Markov Chains
• Mathematics of coarse grained Markov chains
• Mathematics of Coarse grained Markov Chains: The General Case
• A puzzle: origin of the slippy counter
• Where we are so far
• Cellular Automata: Introduction
• Israeli and Goldenfeld; projection and commuting diagrams
• Networks of Renormalization
• Coarse-graining the Lattice
• From Quantum Electrodynamics to Plasma Physics
• Conclusion: Keeping the things that matter
• Homework

#### 1.4 Coarse graining part II - Entropy » Quiz Solution

Which of these distributions has the highest entropy?

A. A coin toss, where the coin can come up either heads or tails, with 50% probability.

B. A coin toss where the coin is heads on both sides.

C. A computer program that spits out one of the 26 letters in the English language uniformly at random.

D. Sticking your hand in Jane Austen's Pride and Prejudice and pulling out a  letter (ignoring whether it is upper or lower-case).

Answer: (C). (B) has zero uncertainty; (A) has more uncertainty than (B) because the coin could come up either way, and we have to go look (gather information) to find out. (C) has more information than (B) because instead of a choice between two options, there's a choice among 26 (another way to see it is that we could get the same randomness in a coin toss by coarse-gaining (C): take a random letter and then asking whether it fell in the first or second half of the alphabet). Finally, (D) has less entropy than (C) because English is biased in favor of some letters (like vowels), and away from others (like z and j); I'm more certain about a letter from an English-language word than I am about a completely uniform distribution. You can also compute the entropy of (A), (B) and (C) exactly, using Shannon's formula; and you can estimate (D) using the original text of Pride and Prejudice (I get around 4.18 bits).