Hello and welcome to unit 5.

This unit is on empirical power 
law distributions.

The central question that we will 
consider is as follows:

suppose you are studying some phenomenon 
and you have a set of data.

How can you figure out

if that data is well described by a power-law?

And if so, what's the best way 
to estimate the power law exponent.

It turns out that these questions 
can be surprisingly subtle and

it's easy to approach them in the wrong way.

I'll begin in the next sub-unit by

looking at different ways to represent data.

We'll talk about histograms but also about

cumulative distribution functions and 
rank frequency plots.

I'll then give a brief survey of 
some of the phenomena

that are thought to be distributed according 
to power laws

and then we'll start looking at

some of these questions of inference.

Given that we suspect that something is a power law,

what's the best way to estimate

the parameters that describe the distribution.

We'll also need to

talk about alternatives, by that

I mean there may be other

distributions that describe

your data better than a power law

and there are some distributions that are often

confused with power-laws in particular

stretched exponentials and log-normals.

So what I'll do is say

a little bit about those other distributions, and

compare and contrast them to power-laws,

and then talk about some statistical techniques

for choosing among those different options.

Before I get started, another word or two about this unit.

I will frequently be referring to

a couple of really key papers in

empirical power laws and inference

with power laws and I've put a list of

those papers along with links to pdfs

in a subunit or a section called

Additional Resources.

So, you can find all the papers that

I mention right there so you

shouldn't have to look around for them.

Additionally, I'm gonna be mentioning

a bunch of statistical techniques and

we're not gonna get so involved that

I'm gonna show you step by step 
how to do them.

I think that's too much

statistics and beyond the scope of this course,

but I suspect, I hope that some of you will want to

try these things out on your own.

So, there's some good software that's out there

that's been developed in the last 5 years or so and

links to those software are also in this

additional resources section.

There's some Python code, some R code, and

a little bit of Matlab code as well.

So, if you want to try some of that stuff out,

those are pretty good places to start.

Ok, so let's begin by

thinking about power law distributions by

looking at how to represent data.

And I'll do so with a simple example that will

lead us through the construction of a

cumulative distribution function and a rank-frequency plot.

You'll see that it's easier to calculate than it is to say.

So, let's get started.