In the previous unit, what I showed you

was a way to rescue MaxEnt models to

describe the language abundance P(n)

by reference to a hidden variable, epsilon

which we referred to as "programmer time"

and what we assumed was that the system

was constrained in two ways:

it was constrained not only to having

a certain average number of projects per

language. We fixed the average popularity

of languages, but we also fixed the average

programmer time devoted to projects

in a particular language. And so this 
distribution, here, looks like this.

in functional form. And when we integrate

out this variable, epsilon, we get 
something that looks like this.

So we get a different prediction for

the language distribution. And we get

a prediction that, I argue, looks
reasonably good.

It certainly looks better than the
exponential distribution.

I feel honor-bound to tell you
about the controversy

that happens when we try to make
these models... and in particular

there is a very different mechanistic
model that looks quite similar.

So this is the Fisher log series.

And the argument behind the Fisher log
series, to explain this distribution,

involves the idea of a "hidden" additional
constraint.

In the open source question, what I've
done is describe that additional

constraint as "programmer time", just
because it seems like it might be a

constraint in the system. OK? 
That the average programmer time

for languages is fixed, not just the 
average number of projects.

But, and so that means that languages
can vary in their popularity, but also

in their efficiency. In the ecological
modelling, this is "species"

languages are species, and this is the 
abundance of species.

So that means the number of particular
instances of the species in the wild.

And also metabolic rate: how much energy
a particular species consumes.

And so in that case, the system is 
constrained to a certain average

species abundance, and a certain average
species energy consumption

Here languages are constrained to a certain
abundance, and a certain consumption of

programmer energy. That's the analogy.

So, we can also build a mechanistic model.

of programming language popularity.

And previously, when we studied the taxi-cab
problem, what we did was,

when we produced the mechanistic model, 
we were able to find a very simple one

that had the same predictions as the
MaxEnt model did.

Here, by contrast, what we're going to find
is that the mechanistic model

is going to produce similar behavior, but
the functional form will actually be

slightly distinct. So, here's the mechanistic
model... we imagine that

languages all start out with a baseline

popularity. Whoever invents the language,
for example, has to write

at least one project. There is at least one
programmer at the beginning

of a language's invention, who knows how

to program in that language, somewhat

by definition. And so there's two ways

that that popularity can grow.

It can grow, for example, linearly.

So, on day 1 there's 1 programmer.

And on day 2, that one programmer is 
joined by another programmer.

And on day 3, those two programmers
are joined by a third.

And so, over time, what you have is

a growth rate that's linear...

in time.

But, perhaps a more plausible model

for how languages accrue popularity

is multiplicative.

At time 1, there's 1 programmer, and he has

some efficiency of converting other

programmers to his cause. So maybe he's

able to double the number of programmers.

And he's able to double the number of
programmers, because his language

is particularly good, and perhaps perhaps 
people who like to program in that language

happen to be particularly persuasive.

And so on the second day, those two
programmers each themselves

go out and convert two people, 
because they are the same as the

original programmer in their effectiveness

and the language itself is just as 
convincing as it was before.

So each of those two programmers goes out

and gathers two for each and we go to 4

And by a similar argument, we go to 8,

and so this would be the 
exponential growth model...

where the number of programmers
as a function of time increases

multiplicatively, as opposed to additively.

So, let's make this model a little more

realistic, and in particular, let's allow
the multiplicative factor,

which in this case we set to 2, we're

going to allow this multiplicative factor
to vary. And in fact, we're going to draw

this multiplicative factor, alpha, from some

distribution. And in fact, it doesn't really
matter what that distribution is

as long as alpha is always greater than 0,

so it's not possible for all programmers
to suddenly disappear.

so it's always greater than zero,

and it's bounded at some point,

so it's impossible for a language to become

infinitely popular after a finite number

of steps. So we're going to draw...

each day we're going to draw a number,
alpha, from this distribution, here.

So, after one day, there are alpha programmers.

After two days, there is alpha... or rather
alpha(1) programmers. (This is the

draw on the first day.) On the second day
there's alpha(2) times alpha(1)

programmers, and so on. 
alpha(3) times alpha(2) times alpha(1)

So this is now growth that occurs through

a random multiplicative process.

It's similar to growth that would happen

through a random additive process

except now, instead of adding a random
number of programmers each day

you multiply the total number of

programmers each day, by some factor alpha

drawn from this distribution.

So, you can always convert this multiplicative

process into an additive process, 
by a very simple

trick of taking the logarithm.

Over time, if we count programmer numbers

we're multiplying, but if you're working in 
log space, we're just adding.

We're adding a random number to the

distribution, as long as alpha is always

strictly greater than zero, these will 
always be well defined.

Now, all of a sudden, it looks like the
additive model in log space.

And what we know from the central limit
theorem, is that if you add together

lots of random numbers, that distribution 
tends towards a Gaussian distribution.

with some particular mean (mu) and some
particular variance (sigma).

Let's not worry about what mu and sigma

are in particular, but rather note that
that growth happens in log space

The distribution of these sums over long

time scales will end up looking like a
Gaussian distribution.

The average boost per day to a language

looks in log space like a Gaussian
distribution. What that means

is that the exponential growth model

with random kicks, random multiplicative
kicks, actually looks like

a Gaussian in log space, or what we call
a log-normal

in actual number space

So, instead of looking at the logarithm of
the popularity of the language,

just look at the total popularity of

the language, and what that means is

that it looks like the exponential of 
log(n) minus some mean squared

over two sigma squared... and then you

just have to be careful to normalize
things properly here.

So, this is the log-normal distribution.

And a mechanistic model where language
growth happens multiplicatively

where a language gains new adherents

in proportion to the number of adherents

it already has, where a language gains

new projects in proportion to the number
of projects it already has

dependent upon the environment - 
that's where the multiplicative randomness

comes from. Alpha is a random number

it's not a constant. It's not 2. It's not
the language always necessarily doubles.

But the fact that it grows through

a multiplicative random process

as opposed to an additive process

means that you have a log-normal growth.

And so now you can say, "OK, let's imagine

that languages grow through this 
log-normal process.

And let's find the best fit parameters

for mu and sigma." And if you do that, you find

that the mechanistic log-normal model looks

pretty good as well.

We were impressed by how well 
the blue line fit this distribution

compared to the red exponential model

the MaxEnt model, constraining only N

That was the red model. Here the blue

model does well... this is the Fisher-log
series.

Unfortunately, a mechanistic model... and 
I've given you a short account

of the mechanistic model, here.

Where what's happening is you're adding

together lots of small multiplicative 
random kicks

The mechanistic model also works [?]

I will tell you that this fits better.

If you do a statistical analysis, both of
these models have two parameters

If you do a statistical analysis, the 
Fisher-log series actually fits better

in particular, it's able to explain these

really high-popularity languages better

these deviations here seem larger

than the deviations here, but you have

to remember that this is on a log scale

So this gets much closer up here

than this does here.

So the mechanistic model, at least 
visually, looks like it's extremely

competitive, with the Fisher log 
series model

derived from a MaxEnt argument

Statistically speaking, if you look at 
these two, this one

is actually slightly dispreferred.

But like many people, what you want is some

ironclad evidence for one versus the other.

And I think the best way to look for

that kind of evidence is to figure out

what, if anything, this epsilon really is 
in the real world.

If we were able to build a solid theory

about what epsilon was, and how we could

measure it in the data, then we could see if

this here, this joint distribution, was 
well reproduced.

If we could find evidence, for example,

for the fact that these two co-vary.

That here we have a term that boosts

the popularity of a language if it becomes

more efficient. So, if this goes down,

this can get higher, and the language

can still have the same probability

of being found with those properties.

And of course, the problem is that we don't

know how to measure this, sort of,
mysterious programmer time...

programmer efficiency.

The ecologists have a much better time
with this.

Because the ecologists, they know what 
their epsilon is.

They know that their epsilon is metabolic
energy units intake.

So this is "how much a particular instance
of this species consumes in energy

over the course of a day, or 
over the course of its lifetime."

And they're able to measure that, and

in fact, they're able to measure this

joint distribution. If we come to study

the open source ecosystem, so far

we don't really have a way to measure this

and so we're unable to measure the joint

and so now we're left with one model that's

mechanistic, right, popularity accrual model

and over here, this model that talks about

there being two constraints on the system

Average number and average 
programmer time