Productivity

Achievement is not normal

Posted on by John

Angela Duckworth gave a 90-second talk entitled Why Achievement Isn’t Normal.

She’s using the term “normal” in the sense of the normal (Gaussian) distribution, the bell curve. With normally distributed attributes, such as height, most people are near the middle and very few are far from the middle. Also, the distribution is symmetric: as many people are likely to be above the middle as below.

Achievement is not like that in many fields. The highest achievers achieve far more than average. The best programmers may be 100 times more productive than average programmers. The wealthiest people have orders of magnitude more wealth than average. Best selling authors far outsell average authors.

Angela Duckworth says achievement is not normal, it’s log-normal. The log-normal distribution is skewed to the right. It has a long tail, meaning that values far from the mean are fairly common. The idea of using a long-tailed distribution makes sense, but I don’t understand the justification for the log-normal distribution in particular given in the video. This is not to disparage the speaker. No one can give a detailed derivation of a statistical distribution in 90 seconds. I’ll give a plausibility argument below. If you’re not interested in the math, just scroll down to the graph at the bottom.

The factors that contribute to achievement are often multiplicative. That is, advantages multiply rather than add. If your first book is a success, more people will give your second book a chance. Your readership doesn’t simply add, as if each book were written by a different person. Instead, your audience compounds. Websites with more inbound links get a higher search engine rank. More people find these sites because of their ranking, and so more people link to them, and the ranking goes up. Skills like communication and organization don’t just contribute additively as they would on a report card; they are multipliers that amplify your effectiveness in other areas.

The log-normal distribution has two parameters: μ and σ. These look like the mean and standard deviation parameters, but they are not the mean and standard deviation of the log-normal. If X is a log-normal(μ , σ) random variable, then log(X) has a normal(μ, σ) distribution. The parameters μ and σ are not the mean and standard deviation of X but of log(X).

The product of two log-normal distributions is log-normal because the sum of two normal distributions is normal. So if the contributions to achievement are multiplicative, log-normal distributions will be convenient to model achievement.

I said earlier that log-normal distributions are skewed. I’ve got something of a circular argument if I start with the assumption that the factors that contribute to achievement are skewed and then conclude that achievement is skewed. But log-normal distributions have varying degrees of skewness. When σ is small, the distribution is approximately normal. So you could start with individual factors that have a nearly normal distribution, modeled by a log-normal distribution. Then you can show that as you multiply these together, you get a distribution more skewed than it’s inputs.

Suppose you have n random variables that have a log-normal(1, σ) distribution. Their product will have a log-normal(n, √n σ) distribution. As n increases, the distribution of the product becomes more skewed. Here is an example. The following graph shows the density of a log-normal(1, 0.2) distribution.

plot of log-normal(1, 0.2) density

Here is the distribution of the product of nine independent copies of the above distribution, a log-normal(9, 0.6) distribution.

plot of log-normal(9, 0.6) density

So even though the original distribution is symmetric and concentrated near the middle, the product of nine independent copies has a long tail to the right.

Related posts

Parkinson’s law

Posted on by John

Yesterday I found a copy of Parkinson’s Law for $1 at a library book sale. This book is best known for it’s opening line: Work expands so as to fill the time available for its completion.

Dust jacket of the book Parkinsons Law and Other Studies in Administration

The name “Parkinson’s law” can mean at least four different things:

  1. The 1957 book by C. Northcote Parkinson
  2. The first chapter of Parkinson’s book
  3. The principle expressed in the book’s opening line, as understood by Parkinson
  4. The principle in the opening line as understood today.

I’d heard of the general principle of Parkinson’s law a few years ago. I only found out about the book more recently. I didn’t know until last night that Parkinson intended his principle to be applied more narrowly than it is applied now.

The full title of the first chapter of the book is “Parkinson’s Law, or The Rising Pyramid.” This chapter explains how work expands to fill the available resources within a bureaucracy and why bureaucracies grow exponentially at a compounding rate of around 5% per year. The subtitle addresses the mechanism for this growth, bureaucrats creating a pyramid of subordinates. Parkinson derives his law from “two almost axiomatic statements”:

  1. An official wants to multiply subordinates, not rivals.
  2. Officials make work for each other.

Nowadays Parkinson’s law is usually condensed to saying work expands to the time allowed. It is applied to individuals as well as a burgeoning bureaucracies. Parkinson discusses this interpretation in his opening paragraph but then limits his attention to organizations.

The total effort that would occupy a busy man for three minutes all told may in this fashion leave another person prostrate after a day of doubt, anxiety, and toil.

Chapter 3 of Parkinson’s law is “High Finance, or The Point of Vanishing Interest.” This chapter is the source of the phrase bike shed arguments. In this chapter Parkinson states what he calls the Law of Triviality:

… the time spent on any item of the agenda will be in inverse proportion to the sum involved.

The idea is that people are more likely to contribute to the discussion of things they understand. A nuclear reactor will sail through the finance committee, but a bicycle shed will cause endless debate because everyone can understand it and everyone has an opinion.

I picked up a copy of Mrs. Parkinson’s Law at the same book sale, also for $1. I’d never heard of it before, but I imagine it will be entertaining.

More corporate culture posts

Email isn't the problem

Posted on by John

I find it odd that so many folks complain about email. Email’s ruining their lives. They wish email had never been invented. Etc.

Say you get 100 email messages in a day. Would you rather have 100 phone calls? The problem isn’t the 100 messages but the 100 new responsibilities represented by the email messages. Email spam is a annoying, but phone spam (i.e. telemarketing) is worse.

There are alternatives to email that work well in their niche. RSS subscriptions beat email newsletters. Instant messaging beats email for quick interaction. Twitter beats email for keeping up with loquacious friends. But I don’t understand those who say “email is dead.”

Can you predict the "20" in 80/20?

Posted on by John

A simplest form of the 80/20 rule says that 80% of results come from only 20% of efforts. For example, maybe the top two people on a team of 10 are responsible for 80% of the team’s output. Maybe the most popular 20% of items on the menu account for 80% of a restaurant’s sales. Maybe you read 10 books on a subject but most of what you learned comes from the best two (or the first two).

The exact numbers 80 and 20 are not special. For example, one study showed that 75% of Twitter traffic comes from the most active 5% of users. That’s still an example of the 80/20 rule. The point is that a small portion of inputs are responsible for a large portion of outputs.

One criticism of the 80/20 rule is that you can only know which 20% was most effective in hindsight. A salesman could call on 100 prospects in a week and only make sales to 20. At the end of the week he could ask “Why didn’t I just call on those 20?” Of course he had to call on all 100 before he could know who the 20 were going to be. Or maybe the best 20% of your stock portfolio accounted for 80% of your growth. Why didn’t you just invest in those stocks? If you could have predicted which ones they were going to be, you would have done just that.

It’s easy to be cynical about the 80/20 rule. There are too many hucksters selling books and consulting services that boil down to saying “concentrate on what’s most productive.” Thanks. Never would have thought of that. Let me write you a check.

At one extreme is the belief that everything is equally important, or at least equally likely to be important. At the other extreme is the belief that 80/20 principles are everywhere an that it is possible to predict the “20” part. Reality lies somewhere between these extremes, but I believe it is often closer to the latter than we think. In many circumstances, acting as if everything were equally important is either idiocy or sloth.

You can improve your chances of correctly guessing which activities are going to be most productive. Nobody is going to write a book that tells you how to do this in your particular circumstances. It takes experience and hard work. But you can get better at it over time.

Related posts

Important because it’s unimportant

Posted on by John

Some things are important because they’re unimportant. These things are not intrinsically important, but if not handled correctly they distract from what is important.

Content is more important than spelling and grammar. But grammatical errors are a distraction. Correct spelling and grammar are important so readers will focus on the content. Typos are trivial (more on “trivial” below) but worth eliminating.

When I was in college, the computer science department deliberately used a different programming language in nearly every course. The idea was that programming language syntax is unimportant, and constantly changing syntax would cause students to focus on concepts. This had the opposite of the desired effect. Since students were always changing languages, they were always focused on syntax. It would have made more sense to say that since we don’t believe programming language syntax is important, we’re going to teach all our lower division courses using the same language. That way the syntax can become second nature and students will focus on the concepts.

Grammar, whether in spoken languages or programming languages, is trivial. It is literally trivial in the original sense of belonging to the classical trivium of grammar, logic, and rhetoric. These subjects were not the goal of classical education but the foundation of classical education. We now say something is “trivial” to indicate that it is unimportant, but in the past this meant that the thing was foundational. Calling something “trivial” meant that it was important in support of something else of greater interest.

When people call something trivial, they may be correct, but not in the sense they intended. They might mean that something is trivial in the modern sense when actually it’s trivial in the classical sense. For example, unit conversions are trivial. Just ask NASA about the Mars Climate Orbiter.

Mars Climate Orbiter NASA photo

For a day or two, make note of every time you hear something called “trivial.” Ask yourself whether it is trivial in the modern sense of being simple and unimportant or whether it could be trivial in the classical sense of being foundational.

Ever feel like a newspaper?

Posted on by John

Why are newspapers going out of business? The simple explanation is that newspaper owners are stupid; the world around them is changing and they’re oblivious. Michael Nielsen has a more interesting explanation. He says that newspapers are in trouble not because they’re stupid now but because they’ve been smart in the past.

Nielsen argues that newspapers are locked into their current business models because they have been so successful. Any small changes will make their businesses less profitable. I don’t know enough about the newspaper industry to say whether Nielsen is right, though I find his argument plausible. (His article is entitled Is scientific publishing about to be disrupted? However, it is about much more than scientific publishing.)

Nielsen argues that newspapers are standing on the top of one hill and profitable online news sources are standing on a higher hill, a hill that didn’t exist 20 years ago. In mathematical lingo, both businesses are at local maxima. Newspapers are trapped because they can’t improve their situation without first making it worse. Anyone who leads a newspaper down its hill in order to climb a new hill will be fired before he starts gaining altitude again.

I don’t care that much about newspapers, but Nielsen’s article struck me because it provides an explanation for many other situations. I feel like some areas of my life are stuck at a local maximum: there’s plenty of room for improvement, but not by making small changes.

The most subtle of the seven deadly sins

Posted on by John

Six of the seven deadly sins are easy to define, but one is more subtle. The seven deadly sins are

  1. lust
  2. gluttony
  3. greed
  4. sloth
  5. wrath
  6. envy
  7. pride.

Sloth is the subtle one.

I discovered recently that I didn’t know what sloth meant. When I first heard of the seven deadly sins, I thought it was odd that sloth was on the list. How would you know whether you’re sufficiently active to avoid sloth? It turns out that the original idea of sloth was only indirectly related to activity.

The idea of a list of deadly sins started in the 4th century and has changed over time. The word in the middle of the list was “acedia” before it became “sloth,” and the word “sloth” has taken on a different meaning since then. So what is acedia? According to Wikipedia,

Acedia is a word from ancient Greek describing a state of listlessness or torpor, of not caring or not being concerned with one’s position or condition in the world. It can lead to a state of being unable to perform one’s duties in life. Its spiritual overtones make it related to but distinct from depression.

In short, “sloth” did not mean inactivity but rather a state of apathy. As Os Guinness says in his book The Call

… sloth must be distinguished from idling, a state of carefree living that can be admirable, as in friends lingering over a meal … [Sloth] can reveal itself in frenetic activism as easily as in lethargy … It is a condition of explicitly spiritual dejection … inner despair at the worthwhileness of the worthwhile …

Sloth and rest could look the same externally while proceeding from opposite motivations. One person could be idle because he lacked the faith to do anything, while another person could be idle because he had faith that his needs would be met even if he rested a while. The key to avoiding sloth is not the proper level of activity but the proper attitude of the heart.

Questioning the Hawthorne effect

Posted on by John

The Hawthorne effect is the idea that people perform better when they’re being studied. The name comes from studies conducted at Western Electric’s Hawthorne Works facility. Increased lighting improved productivity in the plant. Later, lowering the lighting also increased productivity. The Hawthorne effect says that the productivity increase wasn’t due to changes in lighting per se but either the variety of changing something about the plant or the attention that workers got by being measured, a sort of placebo effect.

The Alternative Blog has a post this morning entitled Hawthorne effect debunked. The original Hawthorne effect was apparently due to a flaw in the study design; correcting for that flaw eliminates the effect.

The term “debunked” in the post title may imply too much. The effect in the original studies may have been debunked, but that does not necessarily mean there is no Hawthorne effect. Perhaps there are good examples of the Hawthorne effect elsewhere. On the other hand, I expect closer examination of the data could debunk other reported instances of the Hawthorne effect as well.

The Hawthorne effect makes sense. It has been ingrained in pop culture. I heard a reference to it on a podcast just this morning before reading the blog post mentioned above. Everyone knows it’s true. And maybe it is. But at a minimum, there is at least one example suggesting the effect is not as widespread as previously thought.

It would be interesting to track the popularity of the Hawthorne effect in scholarly literature and in pop culture. If the effect becomes less credible in scholarly circles, will it also become less credible in pop culture? And if so, how quickly will pop culture respond?

Create offline, analyze online

Posted on by John

Sitting at a computer changes the way you think. You need to know when to walk away from the computer and when to come back.

I think mind mapping software is a bad idea. Mind maps are supposed to capture free associations. But the very act of sitting down at a computer puts you in an analytical frame of mind. In other words, mind mapping is a right-brain activity, but sitting at a computer encourages left-brain thinking. Mind mapping software might be a good way to digitize a map after you’ve created it on paper, but I don’t think it’s a good way to create a map.

When I need to sort out projects and priorities, I do it on paper. After that I may type up the results. I like to capture ideas on paper or on my voice recorder but then store them online.

When I do math, I scribble on paper, then type up my results in LaTeX. Scribbling helps me generate ideas; LaTeX helps me find errors. I’ve found that fairly short cycles of scribbling and typing work best for me, a few cycles a day.

In the past, we did a lot of things on paper because we had no choice. Today we do a lot of things on computers today just because we can. It’s going to take a while to sift through the new options and decide which ones are worthwhile and which are not.

Recommended books

Daniel Pink’s book A Whole New Mind has a good discussion of left-brain versus right-brain thinking. As he points out, the specialization between the left and right hemispheres of the brain is more complicated than once thought. However, the terms “left-brain” and “right-brain” are still useful metaphors even if they’re not precise neuroscience.

Also, to read more on how computers influence our thinking, see Andy Hunt’s book Pragmatic Thinking and Learning.

Related posts

Programs, not just projects

Posted on by John

My frustration with personal productivity systems like GTD is that they’re all about projects and tasks. They leave out a third category: programs. GTD thinks of a project as something that can be broken into a manageable number of tasks and scratched off a list. But programs go on indefinitely and cannot be divided into a small number of one-time tasks.

I’m using the word “program” as in an “exercise program” or a “research program.” (I could think of my exercise program as a project, but it’s one I hope not to complete for a few more decades.) Sometimes there is a neat hierarchy where programs spawn off projects that can be divided into tasks. But sometimes you just have programs and tasks.

One of my frustrations with managing software development in an academic environment was the large number of programs disguised as projects. (Sorry, I know it’s confusing to talk about “programs” in the context of software development and not mean computer instructions.) You can’t manage programs as if they were projects. For example, you can’t talk about “after” project is done if it’s not really a project but a never-ending program. You have to either acknowledge that a program is really a program, or you have to have some way to make it into a finite project.