The SoapBoxers

I started subscribing to the Sporting News a few months ago. The subscription rate was cheap, so I figured I would give it a shot.

I have been pretty disappointed. It doesn’t seem to stack up very well against other sports publications and web sites. In particular, its analysis of statistics can be rather poor at times.

The March 30th edition is a case in point. There is an article related to the shortness of a running back’s career. An inset box titled “built for the short run” shows the average years of service for the starters at various positions in week 1 of the NFL season.

I assume that we were to take these numbers and draw a conclusion about the average length of an NFL career for those positions – but I would be really hesitant to do that, since that would be a poor use of the data. It completely ignores bench players and the stage of a player’s career – maybe this year’s draft class just had a great crop of running backs.

Take this example: let’s say that every NFL team had a 10 year veteran at running back in week 1. Then, in week 2, they yanked the veterans and plugged in rookies.

If we run the Sporting News stat, the average years of service would be 10 years for the starters in week 1 and would then slide dramatically to an average of 1 year of service for the starters in week 2. Yet, the cast of players didn’t change, nor did the expected career length.

The model is simply a poor fit – it does not measure what the Sporting News is trying to make it measure. It would be like taking the three members of my family, calculating the average age, and declaring this to be the average life span.

(Note: this article was originally truncated. I apologize for the confusion)

This will likely become a mini-series … statistics are great, but can be abused.

Today’s focus is on the “transitive” property of statistics. My example will be an issue that many people are well aware of. The issue is important and is a problematic societal issue. I am definitely not suggesting that the problem doesn’t exist – but I am suggesting that the arguments have at times been twisted to say things that the number don’t necessarily support.

Fact 1: Members of group A are paid less than members Group B for the same work
Fact 2: Members of group A, on average, have salaries that are 50% of those of members of group B

thus, the creation of
“fact” 3
Members of group A on average make, on average, 50% of members of group B, for the same work.

The only problem with this is that this is overly simplistic. You cannot blend the first two facts. Too many other variables exist.

Let’s illustrate:

Group A, 100 people
6 brain surgeons @ 135,000 = 810,000
94 engineers @ 45,000 = 4,230,000
Total = 5,040,000

Group B, 100 people
50 brain surgeons @ 150,000 = 7,500,000
50 engineers @ 50,000 = 2,500,000
Total = 10M

Let’s test facts 1 and 2

Fact 1: Are members of group A paid less than members of group B for the same work? Yes. They make 10% less.
Fact 2: Are members of group A, on average, paid ~50% of the salary of members of group B? Yes. 50.4%

But when we come to “fact” 3:
Members of group A on average make, on average, 50% of members of group B, for the same work.

This is not true. Members of group A make 90% of what members of group B make for the same work.

Keep an eye open and you’ll see occasional use of the “transitive” property by otherwise reputable sources. Why? Either because the person doesn’t know any better, or because it’s easier than delving deeper to find the “split” statistics – and 90% of people aren’t going to see the problem. The key point here – when you read the news, you should not only scrutinize the facts for accuracy, but also the logic used to piece the story together. The best statistics in the world cannot undo the harm of illogical thinking.

End note: With this particular example, there is the issue of why members of group A tend to have less than members of group B. Is this due to personal preference, lifestyle choices, or a lack of opportunity. This is a good topic for discussion, but it is beyond the point of this article – that people need to analyze the statistics more closely.