CrossFit Scoring – Alternative Perspective

This is my third foray into the world of CrossFit data analysis, this time looking at scoring. Recently, over at the CrossFit Games website, there was an article about Scoring CrossFit Competitions as well as one talking about Scoring Technology. I participated in a number of conversations in the comments section with regards to how competitions are scored. This post is going to be a bit of a summary of those conversations as well as a description of a new extension to the CrossFit Data Explorer that allows people to explore different scoring schemes for any given competition (i.e. alternative perspectives).

For those that are unaware, there are a variety of approaches to scoring competitions. I am not going to go into detail about these different metrics as they are described in both the articles I linked to above. The main thing to keep in mind is that each scoring system has certain “flaws” and depending on how an event is scored, there can be different outcomes.

I wanted to explore some of these different scoring schemes as well as allow others to play with these different metrics. To do this, I added a new tab to the CrossFit Data Explorer called “Scoring tables”. Check out the screenshot below displaying data from the Men’s Northwest Regional.

This view shows a table of the athletes, their results for each event, and it is sorted by the overall placement. On the left-hand side, you can choose how to score the athletes. The currently available scoring options are:

• Ranked-based – each athlete receives points based on their ranking in a workout, 1pt = 1st, 2pts = 2nd, …, 50pts = 50th. The athlete with the lowest overall score wins.
• Proportional – Athletes receive scores based on their relative performance to the top scoring athlete in an event. Highest overall score wins.
• Lowest converted points (LCP) – Timed workouts receive 1 point per second, all other results are subtracted from this score. Lowest overall point total wins.
• Standard score – Athlete receives points based on the distance between their score and the population mean. Highest score wins.

The other feature supported by this new scoring component is that you can filter workouts from consideration in the ranking process. For example, if you thought, “Damn, if that stupid deadlift workout hadn’t been in the regional, I would have made it.” Well, now you can verify whether that’s true :-).

Ok, time to dig into comparing some scores. Keeping with the Men’s Northwest Regional, if we re-rank the athletes based on the four supported scoring systems, we get the following results (top three athletes made games from this region).

System	First	Second	Third
Rank-based	Chris Spealler	Jerome Perryman	Eric O’Connor
Proportional	Chris Spealler	Jerome Perryman	Eric O’Connor
LCP	Jerome Perryman	Jordan Holland	Christopher Dunkin
Standard	Chris Spealler	Jerome Perryman	Jordan Holland

As we can see, there’s a certain amount of fluctuation in the results, especially for the LCP metric. In fact, with LCP, Eric O’Connor finishes 5th and Chris Spealler comes in 10th! (See screenshot below).

The major reason for this is the way the scoring system is designed. Each workout is scored completely independently from all other workouts, so a particular workout can completely dominate the final value. For example, coming in first in workout #1 at this regional gives you a score of 159 while coming in first in the second workout gives you a score of 10,908. That’s a massive difference!

In fact, if we look at the correlation between the LCP ranking and the ranking based purely on the second workout result, there’s a strong correlation of 0.45 (p < 0.01). In comparison to the actual official ranking results, the correlation between workout #2’s ranking and the official ranking is only 0.37 (p < 0.01).

One really nice thing about the proportional, LCP, and standard score is that they “reward” people for being exceptional. For example, consider the Canada Men’s Regional results. Erik Szakaly completed the Wall-ball/Pull-ups workout more than a minute ahead of the next fastest athlete. Check out the picture below, look at how far Erik’s result is separated from all other athletes. Obviously this guy eats wall-balls for breakfast.

He received a low score of 1 point at the regional, but in the standard score system he would have received 2.17 points, almost a full point ahead of the next closest score (see below). This is a fairly significant margin in this scoring system.

Perhaps some kind of hybrid scoring system is needed. I think as CrossFit evolves, so will the scoring metrics, just as we see with other sports.

As fun as it is to play with all this data, there’s a few things to keep in mind. First is that switching the scoring system and re-ranking athletes assumes that the athletes would perform the same regardless of the scoring system. However, that’s quite possibly not true. For example, Garth Prouse won the Canadian Regional run quite easily and received 1 point for coming in first (rank-based system). However, if he knew that he was being scored based on one of the other three systems, he may have pushed his pace harder to give himself a larger lead.

Another assumption that is made is that the people that designed the workouts would include the same set of workouts regardless of the scoring system used. This may not be the case. For example, in the Northwest regional, perhaps the workout designers would have scored workout #2 differently if they were planning on using an LCP system so as to not bias the result in favor of performances in this workout.

The other thing to be aware of is my application has to make certain assumptions. It assumes that all workouts with a time-based result are ranked lowest to highest, and all other workouts go highest to lowest. In the Canadian Regional, the run did not have times recorded, so including this event in the score comparison leads to misleading results.