Tim Swartz, Adriano Arce, and Mohan Parameswaran, “Assessing Value of the Draft Positions in Major League Soccer’s SuperDraft”, The Sport Journal, 16(9): 2013. [PDF]

In this paper, the authors create performance metrics based on minutes played and salary level of drafted Major League Soccer players in order to estimate the relative value of the SuperDraft positions.  Moreover, they create an averaged trajectory of player performance over a football player’s career.

Major League Soccer’s SuperDraft takes place this week, and we begin 2016’s Paper Discussions with a timely read of a 2013 paper on the subject.  The SuperDraft, which is a combination of the former College and Supplemental Drafts, is not as important as it used to be in the early years of the league but it is remains a key element of a club’s talent selection process.  Of interest are not just the selections that each MLS club makes but also the deals and trades between organizations in order to grab additional or more advantageous draft picks.  So any insight of the value of a draft selection — whether actual or perceived — is extremely valuable to club decisionmakers.  The objective of this paper is to assess the absolute and relative value of SuperDraft positions according to the actual performance of past selections.

The lead author is Tim Swartz, who is a professor of Statistics and Actuarial Science at Simon Fraser University in the Vancouver suburb of Burnaby.  His primary research area is in statistical computing and in particular more sophisticated algorithms that are applied to Bayesian analysis.  Over the last 15 years, more of his work has been devoted to statistics in sport, from golf and cricket to football and Highland Games.  I actually met Prof. Swartz at the last NCSSORS conference in 2010 — he was giving a talk on the player auction system in the Indian Premier League (T20 cricket) — and we expressed mutual appreciation of finding a researcher in a sport that wasn’t American football, basketball, or baseball.  Adriano Arce and Mohan Parameswaran were undergraduate students at Simon Fraser at the time; Arce is still a student who graduates in the spring while Parameswaran has graduated and since founded his own company.

The inspiration for this paper finds itself in the draft value charts that have circulated around the front offices of the National Football League and have a common origin: the Dallas Cowboys front office of the early 1990s.  In the book Scorecasting, the authors Jon Wertheim and Tobias Moskowitz tell the story of Mike McCoy, a former Cowboys executive, minority owner, and business partner of Cowboys owner Jerry Jones.  McCoy used draft day transaction data to generate a chart of the relative values of NFL draft picks from the teams’ perspectives, which was then used to inform their trades on draft day.  The results were smashingly successful — in the first five years of the chart, the Cowboys selected 15 future starters and five Pro Bowl players, and they went on to fill important roles in the three Super Bowl wins during that period.  Front-office and coaching staff of league champions become hot properties, and as coaches and front office personnel left for other teams, they took the Cowboys’ draft chart with them.  Today every team in the league has seen and used that chart, and you can find copies of it all over the web.

Can we develop a similar draft value chart for MLS?  Sure, but the type of data collected to create such a chart matters.

Swartz and his students consider over 700 players selected in SuperDrafts between 2000 and 2012, and collect participation and salary data on these players between 2004 and 2012.  (Salary data from the MLS Players Union exists from 2007; further digging yielded salary figures for the 2004, 2005, and 2006 seasons.  Participation data is essentially minutes played, which is a metric common to all players and can be used to observe value, albeit imperfectly.  Salary data indicates the relative value of a player to his peers, even though that is also an imperfect observable and subject to much uncertainty (those who have read my posts on Front-Office Efficiency know about the issues with Players Union salary data).  Players are ranked by minutes played and salary against all other players in the same MLS season, and for each metric two additional metrics are derived: a maximum percentile rank over a player’s career, and the average of the three highest percentile ranks over a career.  These metrics are plotted as a function of draft position, and one such plot is displayed below.  The best-fit line in the plot is used to create the values of the draft chart, which you can find in the paper.

Scatterplot of average max 3-year salary over career vs draft position, with a lowess fit of the data. [From Swartz et al (2013)]

So here are the major findings:

• On average, the top selections in the 2000-2012 MLS SuperDrafts reached the top 80-85% of player participation and compensation over the careers.  If you look at the list of top overall picks in MLS, 80-85% seems high, but with the exception of Steve Shak and Nik Besagno, all of those players made at least 75 MLS appearances.  A few of those players received national team caps, but only Maurice Edu made a World Cup roster (yes, Freddy Adu did play in a couple youth-level World Cups).
• The dropoff in MLS draft selection value is steep, but not that much steeper than other sports.  In the draft chart (which was created in 2012), the last pick of the first round has about a quarter the value of the first pick.  In the original NFL draft chart, the last pick of the first round is worth about 20% of the first pick.  The really steep descent in value comes in the second round of the MLS draft: the last selection is worth 20% of the first.  In the original NFL chart, that didn’t happen until the seventh and last round.
• If you’re drafted beyond the first half of the first round, breaking into a MLS roster is very difficult.  This applies to all other US sports as well, but the draft threshold in those sports comes later.
• The Swartz draft value card represents the expected peak performance of a selected player.  It does a good job of expressing the expected payoff of a draft pick, but it is silent on the perceived value of the pick.  That’s a significant element of the draft transactions that are made on Draft Day.

There’s a lot more that I’d like to say that this paper finds, but to be rigorous I have to say that it doesn’t.  It doesn’t relate draft position to the probability of participating in a MLS match, or the longevity of a player’s career, or the the probability of transfer to an overseas or lower-tier domestic league.  Positional differences are disregarded, and goalkeepers are ignored altogether (although in fairness only 4-5 ‘keepers are selected in an average draft).  The draft position values express maximum value of players, but not their current value which is more relevant when making a transaction.  Finally, how has the value of SuperDraft picks changed over the years, and in particular in a time of increased usage of Homegrown players?

Nevertheless, the paper represents a start of a broader analysis of the SuperDraft’s value in a time of multiple options for recruiting talent in Major League Soccer.