In my previous post, I set up the math that explained the MLS draft pick valuation models.  In this post I will address the second part of the post title — what decisions can be informed and what insights can be gained from the model results.

Relative draft value has changed a lot since 1996

Major League Soccer has undergone a lot of changes over its 20-year history, and the value of the draft to its member clubs has been among those changes.  The most obvious use of the draft pick value model is to estimate the relative values of picks so that front-office officials have a better idea of its exchange power — the famous Dallas Cowboys draft card, for example. I plot the expected career values relative to the maximum value (first pick) of the MLS College and SuperDrafts every other year between 1998 and 2012.  The value of the early drafts was so much higher than in previous years because a majority of players entered the league via the draft system.  The relative draft value of all picks fell at the turn of the century, then stabilized between 2002-2006, which were part of the Golden Era of the drafted player (over 60% of all players in the league were drafted during this period), then dropped again to its current level.  (I’m not sure why the relative draft value saw such a big uptick for the 2008 model.  That would be worth examining!)

The punchline here is that relative draft value can change significantly year-on-year.  It can also remain static, as the periods between 2002-2006 and 2010-present indicate.  But it does mean that holding on to the tattered six-year-old draft card to make trade decisions on draft day make little sense.

Expected career value of MLS College Draft/SuperDraft, valid every other year between 1998 and 2012 with data from draftees up to the previous four years.

Most of a draftee’s career value won’t be to drafting team

This was one finding that popped out at me late in the summer, and it was interesting to see it again when I created a Bayesian model with credible regions and all.  For the draftees taken in the first half of the SuperDraft, there was a significant difference between the expected career value of the Present model and the expected career value of the Club model — there was no overlap in the 95% credible regions of the two curves.  Furthermore, this difference in expected career value has persisted over the history of the league.  Except for some brief moments in the early years of the league, there is an overlap between the expected values of the Present and Club models, which indicates that almost all of the expected value of these late draftees will be with the drafted club.

Comparison of expected career value of draftees from Present and Club draft valuation models, valid for MLS College/SuperDrafts between 1997 and 2003.

Davy Arnaud and Kevin Hartman were the best draft steals ever

Another use of the draft pick value models is to retrospectively assess draft selections and practices.  Here we calculated the draft performance rating of the drafted players using Present or Club valuation models.  Regardless of whether you use the draft to get the best talent or the most compatible talent for the team, Davy Arnaud (5th round pick in 2002 SuperDraft) and Kevin Hartman (next to last pick in 1997 College Draft) were the biggest steals in MLS draft history with over 700 MLS regular season appearances between them.  Jeff Parke (the final pick in 2004 SuperDraft) came close, but Arnaud had almost 100 more appearances.

I’ll post the full list of draft selections and their draft performance ratings on my Project Data repository in the near future.

Nik Besagno is the worst draft selection ever, next to Ben Parry

When analysts and pundits draw up their lists of the worst MLS draft selections ever, Real Salt Lake’s Nik Besagno is almost always at the top of the list.  The selection of a US U-17 international as the first pick of a new MLS franchise (and first pick overall) was stunning at the time, and the subsequent lack of performance brought into question John Ellinger’s judgment.  The draft performance rating for this draft selection in 2005 confirms the popular observation of this pick.  (I shouldn’t be too hard on the guy; he appears to be a smart guy — a math/CS major! — and one who has moved on with his life.)

In my opinion, Ben Parry should get consideration for worst field player selection ever.  The third overall pick from San José Clash in 1997, Parry went straight to injured reserve and then to a USISL touring side called US Project-40, and then never returned to MLS again.  Parry’s draft performance rating (-5.1) puts him at the bottom of the draftee list.

Historically, LA Galaxy dominate the draft

Another use of the draft value model is that we can look at the collective and year-on-year performance of the draft war rooms at each MLS club.

If we look over the College Draft and SuperDraft from 1997 to 2013, and calculate the draft performance rating of all draft selections, we see a couple of interesting patterns.  First of all, if the goal of the drafts is to identify and select the talent with most upside in the league, the Galaxy dominate the draft by a wide margin:

Cumulative draft performance rating of all MLS teams in College/SuperDrafts between 1997-2013.

I find it fascinating that Miami Fusion — a team that was only in the league for four seasons — had a cumulative draft performance rating that was higher than Colorado Rapids, an original MLS team.  (The Rapids are almost certainly ahead on this score now.)

If one views the draft as an opportunity to select players who will make the most contributions for the team selecting him, then the cumulative performance rating is calculated for the Club model and looks like this:

Cumulative draft performance rating of all MLS teams participating in College/SuperDrafts between 1997-2013.

Now the situation changes.  LA Galaxy are still in the mix, but there are now three teams that, according to this model, are good at finding players who deliver big value for them: Chicago Fire, Kansas City Wizards, New England Revolution.  One can identify two or three other groups that describe degrees of success in the MLS draft system.

If one sets aside goalkeepers (the current valuation metric for goalkeepers dominates that for field players), the picture changes further.  When it comes to selecting field players that have high impact for the drafting team, Kansas City Wizards, New England Revolution, and Chicago Fire stand out.

Year by year, the MLS draft is noisy

If one drills down into year-on-year draft performance graphs, he/she will see a lot of noise and a few spikes where teams had exceptional success with that year’s draft.  The median draft performance rating over the history of the league is about 1.2, which usually translates to moderate success at the early draft positions.

Cumulative draft performance rating of clubs in MLS College and SuperDrafts (2007-2013), using a draft pick valuation model (Club). Red line represents median draft performance rating of 1.2.

There are some draft classes that stand out because of the value gained by the team relative to the draft position. They are:

• 2005 Chicago Fire (Gonzalo Segares, Chris Rolfe, Chad Barrett)
• 2002 New England Revolution (Taylor Twellman, Shalrie Joseph, Marshall Leonard)
• 2004 MetroStars (Jeff Parke, Seth Stammler, Zach Wells, Michael Bradley)

Some teams have good consecutive years at the draft, but for the most part that should not be expected, and it seems that in today’s complex player acquisition environment, it’s no longer assumed that the draft will fill all of the team’s needs.

There is a ton of additional insight that can be pulled from these results, but it’s a good idea to stop here.