Bill Gerrard, “Analysing Sporting Efficiency Using Standardised Win Cost: Evidence from the FA Premier League, 1995-2007”, International Journal of Sports Science and Coaching, 5(1): 13-35, 2010. [Citation]

Summary: This study proposes the use of standardised win cost as a simple and transparent measure of sporting efficiency in professional team sports. Standardised win cost measures the ratio of sporting performance (i.e., output) to financial expenditure (i.e. input). The specific version developed for league performance in football is the wage cost per league point standardized across seasons to allow for changes in the general level of player wages as well as league restructuring. The use of standardized win cost to analyze team performance and efficiency is illustrated with data from the FA Premier League between the 1995/96 and 2006/07 seasons.

I’ve been writing a lot about team efficiency in recent posts, and I’m sure that you’re starting to get more than a little sick of the subject.  For my part, I’m ready to move on to something else.  But before I do, I’d like to discuss the paper that informed a lot of my thinking about team efficiency and proved to be the foundation of my work – a publication by Bill Gerrard that was published a little over five years ago.

Bill Gerrard is a giant in the sports analytics field and his name can be found in the earliest posts of Soccermetrics.  He is Professor of Business and Sports Analytics at Leeds University Business School and has worked with the Oakland A’s in Major League Baseball, Leeds United, and Saracens in English rugby union.  He was the subject of an interview for the Soccermetrics Newsletter two years ago.

This particular paper follows on from a benchmarking and regression study that Gerrard conducted several years before (and which was the subject of a previous Paper Discussion), and is part of a broader study of organizational performance that considers the resources available to an organization, its deployment of those resources, and the efficiency of their utilization. One such tool for examining organization performance is ratio analysis, and in his earlier paper Gerrard develops marginal payroll cost per win in Major League Baseball above a certain pay/performance baseline.

So why did Gerrard opt to use a simpler analysis method instead of a relatively more sophisticated regression analysis?  The reason lay in the multivariate studies’ lack of a standard measure of efficiency.  In referencing a related study on the English Premier League, he states that “the measured efficiency of a football club in any particular season is sensitive to both
the model specification and the statistical methods employed.”  Therefore the objective of the current paper is to create a standardized metric that communicates sporting efficiency or win cost — the ratio between the total payroll of a club and its total points won in the competition — independent of nominal prices or changes in competition structure.

The first major section of the paper presents the theoretical background to win cost metric and its standardization process.  The major assumption is that there is a direct proportional relationship between performance \(P\) and resource input \(R\), or to put it in an equation:

\[ P = eR \]

where \(e\) is the efficiency measure which is better known as the performance-to-resource ratio.  This equation expresses the two paths to increased organization performance: either increase resource input, or increase efficiency.  Of course, it is possible to throw huge amounts of money at an organization without an improvement in results; just ask Toronto FC.  Gerrard also recognizes the nonlinear relationship between resource and performance at the extremes; however, he recommends that the departure from linearity be treated as an analysis issue while preserving the simplicity of the overall approach.

In practice, sporting efficiency is expressed as the inverse of \(e\), so that we discuss the ratio of player salaries (our resource) to league points (our performance).  This is the cost of winning:

\[ \mathcal{C}_{win} = \frac{1}{e} = \frac{S}{Pts} \]

Gerrard presents sporting efficiency with the goal of comparing the on-field and front-office performance of clubs over time.  It is only possible to do so if one adjusts for wage inflation and restructuring of the competition.  Gerrard’s research focuses on the period of English Premier League history in which the number of teams was kept constant, so he writes exclusively about inflation adjustment.  If you have read my posts on Front-Office Efficiency in which I discuss adjustments for inflation, it’s the approach that Gerrard presents in his paper.

The remainder of the paper presents an application of standardized win-cost to performance and salary data for the English Premier League between the 1995/96 and 2006/07 seasons.  The 1995/96 season is significant in that it is the first season in which the competition was fixed to the current 20 teams/38 match schedule.  In case you don’t already know, it is much more difficult, if not impossible, to obtain team payroll or individual salary information in European football.  Gerrard went around this restriction by accessing the total wage cost data in the annual financial reports of the football clubs (as compiled by Deloitte Annual Review of Football Finance).  Total wage costs include salaries paid to players, staff, and executives, but because player wages form a large majority of those costs, they are seen to be a good proxy for total payroll.

The analysis is descriptive in nature and is carried out at two levels.  The first is a season-level analysis of win cost and standardized win cost.  The second is an examination of standardized win cost using a cluster analysis of five performance groups, for which length of tenure in the Premier League is the criterion.  The teams in these clusters are the following (and keep in mind that these groups reflected the state of English football in 2006):

• Big Four (Tier 1): Arsenal, Chelsea, Liverpool, Manchester United
• Continuous participation in Premier League (Tier 2): Aston Villa, Everton, Newcastle United, Tottenham Hotspur
• In 75% of Premier League (Tier 3): Blackburn Rovers, Leeds United, Middlesbrough, Southampton, West Ham United
• In 50% of Premier League (Tier 4): Bolton Wanderers, Charlton Athletic, Coventry City, Derby County, Fulham, Leicester City, Manchester City, Sunderland
• Less than 25% of Premier League (Tier 5): Barnsley, Birmingham City, Bradford City, Crystal Palace, Ipswich Town, Norwich City, Nottingham Forest, Portsmouth, QPR, Reading, Sheffield United, Sheffield Wednesday, West Bromwich Albion, Wigan Athletic, Wimbledon, Wolverhampton Wanderers

These were the main results of the analysis:

•  It requires a constantly increasing amount of money to win points in the Premier League, but after adjusting for inflation the average win cost has stayed constant over the last twelve years.  Gerrard found that the average win cost rose almost 17% per annum, with some skewness present due to the large budgets of the major clubs.    Standardized win cost removes the inflationary trend, which is a good thing — one wants to use a metric that is unbiased and captures underlying behavior.
• The most efficient performances over the 12-year period were made mostly by newly promoted teams that avoided an immediate relegation.  These sides exhibited win costs 40-55% below the average win cost in the League in 1995/96.  There are some memorable overachieving sides in this group, such as Wimbledon in 1996/97 who reached the semifinals of the League Cup and the FA Cup before finishing eighth in the League, Ipswich Town who won promotion and finished 5th in 2000/01 (George Burley won Manager of the Season), and Bolton who finished sixth in 2004/05 and missed out on a Champions League qualifying place on goal difference.  Barnsley rates particular mention because their budget should have doomed them to an early relegation, yet they only succumbed after the penultimate round of matches.
• The most inefficient performances were made mostly by big-spending sides who finished below expectations.  There are a few teams that actually finished high in the table (Chelsea in 1999/00, 2003/04 and 2006/07, Liverpool in 1998/99).  In these cases, player payrolls were greater than those of the eventual champions, yet they finished at least five points behind.  In the case of Liverpool, they finished 25 points behind the champions Manchester United.  Other teams spent heavily on talent, such as Blackburn Rovers, Newcastle United, and Leeds United, yet have little to show for it.  The least efficient side was Sunderland in 2002/03, which isn’t too surprising as they are viewed as the worst team to ever appear in the Premier League.  Overall the win cost for these teams was 1.5 to 2.0 times the cost of winning for an average side in 1995/96.
• The Big Four teams are examples of “high cost, high performance” teams.  On average the Big Four sides spent 28% than the baseline league average in order to win a league point.  They also contended for Champions League places and finished on 74 points.  No other group had as much distance between themselves and adjacent groups as the Big Four group.
• The clubs that have continuously participated in the Premier League and those who have been in it for most of its history are direct rivals to each other.  At least in terms of sporting efficiency.  The best of the teams in Tier 2 are 25-30% more efficient than the baseline average, which corresponds to a top six finish over the period.  Everton finished in the top four twice, and so did Aston Villa during the period.
• Newcastle United spent a huge amount of money on players in the mid-1990s and mid-2000s.  Premier League fans of a certain age will remember that 1995/96 Newcastle United team with David Ginola, Warren Barton, Les Ferdinand, and Faustino Asprilla who led for most of the season only to fade late to Manchester United.  Newcastle then went out and paid £15 million for Alan Shearer (a world record fee at the time), yet aside from 1996/97, they never approached the highs of that 95/96 team.  If you look at standardized efficiency for Tier 2 teams, the most inefficient teams were Newcastle United teams at the turn of the century.
• It is possible to be inefficient with a small budget.  This could be a feature of the Premier League since it doesn’t have a salary cap and therefore permits greater stratification of squad payrolls.  But consider Sunderland in the 2002/03 season.  They won just four games on their way to 19 league points, one of the worst performance by a Premier League side.  They did have a low payroll, so a finish in the lower half of the table would not have been unexpected, but the team still had international veterans like Claudio Reyna, Phil Babb, and Tore André Flo, as well as club goalscoring legend Kevin Phillips. Little wonder that their efficiency was 2.5 times worse than the baseline average.

Gerrard’s analysis is great (like he needs me to tell him that), but it is just a snapshot in time, and it only concerns a European football league that is operated differently from a North American or Australian sports league.  Gerrard provides some directions for future research, and I have some others that come to mind:

• For those teams that do operate at the extremes, is it possible to identify those characteristics that cause differences in the win cost?  Gerrard recommends a regression or frontier analysis as part of a broader case study (not too different from the Managing Manchester United case study at Harvard Business School).
• How do squad size, rotation, and injuries affect win cost?  I’ve done work on player utilization which takes into account these factors, but it’s much easier to perform such calculations for league competitions in which all player salaries are publicly available.
• How can capital, development, and acquisition costs be incorporated into win cost?
• There has been some movement in and out of these Tiers since the 2006-07 season — Newcastle United has been relegated, Leeds United has yet to return to the Premiership, and Manchester City has returned to the Premiership with foreign ownership and the billions that come along with it.  How would these Tiers be organized now?

This review is reaching the 2,000-word frontier, so I will stop here.  Go read the paper; it is a simple analysis for sure, but it’s analysis that motivates further analysis.