ter Weel, B., "Does Manager Turnover Improve Firm Performance? Evidence from Dutch Soccer, 1996-2004", De Economist, 2001, to appear. [Link]

Heuer A, Müller C, Rubner O, Hagemann N, Strauss B "Usefulness of Dismissing and Changing the Coach in Professional Soccer", PLoS ONE, 6(3): e17664, 2011. [Link]

These two papers examine the impact of manager turnover on club performance — ter Weel studies the Dutch Eredivisie, while Heuer et al. focus on the German Bundesliga.  This kind of study has applicability to research on manager performance and firm outcomes, but sports data are more readily available and observable than business data.  Both sets of research indicate that manager turnover has little effect of team performance beyond a very short-term improvement.  ter Weel's results indicate that manager quality has very little impact in manager turnover, but tenure and involvement in the transfer market do.

Review after the jump.

———–

[I found out after writing this post that Chris Anderson had already written about the ter Weel paper four months ago!  You can find his articles here and here.  Ah well…what I say isn't all that different, but keep reading if you're interested.]

Two papers provide some up-to-date analysis of the effects of manager turnover on a football team's performance, and the results are similar to that of previous research: mid-season manager turnover has little to no effect on team performance.  The first paper is written by an economist with the Dutch Bureau for Economic Research; a group of professors from the Universities of Münster and Kassel in Germany authored the second.  Jay Williams at the Science of Soccer website summarized the second paper very well.  I will take a look at the first and discuss the similarities and differences of both groups' research.

Bas ter Weel's study focuses on the Dutch Eredivisie between the 1986-2004 seasons and considers 184 managerial turnovers among 19 teams who have been in the Eredivisie for nine seasons or more.  Of these turnovers, 81 of them are forced (firings, forced resignations), while the remainder involve voluntary separations due to end of contract, retirement, or a change in employment.  In an average season, half of the teams in the top flight replace managers, and half of those teams do so via forced separations.  To evaluate the impact of manager turnover on team performance, ter Weel introduces a 4-match moving average of league points earned per game as a performance metric.  This metric produces a smoother and less volatile time-history than points per game.  To obtain a relative measure manager performance, ter Weel uses the average point per game of a typical manager, which I infer to be the mean of all of the moving averages for all Eredivisie teams. He then compares team performance in four-match intervals before and after a separation event, between clubs that experienced a decline in performance and then dismissed their managers go and those who were also experiencing a decline in performance but kept their managers in their posts.

ter Weel uses two statistical analysis techniques, commonly used in econometrics, in his study.  The first is called difference-in-difference estimation, and it assesses the impact of a manager change on team performance (the difference between performance before and after the switch) relative to a control group that did not experience a manager change.  The second is two-stage least-squares regression, or 2SLS regression, which calculates the regressor coefficients in two distinct steps thus reducing the biasing of ordinary least squares.  This technique is used to establish causal relationships between treatments (in this situation, manager characteristics and other factors such as contract time remaining) and the outcome (in this situation team performance).

Here is an outline of some of the main findings:

• Team performance is time-invariant.  ter Weel plotted his performance metric for the Eredivisie teams over multiple seasons, and he said that the most surprising result was that average points won per game, as a moving average, remains relatively constant.  Despite player and manager turnover this appears to be the case.  It indicates that club management tend to recruit managers with similar styles of those recently departed.  It must be emphasized that this observation does not include yo-yo teams or those who only appeared in the Eredivisie for one or two seasons.
• Manager turnover has a very short-term effect.  One match, in fact.  ter Weel's results showed that for teams whose managers have replaced a fired one, there is an increase in performance immediately afterward — the shock effect — but it regresses back to pre-sacking levels by the following game.  
• The ability to turn around a team has nothing to do with manager's experience or playing career. This sounds counterintuitive to people but there are a number of highly successful managers who were not successful football players in their time (José Mourinho immediately comes to mind). When ter Weel isolates for manager experience, expressed as years of service, or his tenure as a professional player or even a national team player, he finds that the performance difference is nonexistent.  
• Time served on a contract and the number of players recruited do influence a team's willingness to fire its manager.  This works in different ways.  ter Weel found that teams whose managers have been actively involved in the transfer market are more likely to fire them when results go bad. Those managers are under more pressure to perform because of the club's financial committment.  On the other hand, it was found that the longer the time remaining on a manager's contract, the less likely a club is willing to sack him if results go bad because of the amount of money invested.  This might seem contradictory, but it is possible to recoup the investment made in a player by selling his contract.  Such a situation does not exist for managers.
• Expectations of team performance weigh heavily in a club's decision to sack a manager.  ter Weel extended his regression analysis with variables for the average team performance in a league and the difference between actual team performance and average performance over the previous three seasons.  He found that a significant drop in performance relative to past performances increased the probability of a turnover.

If you compare the two papers, the German study used goal difference as a metric of team performance, and obtained very similar results.  They found that team performance remains somewhat constant across all teams, and that manager changes result in little change in the team beyond the first match after the switch (and possibly the second). 

What both papers indicate is that in-season changes in the club manager are not very likely to cause any change in the team results.  Any positive changes in the results will be short-lived, and in the medium-term will be washed away by the ebb and flow of the season.  The only situation where a managerial change might be beneficial would be in a club fighting against relegation, as a 3-5% change in performance — one to two points essentially — could make the difference between staying up and dropping down to a lower division.  These two papers illustrate (1) the need for managerial analytics and (2) the need for clubs to develop a systematic plan for club performance independent of the manager.