WHAT CAN MOSHBODS’ TEAM RATINGS AND VeNUE PERFORMANCE VALUES TELL US ABOUT TEAMS’ CHANCES oF WInning HOME-AND-AWAY GAMES ACROSS V/AFL HISTORY?

For this analysis we'll ask a simple question: how do teams' home-and-away game winning rates vary with their MoSHBODS Offensive and Defensive Ratings and their Venue Performance Values, and how has the relationship between victory and these values varied from era to era? 

THE DATA

We have home-and-away game-by-game MoSHBODS data and game outcomes for the entire history of the V/AFL, which we'll split up into 5-season eras (including a slightly longer inital 1897 to 1904 era) with a view to ultimately taking the MoSHBODS pre-game Offensive and Defensive Ratings and Venue Performance Values of every team in every game and then fitting a binary logit to the outcomes from the home team’s point of view for each 5-season era.

(Note, firstly, that determining home team status can be a vexed issue in some games, for example those played overseas or those where a team sells a home game and so ends up playing at an away venue for a designated home game. For the most part, I believe I’ve used the AFL’s designation to determine home status, which will be laughably wrong in some instances, but is an issue deserving of further analysis on another day and is unlikely to materially distort the results here.

Also note that, because I’m fitting binary logits, draws are problematic, so those games are excluded from the analysis).

For the purposes of model-fitting and performance analysis, each era’s data will be randomly split 50:50 into a training and a testing set.

THE MODEL

We could fit a variety of classification models to the home win / home loss data we have, but for now we’ll settle for:

  • glm(Home Result ~ Home Offensive Rating + Home Defensive Rating + Away Offensive Rating + Away Defensive Rating + Home Venue Performance Value + Away Venue Performance Value, family = “binomial”)

To ascertain the relative importance of each input in estimating home teams’ winning chances in a particular era, we’ll use the varImp function from the R caret package on the model for that era, and standardise the importance values such that the most important variable always has an importance of 1. (I’ve never been able to determine if the absolute magnitudes of importance values carry any useful information.)

To measure the performance of the models we’ll use the following measures from the MLmetrics package:

  • Accuracy - % of home and away winners correctly predicted. Higher is better.

  • AUC - measure of a model's ability to differentiate between home wins and away wins. Represents the probability that the model, if given a randomly chosen positive and negative example, will rank the positive higher than the negative. Higher is better.

  • LogLoss - mean value of the sum of -log(probability attached to winning team [base 2]). Lower is better.

  • F1 Score - 2 x Correctly predicted away wins/(2 x Correctly predicted away wins + Wrongly predicted away wins + Wrongly predicted home wins) = 2 x (Precision x Recall)/(Precision + Recall). Considered a single measure of model performance for the away win class (see this link). Higher is better.

  • Precision (also Positive Predictive Value) = proportion of predictions of away team win that are correct. Higher is better.

  • Specificity - proportion of all home wins correctly predicted. Higher is better.

  • Sensitivity (also Recall) - proportion of all away wins correctly predicted. Higher is better.

We apply these metrics to the predictions for the training and test samples, and use any significant difference between the same metric when used for the training versus test sample as an indicator of potential overfitting, and the results for the test sample alone as an indication of the general quality of a given model for an era.

RESULTS

The results of the analysis are summarised in the table below (which can be clicked to access a larger version)

Observations on the data

We have at least 260 games in every defined era, which means at least 130 for model-fitting, so we should have acceptably precise estimates of model coefficients and derived values

Home teams have won between 55% and 63% of all decided games in all eras. It was at 61% in the 1995-1999 era and has fallen or remained constant in the five eras since, so much so that we might be heading for an all-time low.

Observations of model performance

Comparing firstly the performance metrics when measured in-sample versus on the test set, we see no obvious signs of massive overfitting in any era and certainly not overall. In general though, as we’d expect, the performance metrics are less-impressive for the test data than they are for the training data, especially in the 2005 to 2019 period.

The models seem to particularly struggle with forecasting away team wins in the test data (relative to how well they do on the training data for the same period) in this period. They are similarly affected in the 1915-1919 and 1940-1944 eras. That could be because of a flaw in MoSHBODS, a reflection of the true difficulty of forecating away wins for the particular games in the test set in those eras, or a combination of both.

Overall, I think it’s fair to conclude that the models generally do well enough on test data that we can take at least some notice of them when it comes to which variables best predict home wins.

Observations of variable performance

One of the most striking features of the table is the Home Team Offensive Rating is the most important variable in only the 1965 to 1969 era. Of the four non-VPV variables it ranks 4th in 15 of the 25 era and, averaged across all 25 eras, it ranks 4th.

The three team rating-related inputs other than Home Team Offensive Rating vary in importance from era to era, but overall wind up being about equally important across the entire history of the sport. That said Away Team Offensive Rating is ranked 1st in 11 of the 25 eras and either of it or HomeTeam Defensive Rating has been most important variable for all but six eras.

Interestingly, Venue Performance Values (which are just a logical extension of measures of Home Ground Advantage) are only, on average, about one-third as important as Away Team Offensive Rating.

IMPLICATIONS AND FINAL THOUGHTS

The first thing to stress is that there is no guarantee that the findings here reflect the reality of the V/AFL. They will only do so to the extent that the MoSHBODS System appropriately models all points in V/AFL history.

If we take that as given, however, for our current purposes, then a clear finding of the analysis here is that home teams’ offensive abilities have played a substantially smaller role in their success or failure than have home teams’ defensive abilities and away teams offensive and defensive abilities across the majority of eras in V/AFL football. That said, Home Team Offensive Ratings were second only to Away Team Defensive Ratings, and virtually as important as them, in the very latest era, which is the first time this has been the case since the 1980-1984 era.

The other significant finding, I’d suggest, is that the impact of game venuing is somewhat smaller than I’d have thought pre-analysis. That finding too though, of course, is dependant on the adequacy of the MoSHBODS System.