MatterOfStats in 2016: Tipsters and Predictors

/

Despite what the completely fictional critics, made up only for the purposes of this lead-in paragraph were saying, MatterOfStats will be returning for the 2016 AFL season.

Not without changes, however, many of them foreshadowed in the preceding blog post. Most dramatic, I'd suggest, is the overall reduction in the number of Tipsters and Predictors, MoS now content to get by with just nine Head-to-Head Tipsters, 10 Margin Predictors, and five Head-to-Head Probability Predictors. MoS will still also have a wagering oar in the Line Market pond, but this season I'll not be presenting probability assessments for that Market.

(If you're new to MatterOfStats, this blog post from last year might provide some useful background MoS history for you.)

To some details then.

HEAD-TO-HEAD TIPSTERS

This year sees the end of two dozen of the Head-to-Head Tipsters from 2015, Combo_NN1 and Combo_NN2, Combo_7, the three ProPred, three WinPred, and two H2H based Tipsters, and all of the Heuristic Tipsters bar Home Sweet Home, Bookie Knows Best, and Consult The Ladder. These three have been retained solely because of their intuitive nature, which might most simply be described as Pick the Home Team, Pick the Favourite, and Pick the Team Highest on the Competition Ladder. 

Amongst the retired Tipsters, a performance-based case for retention - looking at both Head-to-Head tipping and Margin prediction - could only seriously be made for Combo_7. It finished 5th amongst the Margin Predictors in 2015 and 1st in 2014, and also came 2nd amongst the Head-to-Head Tipsters in both 2014 and 2015. Retention of Combo_7, however, would require the continued use of the H2H_Adj algorithm, which seems to me too high a price to pay when my broad goal for the season is to slim down the number of forecasters and to reduce the effort associated with the production of their forecasts. At some point during the season I might take a look at how a Combo_7 would have been faring.

Anyway, the nine Head-to-Head Tipsters for 2016 will be:

  • Consult The Ladder (CTL), which predicts that the team higher on the ladder will win, and which will use the final 2015 ladder for making its Round 1 predictions.
  • Bookie Knows Best (BKB), which predicts that the team made favourite by the TAB Bookmaker will win, using CTL's prediction in the event that equal-favourites are installed.
  • Home Sweet Home (HSH), which predicts that the AFL-designated home team will win.
  • RSMP_Simple (RSMP_S), an ensemble predictor, which makes its predictions on the basis of a simple averaging of the opinions of 11 underlying base learners (see this blog for details). 
  • RSMP_Weighted (RSMP_W), another ensemble predictor, which makes its predictions on the basis of the weighted opinions of six of those same 11 underlying base learners (the same blog has the specifics).
  • ENS_Linear (ENS_L), also an ensemble predictor, but a much more complex one, it forming its opinions by a simple averaging of the outputs of six different statistical algorithms (see this blog for details).
  • ENS_Greedy (ENS_G), the last of the ensemble predictors, it forming its opinions by applying different weights to the outputs of those same six different statistical algorithms (the weights are given here).
  • C_Marg, which is based on the margin predictions created using the ChiPS Team Rating System (the genesis of which is described in this blog post).
  • MoSSBODS_Marg, which is based on the margin predictions created using the MoSSBODS Team Rating System (the latest version of which is described in this blog post and the genesis of which is described in this earlier blog post).

No changes have been made to the underlying algorithms, weights or other parameters for the two RSMP or the two ENS Predictors, so to the extent they might have learned from the 2015 season, such opportunity has been denied them. In truth, I doubt the competition has changed so fundamentally in the last season or two that this amounts to statistical malfeasance.

C_Marg has not been subjected to the same privations, however, it having been permitted to review the Home Ground Advantages (HGAs) it uses for all common team and venue pairings in light of 2015 experience. I'll provide these new HGAs in a subsequent blog post. Motivation for updating them sprang from a nagging suspicion that they'd been derived in a manner that risked overfitting.

Of the remaining Predictors, MoSSBODS, having only recently been created and using, as it does, the entire history of the sport, is already as informed as it could possibly be, while CTL, BKB and HSH remain resolutely unoptimisable given their one-dimensional nature. 

MARGIN PREDICTORS

Like the Head-to-Head Tipsters, the Margin Predictors have been subjected to similar austerity measures for 2016, last year's 19 competitors having been slimmed to just nine.

Eleven Predictors were culled, the list of deletions similar to that for the Head-to-Head Tipsters, specifically Combo_NN1 and Combo_NN2, Combo_7, the two ProPred, two WinPred, and the four H2H based Predictors.

The nine remaining Predictors include eight returning from last season. Five of those eight are also used as Head-to-Head Tipsters on the basis of the sign of their margin prediction (details for which can be accessed via the links already provided):

  • RSMP_Simple
  • RSMP_Weighted
  • ENS_Linear
  • ENS_Greedy
  • C_Marg

The four other Margin Predictors are:

  • Bookie_LPSO, which uses the TAB Bookmaker's Head-to-Head prices to come up with a margin forecast (see this blog post for the derivation). For all the grandiosity at which its name hints, Bookie_LPSO simply sets its Home Team probability prediction (Prob) equal to 1/Home Team Price - 1.0281%, then converts this probability to a margin prediction by calculating 19.83 x ln(Prob/(1-Prob)).
  • Bookie_3, which also uses the TAB Bookmaker's Head-to-Head prices to come up with a margin forecast. It was created a few years back using the Eureqa modelling tool, and, like Bookie_LPSO, derives its margin prediction by first estimating a home team victory probability. It does this via the overround-equalising method and then converts the resulting probability into a margin prediction by calculating 25.72 x ln(Prob/(1-Prob)).
  • Bookie_9, which is similar in approach to Bookie_3, including in the manner of its estimation of the home team victory probability via the overround-equalising method. It has a more complex equation for converting that probability into a margin prediction, however, it using 2.22+17.73 x ln(Prob/(1-Prob)) + 2*Prob.
  • MoSSBODS_Marg, which, as described earlier, is based on the latest version of the MoSSBODS Team Rating System. For details, please follow the link already provided in the Head-to-Head Tipsters section.

None of the algorithms used for margin prediction (except C_Marg, as described earlier) has been subject to any tuning for the 2016 season.

HEAD-TO-HEAD PROBABILITY PREDICTORS

Last season, eight Head-to-Head Probability Predictors provided their weekly opinions, only the best four of which will be permitted to do the same in 2016.

They are:

  • Bookie Overround-Equalising (Bookie-OE), which derives its home team probability predictions from TAB head-to-head prices as Away Team Price / (Away Team Price + Home Team Price). This is the overround-equalising method alluded to earlier.
  • Bookie Risk-Equalising (Bookie-RE), which derives its home team probability predictions from TAB head-to-head prices as 1/Home Team Price - (1/Away Team Price + 1/Home Team Price - 1)/2 (the rationale for this predictor is provided in this blog post). 
  • Bookie Log Probability Score Optimsing (Bookie-LPSO), which derives its home team probability predictions from TAB head-to-head prices as 1/Home Team Price - 1.0281% (refer to the Bookie_LPSO link provided earlier for details of its derivation).
  • C_Prob, which is based on the margin predictions created using the ChiPS Team Rating System (see the C_Marg link provided earlier for details), converted to home team probability estimates by assuming that margins are distributed as random Normal variables with mean equal to the predicted margin and a standard deviation of about 36.5 points. 

These four Predictors will be supplemented with one further Predictor:

  • MoSSBODS_Prob, which is based on the margin predictions created using the MoSSBODS Team Rating System (see MoSSBODS_Marg link provided earlier for details), converted to home team probability estimates via the equation 1/(1+exp(-Expected Margin^0.184)). Note that the Expected Margin here is measured in Scoring Shots.

OTHER PREDICTIONS

The MoSSBODS Team Rating System, as well as providing, Head-to-Head, Margin and Head-to-Head Probability assessments, will also be used to predict the Home team, Away team and Total scores in each game, this latter prediction forming the basis for MatterOfStats' first foray into the Over/Under wagering market.

In fact, MoSSBODS will power all three of the MoS wagering Funds this year, which will comprise a Head-to-Head, Line and an Over/Under Fund. Information about the wagering and staking strategies for these Funds will appear in a future post.

SUMMARY

In short, MatterOfStats will be a little less cluttered this year and will only include (with the exceptions of CTL and HSH amongst the Head-to-Head Predictors) forecasters with genuine potential to finish best-in-class. It's something of a wrench to let go of the Heuristic Tipsters in particular, but their performance dropped off dramatically coincident with the introduction of byes. I might some day look at better ways of handling byes in the context of this class of Tipster.

The algorithms that remain encapsulate much of the learning that 10 years of statistical modelling - of data sets generally, but of AFL in particular - have afforded me. I hope the performance of these models ultimately reflects that.