Evaluation of the Model: Thrashed in R6 of the Champions League

For the 6th round of the Champions League we published two sets of forecasts:

  1. The Scorecasting Model.
  2. Jean-Louis Foulley’s forecasts, the method of which is described here.

Based on just 16 matches we can provide a rough evaluation of these two methods.

For the most likely result of matches, the Scorecasting Model forecast 7 correct. Foulley’s model however had the most likely result agreeing with the actual outcome in 8 matches.

For the most likely scoreline, the Model got one correct. However, Foulley’s model got 4 correct scorelines!

If we just take the scoreline forecast of each model as a single point forecast, and then award 40 points for a correct scoreline and 10 points for a correct result only, then the Model would score 100 points from the 16 matches. On the other hand, Foulley’s model scores 220 points.

Foulley’s forecasts for past Champions League rounds are also published here.

 

 

Advertisements

Women’s Super League, R12 (9 Dec)

Our WSL and WC forecasts for R12 are in the the table below:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw
Forecast Win (%)
Score Pr (%) P(H) P(A)
WSL
Brighton Chelsea 0-3 14 7 85
Everton Bristol 1-0 11 41 38
Man City Birmingham 1-0 23 76 12
Reading Liverpool 2-0 12 71 15
Yeovil West Ham 0-1 16 11 78
WC
Aston Villa Sheffield 0-1 9 23 60
Durham Manchester Utd 0-2 20 28 53
Lewes Charlton 1-2 9 26 55
London Bees Leicester 1-2 6 35 45
Spurs Crystal Palace 2-1 10 71 15

 

Championship, R20 (7-8 Dec, 2018)

After 7 years in the Premier League, you would think Swansea would be well-placed to bounce straight back after relegation. But currently they sit in the bottom half of the table (13th) and have only won 7 of their first 20 games. The Swans travel to Brentford this weekend, who are perhaps similarly under-performing compared to pre-season expectations. The Model suggests Swansea could slip even further down the table, with a 1-0 (15%) defeat on the cards.

Our local club, Reading F.C., sit perched above the drop zone as we head towards the mid-point of the season. They welcome Sheffield Utd to the Madejski tomorrow evening, who themselves are on a dip in form. The Model forecasts an even game, and 1-1 (12%) is the most likely outcome.

The rest of our Championship forecasts are in the the table below:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw
Forecast Win (%)
Score Pr (%) P(H) P(A)
West Brom Aston Villa 2-1 10 43 29
Birmingham Bristol City 1-0 10 49 24
Brentford Swansea 1-0 15 45 27
Leeds QPR 1-0 12 53 21
Middlesbrough Blackburn 1-0 16 53 21
Millwall Hull 1-0 11 43 28
Norwich Bolton 1-0 20 66 13
Notts Forest Preston 1-0 15 49 24
Reading Sheff Utd 1-1 12 35 36
Sheffield Wed Rotherham 2-0 11 42 29
Stoke Ipswich 1-0 19 59 17
Wigan Derby 0-1 11 32 39

 

Premier League, R16 (8-10 Dec)

Claudio Ranieri has vowed to “tinker clever” in his endeavours to keep Fulham up. He must be doing something right as the Cottagers have taken 4 points from his first 3 games in charge. Fulham face Man Utd on Saturday. The Model however has Utd to win comfortably 3-0 (9%).

The game of the round is undoubtedly at Stamford Bridge between Chelsea and Man City. If City take 3 points form these kinds of games, then there is surely no stopping them taking the title. Both sides rested players in midweek. City scraped a win at Watford, but Chelsea slipped up against Wolves. The Model suggests City will edge this one 0-1 (14%).

The table below gives the rest of the Model’s forecasts for Round 16 of the Premier League:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw
Forecast Win (%)
Score Pr (%) P(H) P(A)
Arsenal Huddersfield 2-0 12 67 13
Bournemouth Liverpool 0-2 12 30 41
Burnley Brighton 1-1 9 34 37
Cardiff Southampton 1-1 13 35 36
Chelsea Manchester City 0-1 14 31 40
Leicester Spurs 1-2 9 31 40
Manchester Utd Fulham 3-0 9 63 15
West Ham Crystal Palace 1-0 11 44 28
Newcastle Wolves 1-0 17 44 28
Everton Watford 1-0 20 50 24

 

Evaluation – Quick Update – “The Model vs Lawro vs Merse” – Model takes the Lead

The featured image (and below) plots the cumulative forecasting performance of “The Model vs Lawro vs Merse” since round 10  in the Premier League, and as of the midweek fixtures just gone, according to the BBC Sport scoring metric (40 points for a perfect scoreline forecast, 10 points for a correct result only).

The Model got off to a good start, but by round 5 tipster Merson had overtaken the Model, and by round 9 tipster Lawro was getting close (see here).

But now the Model has taken the lead again, overhauling Merson and pulling away form Lawro – and we wouldn’t bet against that trend continuing till the end of the season!

fcast

 

Premier League, R15 (4-5 Dec)

It is Man U v Arsenal tomorrow night. Throughout the Premier League era, surely this has been the most exciting fixture of the season, not least due to the League’s two greatest managers, who had an intense but (usually) respectful rivalry. Utd’s current manager is in some bother, describing his star player recently as a “virus”. After seemingly arresting Utd’s decline last season, the lack of return from the transfer market and poor results this season puts Mourinho in danger of spending the whole Christmas period at home with his family. The Model thinks matters won’t get any easier for Mourinho anytime soon, and forecasts a 1-1 (10%) draw, with Utd slightly ahead in terms of chances of winning the game outright (39%).

Southampton have just sacked their manager – who apparently has been “beleaguered” for a some time, across several clubs and decades. Instead of opting for the sort of safe pair of hands which kept them up last season, Saints look set to appoint somebody who has never managed in England as their new manager. It seems unlikely that Southampton will get an immediate post-sacking (reversion to the mean) bounce on Wednesday night, as they travel to Wembley to take on Spurs. The Model has this one down as a 2-0 (17%) win to Spurs.

The table below gives the rest of the Model’s forecasts for Round 15 of the Premier League:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw

 

Forecast Win (%)
Score Pr (%) P(H) P(A)
Bournemouth Huddersfield 2-1 10 57 19
Brighton Crystal Palace 1-0 10 41 30
Watford Manchester City 0-2 12 20 55
West Ham Cardiff 2-1 10 53 21
Burnley Liverpool 0-1 19 20 55
Everton Newcastle 1-0 18 52 22
Fulham Leicester 1-2 8 32 39
Manchester Utd Arsenal 1-1 10 39 33
Spurs Southampton 2-0 17 58 18
Wolves Chelsea 0-1 18 24 49

Which English Football League is the most “predictable” in 2018/19?

After watching their defender boot the ball out of the stadium, or the centre forward hit the corner flag with a shot from the edge of the box, lower league football fans can reassure themselves that the reason they turn up at dilapidated stadiums each week instead of the Emirates or Stamford Bridge really is because the Premier League is just too “predictable” (boring).

At least, that is what an unqualified comparison of the Model’s forecasts and actual outcomes so far this season suggests.

[There are many better ways to answer this question, e.g using a range of forecasts including bookmaker odds, or using the forecast density of all possible outcomes rather than just point forecasts. And not least because we have in fact tinkered with our Model design all season (probably making it worse) But you would probably just get a similar result anyhow… let us know if you don’t!]

The table below compares the Model’s forecast performance across all the English Football Leagues and Women’s Super League so far this season (up to date as of 12 noon, 2nd December).

The Model has forecast correctly 53% of results in the Premier League, the third highest of all six leagues looked at, only bettered by the relatively more “predictable” Women’s Super League and Women’s Championship. The worst forecasting performance by the Model has been in League One with just 40% of results correct, closely followed by the Championship with 41%.

In terms of exact scorelines, the Premier League is the most “predictable” in England, with 13% of match scores being forecast exactly right by the Model. Despite being the most predictable in terms of results, the Women’s Super League is least predictable in terms of exact scorelines, probably explained by the high variance of goals scored across matches.

In terms of average “Lawro” points per game, which in effect put weights on getting the correct result relative to also getting the correct scoreline in any match with a single forecast, the Premier League is comfortably the most “predictable”, with on average 9.5 points per game achieved by the Model so far this season (note, tipsters Mark Lawrenson and Paul Merson perform similarly highly on this metric at 8.1 and 9.6 points per game, respectively).

Scores (%) Results (%) “Lawro” p.p.g. # matches
Premier League 13% 53% 9.5 137
Championship 11% 41% 7.8 238
League One 5% 40% 5.6 237
League Two 12% 44% 8.1 237
Women’s SL 4% 73% 8.6 49
Women’s Champ 8% 56% 7.5 48