Evaluation of the Model: Thrashed in R6 of the Champions League

For the 6th round of the Champions League we published two sets of forecasts:

  1. The Scorecasting Model.
  2. Jean-Louis Foulley’s forecasts, the method of which is described here.

Based on just 16 matches we can provide a rough evaluation of these two methods.

For the most likely result of matches, the Scorecasting Model forecast 7 correct. Foulley’s model however had the most likely result agreeing with the actual outcome in 8 matches.

For the most likely scoreline, the Model got one correct. However, Foulley’s model got 4 correct scorelines!

If we just take the scoreline forecast of each model as a single point forecast, and then award 40 points for a correct scoreline and 10 points for a correct result only, then the Model would score 100 points from the 16 matches. On the other hand, Foulley’s model scores 220 points.

Foulley’s forecasts for past Champions League rounds are also published here.

 

 

Advertisements

Champions League R6 (11-12 Dec)

It’s crunch time in the Champions League – the final set of group matches. Who will progress? Who will slip into the Europa League? This week, in addition to our usual forecasts, we also list the outcome forecasts created by Jean-Louis Foulley – see below. These are forecasts posted here, and Jean-Louis has also written an important paper on the ways in which we might penalise scoreline forecasts – like the ones we create.

Tottenham look set to drop out on Tuesday night, losing 2-0 to Barcelona (11%), while Liverpool look set to progress, beating Napoli 3-0 (9%).

Forecast Win (%)
Tuesday Score PR(%) P(H) P(A)
Barcelona Tottenham 2-0 11% 48% 26%
Brugge Atl Madrid 0-1 21% 22% 54%
Galatasaray Porto 0-1 23% 24% 51%
Inter PSV 1-0 20% 48% 26%
Liverpool Napoli 3-0 9% 58% 19%
Monaco Dortmund 1-2 9% 22% 54%
Red Star Paris St-G. 0-2 10% 29% 45%
Schalke L Moscow 2-0 15% 56% 21%

On Wednesday, in the only group where qualification hasn’t already been resolved, Shaktar will beat Lyon into the knock-out stages…

Forecast Win (%)
Wednesday Score PR(%) P(H) P(A)
Ajax B Munich 1-0 10% 41% 32%
Benfica AEK 1-0 18% 57% 20%
Man City Hoffenheim 3-1 8% 78% 8%
R Madrid CSKA 3-0 15% 68% 13%
Shakhtar Lyon 1-0 10% 49% 25%
Valencia Man Utd 0-1 23% 28% 45%
Plzen Roma 0-3 10% 36% 37%
Young Boys Juventus 0-3 9% 26% 48%

Foulley’s forecasts:

We’re going head to head against Foulley’s model, so here are his outcome probabilities, including the most likely socreline. For the forecast scoreline, we disagree on three matches (Ajax vs Bayern, Shakhtar vs Lyon, & Valencia vs Man U), and on all other matches, we are more conservative in our estimated probabilities. We will continue to post these comparisons, as with probabilistic forecasts, it is fairly meaningless to compare a small number of probabilities as those listed here.

Foulley (Bayesian) forecasts Win %
Score PR(%) P(H) P(A)
Barcelona Tottenham 2-0 10% 69% 13%
Brugge Atl Madrid 0-1 12% 17% 62%
Galatasaray Porto 0-1 13% 20% 56%
Inter PSV 1-0 12% 61% 17%
Liverpool Napoli 1-0 11% 53% 23%
Monaco Dortmund 0-1 13% 18% 59%
Red Star Paris St-G. 0-2 12% 11% 73%
Schalke L Moscow 1-0 13% 64% 15%
Ajax B Munich 1-1 12% 30% 44%
Benfica AEK 1-0 12% 68% 13%
Man City Hoffenheim 2-0 10% 84% 6%
R Madrid CSKA 2-0 13% 83% 5%
Shakhtar Lyon 1-1 11% 48% 29%
Valencia Man Utd 1-0 11% 37% 35%
Plzen Roma 0-2 10% 18% 61%
Young Boys Juventus 0-2 13% 12% 70%

 

Model vs the Experts (R16)

As usual, Lawro and Merson have got their forecasts in. We’ve been doing well so far this season against the experts (if not our own challengers), let’s see if we can keep it up…

Model Lawro Merson
Arsenal Huddersfield 2-0 3-0 3-0
Bournemouth Liverpool 0-2 0-2 1-3
Burnley Brighton 1-1 2-0 1-1
Cardiff Southampton 1-1 2-1 2-0
Chelsea Man City 0-1 0-2 1-2
Leicester Tottenham 1-2 0-2 1-3
Man Utd Fulham 3-0 2-0 2-0
West Ham C Palace 1-0 2-0 2-0
Newcastle Wolves 1-0 1-1 2-2
Everton Watford 1-0 2-0 2-1

Women’s Super League, R12 (9 Dec)

Our WSL and WC forecasts for R12 are in the the table below:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw
Forecast Win (%)
Score Pr (%) P(H) P(A)
WSL
Brighton Chelsea 0-3 14 7 85
Everton Bristol 1-0 11 41 38
Man City Birmingham 1-0 23 76 12
Reading Liverpool 2-0 12 71 15
Yeovil West Ham 0-1 16 11 78
WC
Aston Villa Sheffield 0-1 9 23 60
Durham Manchester Utd 0-2 20 28 53
Lewes Charlton 1-2 9 26 55
London Bees Leicester 1-2 6 35 45
Spurs Crystal Palace 2-1 10 71 15

 

Lower Leagues, R21 (Dec 8)

There have been 31 goalless draws so far in League One and League Two. Indeed we Scorecasting Economists saw one of them earlier in the season. Will there be any more this weekend? If The Model is to be believed, then no.

The eye-catching match in League One, surely, is Sunderland’s visit to Accrington Stanley. This is the first meeting between the two clubs for a mere 124 years, and the first league meeting for 126 years. When they met back in 1892, Sunderland routed Stanley 6-0. The Model thinks it’ll only be 1-0 to Sunderland tomorrow (10%).

Forecast Win (%)
League One Score PR(%) P(H) P(A)
Accrington Sunderland 0-1 10% 28% 44%
AFC W Rochdale 2-1 8% 45% 27%
Blackpool Charlton 1-0 10% 60% 17%
Bristol R Doncaster 0-1 9% 22% 52%
Burton Shrewsbury 1-2 5% 25% 47%
Luton Fleetwood 1-0 10% 53% 21%
Peterboro Oxford 2-1 8% 50% 23%
Plymouth Bradford 0-1 3% 32% 39%
Portsmouth Southend 1-0 11% 69% 12%
Scunthorpe Gillingham 2-3 3% 29% 42%
Walsall Coventry 1-0 6% 58% 18%
Wycombe Barnsley 0-1 14% 21% 53%

Sol Campbell gets going with Macclesfield at the bottom of the Football League this weekend, as Macc make the tough trip to Essex’s finest, Colchester United. The Model doesn’t think Sol will have solved Macc’s defensive problems just yet, predicting a 2-0 defeat (10%). The Great Western Railway Derby between Swindon and Newport looks set to go the Welshmen’s way, 1-0 (9%).

Forecast Win (%)
League Two Score PR(%) P(H) P(A)
Bury Exeter 1-0 5% 47% 26%
Cheltenham Grimsby 0-1 11% 31% 41%
Colchester Macclesfield 2-0 10% 61% 16%
Crawley N’hampton 1-2 6% 30% 41%
Crewe Oldham 0-1 9% 19% 57%
Mansfield Notts Co 1-0 11% 59% 17%
MK Dons Carlisle 1-0 12% 50% 23%
Morecambe Port Vale 0-1 12% 31% 40%
Stevenage Lincoln 0-1 11% 18% 57%
Swindon Newport Co 0-1 9% 25% 47%
Tranmere Cambridge 2-0 11% 64% 14%
Yeovil Forest Green 1-1 12% 36% 35%

Championship, R20 (7-8 Dec, 2018)

After 7 years in the Premier League, you would think Swansea would be well-placed to bounce straight back after relegation. But currently they sit in the bottom half of the table (13th) and have only won 7 of their first 20 games. The Swans travel to Brentford this weekend, who are perhaps similarly under-performing compared to pre-season expectations. The Model suggests Swansea could slip even further down the table, with a 1-0 (15%) defeat on the cards.

Our local club, Reading F.C., sit perched above the drop zone as we head towards the mid-point of the season. They welcome Sheffield Utd to the Madejski tomorrow evening, who themselves are on a dip in form. The Model forecasts an even game, and 1-1 (12%) is the most likely outcome.

The rest of our Championship forecasts are in the the table below:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw
Forecast Win (%)
Score Pr (%) P(H) P(A)
West Brom Aston Villa 2-1 10 43 29
Birmingham Bristol City 1-0 10 49 24
Brentford Swansea 1-0 15 45 27
Leeds QPR 1-0 12 53 21
Middlesbrough Blackburn 1-0 16 53 21
Millwall Hull 1-0 11 43 28
Norwich Bolton 1-0 20 66 13
Notts Forest Preston 1-0 15 49 24
Reading Sheff Utd 1-1 12 35 36
Sheffield Wed Rotherham 2-0 11 42 29
Stoke Ipswich 1-0 19 59 17
Wigan Derby 0-1 11 32 39

 

Premier League, R16 (8-10 Dec)

Claudio Ranieri has vowed to “tinker clever” in his endeavours to keep Fulham up. He must be doing something right as the Cottagers have taken 4 points from his first 3 games in charge. Fulham face Man Utd on Saturday. The Model however has Utd to win comfortably 3-0 (9%).

The game of the round is undoubtedly at Stamford Bridge between Chelsea and Man City. If City take 3 points form these kinds of games, then there is surely no stopping them taking the title. Both sides rested players in midweek. City scraped a win at Watford, but Chelsea slipped up against Wolves. The Model suggests City will edge this one 0-1 (14%).

The table below gives the rest of the Model’s forecasts for Round 16 of the Premier League:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw
Forecast Win (%)
Score Pr (%) P(H) P(A)
Arsenal Huddersfield 2-0 12 67 13
Bournemouth Liverpool 0-2 12 30 41
Burnley Brighton 1-1 9 34 37
Cardiff Southampton 1-1 13 35 36
Chelsea Manchester City 0-1 14 31 40
Leicester Spurs 1-2 9 31 40
Manchester Utd Fulham 3-0 9 63 15
West Ham Crystal Palace 1-0 11 44 28
Newcastle Wolves 1-0 17 44 28
Everton Watford 1-0 20 50 24