Evaluation of the Model: Thrashed in R6 of the Champions League

For the 6th round of the Champions League we published two sets of forecasts:

  1. The Scorecasting Model.
  2. Jean-Louis Foulley’s forecasts, the method of which is described here.

Based on just 16 matches we can provide a rough evaluation of these two methods.

For the most likely result of matches, the Scorecasting Model forecast 7 correct. Foulley’s model however had the most likely result agreeing with the actual outcome in 8 matches.

For the most likely scoreline, the Model got one correct. However, Foulley’s model got 4 correct scorelines!

If we just take the scoreline forecast of each model as a single point forecast, and then award 40 points for a correct scoreline and 10 points for a correct result only, then the Model would score 100 points from the 16 matches. On the other hand, Foulley’s model scores 220 points.

Foulley’s forecasts for past Champions League rounds are also published here.

 

 

Advertisements

Champions League R6 (11-12 Dec)

It’s crunch time in the Champions League – the final set of group matches. Who will progress? Who will slip into the Europa League? This week, in addition to our usual forecasts, we also list the outcome forecasts created by Jean-Louis Foulley – see below. These are forecasts posted here, and Jean-Louis has also written an important paper on the ways in which we might penalise scoreline forecasts – like the ones we create.

Tottenham look set to drop out on Tuesday night, losing 2-0 to Barcelona (11%), while Liverpool look set to progress, beating Napoli 3-0 (9%).

Forecast Win (%)
Tuesday Score PR(%) P(H) P(A)
Barcelona Tottenham 2-0 11% 48% 26%
Brugge Atl Madrid 0-1 21% 22% 54%
Galatasaray Porto 0-1 23% 24% 51%
Inter PSV 1-0 20% 48% 26%
Liverpool Napoli 3-0 9% 58% 19%
Monaco Dortmund 1-2 9% 22% 54%
Red Star Paris St-G. 0-2 10% 29% 45%
Schalke L Moscow 2-0 15% 56% 21%

On Wednesday, in the only group where qualification hasn’t already been resolved, Shaktar will beat Lyon into the knock-out stages…

Forecast Win (%)
Wednesday Score PR(%) P(H) P(A)
Ajax B Munich 1-0 10% 41% 32%
Benfica AEK 1-0 18% 57% 20%
Man City Hoffenheim 3-1 8% 78% 8%
R Madrid CSKA 3-0 15% 68% 13%
Shakhtar Lyon 1-0 10% 49% 25%
Valencia Man Utd 0-1 23% 28% 45%
Plzen Roma 0-3 10% 36% 37%
Young Boys Juventus 0-3 9% 26% 48%

Foulley’s forecasts:

We’re going head to head against Foulley’s model, so here are his outcome probabilities, including the most likely socreline. For the forecast scoreline, we disagree on three matches (Ajax vs Bayern, Shakhtar vs Lyon, & Valencia vs Man U), and on all other matches, we are more conservative in our estimated probabilities. We will continue to post these comparisons, as with probabilistic forecasts, it is fairly meaningless to compare a small number of probabilities as those listed here.

Foulley (Bayesian) forecasts Win %
Score PR(%) P(H) P(A)
Barcelona Tottenham 2-0 10% 69% 13%
Brugge Atl Madrid 0-1 12% 17% 62%
Galatasaray Porto 0-1 13% 20% 56%
Inter PSV 1-0 12% 61% 17%
Liverpool Napoli 1-0 11% 53% 23%
Monaco Dortmund 0-1 13% 18% 59%
Red Star Paris St-G. 0-2 12% 11% 73%
Schalke L Moscow 1-0 13% 64% 15%
Ajax B Munich 1-1 12% 30% 44%
Benfica AEK 1-0 12% 68% 13%
Man City Hoffenheim 2-0 10% 84% 6%
R Madrid CSKA 2-0 13% 83% 5%
Shakhtar Lyon 1-1 11% 48% 29%
Valencia Man Utd 1-0 11% 37% 35%
Plzen Roma 0-2 10% 18% 61%
Young Boys Juventus 0-2 13% 12% 70%

 

Champions League, R6 (Nov 27-28)

The Champions League returns this week. Our forecasts are posted in two tables, one for each night of games.

Forecast Win (%)
Score PR(%) P(H) P(A)
AEK Ajax 0-1 18% 28% 45%
B Munich Benfica 2-0 16% 68% 14%
CSKA Plzen 2-0 13% 58% 19%
Hoffenheim Shakhtar 3-1 8% 43% 30%
Juventus Valencia 2-0 20% 73% 11%
Lyon Man City 1-2 8% 21% 54%
Man Utd Young Boys 2-0 15% 68% 13%
Roma Real Madrid 1-0 7% 38% 35%
Forecast Win (%)
Score PR(%) P(H) P(A)
Atl Madrid Monaco 1-0 24% 77% 9%
B Dortmund Club Brugge 3-0 10% 74% 10%
L Moscow Galatasaray 1-0 12% 46% 28%
Napoli Red Star 1-0 16% 61% 17%
Paris St-G. Liverpool 2-1 6% 46% 28%
Porto Schalke 1-0 18% 62% 17%
PSV Barcelona 0-2 13% 34% 39%
Tottenham Inter 1-0 12% 69% 13%

Champions League, R5 (6-7 Nov)

The Champions League returns again tonight and tomorrow. In the usual format, we’ve produced forecasts for this week’s matches in the two tables below.

As with most weeks, it’s forecast to be a good week for English teams – will they live up to expectations this week?

Forecast Win (%)
6th Nov Score P(H) PR(%) P(A)
Atl Madrid B Dortmund 1-0 16% 51% 24%
Inter Barcelona 1-2 9% 29% 44%
Monaco Club Brugge 2-0 16% 40% 33%
Napoli Paris St-G. 2-1 6% 41% 32%
Porto Lok. Moscow 5-0 13% 65% 15%
Red Star Liverpool 0-3 13% 25% 50%
Schalke Galatasaray 1-0 27% 55% 21%
Tottenham PSV Eindhoven 2-0 15% 62% 17%

 

Forecast Win (%)
7th Nov Score P(H) PR(%) P(A)
B Munich AEK Athens 3-0 14% 71% 12%
Benfica Ajax 0-1 14% 34% 39%
CSKA Mosc. Roma 1-0 7% 37% 36%
Juventus Man Utd 1-0 23% 57% 20%
Lyon Hoffenheim 3-1 8% 53% 23%
Man City Shakhtar 2-0 14% 74% 10%
Valencia Young Boys 1-0 12% 41% 32%
Viktoria Plzen Real Madrid 0-4 11% 27% 47%

Champions League, R3 (Oct 23-24)

Competing with League One and League Two this week is the Champions League. How will Christiano Ronaldo fare on another of his returns to Old Trafford? Looks like he might struggle, a 1-0 win for United 17% likely. Can Bayern, Real and Barca get  their seasons back on track? It looks likely, with wins in Athens for Bayern (3-0, 10%), at home against Plzen for Real (7-0, 11%), and at home against Inter for Barca (2-1, 10%).

Liverpool also look set to bounce back from defeat in Naples last time out by beating Red Star 5-0 (13%).

Forecast Win (%)
Score PR(%) P(H) P(A)
AEK Athens B Munich 0-3 10% 27% 47%
Ajax Benfica 1-0 16% 49% 25%
Hoffenheim Lyon 2-1 10% 41% 32%
Man Utd Juventus 1-0 17% 41% 32%
R Madrid Plzen 7-0 11% 72% 12%
Roma CSKA 2-0 14% 52% 23%
Shakhtar Man City 1-2 9% 25% 50%
Young Boys Valencia 1-0 33% 52% 23%
Dortmund Atl Madrid 1-0 12% 40% 33%
Barcelona Inter 2-1 10% 68% 14%
Brugge Monaco 1-0 4% 53% 22%
Galatasaray Schalke 1-0 20% 47% 27%
Liverpool Red Star 5-0 13% 72% 11%
L Moscow Porto 0-3 21% 34% 39%
Paris St-G. Napoli 2-0 9% 55% 21%
PSV Tottenham 0-1 14% 35% 38%

Champions League, Gameweek 2 (Oct 2-3)

Although we are certified Football League fans, it is hard to avoid the Champions League. And it’s good fun to make forecasts, even if we get walloped by Lawrenson and Merson in the Premier League. So we’ve cranked out some forecasts for the Champions League matches taking place this evening, and tomorrow night.

It looks like it should be a good week for the English teams in the competition, although Man City might want an easier night than may be in store for them: a 3-2 win, although the 5% probability shows the spread of likely scores.

Forecast Win (%)
Score PR(%) P(H) P(A)
Tue AEK Benfica 1-0 9% 46% 28%
Tue Munich Ajax 3-0 9% 69% 13%
Tue CSKA Real M 0-3 9% 26% 49%
Tue Hoff Man City 2-3 5% 22% 54%
Tue Juve Yng Boys 4-0 10% 69% 13%
Tue Lyon Shakhtar 1-0 12% 45% 28%
Tue Man Utd Valencia 1-0 22% 70% 12%
Tue Roma Plzen 6-0 7% 63% 16%
Wed Atl M Brugge 2-0 23% 70% 12%
Wed Dortmund Monaco 2-1 9% 64% 16%
Wed Lok. Mos Schalke 1-0 0% 40% 33%
Wed Napoli L’pool 1-3 7% 32% 41%
Wed PSG Red Star 1-0 0% 67% 14%
Wed Porto G’tasaray 2-0 14% 57% 20%
Wed PSV Inter 2-1 4% 56% 20%
Wed Spurs Barca 1-0 11% 46% 27%

Champions League, R1 (18-19 September)

Better late than never…  The Model is getting ambitious. Tonight it is forecasting the Champions League. This is experimental, so the methods may follow later.

The rest of our Champions League forecasts are in the the table below:

  • The Forecast scoreline, with the % chance of that happening
  • The % chance of a win by the Home team, P(H), or Away team, P(A) (with one hundred minus those two numbers giving the % chance of a draw
Forecast Win (%)
Score Pr (%) P(H) P(A)
Barcelona PSV Eindhoven 2-0 16 82 5
Club Brugge B Dortmund 0-3 12 2 91
Galatasaray Lok. Moscow 0-0 14 35 34
Inter Tottenham 0-1 20 15 58
Liverpool Paris St-G. 2-1 9 65 18
Monaco Atl Madrid 0-0 15 27 41
Red Star Napoli 0-1 22 16 53
Schalke Porto 0-1 18 23 45
Ajax AEK Athens 1-0 15 45 26
Benfica B Munich 0-2 11 13 69
Man City Lyon 2-0 17 77 6
Real Madrid Roma 3-0 8 84 6
Shakhtar Hoffenheim 1-0 20 50 19
Valencia Juventus 0-1 23 11 62
Viktoria Plzen CSKA Mosc. 1-2 9 18 64
Young Boys Man Utd 0-2 10 16 65