Tournament Scoring - Part 6: Summarising the Problem with Scoring Systems

Before I move on to look at my suggestion for a scoring system I want to summarise the problems with scoring systems that are out there already.  If we can work out what is wrong with them, we can use this to design a system that might address these problems.

When I first started playing on online I remember raising a question: Why are draws scored equally? Why isn't the number of supply centres you hold at the end of the game taken into account?

The answer that came back from a site mod was along the lines of the argument I use now.  At the end of a game of Diplomacy you have achieved one of three results: you've won, you've drawn, or you've lost.  If you've won, then you get all the points in the game.  If you've drawn, it doesn't matter what number of SCs you've finished the game on, the result is the same.  If you've lost... well.

At the time, that site used a game result system to score games:

• 12 points for a win
• 6 points for a 2-way draw
• 4 points for a 3-way
• 3 points for a 4-way
• 0 points for anything else.

You also got a bonus point for taking over a surrendered position and not surrendering, and if you surrendered from a game you lost a point.  This system was later modified to add 5-way draws at 2 points.  

It was a simple system but it encouraged cherry-picking; that is, players would look for games with newbies in which gave them a decent chance of scoring well, or who even looked to pick up surrendered positions which had a decent chance of a good result.  I remember that, before the system was changed completely (to an Elo-esque system modified for Diplomacy), the site leader was a cherry-picker.  He dropped from the site when cherry-picking became effectively penalised by the new rating system.

The great and good ABC had the same idea: At the final outcome, the only thing that matters is that you've won, you've drawn or you've lost.  Nothing else has any impact on the game.  If that's what Calhamer thought, then that carries more weight with me than anything else.

There is an argument to say that Calhamer wasn't thinking about tournaments specifically with this, but rather in scoring an on-going series of games.  Tournaments are different: they're short, finite series of games and scoring needs to reflect this.  Whereas an on-going series will have a large number of games to differentiate between players, a draw-based scoring system won't easily work for tournaments.  However, DSS systems reflect more of what a game of Dip should be about than SCS systems.  You play for the result, not the number of SCs you can grab.

I've just sumarised what doesn't work with the tournament scoring systems that are often used.  Leaving aside the hybrid systems (that combine the good and bad of both DSS and SCS, and make them unclear as to what you should be doing), neither DSS nor SCS work.

Draw Size Scoring systems just don't produce enough of a difference between players over a short series of games.  Simply, points awarded are usually too similar.  Add to this the fact that players are likely to play for a decent draw, with as few players as possible, and that alliances are likely to be maintained as a way to get this result, DSS produces boring games and a poor scoring system.

Supply Centre Scoring systems, and Ranking or Placement systems which use SC count, pervert the objectives of Diplomacy.  Where did this idea of counting SCs in a draw come from?  Where did the idea that is so pervasive in the thinking of so many experieced players - topping the board - come from?  From the failure of DSS.

SCS systems produce more dynamic games.  They produce more differentiated scoring.  But games degenerate into a grab for SCs.  And Diplomacy was never intended to be about grabbing SCs... unless you're homing in on a solo, of course.  If you win the game, the number of SCs I end the game on doesn't matter: I lost.  If we draw the game, neither of us has achieved any better result than the other.  If I'm on 2 SCs and you're on 15, you've gained more SCs than me... but you weren't able to beat me.

Tournaments might be easier to score using a system that utilises some form as SC count, but that isn't what Diplomacy's about.  We're sacrificing the game for the sake of scoring it.  That's a sad thing.

If we're going to solve the problem, we need some more creative thinking.  How can we differentiate between players while using a draw-based system?

First, we need to change our thinking.  Draw-based systems meet the objectives of Diplomacy much more effectively than SC count systems; that ought to be clear.  But a draw-based scoring system that differentiates between the number of players in the draw doesn't work.  A draw's a draw... when it comes down to it, does it really matter if it's between 2 or 5 players?  So, instead of DSS - Draw Size Scoring - let's just award points for a win, and a draw of any size.  It should be enough to have drawn and not lost.

Second, we need to find a better solution to tie-breakers that recognise SC count.  We're trying to get away from that, after all.  So we need a different way of comparing players who have the same number of points under draw-based system.

And we don't need bonus points.  Bonus points are based on false results.  So let's get rid of them.

The prototype system I've come up with uses a draw-based system, rewards solos but not until all the games in the tournament have been completed, and uses a variation on a very different type of scoring system to those we've looked at previously.

That's what I'm going to look at next: the Mystery scoring system.

TOURNAMENT SCORING series:

1. Tournament Games are Variants
2. What should a good scoring system do?
3. Draw Size Scoring (DSS)
4. Supply Centre Scoring (SCS)
5. Other Scoring Systems
6. Summarising the Problem with Scoring Systems
7. The Mystery Scoring System Explained
8. Does Mystery Scoring meet the objectives?