One of the things we've been thinking about for a while now is how to improve the player experience around finding a server to play on. It's a tricky problem because our master servers need to ask a game server for its details, and that server can lie to us if it wants to. We decided we needed to find a way of scoring servers, with a goal of finding and delisting ones we considered "bad". The scoring system had to penalize lying without penalizing custom game rules, because some players like custom game rules. Best case, the system needed to work entirely from data that didn't come from the servers themselves, so they couldn't lie to us in any way to affect it.
After kicking around some proposals, we came up with a simple system built around the theory that player time on a server is a useful metric for how happy the player is with that server. It's game rules agnostic, and we can measure it on our steam backend entirely from steam client data, so servers can't interfere with it. We already had this data for all the TF2 servers in the world, allowing us to try several different scoring formulas out before settling on this simple one that successfully identified good & bad servers:
- New servers start with a score of 0 points
- Each time a player connects to a server, it loses 15 points
- For each minute the player stays on the server, it earns 1 point (up to a max of 45 points per player)
In short, servers that have lots of players joining & leaving rapidly will score badly. Servers that consistently have players join and stay on for long periods of time will score well.
Here's a graph showing server scores for all the TF2 servers in the world as of last week. For the purpose of scoring, we ignored all password-protected servers, and all servers that had fewer than 200 connections a day. The blue line in the graph represents the scores for all the TF2 servers. The red line is the matching player connection count for each server.
[IMG]images/posts/ServerScoring.jpg[/IMG]
As you can see, the bulk of servers in the world are doing a pretty good job of providing an experience that's expected by the people joining them. More importantly, it's really easy to see what servers are bad. Overlaying the number of players connecting to the servers illustrates how nasty an effect these bad servers are having on players. The very worst servers attract a large number of connections, mostly because they're lying in ways that make them look like a very attractive server at all times.
Our first step in improving this part of the player experience has been to delist all the really bad servers. The master server will simply stop giving these to you when you fire up the serverbrowser. After that, we're going to keep improving our ability to measure this kind of problem.
