How does scoring work?
-
Seems quite counter intuitive.
Seems like you're saying you get scored on your rating, not your game score, i.e. units killed, resources generated.
Is that right?
(Just to be clear, by 'score' I mean how many points your rating goes up or down after the match) -
The only way a performance based rating will work is if you black box the calculations that go into rating and NDA the devs implementing it. Since that isn’t happening in an open-source project, any in game metric (acu kills, unit kills, resources created) will be exploited by players once the formula is common, public knowledge.
Only things that matter in FAF are whether you win or loss, who you won or lost against, and how certain the system is of your placement.
-
Every other game on the planet (video game, or sports game) publishes the scoring rules, so I'm struggling to understand your reasoning.
I disagree that players attempting to max out metrics like ACU kills, unit kills, is "exploiting" the game, since that's kind of the whole point of the game, and if anything would encourage people to not turtle.
Is there any other video game or sport example you can cite where the scoring rules are kept secret? - like, shouldn't they reflect performance of players in the game, make sense, and be public
By analogy, what you're saying that in a grand slam tennis match, even if the less ranked player won, they'd award 1st place to the top seeded player because "they're the top seed", not because of who actually won the match.
Clearly nonsense.
-
@ginandtonicbot said in How does scoring work?:
disagree that players attempting to max out metrics like ACU kills, unit kills, is "exploiting" the game, since that's kind of the whole point of the game, and if anything would encourage people to not turtle.
It would encourage people to get the most amount of kills etc. and not to win the game, which incites a certain competitiveness between teammates.
Also FAF rules are not secret, that's the point. Only winning matters.
His point was, that if you wanted to introduce a scoring system based on game stats, that system would need to be secret.The amount of points you get is only different, because the system was more sure of your previous rating (as you probably had already played more games) than your teammates. It's not like you get more rating, if you already have more or something.
Only your variance (the systems uncertainty) matters and the rating difference between the teams.@ginandtonicbot said in How does scoring work?:
Every other game on the planet (video game, or sports game) publishes the scoring rules, so I'm struggling to understand your reasoning.
I believe almost every other game also uses this approach. In chess you don't get more rating for getting more pieces or winning in a more favorable position.
@ginandtonicbot said in How does scoring work?:
By analogy, what you're saying that in a grand slam tennis match, even if the less ranked player won, they'd award 1st place to the top seeded player because "they're the top seed", not because of who actually won the match.
This system is also for ranking players by skill and not to decide a winner.
The rating system has no winner, who has the most rating, but it should simply reflect how good each player is. -
No, games like League use performance based metrics but they are kept black boxed. I’m not 100% sure but I believe CS:GO is the same. Games like chess don’t. My point is more that there is no open-source example of a large playerbase game successfully implementing metric based rating.
-
FtXC said the rules would need to be kept secret. I don't agree.
Thanks for the post. I appreciate you're making an argument, but this statement doesn't seem to be true -> "...simply reflect how good each player is."
Because if I essentially win the game for my team by having the most kills and most resource gen, it doesn't reflect my performance in that game.
I am pretty sure this is why players go smurfing, set up multiple accounts, and basically cheat by playing themselves to get their ranking up.
I think the FAF community is pretty strong, so would you be against running a vote to see if there is consensus to introduce at least some partial factoring of "How well did the player perform in that game" to determine their rating change? Minimally, why not give all players on the winning team the same rating change?
I seem to be constantly in games where my team collapses, probably due to smurfing as mentioned above, my rating tanked, and now it's seemingly impossible to get any decent rating increases.
-
This post is deleted! -
I can keep the game going perpetually and make myself have 100,000 mass income in generation. I could also ignore the situation in the game and make 100,000 mass income. I could also inflate my score by overbuilding energy. I could refuse to gift units even if it would be optimal to win a game (multiple people having asf) because I want the score from killing enemy air given to me since it gives more rating.
I will always be against performance based rating because it will immediately ruin the games I play by causing tension between playing the game and playing the system. I do not care what a poll says and I would not run a poll because only a few people actually understand how the current system works or what the drawbacks/benefits of other systems would be.
If you think you’re stuck in ELO hell then play 1v1s and 2v2s where the system can be more confident you are the result of the game result and therefore awards more rating/loss in uncertainty. If you are underrated, it’ll be easy to gain rating. If you aren’t then you’re right where you should be and the system is working fine.
-
@ginandtonicbot said in How does scoring work?:
Because if I essentially win the game for my team by having the most kills and most resource gen, it doesn't reflect my performance in that game.
And it's not supposed to do that. Your rating is not reflecting your performance from your last game, but "slowly" approaches your true rating over time.
@ginandtonicbot said in How does scoring work?:
Minimally, why not give all players on the winning team the same rating change?
This is basically happening, but for people with more uncertainty the change affects their rating more and completely equalizing the point gain would also be weird, because the ratings in a team can be quite different and giving the 1k rated player +100 for the win would be as weird as giving the 0 rated guy +5.
@ginandtonicbot said in How does scoring work?:
I seem to be constantly in games where my team collapses, probably due to smurfing as mentioned above, my rating tanked, and now it's seemingly impossible to get any decent rating increases.
If your constantly winning your rating gains will also increase, to converge to your true rating faster.
@ginandtonicbot said in How does scoring work?:
How well did the player perform in that game
Another problem with supcom is, that it's possible to single-handedly win the game, if your far above your opponents, so it will quickly introduce toxic behavior of people who believe themselves to be that much better, so they can gain more rating, as (almost) all stats can be increased faster if all resources belong to you.
@ginandtonicbot said in How does scoring work?:
I am pretty sure this is why players go smurfing, set up multiple accounts, and basically cheat by playing themselves to get their ranking up.
I'm also pretty sure, there are not that many smurfs around in FAF (as it's banned compared to most other games) and i believe the mods are banning them quite well. It's also not just "basically" cheating, as this is also rating manipulation, which is also banned.
@ftxcommando said in How does scoring work?:
No, games like League use performance based metrics
They do? Thought it was also just win/loss.
But i guess it makes sense as the game sees little actual teamplay. -
@ftxcommando I might have to do 1v1 and 2v2, etc, because it's going to take 3000 years to get my rating back up otherwise.
Maybe at the same time you might consider a partial fix to what I still consider broken game scoring, i.e. someone on my same team gets 20x or more rating increase than me when I was indicated as the game winner -- maybe there could be some sort of smoothing on rating changes given to the winning team where someone can't get 20x+ the rating increase?
-
No, that makes the system worse. People aren’t “gaining” huge rating. They already have it. A new player has 1500 mu and 500 sigma. This gives them a shown rating of 0. If they beat a 1200 mu 100 sigma player, their mu moves slightly while their sigma drastically decreases. This gives them the illusion of “gaining” rating in terms of shown rating when instead they haven’t gained much rating at all in terms of what trueskill is concerned with. Smoothing out rating gains is just a misunderstanding of how new players are integrated into the distribution of players.
-
Thanks for the response.
How is mu adjusted for players? You said 'slightly'. Is it a set amount, and is it adjusted the same for all players, regardless of rating, and then the overall apparent rating change as you explained is based on mu and sigma?
Or is mu adjusted based on the players net rating?
-
Only things that matter in FAF are whether you win or loss, who you won or lost against, and how certain the system is of your placement.
Thats why the tactic of suicide is should be allowed. You killing your opponent no matter what, the fact that you killed him.
-
@eternal Thanks for the info. I get that. I'm now interested in understanding how mu is adjusted. i.e. same for all players on winning team, or different?
-
It's calculated for each player individually. The actual math gets pretty complicated. Here is a very detailed explanation: https://www.moserware.com/assets/computing-your-skill/The Math Behind TrueSkill.pdf
I found a shorter explanation once, but I can't find it right now. You can just google for trueskill and will find all sorts of explanations. -
Is there a simpler answer than a 57-page word doc?
Seems like you're saying that even though no information is collected to assess player performance, players are given different mu adjustments.
-
There is no 'player performance' collected because it is meaningless. It doesn't matter how you won, what is relevant is that you did. If we introduce 'player performance' then people stop playing the game and start playing the rating system.
-
Players are given different mu adjustments when their deviation is different. There is an exact formula for it, but it's complicated and I can't find it quickly. Your best bet is to google it yourself if you want more details, I've been doing the same.
-
Trueskill was developed by Microsoft so I suggest we all contact Microsoft support and tell them to change the algorithm.
-
@ginandtonicbot said in How does scoring work?:
Thanks for the info. I get that. I'm now interested in understanding how mu is adjusted. i.e. same for all players on winning team, or different?
It is different.
Is there a simpler answer than a 57-page word doc?
Yes.
Seems like you're saying that even though no information is collected to assess player performance, players are given different mu adjustments.
The after game rating changes that you see are changes in displayed rating, which can basically be thought of as "base rating (mu) - uncertainty (3 * sigma) = displayed rating". The higher the sigma value, the lower the displayed rating and the more uncertain a rating is. So, new players start at "1500 base rating - 1500 uncertainty = 0 displayed rating". A typical player with a lot of games has around 250-300 rating uncertainty. Playing rated games lowers uncertainty to around 100-350 uncertainty as a stable level whose exact value largely depends on whether the player plays a lot of 1v1's (less uncertainty), 3v3's, 8v8's (more uncertainty), etc. Since players start with 1500 uncertainty, their ratings are initially more volatile and they more readily gain/lose rating. As they play more rated games, the uncertainty decreases and their ratings stabilize more, resulting in smaller rating changes from individual games. Additionally, if a player wins a game that was balanced in the player's favor, then the player gains a smaller amount of base rating (mu) than if the game was balanced against the player's favor (based on rating). Also, smaller game sizes (ie: 1v1's) generally have more impact than larger game sizes (ie: 8v8's).
