@blackyps said in Upcoming rating changes: Gaining global from playing matchmaker:
So we effectively do not trust ladder rating?
No? How do you get that impression?
Because you are assuming that ladder rating is correct and global is not when both rating contexts have built up different predictions of skill difference between two players.
Inserting a prediction into the match that Trueskill never actually made (by replacing ONLY the opponents rating with their ladder rating) is bonkers!
It seems like there is a misunderstanding here. I'll give you a 2v2 example: When performing the adjustment for player 1 we take the tmm ratings of everyone and replace only player 1's rating with global. Then we let trueskill rate this game and if the conditions mentioned in the first post are met, save the new global rating for player 1. Then we do this with all other players. So for player 2 everyone else will have tmm rating again and player 2's global will be used.
We could maybe perform the adjustment with using only global ratings, but we don't know the global ratings of the players at all. The only thing we know is that their tmm ratings are similar enough to give a balanced game, so we use these. Also we have more trust in the matchmaker ratings than global as I explained above.
There is no misunderstanding here. You are using ladder rating numbers, which have no meaning inside the global rating context, as prediction to adjust the prediction of the global rating context, in fact mixing them inside the same calculation. That makes no sense.
If you adjust the global prediction you MUST only use global predicitions as input! The different rating contexts gain inflation and deflation independently of each other, as i have already sid, 500 in global is not guarantueed to be close to 500 in ladder!
Why is it important to know what their global ratings are?? You don't need to! Trueskill was made so you don't have to worry about this, just let it calculate inside its context and improve its own prediction!
I hope you plan to remove global at some point then because if you do it as it is currently planned you are no longer doing things that make sense for having a predictable well-behaved rating system, you are deep inside band-aid mode already.
Yes, I do not believe that global as it is now is a healthy rating system. There are too many variables players can control in custom games to influence their rating. You can see the result in people that are way higher rated than you would expect them to be. Removing global right now would make the gap, setons and astro players rise up in arms, so unless you have an idea how to solve their needs, global stays for now. In the long term people hopefully play the matchmaker more, so global becomes less relevant, but when it has become irrelevant it doesn't really matter if it gets removed or not.
And the interpolation i suggested would let you slowly phase out global by adjusting the global weight over time, which is another advantage it has.
So if i am correct this is the first time we really try to fuck with Trueskill itself!
I really fail to see what we are fucking with. Yes, this change will lead to "communication" between the ratings, but as we all initialize them with 1500+-500 I don't see a problem with that.
The problem is that every single player entering or leaving the rating context shifts that context ratings range! If a 2000 rated player stops playing global, then the global rating pool just lost 2000 rating! So right from the start the averge mean rating will start to diverge. The default values only guarantuee that deviation has the same meaning, mean rating is not predictably related!
The proposed change would also make simulating a change to global rating parameters with past data much harder i think, because global game outcomes are no longer enough to redo all global rating calculations.
Does anyone actually want to do this? This is a genuine question. I know that in the past it was demanded to do this to test changes to trueskill, but to me it seemed more like a ostensible reason to prevent changes to the rating system. There has been a change of the tau value in the past and I don't think it was simulated beforehand with all the rating data we have.
Yeah that Tau value was changed when we still had a person that had a lot of confidence in its own understanding of Trueskill. But if i remember correctly, Tau does not really do much, it basically just changes the average amount of mean rating gain/loss. So an very non-risky change to make.
Also, how about we (as a community) discuss such intrusive changes BEFORE they are implemented?
There has been discussion about this. I don't remember everything where it took place, but there was discussion on the issue: https://github.com/FAForever/server/issues/845
We also had a voice call with Morax as the player councillor, some developers and some ladder team members.
So no open discussion in the forum where such things need to take place if you want others than developers and councillors to give input on these things.