Ladder needs some form of decay
-
There is a curious phenomenon that exists when you take a look at the ladder league data: the 1800+ league basically has zero activity. As of the 27th, only 2 players with ratings above 1800 have played more than 30 ladder games over this month. Why is that?
I believe that it is because the incentives to play ladder are broken. A high ladder rating is valuable because it guarantees favorable seeding in tournies, which usually have money on the line. Inactivity guarantees this ladder rating will be maintained, while activity will run the risk of potentially lowering this rating.
As the title of this post suggests, I believe the solution to this issue is the implementation of some form of rating decay. Whether it is necessarily decay in the actual rating, or just the uncertainty, there needs to be some incentive to have people continue to play ladder. Back when I had played Heroes of the Storm, my rating would undergo a soft reset every 3 months - people who had achieved the top rating would need to play consistently to maintain that rating. This is the case for many other online games, and should be the case for FAF.
--
My proposal: add in a rating decay system based on a visible rating and an invisible rating.v = Visible rating
i = Invisible rating
t = Inactivity trackerv = i*0.96^t
Where t=t+1 for every month that passes without 10 games played and t=t-1 for every 10 games played.
--
It's a pretty simple and straightforward system that adds decay based on how long a person hasn't played for. This can go hand in hand with the new "divisions" system, which displays a visible division rank (visible rating) on top of the actual player rating (invisible rating), but also works directly with the current rating system.
A returning player should find that their rating rises quickly with activity.
This also fixes the issue FtX is facing with tourney seeding - instead of having to give completely inactive players a rating of 0, the rating system already accounts for the decay that should happen.
-
Well the problem is much more deeply rooted than just a simple "people don't want to play to lose tourney seeding" and you can honestly see the reason why by just looking at the league system.
https://www.rankedftw.com/stats/leagues/1v1/#v=2&r=-2&sx=a
Let's compare it to sc2 league distribution.
FAF has 2170 unique users playing ladder this month (first time we broke 2000 in all of FAF history)
Our 1800+ (.1% of monthly activity) is mid-tier grandmaster.
1300-1800 (.5% of monthly activity) is basically the rest of grandmaster and most of the master division.
800-1300 (13% of monthly activity) is mid tier diamond and better.
300-800 (25% of monthly activity) is below mid tier diamond to mid tier platinum.
<300 is everything else.Now, with our Trueskill implementation, we would expect 1500 mu (or about 1200/1250) to be somewhere at uppercrust of Gold, instead it's like Diamond I.
I believe there's just a fundamental flaw with the rating distribution that revolves around the fact it was implemented in late 2012 when the dataset for ratings was essentially all the GPG vets that Zep knew and invited onto the client or cared about FA so much that they managed to find an alternative way to play once GPGNet closed. This initially biased sample has caused trueskill to have a skew towards what it expects the initial 1500 mu player to play as and so we have an absurdly large general deflation on FAF that makes the problem of finding players at the upper tier quite extreme. This is further accentuated by the fact that a lot of these older players are now also inactive and the system itself cannot do any sort of correction by redistributing rating from older players amongst newer players, or taking the rating from newer players and giving it to older players.
I'd rather just do a general rating reset frankly rather than a decay, I see the problem as a lot more deeply rooted into the structure of the rating system than what a decay is going to be able to solve.
-
Though I guess I should get some data on accounts that played more than 10 games this month and see how that impacts the distribution.
With regards to the soft reset stuff, that's already planned to be including with the new division system. We generally do not want to touch the trueskill value itself with periodic resets but rather adjust the structure on top of it.
-
The question is will there be a reward for having high rating that is shown on top of true skill like for example seeding in tourneys based on your rank and not TrueSkill, Ladder League winner being the guy that has rank 1 instead of the guy that has 60h during that month and is ~1.8k-2k rating so he actually can find any opponents.
Over all I think that with the roll out of the new division system it would be healthy to increase the sigma for all players so that in order to get any visual rank they would need to play some games and thus the true skill would be adjusted as well. -
Increasing sigma should have no real impact unless you think there is a large amount of inaccuracy in the confidence interval of the rating as it is. For example, you do it with a transfer of global to 2v2 tmm because there is a degree of inaccuracy in expecting a guy that played dual gap for 1000 games and got 1600 to play that level in 2v2. However, you could expect that guy to be able to play a 2v2 at around 900 level if not better, and he should also be able to improve fairly quickly as he has a general foundation in the game. Doing this with periodic season resets is pointless. Petric with a +100 sigma increase will still be matching against people close to his original mu and get back to his roughly original confidence interval in 4-5 games.
I also don't really understand why you need a sigma increase to remove a visual rank? You could just require 10 games in the current season to get your league badge.
-
@FtXCommando Completely agree with the need for a one-time hard rating reset, especially given what you had said. I suspected that because a lot of rating was locked away in old inactive accounts that the curve was shifted so far to the left, and it seems a hard reset is the only real solution to that particular problem. Also, it seems like the new division system is the best opportunity to actually do a hard reset.
@Tagada My understanding is that your underlying rating will be unchanged and you will just have some minimum amount of activity to get your visual rank to match your underlying rating. If you play some minimum amount of games during that month there is no reason that one 1.8k guy with 60h should seed higher than you.
-
on the faf promotional discord I did have a look at the ladder rating distribution over time. And found it did decay in the fist couple of years quite a bit and after that has stabilized somewhat. Trueskillrating in itself has a tendency do deflate with new, unexperienced players joining the pool. That is in my opinion the most probable explanation for the avrg µ droping about 200 points from 2016 to 2017. But there have also been some shinanigans with the trueskill parameters in that time. The plot shows µ:
The problem with rating decay is that you add an additional mechanism for rating deflation. I have to agree that a hardreset wouldn't be that bad since trueskill is one of the fastest converging rating systems out there. At least according to the white paper that microsoft released.
-
The league system will solve this problem without the need to touch trueskill. The trueskill value will remain the same but you will not have it displayed anywhere. Instead you have your league and points displayed which will reset every season.
-
There's actual game balance value in increasing people's search range when they haven't been seen on the relevant ladder for a while in a while - that person could be improving in custom games or be afk and now be 300 points worse than they were. If you're well over 2k you would only really have to search once to avoid this and keep your rating and crush whatever 1800 you can find.
Seems very related, but what's the plan with ladder divisions/the general vision for ladder? Feel like dumping that all in one spot on the forums (along with things like TMM's future plans) would be helpful - can't find much from searching. Don't have to have timeframes, but letting people know what the plan is and what needs to be done (perhaps prodding people to help who can) might be good.
-
Most stuff was on the old forum in the TMM thread by me and the divisions thread by BlackYps. Maybe one day I'll have motivation to transfer stuff here but currently I don't.
What needs to be done:
https://github.com/orgs/FAForever/projects/7But we're quite close to a "minimum viable product" as it is and a lot of the issues left I believe are mostly QOL stuff (leaderboards, me being able to actually change pools without bothering brutus, displaying stuff)
-
@Askaholic said in Ladder needs some form of decay:
The league system will solve this problem without the need to touch trueskill. The trueskill value will remain the same but you will not have it displayed anywhere. Instead you have your league and points displayed which will reset every season.
Was talking about the idea of a reset with BlackYps the other day and we basically arrived at the conclusion that while the trueskill distribution on FAF is a problem, we basically incidentally have solved the problems with it without, well, actually solving it.
Mainly:
The problem of new players having 1500 mu in a distribution where the median is more like 700? We have a new linear extrapolation matchmaking process for new players where they climb UP to their actual mu after 10 or so games. This will result in a less efficient matchmaking algorithm based on the theoretical expectations of trueskill, but it'll make a better experience for new players and actually match the median user against new players earlier on.The problem of leaderboards full of inactivity? Well, as you mentioned, the new season based division system will resolve the issue.
With those issues resolved, the reset kind of doesn't bring us anything aside from prettier graphs.
-
@FtXCommando said in Ladder needs some form of decay:
What needs to be done:
https://github.com/orgs/FAForever/projects/7This is just TMM right, is there this kind of thing for the divisions? Also I just searched Blackyps posts in the old forums and couldn't find any thread he started or post he made about this, also couldn't find anything when searching for Ladder Divisions (other than a conversation you and I had about ladder more broadly in February which is somewhat outdated now and was quite vague regarding divisions) - could you point me in the right direction please?
I really do think it would be worth consolidating some of the roadmap/vision/what's in development stuff somewhere easy to find on the forums or in the client.
-
Yeah turns out it's mostly spread out over some general posts.
https://docs.google.com/document/d/1U60RylfoWdKADorXZmiXs3pq7MYn9cCXy1UMp-js4U0/edit?usp=sharing
This is the general structure.
Also don’t think it needs much help, it’s been done for a bit and mostly is waiting on tmm.
-
Thanks, interesting read. So if it's done and just waiting, the divisions would start getting rolled out once whatever teething problems TMM has are dealt with after implementation?
-
From what I know, TMM and divisions/leagues are completely separate. Neither is waiting for the other, they just both happen to have a similar expected completion date. Both of them are pretty much complete, and are facing the usual logistical delays of finding time for server updates, code reviews, testing, etc.
