• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Login
FAForever Forums
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Login

[Rating] Inconsistency when drawing

Scheduled Pinned Locked Moved General Discussion
16 Posts 8 Posters 1.3k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • F
    Fichom
    last edited by Fichom 5 Jun 2024, 20:40 6 May 2024, 20:34

    Correct me if I'm wrong, but any game sub 5 minutes is unrated. This should be the case for draws as well!
    8aa98e3b-9f25-4f22-8b9e-204867aaf357-image.png

    To make it extra clear, the game in question was ended after 34 seconds because of a DC, and it was done with a Draw call. In no situation ever in any respectable system should this result in rating change.

    1 Reply Last reply Reply Quote 0
    • B
      BlackYps
      last edited by 6 May 2024, 21:06

      matchmaker games are immediately rated

      F 1 Reply Last reply 6 May 2024, 21:18 Reply Quote 0
      • F
        Fichom @BlackYps
        last edited by Fichom 5 Jun 2024, 21:18 6 May 2024, 21:18

        @blackyps I see. That's dumb. It should at least consider game length, and anything say sub N minutes should be unrated, where N is number of players / team (so 1 min for 1v1), regardless of how the game came to an end (recall, draw, or people simply leaving). Or you can say N/2 minutes.

        Unless it's some kind of a system limit, which I highly doubt, I'd really like to hear a good reason why a game that lasted 30 seconds should be considered for rating (again, you pretty much have time to build a single factory and nothing else). Even more so for team games where maps tend to be larger.

        1 Reply Last reply Reply Quote 0
        • F
          FtXCommando
          last edited by 6 May 2024, 21:27

          Encourages people to just leave games if they don’t like the map

          F 2 Replies Last reply 8 May 2024, 14:54 Reply Quote 1
          • A
            Askaholic
            last edited by 6 May 2024, 21:29

            The rating grace period doesn’t exist in matchmaker games to prevent people from manipulating their rating by just quitting games when they get a map that they don’t like or think they are bad on. Also if they get matched against an opponent they don’t want to play against. Same reason you’ll still be matched against people you’ve foed.

            In customs you can control all that stuff before the match anyway so it makes more sense to have a grace period for disconnects.

            1 Reply Last reply Reply Quote 0
            • F
              Fichom
              last edited by Fichom 5 Jun 2024, 21:39 6 May 2024, 21:35

              Ok, that's a good reason. But couldn't a system that punishes such behavior be put in place? Like keep track for each player what maps they get and if they DC. If someone DCs >10 times from a particular map (and always on that map), but almost never from others, the likelihood of that not being on purpose is really low. Increase the "10" number to reduce chances of false positives.

              Or, could there be a detection system that sees who DCed, that person gets penalized, and the rest of the players don't. I understand it would be hard to pull maybe in 1v1 if a player comes up with some kind of DoS attack on the opponent, but in 2v2, 3v3 and 4v4... idk, 60 seconds feels like a safe bet game-wise (as in nothing really major can happen in the first 60 seconds).

              Edit:
              Like, I'm thinking how other games get around this issue. I'd assume a time-penalty system would work as well, but I feel such a system would probably punish willing players that just had an unlucky DC too much.

              1 Reply Last reply Reply Quote 0
              • P
                PViddy
                last edited by 7 May 2024, 03:00

                Even when all players agree to a Draw, there is still a rating change for TMM? I've experienced the same thing, and was surprised.
                How can you determine a winner/loser if it was a Draw?

                But then I checked the games a day later, and they correctly say 'DRAW' and no rating change.
                So it all worked out in the end, but that is a bit a confusing!

                Fichom - Have you rechecked the game results a day or two later? They may correctly appear as a Draw.

                B F 2 Replies Last reply 7 May 2024, 07:26 Reply Quote 0
                • A
                  Askaholic
                  last edited by 7 May 2024, 04:01

                  Of course there “could” be but who is going to have time to build that. Detecting who is causing connection issues is also quite difficult and sometimes impossible since how can you decide which end of the pipe is responsible for the connection issue in a peer-to-peer system? That’s assuming you can even see which connections are having issues, which the lobby server can’t. I think nowadays the telemetry server often has that data and you could conceivably send it to the lobby server but there’s yet another nontrivial piece of code someone’s gotta write. All’s to say, yea it’s possible if you got enough devs with enough spare time.

                  1 Reply Last reply Reply Quote 0
                  • B
                    Brutus5000 FAF Server Admin @PViddy
                    last edited by 7 May 2024, 07:26

                    @pviddy said in [Rating] Inconsistency when drawing:

                    Even when all players agree to a Draw, there is still a rating change for TMM? I've experienced the same thing, and was surprised.
                    How can you determine a winner/loser if it was a Draw?

                    Wrong assumptions on your end. A draw can cause rating changes and win of one player might - in theory - not cause a rating change. This depends on what the expected result is.
                    The rating we should is just a simplified number of the underlying metrics mean (your assumed skill) and the deviation (the probability or confidence your mean is correct).
                    We calculate the shown rating as rating = mean - 3*deviation.

                    So if the mean doesn't change because the game predicted this result (and a draw could also be predicted!), the deviation might still be reduced because we are more sure, that the mean matches the true players skill.

                    And I would still love somebody to build a trueskill calculator to explain this to people in a way they can play around with the values and see themselves.

                    He said, "I've been to the year 3000
                    Not much has changed, but they live underwater
                    And your great-great-great-granddaughter
                    Is playin' FAF, playin' FAF"

                    1 Reply Last reply Reply Quote 0
                    • N
                      Nomander
                      last edited by 7 May 2024, 08:19

                      How about not rating matchmaker games if they result in a draw before the same grace period as in custom games?

                      1 Reply Last reply Reply Quote 0
                      • B
                        BlackYps
                        last edited by 7 May 2024, 08:31

                        You mean like this? https://github.com/FAForever/server/issues/948

                        1 Reply Last reply Reply Quote 0
                        • N
                          Nomander
                          last edited by 7 May 2024, 10:25

                          Pretty much. Your idea in the issue is to not rank games ended by a draw, and I want to add on top of that the grace period condition.

                          As for the rating manipulation possibilities mentioned there:

                          • In 1v1, draw should never derank because connection issues aren't frequent enough.
                          • In team games, imo there are enough participants for someone to make a report against people who leave when they don't like their opponents/the map, and the unranked draw would be relieving for many players who get connection issues.
                          1 Reply Last reply Reply Quote 0
                          • B
                            BanthaFodder
                            last edited by 7 May 2024, 10:47

                            TMM games, like all global games should be unrated within that first 8 minutes.

                            Regardless of who is "expected" to win.

                            Surely that is easy to change. Apply this rule after 8 minutes.

                            It's plain stupid that rating changes occur during any kind of draw. It defies the definition of the word draw.

                            1 Reply Last reply Reply Quote -2
                            • F
                              Fichom @PViddy
                              last edited by 8 May 2024, 14:48

                              @pviddy That's because of the most recent client update that now doesn't show rating changes in matchmaker games at all. The change still happens, you just don't get informed on it 😉

                              1 Reply Last reply Reply Quote 0
                              • F
                                Fichom @FtXCommando
                                last edited by 8 May 2024, 14:54

                                This post is deleted!
                                1 Reply Last reply Reply Quote 0
                                • F
                                  Fichom @FtXCommando
                                  last edited by 8 May 2024, 14:55

                                  @ftxcommando I forgot to add, which is quite important: that is a valid reason. But, at least the Draw mechanic should have a timeout so it's unrated for a sufficiently short games.

                                  1 Reply Last reply Reply Quote 0
                                  10 out of 16
                                  • First post
                                    10/16
                                    Last post