FAForever Forums
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Login

    Private coturn server for my games.

    Scheduled Pinned Locked Moved
    General Discussion
    8
    22
    940
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Brutus5000B
      Brutus5000 FAF Server Admin
      last edited by

      The Forged Alliance engine will alway open one udp port per player in the game and send data multiple times.

      In theory you could try to reverse engineer the binary network protocol and remux the streams that some intermediate software merges outgoing traffic into 1 connection and splits up incoming traffic into "per player" traffic.

      The is actually a developer on Discord trying to do exactly that. I'm not sure if this really solves network connection issues. What I am sure is that it will cause latency issues for all users geographically far away from the central server.

      2 players from australia don't take the direct route but sent over a gateway in europe adding 500ms+ latency.

      We don't know much about the binary data protocol

      He said, "I've been to the year 3000
      Not much has changed, but they live underwater
      And your great-great-great-granddaughter
      Is playin' FAF, playin' FAF"

      I 2 Replies Last reply Reply Quote 2
      • I
        IamFromRussia @Brutus5000
        last edited by

        @brutus5000 If we know so little about the data protocol, how do you get information about the game starting/ending? How do you get messages from the chat?

        If it's not too much trouble, send the discord of the modder who is trying to do this )

        Brutus5000B 1 Reply Last reply Reply Quote 0
        • S
          Surfer
          last edited by

          Game using peer-to-peer (P2P) architecture where every player connected to each other.
          Thats good for LAN party and terrible for modern networks with all those firewalls and VPNs.
          When two players can't establish direct connection, coturn/stun server help them to find the route using some smart tricks.
          If you experience connectivity problems consider buying static IP address from your ISP.

          As for single game server, that should improve connectivity, theoretically.
          For example in 8v8 the total connection count is 49 (7 per player), with single server its only 7 (1 per player).
          Chance of connection lost is much lower with less connection count.
          But again, if host act as a server and unlucky with ISP all that doesn't matter.

          If server running by FAF org or privately another problem arise:

          • hardware and traffic cost
          • server require maitance
          • increased latency depending on player location (nogo for Oceanic players)

          I made few tests but noting real, the code is available on github

          https://github.com/anykey111/fa-mp-test

          1 Reply Last reply Reply Quote 1
          • Brutus5000B
            Brutus5000 FAF Server Admin @IamFromRussia
            last edited by

            @iamfromrussia said in Private coturn server for my games.:

            @brutus5000 If we know so little about the data protocol, how do you get information about the game starting/ending? How do you get messages from the chat?

            Because there are different datastreams 🙂

            There is a more "high-level" protocol where the game gives information about the game state. This is called GPGNet protocol and it's only sent between Game <-> FAF Client <-> FAF lobby server. This is well known.

            Then we have the game datastream (which is basically the replay format). This is except for few unknown bits well known by now.

            The part you are asking for is the network metadata between the UDP connections of the games and this was never in the focus. If you look at @Surfer's git repository, he reverse engineered a few pieces. But that is basically new work from the last few months.

            If it's not too much trouble, send the discord of the modder who is trying to do this )

            Its @Surfer or anykey111 in Discord.

            He said, "I've been to the year 3000
            Not much has changed, but they live underwater
            And your great-great-great-granddaughter
            Is playin' FAF, playin' FAF"

            1 Reply Last reply Reply Quote 2
            • I
              IamFromRussia @Brutus5000
              last edited by

              @brutus5000 said in Private coturn server for my games.:

              In theory you could try to reverse engineer the binary network protocol and remux the streams that some intermediate software merges outgoing traffic into 1 connection and splits up incoming traffic into "per player" traffic.

              @Surfer @Brutus5000 Why do we need to reverse engineer the binary protocol? Do we need to know what's inside the binary data stream? Why can't we work with binary stream packets like black boxes? Just passing them on without understanding what's inside.

              S 1 Reply Last reply Reply Quote 0
              • S
                Surfer @IamFromRussia
                last edited by

                @iamfromrussia You don't need deep protocol knowledge for such task.

                See the patch fa-mp-test/blob/main/0001-add-proxy-server-draft.patch that completely remove P2P exchange from the ice-adapter and use well known server (main.c)

                Please, state cleary what is the point?
                If you need private server, just host multiplayer game and tell firends your IP, they will join and thats it.

                I 1 Reply Last reply Reply Quote 0
                • I
                  IamFromRussia @Surfer
                  last edited by IamFromRussia

                  @surfer said in Private coturn server for my games.:

                  Please, state cleary what is the point?

                  We usually play in a small group of 4-5 people. When I say: "Let's try to play 8vs8." I usually get the answer: "We'll wait a long time for the game, and then we'll still leave because of the lags."
                  If I could say something like: "I have my own private server and there will be no lag" That would be a strong argument.

                  If FAF has the ability to register your own server in the client through which traffic would pass instead of a P2P connection.

                  If a person who has registered his own relay server creates a game, then everyone who connects to this game will redirect their traffic to the relay server.

                  If this is a high-quality VPS with a static IP, then the connection will be more stable.

                  As an option, you can send an excessive number of UDP packets to the server, for example, send two duplicates for each packet, which will increase the probability of reception on the player's side and remove duplicates in ICE. But here I am not sure about the possibility of implementing this. https://gafferongames.com/post/deterministic_lockstep/ (Yes, this is much more complicated than a simple Relay server)

                  This is a very brief description of the feature, so please don't beat me up XD

                  1 Reply Last reply Reply Quote 0
                  • Brutus5000B
                    Brutus5000 FAF Server Admin
                    last edited by

                    You assume that running your own stable coturn solves all connection issues. Our reports from ICE adapters tell otherwise.
                    The problem is not necessarily unstable coturns, but with issues making a connection even with coturn.

                    He said, "I've been to the year 3000
                    Not much has changed, but they live underwater
                    And your great-great-great-granddaughter
                    Is playin' FAF, playin' FAF"

                    1 Reply Last reply Reply Quote 0
                    • S
                      Surfer
                      last edited by

                      Keep in mind with your own coturn server all 16 players should agree beforehand to use same settings (FAF Client -> "Preferred coturns for ICE").
                      With centralized game server they must also use custom FAF client modification.

                      1 Reply Last reply Reply Quote 0
                      • I
                        IamFromRussia
                        last edited by IamFromRussia

                        @surfer said in Private coturn server for my games.:

                        As for single game server, that should improve connectivity, theoretically.
                        For example in 8v8 the total connection count is 49 (7 per player), with single server its only 7 (1 per player).

                        I still can't understand, does it make sense to combine binary traffic? If it's such a difficult task, is it worth spending time on? I'm not an expert in network programming, but isn't UDP a connectionless protocol https://docs.oracle.com/cd/E19620-01/805-4041/6j3r8iu2f/index.html ?
                        Is there a difference whether the client will receive information on one port, on 16 or on 16,000 ports that the server listens on in case of UDP connection?

                        S 1 Reply Last reply Reply Quote 0
                        • YewY
                          Yew
                          last edited by

                          FAF has been operational for over a decade, and has yet to establish reliable consistent connections between players.

                          You rarely see this issue occur in steam, and the best part in steam is that, if someone does disconnect, you can simply wait for them to reconnect. 9/10 times they successfully reconnect too. I remember when you could do this in faf many moons ago, but sadly this is no longer the case. And you can't reconnect in the client either to try salvage it.

                          Idk what the problem is with faf and players not being able to reliably connect to eachother in game. It's very painful going through 32 minutes of video gameplay only to have some random player randomly disconnect for no apparent reason.

                          I doubt faf will ever fully resolve this issue, I don't think they even know how.

                          Brutus5000B 1 Reply Last reply Reply Quote 0
                          • Brutus5000B
                            Brutus5000 FAF Server Admin @Yew
                            last edited by

                            @yew said in Private coturn server for my games.:

                            I doubt faf will ever fully resolve this issue, I don't think they even know how.

                            This is correct. We don't know. None of us are professional game and/or network engineers. We can not afford to pay for a company to fix it for us. And making ourselves dependent of Steam is also not an option (even all the legal issues aside).

                            He said, "I've been to the year 3000
                            Not much has changed, but they live underwater
                            And your great-great-great-granddaughter
                            Is playin' FAF, playin' FAF"

                            1 Reply Last reply Reply Quote 0
                            • S
                              Surfer @IamFromRussia
                              last edited by

                              @iamfromrussia said in Private coturn server for my games.:

                              I still can't understand, does it make sense to combine binary traffic? If it's such a difficult task, is it worth spending time on? I'm not an expert in network programming, but isn't UDP a connectionless protocol https://docs.oracle.com/cd/E19620-01/805-4041/6j3r8iu2f/index.html ?
                              Is there a difference whether the client will receive information on one port, on 16 or on 16,000 ports that the server listens on in case of UDP connection?

                              Just for optimization to reduce the total number of connections and the numer of individual packets flying through. Every UDP packet is independent, for example 8v8, its about 30 packets on every tick (lets say ~250 milliseconds). The total number of packets is 30164=1920 per second, for each player server should open 15 ports, 15*16=240.

                              For now the biggest problem for me is testing, i need atleast 3 players. using my main PC as host, remote desktop on another PC and old PC under the table.

                              1 Reply Last reply Reply Quote 0
                              • S
                                snoog
                                last edited by

                                If you need help testing feel free to ping me. I work from home so I'm available most of the day anyway.

                                1 Reply Last reply Reply Quote 1
                                • S
                                  Surfer
                                  last edited by Surfer

                                  Ok, now we can mythbust the proxy server approach.
                                  I changed the faf-ice-adapter to use a single TCP socket for all communications.

                                  diag.png

                                  faf-ice-adapter repo
                                  proxy-test repo

                                  After serveral mock tests i didn't see any difference. On weekend i plan to rent a basic VPS and run test in the wild.

                                  1 Reply Last reply Reply Quote 2
                                  • Brutus5000B
                                    Brutus5000 FAF Server Admin
                                    last edited by Brutus5000

                                    In mock tests ICE also always works perfectly 😛

                                    Nobody ever questioned whether one large plain proxy would work. The ICE adapter laid foundations for rerouting the game traffic anywhere.
                                    However it has drawbacks in operational costs and in latency.
                                    And in theory plain ICE connections should always work (with relay as a fallback which basically is single-connection proxy). As such in theory it is also superior to a proxy-solution that does not traffic deduplication.
                                    But in practice (especially Windows and/or security software interference) and for the given software libraries available we see, that it does not hold up the promises.

                                    However under current DDOS situation a proxy server becomes more interesting if it were tunneled through cloudflare websockets...

                                    He said, "I've been to the year 3000
                                    Not much has changed, but they live underwater
                                    And your great-great-great-granddaughter
                                    Is playin' FAF, playin' FAF"

                                    1 Reply Last reply Reply Quote 3
                                    • S
                                      Surfer
                                      last edited by

                                      I have first feedback from 2v1 game, testers say it didn't feel better.

                                      How to join testing?

                                      • backup your FAF Client/natives/faf-ice-adapter.jar
                                      • replace with https://github.com/user-attachments/files/17532274/faf-ice-adapter.zip or build from the source (https://github.com/anykey111/faf-star-adapter)
                                      • host game, invite firends (they must use the same version of faf-ice-adapter.jar as you're using)

                                      Why should i participate?

                                      For the sciene, even if we know that this solution is inferior there is no proofs and people keep returning to this idea.

                                      1 Reply Last reply Reply Quote 1
                                      • S
                                        Sturmgewehr
                                        last edited by

                                        1. Can I see in the logs/debugger which player is in my lobby and has coturn server connections?
                                        2. Can I prevent this by not allowing coturn server connections (so, I could have the IPs and block them as hosts in my firewall, then no more players can get into my lobby - there are only 4 Germany 1 / Finland / USA East / Global of them - or are there several and only the key regions are listed)?
                                        3. When you switch from the lobby to the actual game, are all connections renegotiated, right or wrong? And what happens in the lobby is just a rudimentary check to see if it works at all with the connections?
                                        4. What about the option “Allow the ICE adapter to use IPv6”? It says that it causes connection problems for some players, but what advantage does it have in general (more stable connections when a connection is established?)?
                                        5. What actually causes the lag in a DDoS when players are in the game, if it's just P2P connections? If players are connected to coturn servers and these are then attacked (the coturn servers - are they attacked at all?), or are there other services that limit the game (live replay?)? Does the player still have connections to FaF service servers during the game, which can cause lag (in-game behind issues)?
                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          Sturmgewehr
                                          last edited by

                                          If you block the faf coturn server selected in the settings, you cannot connect to other players.
                                          So a coturn server is always used?

                                          Germany 1 = Hetzner (Dedicated Server?) always same IP
                                          Finland = Hetzner (Dedicated Server?) always same IP
                                          US East = Hetzner (Cloud?) always same IP although cloud?
                                          Global = xirsys / DigitalOcean (Cloud?) mostly always a different IP

                                          As the only coturn server, what does Global actually want with port 80?

                                          Example:

                                          Germany 1:
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [94.130.225.99:3478/tcp, 94.130.225.99:3478/udp] (c.f.iceadapter.ice.PeerIceModule:177)

                                          Global:
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)
                                          INFO Using all viable ice servers: [207.154.234.214:3478/tcp, 207.154.234.214:3478/udp, 207.154.234.214:80/udp, 207.154.234.214:80/tcp] (c.f.iceadapter.ice.PeerIceModule:177)

                                          1 Reply Last reply Reply Quote 0
                                          • AskaholicA
                                            Askaholic
                                            last edited by

                                            Can’t say for sure but using a common web port like 80 or 443 is often an easy way to get through a network firewall since those ports are often allowed in order to allow web traffic. Could be because of that.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post