Is it worth me writing a program to check the network?
-
No we lost a few message on a forced server restart.
-
@youthenoob said in Is it worth me writing a program to check the network?:
I already work in software development and networking, and am now wondering whether it is worth my time (perhaps a day to two) to write some software to test/record the network performance.
It's always nice to have more data on network performance for investigation.
So before I start investing/wasting too much time on this I thought i would get your opinions. The things I was looking at testing were:
- Ping times to the local gateway, to see if there is jitter (caused by the wifi/bad cabling/s**t router).
- Ping times to a distant IP address to see if there are ISP issues
You can do some latency tests, although I usually just point people to ping.canbeuseful.com for running a latency test during the game, I think weak WiFi links aren't the issue in 99.9% of games, most people are smart enough to think of that themselves.
- iPerf (bandwidth tester), to see the max bandwidth the user has
That would be quite interesting, you could add information to your tool that tells the user what games they will be able to run (2v2, 4v4, 5v5, 6v6, ...). Bandwidth scales rather linearly with peer count, up / down bandwidth is symmetrical.
- The bandwidth used during gameplay, to see if something else is taking all of it (vs total bandwidth).
If you can monitor / obtain info about other system traffic that might be interesting to discover background load, like steam updates.
Traffic in FA consists of 2 packets per peer per tick. 10 Ticks per second. One of those packets is ~15 bytes, the other one is ~25-70 bytes.
- Nvidia driver version (to test for the old driver issue)
- Whether the user is using Global Carrier NAT; games get routed through a relay and ping times are added.
That would also be quite interesting.
A lot of data is also available from the ice adapter debug window, which is currently not visible due to the client not having a JRE with JavaFX.
The ICE adapter already measures latency to each peer.
It can also tell you which candidate pair it chose for each peer (host, prflx, srflx, relay), from which you can infer information about network and NAT architecture.The best option would probably to add some of those measurements to the ice adapter itself, as it has already gotten access to the traffic as well as quite a bit of other debugging information. You can e.g. just start counting game traffic here: https://github.com/FAForever/java-ice-adapter/blob/master/ice-adapter/src/main/java/com/faforever/iceadapter/ice/PeerIceModule.java#L390
Not sure if that's an option for you though, language-wise as well as taking into account you want to access system traffic info and want to perform measurements to local gateway.
-
So my takeaway from this:
- Yes it's worth me doing, even if it isn't in Java.
- It would be much much better if I did it in Java. If I did it in Java it could be incorporated into the client.
- People have already been looking at in-game networking tools and changing networking settings to improve performance, but there doesn't appear to have been a lot that has been implemented or is available for "non-techies".
Thanks for the help guys. I'm semi-tempted to learn Java just to help with FAF. I've spent literally thousands of hours playing it (i'm still not any good) and I haven't paid a penny, so a bit of my spare time to help seems like the least I could give.
-
Yes, pay with your soul
-
This is a really cool idea! would be great if it was added to the client lobby.
There are many aspects to how well a network is running, it is not always oops i was downloading it could be faults in the line or issues with the ISP. Some people think i can watch youtube my connection is working perfectly however this is not always the case. -
Famous last words... "it will take me a few days"....
To make sure people don't think that I've just forgotten about this, I'm still working on this but as a small project, it's actually a bit more complicated than I first thought. I am continuing with this but it's going to take me a bit longer to make something worthwhile.
FYI on the on challenges:
Checking for network jitter
- Creating a program to send out multiple pings a second to multiple IP addresses is actually a bit more complicated than I first thought. Sending a single ping is easy, sending 15 a second, to each hop, to a distant address, is far more complicated. It involves using multiple threads and then ensuring the task is disposed of after it has been run, otherwise, the program uses increasing resources. I understand why there are 0 freely available tools to diagnose this stuff, since it's not a 5-minute job.
Baby steps, the current target is to just ping the local gateway and a remote IP address 10 times a second, this should give a good indication of network jitter.
Driver Check
- So my code runs a windows WMI query to get driver details by searching for "Nvidia" but there are also audio drivers so working out which one is the graphics card is still something im pondering. I personally know my model so I can easily see it, however making sure this works with every Nvidia graphics card is something I still need to work out.
Realtime reporting
- I looked at having a small panel that would overlay the active game, showing network performance but this means that DirectX needs to be used directly, instead of using a simple Winform as Forged Alliance overrides the display to show the game.
Wifi or Cable
- Even this isn't a simple task. This involves more than a few things to get right. It's not as easy as just checking a network interface status, since you can have wifi and cables both connected. I have to write some code to look at the windows routing table to work out which interface is being used when both interfaces are on the same network.
So as a little project, this is still ongoing. The amount of time that it is taking, I might just learn Java.
Does anyone know how an easy way to get the other players IP address? I was looking at pinging the other player's IP addresses to confirm if they are still reachable.
-
It involves using multiple threads and then ensuring the task is disposed of
I prefer using an async event loop. There are lots of different ones out there in lots of different languages. Netty for Java, asyncio for Python, tokio for Rust, and Go has it built in. When I see threads in networking code that kinda raises a red flag for me since it’s gonna be a lot more resource hungry and harder to get right than a simple async app. Just my opinion.
BTW, I just saw that Downlord was making a basic status checking tool to integrate into the FAF client. https://github.com/FAForever/faf-status
Not sure exactly how it’s gonna work. -
BTW, I just saw that Downlord was making a basic status checking tool to integrate into the FAF client. https://github.com/FAForever/faf-status
Not sure exactly how it’s gonna work.It's basically just an uptime monitor. It won't tell you anything about your network connection.
-
@askaholic Thanks for the info. I won't bore you with all the details but there are lots of little things that make this a bit more complicated.
C# has async methods too. They return an object called a Task, which creates a new thread for the method but I need to manually dispose of them due to hundreds of pings running at the same time and all returning a value after different timespans. In C# the issue becomes one of disposing of the task once itself has finished (received the ICMP response). What makes this more complicated is
- ensuring that a ping is sent every 100ms (no time drift)
- resources are cleared up
- ping responses are stored in an array with the correct position (accounts for misordered responses).
- Writes to disk the result once all the pings have been received or timed out.
Microsoft has some basic info here but the problem with these examples is that the tasks are done once. My tasks need to repeat without knowing when the response will be finished.
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/async/I think the way I will do this, is that I will have to stagger the tasks so that there are three threads doing this. One will be pinging, one will be waiting and one will be writing to disk and clearing up resources.
There might be a much much easier way to do this and I'm not seeing the wood for the forest.
-
So still working on this. Wife isn't too happy because I get a bit obsessive and spend longer on these things than I really should.
So some technical questions:
- How does FAF decide whether to use a relay or a direct connection to the other users? Eg does it do it on ping responses? Is there a response from the client that is expected?
- If it is down to a response from the client, what is the best way to get a list of all the other IP addresses of the other users/how are the connections managed?
- Is there any documentation on how the FAF client/Ice adapter works? Apologies but I don't know Java and even if I did, looking through the whole project for find these facts might not be the most time-efficient.
- What is the best way to speak to someone involved in the actual project? I can see the Discord group has a list of developers and I get the feeling that I will have a lot more questions, at the same time I don't want to bother anyone by constantly asking questions, hence why I'm asking here.
The reason I need this information is I want my program to be able to advise the user on how to modify their system so the relay isn't needed (direct connection) and how to improve jitter. Living in Australia a user using the relay can add a lot of latency.
-
@Geosearchef is the author of the ice adapter and knows the most about the connection space.
-
Dev talk happens mostly on our zulip chat. You can ask @Brutus5000 or @Giebmasse for an invite. People would be happy to answer any questions on there.
-
@youthenoob If you look on the https://wiki.faforever.com/index.php?title=Connection_issues_and_solutions
Another reason for this behavior could be your NAT (in your router) assigning packet traveling out a random port instead of the same port as used in the local network. You can try finding out if your router does this and if it does take a look at its configuration if you're able to set it to mapping ports "static"ally.
So a good test is to check if the Outbound NAT is using a static source port.
This is how you fix it in pfsense
-
Another thing that could be added is showing the NAT TYPE like in Start, select Settings > Gaming > Xbox Networking, and look for NAT Type under Xbox Live multiplayer
-
No, it is not too late. And It won't be untill all system level software will use some other language. In my humble opinion,one should not expect such event ...
Networking adds a lot of power to simple programs. With networks, a single program can retrieve information stored in millions of computers located anywhere in the world. A single program can communicate with tens of millions of people. A single program can harness the power of many computers to work on one problem.What does a network programmer do? Network programmers make it possible for gamers to play with each other. They develop the code behind online multiplayer games so that everything that happens in a game gets transmitted from one machine to another.
https://192168l254.com.mx/ES
