Why would you have left FAF?

Blodir

@redx said in Why would you have left FAF?:

Even if you could somehow convince Nordic to care about FAF, you would have to completely rewrite the game to completely solve lag and connection issues. The reason the game is so susceptible to connection issues is literally the very core of how the game's simulation engine works. You'd literally be better off porting all of the game mechanics and units to Spring or something than trying to change SC:FA into a client/server game.

Spring uses deterministic lockstep like supcom, besides you don't want an authoritative server RTS regardless.

Connection issues are in no way 'core' to how the game works. You wouldn't need to rewrite the game to solve network lag.

You'd most likely have to make significant changes to increase the sim tickrate (source of most of what appears as input lag) gracefully though.

Nex

@blodir Isn't the input lag something that's deliberate (and not related to the sim)?
Because you need the input lag, to compensate for potential network lag.
That's why the game has a standard input lag of X ms, so people that have X ping to each other can still play together without stuttering.
That is because the engine expects input from every player on every tick, so if you have a higher ping than the input lag, the game micro freezes till all inputs arrive.
Changing the simulation tick rate shouldn't matter, but you'd need to rewrite how the engine works to be able to solve this lag problem. You'd need to introduce some kind of rollback code, that when there is no input from a player continues the simulation, but once the input for an earlier frame arrives, you'd need to roll back to that state and recompute up to the current tick (that's how most fighting games work), but that's a pretty heavy workload for a complex simulation like supcom (that doesn't even manage to run in real time sometimes).

As far as I understand springRTS, it handles it by having a server-client architecture, with the server as the single source of truth, so when someone lags behind they'll just get updated to the state of the server.

Blodir

@nex said in Why would you have left FAF?:

@blodir Isn't the input lag something that's deliberate (and not related to the sim)?
Because you need the input lag, to compensate for potential network lag.
That's why the game has a standard input lag of X ms, so people that have X ping to each other can still play together without stuttering.
That is because the engine expects input from every player on every tick, so if you have a higher ping than the input lag, the game micro freezes till all inputs arrive.
Changing the simulation tick rate shouldn't matter, but you'd need to rewrite how the engine works to be able to solve this lag problem. You'd need to introduce some kind of rollback code, that when there is no input from a player continues the simulation, but once the input for an earlier frame arrives, you'd need to roll back to that state and recompute up to the current tick (that's how most fighting games work), but that's a pretty heavy workload for a complex simulation like supcom (that doesn't even manage to run in real time sometimes).

As far as I understand springRTS, it handles it by having a server-client architecture, with the server as the single source of truth, so when someone lags behind they'll just get updated to the state of the server.

Tick n:

collect pooled inputs during the time between tick n-1 and now and send to other players. Mark these inputs as destined for tick n+1
run simulation with inputs destined for tick n (if inputs for tick n have not been received from all players, then pause)

So with 500ms sim tick length if you enter input (eg. give a move command to a unit) 10ms into tick n-1, then the input will be sent at 500ms, and executed at 1000ms. That is a 990ms input delay, which is tied to the sim tick rate.

Blodir

@nex said in Why would you have left FAF?:

As far as I understand springRTS, it handles it by having a server-client architecture, with the server as the single source of truth, so when someone lags behind they'll just get updated to the state of the server.

I don't know much about springRTS, but based on quick googling this is incorrect and it works the same as supcom and just about every other RTS. Note that the client-server network topology is not mutually exclusive with deterministic lockstep. When people refer to a server in the context of games like eg. sc2 they are referring to a proxy server that facilitates connections between the players (just like what we have in faf!) (though it is possible to have some extra functionality too, just not the kind that you are describing)

Nex

@blodir the current tick rate is 100ms tho, so doubling(halving) that to 50ms will only give you a marginal increase in responsiveness compared to the current 150(?) ms of input lag

Nex

@blodir said in Why would you have left FAF?:

@nex said in Why would you have left FAF?:

As far as I understand springRTS, it handles it by having a server-client architecture, with the server as the single source of truth, so when someone lags behind they'll just get updated to the state of the server.

I don't know much about springRTS, but based on quick googling this is incorrect and it works the same as supcom and just about every other RTS. Note that the client-server network topology is not mutually exclusive with deterministic lockstep. When people refer to a server in the context of games like eg. sc2 they are referring to a proxy server that facilitates connections between the players (just like what we have in faf!) (though it is possible to have some extra functionality too, just not the kind that you are describing)

Might entirely be I just looked at what was posted above
https://springrts.com/phpbb/viewtopic.php?t=37637
and how the syncing works:
https://springrts.com/wiki/Syncing_System
these all mention a server/host, which must not necessarily be a dedicated server, but there is still a host and clients in the architecture

Blodir

@nex said in Why would you have left FAF?:

@blodir said in Why would you have left FAF?:

@nex said in Why would you have left FAF?:

As far as I understand springRTS, it handles it by having a server-client architecture, with the server as the single source of truth, so when someone lags behind they'll just get updated to the state of the server.

I don't know much about springRTS, but based on quick googling this is incorrect and it works the same as supcom and just about every other RTS. Note that the client-server network topology is not mutually exclusive with deterministic lockstep. When people refer to a server in the context of games like eg. sc2 they are referring to a proxy server that facilitates connections between the players (just like what we have in faf!) (though it is possible to have some extra functionality too, just not the kind that you are describing)

Might entirely be I just looked at what was posted above
https://springrts.com/phpbb/viewtopic.php?t=37637
and how the syncing works:
https://springrts.com/wiki/Syncing_System
these all mention a server/host, which must not necessarily be a dedicated server, but there is still a host and clients in the architecture

First link the first reply clarifies that it's regular ol' deterministic lockstep. However, the second link appears to describe a re-syncing feature. Yea, this is a hybrid approach where the system falls back on authoritative server approach if the parallel simulations get out of sync. As a downside whoever is selected as the authority gets to cheat to their heart's content

Blodir

@blodir said in Why would you have left FAF?:

@nex said in Why would you have left FAF?:

@blodir said in Why would you have left FAF?:

@nex said in Why would you have left FAF?:

As far as I understand springRTS, it handles it by having a server-client architecture, with the server as the single source of truth, so when someone lags behind they'll just get updated to the state of the server.

I don't know much about springRTS, but based on quick googling this is incorrect and it works the same as supcom and just about every other RTS. Note that the client-server network topology is not mutually exclusive with deterministic lockstep. When people refer to a server in the context of games like eg. sc2 they are referring to a proxy server that facilitates connections between the players (just like what we have in faf!) (though it is possible to have some extra functionality too, just not the kind that you are describing)

Might entirely be I just looked at what was posted above
https://springrts.com/phpbb/viewtopic.php?t=37637
and how the syncing works:
https://springrts.com/wiki/Syncing_System
these all mention a server/host, which must not necessarily be a dedicated server, but there is still a host and clients in the architecture

First link the first reply clarifies that it's regular ol' deterministic lockstep. However, the second link appears to describe a re-syncing feature. Yea, this is a hybrid approach where the system falls back on authoritative server approach if the parallel simulations get out of sync. As a downside whoever is selected as the authority gets to cheat to their heart's content

From the first link it does sound like they have an ad-hoc server implementation that gets to determine the destination tick of each input packet so the high ping players experience more input delay. Unfortunately this would require supcom source code and a new proxy server implementation (but it is still the same network model and only a minor optimization). Again didn't look too closely tho just skimmed...

Nex

yeah it all basic lockstep, but it uses some proxy like host-server (that doesn't run it's own simulation), that decides how fast the simulation runs, who is out of sync and it can somehow reorder the clients inputs.
Also apparently the lag just increases for slower cpus over the whole game time

players with slower-than-average CPU's will fall further and further behind the reference time but can stay ingame with increasing order latency

I don't know how long bar games can last, but a lot of supcom games are played on minus-speed for extended periods of time, so this approach also might break the game for a lot of people

BlackYps

yeah if I understand this correctly it essentially means that people will fall behind further and further until they are several minutes behind. So this doesn't sound like a feasible solution at all.

ZLO

@blackyps said in Why would you have left FAF?:

yeah if I understand this correctly it essentially means that people will fall behind further and further until they are several minutes behind. So this doesn't sound like a feasible solution at all.

But this does mean that game only lags for a lager instead of lagging for everyone and game does not freeze when someone has lost connection.

it also may be considered to not be feasible that everyone lags when 1 of the 16 players lags.

it also allows players to spectate the game without lagging it to much

in Beyond all Reason i have seen as much as 50 spectators in 16 vs 16 game, or about 60-70 in 8v8 game

yes it does mean that you can fall behind by any number of seconds or minutes
but it does sound fine to me for casual games
in comparison for game being unplayable

also in BAR 8v8 someone can crash for example on minute 30
and players can decide to wait for him to launch the game again and catch up or just take his units and keep playing and if he will logs in again and load the game again and catch up to other players they can give him his base back.

in tourneys i see this fall behind feature be less desirable... but can probably make game pause automatically if players fall behind more than by 2 sec or something like that.

in case of sim lag i think in BAR it works that game does slow down to wait slow PC players but not always... it depends and idk how exactly it works there

Tho idk why were are discussing this here... don't think we can add this functionality to FAF

Nex

@zlo said in Why would you have left FAF?:

it also allows players to spectate the game without lagging it to much

That's basically how FAF Live replays currently work.
You have some server that gets all the info from the players saves that and then distributes it to all spectators at a slower rate.

@zlo said in Why would you have left FAF?:

also in BAR 8v8 someone can crash for example on minute 30
and players can decide to wait for him to launch the game again and catch up or just take his units and keep playing and if he will logs in again and load the game again and catch up to other players they can give him his base back.

Yeah a reconnect mechanic would be cool, but I don't think you can somehow get the game to load a replay, then after some time swap from watching replay into one of the players positions and then somehow make the game believe you are actually connected to an online lobby playing this game, without a lot of work on the engine side.

@zlo said in Why would you have left FAF?:

in case of sim lag i think in BAR it works that game does slow down to wait slow PC players but not always... it depends and idk how exactly it works there

From the link above it seems like it balances the speed around the average cpu rating of all players and people slower than this might fall behind.

@zlo said in Why would you have left FAF?:

Tho idk why were are discussing this here... don't think we can add this functionality to FAF

Well lag and desyncs are something that make people quit the game, but yeah fixing it is probably out of scope of FAF (for now).
But discussing the solutions other projects use is kinda fine here I think. Better than just telling people that make those suggestions "it won't work." without any explanation.

ZLO

@nex said in Why would you have left FAF?:

@blodir the current tick rate is 100ms tho, so doubling(halving) that to 50ms will only give you a marginal increase in responsiveness compared to the current 150(?) ms of input lag

Current inputlag is 500 ms

(BAR players say that they don't like that aspect of FAF)

Nex

@zlo said in Why would you have left FAF?:

Current inputlag is 500 ms

huh interesting
my brain somehow had this number 150 saved for some reason.
Well with 500ms input lag assuming the sim tick "lag" comes on top that is currently 50ms on average (100 max) so you could reduce the input lag by 5-10%, by doubling the sim tick rate.
Doesn't sound like a good trade tbh.

ZLO

@nex said in Why would you have left FAF?:

@zlo said in Why would you have left FAF?:

Current inputlag is 500 ms

huh interesting
my brain somehow had this number 150 saved for some reason.
Well with 500ms input lag assuming the sim tick "lag" comes on top that is currently 50ms on average (100 max) so you could reduce the input lag by 5-10%, by doubling the sim tick rate.
Doesn't sound like a good trade tbh.

i think the inputlag is mostly defined by the ping that users usually get.
i am not sure but it is just if ping is higher than inputlag then the game will start to feel laggy, choppy and stuff like that.

Exselsior

I mostly skimmed through the most recent messages but two things - I think Blodir said this but networking code is separate, so that would be much easier to fix if we had source code access and wouldn’t take a full game engine rework iirc.

A rather important detail that I haven’t seen mentioned though: even if, and that’s a very big, expensive, and unlikely if, we got the rights to SC:FA that doesn’t mean we get the source code. Afaik our current understanding is that the original source code is lost. Meaning we’d still be stuck with assembly patches and reverse engineering which does us little good and makes getting the rights to SC a moot point in terms of making game engine/networking related improvements.

Blodir

@nex said in Why would you have left FAF?:

@zlo said in Why would you have left FAF?:

Current inputlag is 500 ms

huh interesting
my brain somehow had this number 150 saved for some reason.
Well with 500ms input lag assuming the sim tick "lag" comes on top that is currently 50ms on average (100 max) so you could reduce the input lag by 5-10%, by doubling the sim tick rate.
Doesn't sound like a good trade tbh.

This is not correct. Here's some approximations for magnitudes of different latencies:

Sim/physics update/input sampling. Note, as I said earlier, that the worst case scenario is that the length is doubled for an input! So if supcom runs at 10hz (which idk if it does) that would be worst case of 200ms delay! (or more, if the inputs are scheduled to be processed at an even later tick)
Framerate, monitor offsync with framebuffer and vsync. This on the scale of <16ms for 60fps.
Display lag, input devices, drivers, and so on. <5ms

Network latency between two players with reasonable ping is not at all relevant, because the sim tick rate dominates!

Like say you are playing an 1v1 game with someone with a direct connection (without proxy). You can tell from experience that you still experience a comparable amount of input lag. Keep in mind that ping refers to round-trip-time. So if you have 40ms ping between two players, that would amount ot only 20ms one-way latency. So when you are sending an input packet at tick n that is destined for tick n+1, the other player will receive it 20ms later - even though, assuming 10hz tickrate, there's still 80ms of the tick left! So we can easily expect that at low ping and slow sim rate contexts network latency is practically irrelevant.

Also note that if someone is late, the time window might be extended multiple ticks in the future so eg. inputs collected at tick n would be processed at tick n+2 instead of n+1. A higher sim tick rate gives more flexibility in this sense, where a slower one introduces a large additional delay in the worst case.

veteranashe

Unless we plan on re enginein faf into spring I'm not sure how this benefits us at all or the topic of players leaving

nflanders

@ZLO @Blodir @Nex @BlackYps We're completely lost in our dreams XD. We have not yet found a HERO who will negotiate on behalf of FAF. Legal and diplomatic challenges seem more difficult.
We need a hero)

But there's no harm in dreaming, so let's continue)

@zlo said in Why would you have left FAF?:

yes it does mean that you can fall behind by any number of seconds or minutes
but it does sound fine to me for casual games
in comparison for game being unplayable

I note that it is assumed that all players will play through the coturn server and there will be no peer-to-peer connection. Therefore, at the stage of connecting to FAF, we can detect the loss of packets when connecting to the coturn server. I think this solution will already weed out 90% of lagers whose pockets are lost. And the SpringRTS network system that we are discussing will eliminate micro-lags.

As I understand it, if you experience a lag, this does not mean that you will lag behind the entire game; if you have new hardware, for example i3-12100 (good single-threaded performance), then in theory you will catch up with the rest very quickly.

veteranashe

@nflanders said in Why would you have left FAF?:

@ZLO @Blodir @Nex @BlackYps We're completely lost in our dreams XD. We have not yet found a HERO who will negotiate on behalf of FAF. Legal and diplomatic challenges seem more difficult.
We need a hero)

But there's no harm in dreaming, so let's continue)

@zlo said in Why would you have left FAF?:

yes it does mean that you can fall behind by any number of seconds or minutes
but it does sound fine to me for casual games
in comparison for game being unplayable

I note that it is assumed that all players will play through the coturn server and there will be no peer-to-peer connection. Therefore, at the stage of connecting to FAF, we can detect the loss of packets when connecting to the coturn server. I think this solution will already weed out 90% of lagers whose pockets are lost. And the SpringRTS network system that we are discussing will eliminate micro-lags.

As I understand it, if you experience a lag, this does not mean that you will lag behind the entire game; if you have new hardware, for example i3-12100 (good single-threaded performance), then in theory you will catch up with the rest very quickly.

I will do it, what's the budget?