Does Supcom work with windows 11?

Jip · 21 Dec 2021, 07:13

@corsaire said in Does Supcom work with windows 11?:

I mean having two computers giving different results for a same calculation is kind of, bad.

This happens all the time. Particularly floating point numbers (in comparison to doubles) is easy prey to giving different results. It may be small, but if it determines whether something got hit then suddenly you have a desync. You can read this article that scratches the surface of the issue.

Corsaire · 21 Dec 2021, 08:13

@jip That's a nice article you got here.
Interestingly it points to a possible solution for the case we are facing.

Jip · 21 Dec 2021, 11:02

A solution if we had access to the source code of the engine - which we do not :sad_cowboy:

Katharsas · 21 Dec 2021, 22:04

We would not be able to rip out floating points out of the egine even if we had source code access. Maths, graphics and physics libraries for games and even C++ compilers all usually are written explicitely for floating point numbers instead of simulating it with fixed point arithmetic. Switching away from floating point, if at all possible, would probably result in an engine too slow to be usable.

And for little reason, becasue the imprecision of floating point is not really the problem here. A problem only appears if the imprecision varies from machine to machine. If you use the exact same hardware/software, you will still have floating pint imprecision, but it will be the exact same imprecision on every machine, so the game will not desync.

However, when hardware is emulated you often run into problems where the emulation has to do things slightly different than real hardware for performance reasons.

Anyway that doesn't matter much because we still need engine source to find problems with the strictness of FP operations.

Jip · 22 Dec 2021, 12:00

@katharsas said in Does Supcom work with windows 11?:

And for little reason, becasue the imprecision of floating point is not really the problem here. A problem only appears if the imprecision varies from machine to machine. If you use the exact same hardware/software, you will still have floating pint imprecision, but it will be the exact same imprecision on every machine, so the game will not desync.

I think this is incorrect.

In the CPU when processing floats they are processed using full precision instead of 32 bits. However, when the thread is taken off the processor the intermediate result is stored back into 32 bits (as it is a float and not a double). This can cause any computation to divert.

I think in FA they use double precision for everything. I know that at least Lua uses double precision by default.

Corsaire · 22 Dec 2021, 13:14

I was thinking, it is probably a very naive thought but, can the calculation be done by another code, like a mod?
Or is it something that is done by a part that is untouchable?

Dragun101 · 22 Dec 2021, 19:44

@jip said in Does Supcom work with windows 11?:

A solution if we had access to the source code of the engine - which we do not :sad_cowboy:

We do legit have access to alot of the games' code. While not helpful for above problem I do think it is worth stressing that we do have alot going for other games don't.

Corsaire · 23 Dec 2021, 01:02

@dragun101 said in Does Supcom work with windows 11?:

We do legit have access to alot of the games' code. While not helpful for above problem I do think it is worth stressing that we do have alot going for other games don't.

hmmm, so you think that it would be possible to have a look into this matter?

Dragun101 · 23 Dec 2021, 02:32

No, Jip knows the code better than I do. I am just saying how lucky we are in general.

Katharsas · 23 Dec 2021, 09:40

@jip said in Does Supcom work with windows 11?:

@katharsas said in Does Supcom work with windows 11?:

And for little reason, becasue the imprecision of floating point is not really the problem here. A problem only appears if the imprecision varies from machine to machine. If you use the exact same hardware/software, you will still have floating pint imprecision, but it will be the exact same imprecision on every machine, so the game will not desync.

I think this is incorrect.

In the CPU when processing floats they are processed using full precision instead of 32 bits. However, when the thread is taken off the processor the intermediate result is stored back into 32 bits (as it is a float and not a double). This can cause any computation to divert.

I think in FA they use double precision for everything. I know that at least Lua uses double precision by default.

I think you are talking about putting values into/out of registers, because speaking of threads does not make much sense in this context. If your CPU context switches your thread into/out of the hardware, it should also restore/backup any register values.

The only thing that your CPU is doing is executing instructions. If the same instruction does the same thing on every CPU that you want to support, you can safely use that instruction without ever running into problems. I am not sure why expanding a float to a double and back would change anything about that.

If you know to which instructions your code is compiled by your C++ compiler, you can probably make sure that only the ones are used that are consistent for every CPU that is relevant.

Here is an example of the amount of options you have on windows these days:
https://docs.microsoft.com/en-us/cpp/build/reference/fp-specify-floating-point-behavior?view=msvc-170

I have no idea if those options are enough or if they result in code that is too slow, there are usually a ton of trafe-offs to consider here. For a usefull insight we would have to

know which instructions create the problem for the emulator
why and how often the game uses those instructions
if those instructions can be replaced with an equivalent that does not run into the same problem

@Corsaire
In general nothing is really impossible, but you would need to do reverse engineering and answer all the questions stated above. And even if you know the exact problem it it might be unfeasable to patch (for example due to problematic instruction(s) being used everywhere in the code or the replacement instruction(s) being larger than the original instruction).

Jip · 23 Dec 2021, 10:43

@katharsas said in Does Supcom work with windows 11?:

The only thing that your CPU is doing is executing instructions. If the same instruction does the same thing on every CPU that you want to support, you can safely use that instruction without ever running into problems. I am not sure why expanding a float to a double and back would change anything about that.

Because a double has more precision. You can add 0.0000000001 a 1000000000 times to 1000000000 to get 1000000001 when you're working with doubles. But when you work with floating point numbers the number of bits in the mantissa is not sufficient to represent that smaller number in the context of the larger number.

When you work purely with floating points you'll get ~1000000000 .
When you work purely with doubles you'll get ~1000000001
And when you work with floating points that are expanded on the CPU then you'll get something in between the two, depending on when the context switch(es) happened. Context switches are indeterministic and therefore so is the final result.

I've run into this issue in practice. I think this is also why the technical artist of the game mentioned it was difficult to run the same simulation on different systems, to keep them in sync.

Corsaire · 24 Dec 2021, 00:53

Do you think thee is a way around it? Or it is a lost cause.
I can use my setup to do some tests if needed, just keep in mind I'm not a developer myself so I may not have the tools you'd expect immediately at hand.

Another thing I'm curious about, I ran into the problem trying to game with friends that are on PC, none of them has a M1 mac, but how does it behave in a game between two M1 mac?

Jip · 24 Dec 2021, 06:38

This is out of our hands. The expertise, time investment and dedication that is required is just not here.

Katharsas · 3 Jan 2022, 05:14

I still don't get it, a context switch should save the entire size of the FPU register stack (each 80bits) when a context switch happens, and should then restore it when you thread resumes. The context switcher does not even know if the registers contain doubles or expanded floats, so it needs to save all bits anyway.

Since each FPU register is 80 bits on x86, you have expansion anyway regardless of whether you use floats or doubles, because doubles are only 64bit.

Do you have a link where that issue is described in a bit more detail because i feel there is something missing here.

owenj7920 · 3 Jan 2022, 05:14

This post is deleted!