SC: FA (Moho) engine sources

[IndexLibrorum]

If I understand this correctly, with my limited knowledge of FAF dev, this is a pretty huge deal right?
What amazing work. Big props.

[MayeulC]

Thank you for your answers.

Goal and idea of the project is to have byte-to-byte replay and simulation parity, including visuals - which is fully achieved today for whatever I did tested myself.

I did see several mentions of MSVC in the repository readme, so I was a bit worried there, but it is good that the project can already be compiled with clang.

Everything is kept to original, nothing changed for sim or rendering that changes visual look from the original.

Gotcha. It should be possible to put test assets in the original, take a screen capture, and replay with the test assets in the CI to catch visual bugs, like what emulators do, if you really want to ensure 1:1 fidelity.

If you mean to the repo - I will accept changes that fixes bugs that doesn't affect simulation or any other PRs that fixes things I did missed to repeat original game behavior.

I really meant your repository. There are so many improvements I could think of, more or less user-visible. I listed some, but less-visible improvements could include multi-monitor improvements, steam input integration to make it easier to play on Steam Deck, Discord/Steam rich presence integration, better integration with FAF (integrated lobby, control it via IPC to avoid restarting the game, etc).

Testing Linux build also one of the TODOs, since compiler are clang it's possible but will require x87 math adjustments for the platform.

Yeaaah... I imagine that the game relies on system libraries? Probably not on the windows ucrt "universal c runtime" otherwise we would have seen desyncs on wine, but x87 definitely need some focus. It should be possible to load windows libraries (extracted from MSVC redist) with a small shim (or compiler options) to adjust the calling convention. That would not be my prefered approach though, as it makes it more complicated to support non-x86 platforms (arm) with native code. The other option is to link another controlled FP library, which behaves the same across platforms. This would break replays (the old library can still be embedded to read them on Windows, but new replays would be generated with the new one, unless additional trickery is performed). But you probably thought about all this already.

Something else came to my mind: It should even be possible to compile this to webassembly, and, with a few changes, have a replay player in the browser. The hard part is that assets are copyrighted and distributed with the game, but at least event (commander destruction, experimental unit built, game stats) can be extracted, and maybe a strategic view/minimap provided with custom assets? I am fairly new to FAF (not to supcom, but these decomp projects attracted me to FAF), but I imagine that the community would value that?

[IndexLibrorum]

@MayeulC said:

Something else came to my mind: It should even be possible to compile this to webassembly, and, with a few changes, have a replay player in the browser. The hard part is that assets are copyrighted and distributed with the game, but at least event (commander destruction, experimental unit built, game stats) can be extracted, and maybe a strategic view/minimap provided with custom assets? I am fairly new to FAF (not to supcom, but these decomp projects attracted me to FAF), but I imagine that the community would value that?

Some lightweight and fast replay tool would be much appreciated. I imagine that issues with copyright can be solved by simply not using any of the actual assets. A replayviewer with just strategic icons, for example, would already be very powerful.

[Tokyto_]

This is really cool work! I wanted to see what could be possible with this, so I experimented a little bit

Since I was mapping in the past and fixed water to a single level was quite a frustrating limitation, I made it so it's possible to create water height at different levels / regions.

I now have a small proof of concept with polygon-shaped lakes at different elevations. Ships, submarines, torpedoes and naval factories seem to work in my tests. I also tried AI games and a short two-PC LAN game without a desync, although there’s still a lot more testing to do.

Video: https://www.youtube.com/watch?v=UmmmxQOOND8
Code: https://github.com/Tokyto/faf-re/tree/regional-water
Editor + Explanation: https://github.com/Tokyto/faf-re/tree/regional-water/regional-water-demo

Also had to make a small tool that allows you to make water / lakes regional for said map (saves as EXAMPLEMAP.v0001.scmap.regionalwater alongside the actual map):

A few technical details:

• The engine loads polygon regions and their water heights from the map’s sidecar file.
• Water height queries use the local region at a given position. This needed changes in several places, including unit movement, projectile collisions, building placement and some AI queries.
• The renderer creates separate water surfaces for the regions. So far I’ve tested this with D3D9.
• In my tests, ships could navigate along the shores, submarines could dive, torpedoes and torpedo bombers worked, and AI built a naval factory and ships. Transports could also unload units in the raised water.
• The LAN test was done with the same modified engine and map files on both PCs. It’s a promising start, but definitely not proof that longer multiplayer games are desync-free.

This is an experimental demo, not a finished FAF feature. Big thanks to Draiget and everyone who worked on reconstructing the engine — I hope this shows some of the interesting possibilities and encourages more experiments!

Feel free to use my vibe coding as you see fit

[Nooby]

@IndexLibrorum

This would really speed up my work with

https://forum.faforever.com/topic/7746/unit-logger-mod?_=1793094473252

I currently have a 169 GB postgres database of unit information through time every few ticks of thousands of replays and the game result. I am sure some interesting strategy and balance information could be extracted from this. Currently it is run on a Linux machine through wine but it does have issues with black-screens and removing the graphics entirely would make it much smoother and faster.

[IndexLibrorum]

@Tokyto_ said:

I now have a small proof of concept with polygon-shaped lakes at different elevations. Ships, submarines, torpedoes and naval factories seem to work in my tests. I also tried AI games and a short two-PC LAN game without a desync, although there’s still a lot more testing to do.

Very cool. If you're looking for another related concept, mapmakers have been complaining for years that there is no kind of terrain or option to allow both naval and land units to cross the same spot. Think something like bridges, or 'shallows' like AoE has.

Something like that would be really interesting for us mappers.

[Jip]

This is the most promising step forward for FAForever, with no intentions to refute other work like the alternative version of the game in written in Rust which is also impressive. @draiget and the others that are or were involved - hats off to all of you. This is such a milestone that it may almost make me come back to see it through. The fact that we can now fix fundamental problems with this engine makes for a promising future for FAForever. The problems that Draiget mention with regards to hardening the application are real. Being able to run the game at x64 instead of x86 is also not to be underestimated, as I assume this would also relieve the RAM limit that we often experience. This truly opens the flood gates:

• We can actually debug and understand the engine now. No more going in circles and guessing.
• We can fix networking problems where they occur. No more wrappers like ICE adapters.
• If I read it correct, the whole thing is even backwards compatible (!!)

These two things on their own are so fundamental. Networking problems have been the Achilles heel of this community. It's literally what's been holding us back. Having more, decent performance is a huge plus. All the other things are not quite as relevant. Having water at different elevations is nice, but (and don't take offense by this, please) it is feature creep. Just like 10k vs 10k ASF battles. These are not the things we need. We need a more stable game, where there's less lag. Matchmaker can just launch, without it being a gamble on 4v4 or higher queues. The game is great (and sure, limited to some extend) on its own. It's why all of you are still here. So whatever happens next, please focus on the problems that actually matter first. The problems that we could never really solve, only apply bandaids for.

[Giebmasse]

Welcome back Jip, you know you want to tinker with the engine a bit

[Nombringer]

A question because I'm interested, with the caveat that I'm not sure if this can be actually answered on a public forum:

Do we run into any potential legal issues with this? Is there anything that needs to be considered?

[Draiget]

@Nombringer my repository is a potential legal issue tbh, less that it was initially because I stripped CRI middleware and replaced it with the open source FFmpeg, but still it does do almost 1:1 reassembly of original binary. Yes that's not "original" sources but it's very close. The way we can make it cleaner, is to remove all mentions of "Address: 0x0..." that I've added in every engine function during restoration process and rename every single class\namespace to a custom one, better a clean-room implementation of moho engine with same replay-parity (which is achievable and I did created specs for this already for future). But since FAF model is to require to buy\have original game before playing - I don't think we someone will even be looking on us with lawyers, but who knows for sure.