Supcom/Moho engine rewrite

[hobosapien]

God DAMN this is AWESOME

[Xiaomi]

@slightlygrilled Hi. I'm very interested in the technical details:

• Is the simulation single-threaded?
• Did you use ECS?
• How much slower is fixed-point math?

[SlightlyGrilled]

@Xiaomi Always happy to talk tech

is the simulation single-threaded?
Hell no! I want every core i paid for to be used!
However to be a bit more technical, there are parts that are threaded, and there are parts that are single threaded.

So the tick runs in stages, which you can see listed here:
https://github.com/ThomasMonkman/rts-core-engine/blob/develop/docs/architecture/sim.md#the-tick
normally the way it works is, each stage has calculations done in rust in a threadpool, then lua callbacks happen

so you end up with a spiky pattern of cpu usage,

rust multicore --> lua single thread -> rust multicore -> lua single thread

this is just the nature of how the game was designed, and i am staying combatible to supreme commanders lua.

I do have ideas for a threadpool lua api, that modders and such could use at will to distribute work to as many cores as they please.

steps that are expensive enough to justify giving work to a threadpool have that work split up across how ever many cores you have.

you can see this talked about in this post
https://forum.faforever.com/post/75138

with the image below you can see the thread usage

• each row is a thread
• workers 0-17 is the general thread pool (i have an 18 core machine)
• there is a dedicated sim thread
• dedicated threads for each window/screen
• 6 render worker threads (these and the window threads have higher OS priority so the sim can't stave out the graphics from actually appearing...)

you can see in the screenshot below (this is a 64k airfight so its extremely taxing on the lua)
also its super pixerlated...
the movement, intel, weapons, projectiles, and transfer sim state to the renderer are all threaded

Not to say this couldn't be improved though! always want more core usage!

Did you use ECS?
No, right at the very start i looked at it, and as much as i think its a fantastic data structure, it just doesn't fit supcoms design, as remember i wanted it to be a drop in replacement.
And in the lua phases there is a ton of random access.

I do however take in to account the ideas of ECS in terms of cpu cache awareness, and all the "hot" data (data that is accessed in tight loops) is all packed in to dense arrays, with "cold" data stored elsewhere.

Fixed point vs floating point

I just ran a quick benchmark for you, of course this is a mirco benchmark, so take with a massive massive grain of salt, as normal code isn't like this, but its the raw numbers.

│ Operation │ Fx (I32F32) │ f32 │ f64 │ Fx vs f32 │
│ add │ 0.48 ns │ 0.49 ns │ 0.50 ns │ ~1× │
│ mul │ 0.50 ns │ 0.49 ns │ 0.49 ns │ ~1× │
│ div │ 3.9 ns │ ≤0.49 ns │ ≤0.49 ns │ ≥8× │
│ sqrt │ 5.4 ns │ ≤0.48 ns │ ≤0.47 ns │ ≥11× │
│ atan2 │ 12.9 ns │ 2.1 ns │ 4.6 ns │ ~6× │
│ sin_cos │ 10.4 ns │ 2.3 ns │ 3.5 ns │ ~5× │
│ vec3 normalise │ 15.5 ns │ 1.0 ns │ 1.1 ns │ ~16× │
│ quat multiply │ 5.5 ns │ 1.1 ns │ 1.6 ns │ ~5× │

as you can see it depends what you are doing.

But what i would say, from my posts above, i don't really believe it matters even though floating point is faster, my engine is still super fast, and im almost fully limited by the single thread nature of supcoms lua.

[SlightlyGrilled]

@hobosapien Thank you

[SlightlyGrilled]

@The_Janitor Can you send me the log file please? it should be right next to the exe

[SlightlyGrilled]

@IndexLibrorum Hello! I think i have actually spoke to you a bit on the discord, when I've posted links, i think you had ideas on the keyframe stuff with the replays right? well know there in!

I'm always happy to chat tech, or anything like that

On the speeding up tests, that's actually a great idea and didn't occur to me! thank you
I spent a bit of effort to load in the maps as fast as possible before, a main bottleneck for me is some of fafs start up lua is quite slow, although i did some work to speed that up, and the tests do run faster now.
In reality i run the tests in parallel without the window, that i just measured, completes in 9.49 seconds on my machine.

I did the screen recording just for visuals

But faster is always better!

[The_Janitor]

@SlightlyGrilled
https://limewire.com/d/DWWf2#9gbKJxH9UW
game.log is uploaded on the link, text size too big to place here, sorry.

[SlightlyGrilled]

@IndexLibrorum you peaked my interest on the map loading, so i went and actually timed it.

Both the total time with graphics running in a window.
and if i run it headless without the graphics.

Theta Passage (SCMP_012), 5 km
total 0.89 s | map textures ready 0.71 s | sim running 0.75 s
(headless total 0.47 s)

Open Palms (SCMP_007), 10 km
total 0.93 s | map textures ready 0.80 s | sim running 0.78 s
(headless total 0.50 s)

Seton's Clutch (SCMP_009), 20 km
total 1.04 s | map textures ready 0.88 s | sim running 0.87 s
(headless total 0.58 s)

Gentleman's Reef (SCMP_005), 40 km
total 1.38 s | map textures ready 0.87 s | sim running 1.20 s
(headless total 0.83 s)

Betrayal Ocean (SCMP_029), 81 km
total 1.51 s | map textures ready 1.06 s | sim running
(headless total 0.91 s)

Frostmill Ruins (SCMP_030), 81 km
total 1.51 s | map textures ready 1.02 s | sim running 1.34 s
(headless total 0.94 s)

Debris (X1MP_012), 81 km
total 1.45 s | map textures ready 1.21 s | sim running
(headless total 0.82 s)

[SlightlyGrilled]

@The_Janitor Thanks I will have a look!

[SlightlyGrilled]

@Dim-A-miD yeah its super cool!

In case three isn’t enough and you’d like to bring the total to five...

I think this, and your link goes way over my head?

[SlightlyGrilled]

@The_Janitor ok found it, i will release a new version that fixes this, but in the mean time, in your mod folder you have a file mods/auto-upgrading mass extractors.7z

my engine is picking this up and thinking its a zip file. if you move that file it should work

but i will also fix this in the next version.