So this is a really big update from my side, tons of things, internally I use issues to track what I have and haven't done, so this time around I've put the issue numbers on what I've done
Open source
First off I've finally released the source code, it took a ton of effort to make this ready for other people, and I'm sure it still needs more, so please be patient with what you find
I have made it as a fresh repo, as understandably when you work on something yourself which you're never sure may see the light of day, you can get sloppy, and I didn't want to risk passwords or what ever being public.
I'm happy for people to raise issues for bugs and such
and I hope people have fun playing around
source code
https://github.com/ThomasMonkman/rts-core-engine
exes:
https://github.com/ThomasMonkman/rts-core-engine/releases/tag/v0.1.0
#367 Replay keyframes: fast seeking, played ahead in the background
Replays now support quick rewinding! and skip forward (to degree)
when starting a replay, a second process now starts up in the background and plays the replay at max speed saving out "keyframes" of the game. You can see this in the video below as a buffer bar.
You can now skip to anywhere in this buffer bar. a brief delay as the game reloads in at that time.
But you can now go back and forth as you please.
and thanks to differential compression it uses very little storage.
These keyframes stay even after closing the game, so you can quickly look through a replay or preload it, and then scan through the whole replay later with no background processing
Video:
https://youtu.be/0MWgoXI4nQE
this can all be controlled from settings.
[image: 1793295256836-81fa8526-ab13-4bc8-a4f0-610959578e51-image.png]
#386 The lobby downloads a missing map from FAF's vault
The lobby will now download maps from faf that are missing from clients, this happens as they are selected by the host, so no reloading needed.
Testing
An important part of any modern software development is testing.
As such this project has a few different types.
unit test (very isolated, does the function add(2, 3) equal 5)
benchmarking
end to end, or real scenarios tests
the scenarios tests are what I want to show below, it's massively sped up so you can actually see it somewhat in a lifetime.
they each test something fairly specific, like "can this tank attack this tank", "is this tanks shot blocked by terrain"
i did spot a few errors during this recording, but that's all part of the process
https://youtu.be/edDiLLQhGL4
Naval
#351 Torpedoes run under the water, naval units start on the surface
#355 Submarines start submerged
#357/#358 Submarines surface on their own for carrier loads, unloads and AutoSurfaceToAttack
#356 Torpedoes fired from a still sub start at rest
#457 Submarines dive and surface bow first
#455 Carriers launch aircraft from their launch points, one a tick, without moving
#418 The cheat menu spawns ships on water
Movement and pathing
#475 Units ease into a stop and coast to a halt on arrival
#327 Formations run at 0.85× their slowest unit, and members steer for their final slots
#329 Units only push moving units on other orders, as fading impulses
#406 Land units no longer path over cliffs into or out of the water
#469 Hover units lean a tenth as far and level out after stopping (the seraphim where just bobbing around all over the place )
#467 Finished ground units stand on the ground under them
#361 Walk animations follow the unit's speed and turns
Issue* orders skip units that can't do the order, such as an shield generator being told to attack, or tanks being told to reclaim
Combat and vision
#347 We now draw nice fog of war instead of pixelated from before
#345 Weapon target searches skip targets the terrain blocks
#348 Weapons on hold fire search for no targets
#342 Aggressive moves and patrols engage a task step late
#420 Turret pitch is aimed from the barrel bone's own pose
#370 Persistent blips (FAF's Blip class, destroy hooks, real-game lifetime)
#468 Fog of war's clear ground is drawn as vision circles
#499 Czar and some other beams had effects sideways
Overcharge works
Weapons in general work better and more faithfully to the base game
Energy drains are actually paid, so T2 artillery charge-up, teleport and overcharge wait for energy
Weapons fire the beat after they take a target, as in the real game (i feel like this is something maybe should be changed about the game?)
Homing missiles steer by their facing. The Corsair's AA darts now fly level and hit, where before they flew into the ground
Solace torpedo bombers now actaully do damage
TMDs now actaully work correctly
Fire Beetle detonates on attack orders and no longer blows up at game start (this was more when doing testing, as you never start the game with fire beetles, but they also had other bugs)
HARMS sits on the sub layer
Weapon/damage callbacks now get the real tick number it happened on, instead of the tick n - 1
GiveStorage / GiveResource apis behave as in the real game
Anti torpedo weapons now work
Networking, replays, lobby
#449 Lockstep desync fixes (local-only input, ejected turns, command text, observer sources) and a desync dialog
(this was a nasty little bug, i was double reading local player input once, before the network tick and once after, so it was possible to input commands in to your local sim without sending to others!!)
#454 Desync detection: a checksum every tick, in parts, covering motion, orders and economy
While i've made efforts to build the sim in such a way desyncs should be hard to get, its always still good to have logging for it
#380 One broken map no longer empties the lobby
Building, engineers, factories
#403 Build sites snap to the cell grid
#464 Assisting factories share the assisted queue and keep building their own
#465 An engineer's assist ends when an order is queued behind it
#451 Engineers parking to assist slide off each other instead of stacking
#456 Assisting engineers show the progress of what they help build or repair
#452 Build beams go away when an unfinished structure is destroyed
#463 Factories keep a queue of rally orders, shown with Shift and kept on upgrade
#416 The construction panel queue works for mobile factories and ACU enhancements
#417 One waypoint ETA per structure, at its builders' combined rate
#459 The ACU upgrade ETA shows without selecting the ACU
#466 Reclaiming a unit takes its health down with its fraction
#453 SetMaxHealth leaves health alone
#458 Excess resources overflow to allies
#365 Enhancement tabs are in the real game's order
#507 engineers assisting one of themselves leave its own build queue alone
Camera and interface
#390 Camera spin (Space + mouse, Insert / Delete, cam_Free)
#389 Camera field of view and zoom works as in moho
#394 You can now scroll in and out when tracking a unit with "T"
#450 Camera SnapTo / MoveTo / RestoreSettings honour heading and pitch
#360 Queued waypoints and build placements can be dragged to move them
#414 A test that dragging the build queue reorders the factory queue
#382 Unit stats for the end-of-game screen
#385 The stats bar pause button no longer shifts
#404 The in-game exit says "Exit" on platforms other than Windows
Minimap shows an earlier game's map, if you play a second game without restarting
Wrong army in later games, if you play a second game without restarting
Game freezes when the other player leaves (no clickable buttons, black screen after the score view)
Process stays running after the window closes → exits on its own after 3s
Map download "builder error" on Linux, i've added more logging, and a failed download now blocks that player's launch
Visuals
#470 Seraphim build mesh and Seraphim / Aeon build beams match the real game
#474 Cybran structures under construction hide what's behind them
#461 Mesh LOD fixes
#492 Collision beam's far end starts at its target when enabled (Monkeylord laser)
#493 Ythotha death: energy being draws, wreck keeps its death pose, no white beam slabs
Platform
#383 The join code now copies to the clipboard on sway / Wayland
#384 FAF and Steam are found where faf-linux looks for them
#413 A hidden window no longer stops the other window drawing, plus you can now window the other window
and in general multi monitor should be much better supported, although their are limitations with wgpu and different platform support
Wayland terminal flooded with clipboard log lines
No vsync cap on Wayland (1000 FPS in the menu), wayland wasn't giving a framerate limit at startup, we now find the highest refresh rate screen and use that until we get the correct refresh rate
Performance and internals
#338 Map decals skip lighting their transparent pixels
#339 Terrain shadow is worked out once a pixel
So one of the only graphics changes i have made to the project is using SMAA for AA and PCF (percentage closer filtering) for shadows, along with up to 2k shadow maps instead of 1k.
the downside to that shadow improvement is the shadow calculations becomes more expensive on the gpu.
While playing around I noticed with just looking at the map was dropping my frame rate which absolutely shouldn't be happening on a game from 2007!
well time to fire up the graphics profilers and have a look.
got to say im very impressed with xcodes metal debugger.
and its profiling instantly shows almost all the gpu time is spent drawing the decals on the map!
you can see here in this image up at the top left hand side that vs_decal/fs_decal_albedo is at 68% of the world stage which its self is 90% of the frames work!
[image: 1793295464797-84f65fd2-b2ed-4705-aa81-bdfbcb58c529-image.png]
well it turns out now calculating shadows is much more expesnive and its done for all the decals across the whole map.
well the fix is easy, most of the decals texture is transparent.
so right at the start of the shader, if the pixel is transparent stop.
if (alpha <= 0.0) { discard; }
this took my frame time from 15.46ms to 11.10ms!
[image: 1793295478154-68aaadf9-98d1-4b2a-8b6b-aefbecffdc01-image.png]
next still for every decal we are calculating the shadows again, as decals can stack on top of each other, this is a lot of extra work.
so we now precalculate the shadows and share between the terrian and decals
this takes 11.10ms to 7.96ms!
~twice as fast as before!
[image: 1793295491155-3d21f92b-4796-4cb6-9467-8f78c5b4fbda-image.png]
But we can go further! I noticed at large zoomed out views my fps again was dropping. Unacceptable!
Most of the map drawing is dominated by reading lots of textures, but for when the texture is far away, you could use smaller textures right? you see less of it so who cares if its smaller?\
well thats a well known thing in graphics already, called mipmaps. you store with the texture a version of it halfed and halfed until it gets to 4x4.
like so
512
256
128
64
32
16
8
4
then the shader uses the size appropriate in the moment, this saves a lot of memory bandwidth.
this gives me 5.6ms to 3.88ms! (on a different map to the one above, so just look at the change)
In fairness, the base game also does this, i just assumed all textures shipped with mipmaps...
These mipmap textures are saved out to a cached folder, which can be configured both in size and expire date in the settings.
it will make first startup slower
#374 Faster weapon target searches (this really only effects the 64k stress test, but weapons are now twice as fast)
#376 Effect expiry uses an index instead of scanning every effect
#368 Landing for aircraft searches are much faster now
#407 Mouse moves are coalesced, so hovering hit-tests once a frame (especially useful with super high hz mice)
#421 The interface keeps control layouts between frames (this is very much targeted at the restrictions page which is very heavy on the ui, now its atleast twice as fast)
maps can now be saved loaded when compressed with zstandard (god I love a good compression algo, this shrinks a 81km map to like 5-24x smaller)
here is a grid of all the inbuilt maps which really shows you how good compression is, and to think they shipped the game on dvd!
Base game map sizes, raw .scmap vs zstd level 19
SCMP_001 Burial Mounds (20 km) 9.4 MB -> 2.5 MB (3.7x)
SCMP_002 Concord Lake (20 km) 7.9 MB -> 2.5 MB (3.2x)
SCMP_003 Drake's Ravine (20 km) 8.8 MB -> 1.7 MB (5.2x)
SCMP_004 Emerald Crater (20 km) 10.8 MB -> 3.0 MB (3.6x)
SCMP_005 Gentleman's Reef (40 km) 37.9 MB -> 5.9 MB (6.4x)
SCMP_006 Ian's Cross (20 km) 10.8 MB -> 2.9 MB (3.7x)
SCMP_007 Open Palms (10 km) 2.7 MB -> 0.7 MB (3.7x)
SCMP_008 Seraphim Glaciers (20 km) 9.9 MB -> 2.0 MB (5.0x)
SCMP_009 Seton's Clutch (20 km) 9.6 MB -> 2.6 MB (3.7x)
SCMP_010 Sung Island (20 km) 9.9 MB -> 2.3 MB (4.4x)
SCMP_011 The Great Void (40 km) 31.2 MB -> 1.9 MB (16.1x)
SCMP_012 Theta Passage (5 km) 1.0 MB -> 0.2 MB (4.0x)
SCMP_013 Winter Duel (5 km) 0.9 MB -> 0.3 MB (3.5x)
SCMP_014 The Bermuda Locket (20 km) 11.1 MB -> 2.2 MB (5.0x)
SCMP_015 Fields of Isis (10 km) 3.9 MB -> 1.2 MB (3.4x)
SCMP_016 Canis River (5 km) 1.4 MB -> 0.4 MB (3.2x)
SCMP_017 Syrtis Major (10 km) 3.0 MB -> 1.0 MB (2.9x)
SCMP_018 Sentry Point (5 km) 2.0 MB -> 0.6 MB (3.5x)
SCMP_019 Finn's Revenge (10 km) 3.5 MB -> 0.8 MB (4.2x)
SCMP_020 Roanoke Abyss (20 km) 10.3 MB -> 2.3 MB (4.4x)
SCMP_021 Alpha 7 Quarantine (40 km) 32.2 MB -> 7.6 MB (4.2x)
SCMP_022 Arctic Refuge (10 km) 2.5 MB -> 0.8 MB (3.0x)
SCMP_023 Varga Pass (10 km) 2.5 MB -> 0.8 MB (3.1x)
SCMP_024 Crossfire Canal (20 km) 12.0 MB -> 3.5 MB (3.4x)
SCMP_025 Saltrock Colony (10 km) 2.7 MB -> 0.6 MB (4.3x)
SCMP_026 Vya-3 Protectorate (10 km) 4.0 MB -> 1.0 MB (4.0x)
SCMP_027 The Scar (20 km) 9.2 MB -> 2.8 MB (3.3x)
SCMP_028 Hanna Oasis (40 km) 32.5 MB -> 7.7 MB (4.2x)
SCMP_029 Betrayal Ocean (81 km) 122.0 MB -> 20.3 MB (6.0x)
SCMP_030 Frostmill Ruins (81 km) 126.8 MB -> 16.6 MB (7.6x)
SCMP_031 Four-Leaf Clover (10 km) 2.6 MB -> 0.9 MB (3.0x)
SCMP_032 The Wilderness (10 km) 3.6 MB -> 1.0 MB (3.5x)
SCMP_033 White Fire (10 km) 3.6 MB -> 0.7 MB (5.0x)
SCMP_034 High Noon (10 km) 2.9 MB -> 0.7 MB (4.2x)
SCMP_035 Paradise (10 km) 3.0 MB -> 0.8 MB (3.6x)
SCMP_036 Blasted Rock (5 km) 1.0 MB -> 0.3 MB (3.0x)
SCMP_037 Sludge (5 km) 0.8 MB -> 0.3 MB (2.8x)
SCMP_038 Ambush Pass (5 km) 1.3 MB -> 0.4 MB (3.3x)
SCMP_039 Four-Corners (5 km) 1.2 MB -> 0.3 MB (3.6x)
SCMP_040 The Ditch (20 km) 9.7 MB -> 2.4 MB (4.1x)
X1CA_001 campaign (20 km) 8.7 MB -> 2.3 MB (3.7x)
X1CA_002 campaign (20 km) 8.5 MB -> 2.7 MB (3.2x)
X1CA_003 campaign (40 x 20 km) 17.2 MB -> 4.5 MB (3.8x)
X1CA_004 campaign (20 km) 7.9 MB -> 1.5 MB (5.4x)
X1CA_005 campaign (20 km) 8.5 MB -> 2.1 MB (4.0x)
X1CA_006 campaign (20 km) 8.4 MB -> 2.1 MB (3.9x)
X1MP_001 Crag Dunes (5 km) 1.2 MB -> 0.4 MB (2.7x)
X1MP_002 Williamson's Bridge (5 km) 1.0 MB -> 0.4 MB (2.5x)
X1MP_003 Snoey Triangle (10 km) 2.4 MB -> 0.7 MB (3.3x)
X1MP_004 Haven Reef (10 km) 3.1 MB -> 0.8 MB (3.7x)
X1MP_005 The Dark Heart (10 km) 2.8 MB -> 1.0 MB (2.9x)
X1MP_006 Daroza's Sanctuary (10 km) 2.2 MB -> 0.7 MB (3.0x)
X1MP_007 Strip Mine (20 km) 8.6 MB -> 1.7 MB (5.0x)
X1MP_008 Thawing Glacier (20 km) 9.1 MB -> 1.8 MB (5.0x)
X1MP_009 Liberiam Battles (20 km) 8.5 MB -> 2.2 MB (3.8x)
X1MP_010 Shards (40 km) 33.9 MB -> 7.8 MB (4.3x)
X1MP_011 Shuriken Island (40 km) 32.4 MB -> 4.6 MB (7.0x)
X1MP_012 Debris (81 km) 122.2 MB -> 5.0 MB (24.5x)
X1MP_014 Flooded Strip Mine (20 km) 10.4 MB -> 2.5 MB (4.2x)
X1MP_017 Eye of the Storm (10 km) 3.0 MB -> 1.0 MB (3.1x)
Total, 60 maps: 862.1 MB -> 152.8 MB (5.6x)
