M28AI Devlog (v95)

Mainly in cases where air battles started

I didnt notice when I was watching the replay but I stopped after 41m as I had enough points to look at by then, do you know roughly the timestamp where it went to -6? It makes sense that it would be slower in a big air battle although I wouldn't have expected -6 (assuming everyone has average to good cpus) so it could be an indication of something gone wrong with the code. It's also possible it's a result of some of the additional checks that have to be run in campaign maps that will add to the overhead (although again I wouldn't have expected such a big impact)

The game suddenly freezes completely after a certain time. Without error message and without replay. Whether with 2 or 4 players against an M28AI. You then have to end the game via the Task Manager. However, this only happens when you have reached the maximum of 1000 units (i.e. when you go on the attack). And it happens in every game. I have also noticed that the speed sometimes drops to -6.

We play with the following mods:

• All Faction Quantum Gate
• M28AI
• Nuclear Repulsor Shields
• Total Mayhem
• Total Mayhem Lite
• 5x Build Range

Are you aware of any incompatibility with any of these mods?

@klecks Not individually (some like Quantum gate, total mayhem and build range I've seen work fine, theo thers I've not specifically tested), but issues are much more likely to occur with multiple mods, either due to an incompatibility with a particular combination of mods; a bug with the mod that M28's logic highlights, or memory issues (which are more likely with more mods), in addition to the potential for a bug with M28.

If it happens with just one of those mods enabled then I can take a look to figure out. The other thing you could do is see if there's a log file (even if there's no replay - although I'd have normally expected a replay to be available for games played in FAF even if the game crashes part-way through).

For example logs are saved for me at the following location:
C:\Users[username]\AppData\Roaming\Forged Alliance Forever\logs

Have you always had these issues or has it only been since the latest release (last 24 hours)?

Also enable more detailed logging in the settins in the client, which increases the chance that the cause of the crash can be identified from the log:

v59 Update
A mix of campaign, replay and vs AI based changes with this update:

• Fighting M27 - 9 changes; interestingly M28 beat M27 on 4 different 5km and 10km maps that M27 had been tested on extensively (theta passage, polar depression, twin rivers, and eye of the storm), making me think M28 had finally reached the point of convincingly dethroning M28. However, M28 lost on 2 20km maps (setons and burial mounds), contrary to what I'd have expected. The main change added was improving M28's ability to handle an early strat if it has asfs.
• 8 changes based on replay comparisons from Radde between M28 and M27 on Xander, including making land experimentals more likely to move towards an enemy instead of attack-move, and fixed some issues with hos mongoose and MAA would retreat from units (the former being too cautious in some cases due to a bug, the latter ignoring nearby tanks due to a bug in some cases)
• 5 changes from 2 mapgens where Radde and Mhad beat 1.5 resource M28s, mainly trying to make M28 slightly more resistant to air snipes
• 9 changes from an M28 vs 1.2 RNG game on twin rivers (normally something M28 won but this time it lost after an epic 2 hour game due to poor fatboy usage) - the main change is making M28 retreat fatboys much sooner so they're less likely to be caught up to and killed, and increasing the extent to which M28 will eco when behind on eco.
• 11 changes for Saver's FA mission 1 of the campaign to make M28 more aggressive with navy, have a better unit mix, send GCs at the enemy start position (instead of keeping them on patrol), and being more likely to build T3 arti/similar
• 4 changes from a replay by Bart on gap of rohan, mainly fixing issues with overbuilding T2 arti and SAMs

Trophy awards

• Radde trophy - Awarded to Radde and Mhad for beating 1.5 resource M28 on both a mixed land+navy mapgen, and a navy mapgen
• Sladow trophy - Current holder is triangleleaf for beating v57 M28

Acknowledgements

• Radde - sending various replays including a comparison of M27 and M28 to help me identify significant differences in their approaches that make M27 seem tougher for a human player
• Saver - Replay against M28 as a hostile AI in campaign which highlighted a number of campaign specific issues
• Bart - Replay in a norush gap of rohan high eco game which highlighted some flaws with M28's arti and SAM logic

@klecks said in M28AI Devlog (v59):

The game suddenly freezes completely after a certain time. Without error message and without replay. Whether with 2 or 4 players against an M28AI. You then have to end the game via the Task Manager. However, this only happens when you have reached the maximum of 1000 units (i.e. when you go on the attack). And it happens in every game. I have also noticed that the speed sometimes drops to -6.

We play with the following mods:

• All Faction Quantum Gate
• M28AI
• Nuclear Repulsor Shields
• Total Mayhem
• Total Mayhem Lite
• 5x Build Range

Are you aware of any incompatibility with any of these mods?

What you're talking about is probably related to memory. Maybe next time you can take a look at the memory usage for high unit quantities (via task manager).

@zhanghm18 We all have 32GB of RAM, so I don't think that's the reason for this

@klecks FAF is prone to crashing if you reach over 2GB of memory usage; i.e. if you have taskmanager open and you see it approaching 2gb, then it's much more likely to crash.

The memory usage is increased significantly by unit mods (along with various other factors), and has been known to crash with AI (including but not limited to M28) since AI also use up more memory.

@maudlin27 said in M28AI Devlog (v58):

AF is prone to crashing if you reach over 2GB of memory usage; i.e. if you have taskmanager open and you see it approaching 2gb, then it's much more likely to crash.
The memory usage is increased significantly by unit mods (along with various other factors), and has been known to crash with AI (including but not limited to M28) since AI also use up more memory.

OK, I understand. This is a screenshot from Task Manager when the game was frozen (two AI und only me). So the prolem with memory depends on the 32bit game

@maudlin27
Hello unfortunately I didn't write down any timestamps. I can only make the following statements about the PC used, the three of us use similar components: processor i7-8700 3.19GHz, 16 - 32GB RAM, Geforce 1080GTX and higher. I will also look at the replay to provide possible timestamps. Thank you again for your great work.

@maudlin27 Replay timestamp https://replay.faforever.com/21362409
min 43: -3, min 44: -4, min 45.20: -4, 45.40: -4, min 46.50: -5. Unfortunately I couldn't find -6. However, I only watched the end of the first section. I hope the data can be of some help to you.

@Klecks Dont use total mayhem lite (crashing/freezing random all the time), or total mayhem with this shieldmod. Better use some other mods if you want this t4 shields~ U can check my mod list some posts above.

Ansonsten frohe Festtage

m28ai seemed to crash twice when we used it, 2v2ai 3v1ai
33 min first game, 44 min second game
mods used / yes
https://replay.faforever.com/21423105 - 1st game
https://replay.faforever.com/21423406 - 2nd game

debug: Current gametime: 00:32:30
warning: false M28ERROR Count=54: GameTime 1951: Dont have a valid upgrade ID; UnitID=bsb0002
warning: stack traceback:
warning: [C]: in function assert' warning: [C]: in function pcall'
warning: ...r forged alliance\mods\m28ai\lua\ai\m28utilities.lua(55): in function ErrorHandler' warning: ...der forged alliance\mods\m28ai\lua\ai\m28economy.lua(137): in function UpgradeUnit'
warning: ...mander forged alliance\mods\m28ai\lua\ai\m28team.lua(2659): in function GetAnyMexOrFactoryToUpgrade' warning: ...mander forged alliance\mods\m28ai\lua\ai\m28team.lua(2880): in function ConsiderNormalUpgrades'
warning: ...mander forged alliance\mods\m28ai\lua\ai\m28team.lua(2930): in function <...mander forged alliance\mods\m28ai\lua\ai\m28team.lua:2908>
warning: false M28ERROR Count=3: GameTime 1953: After filtering to a faction we have no available engineers - this shouldnt be possible
warning: stack traceback:
warning: [C]: in function assert' warning: [C]: in function pcall'
warning: ...r forged alliance\mods\m28ai\lua\ai\m28utilities.lua(55): in function ErrorHandler' warning: ...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua(4398): in function FilterEngineersOfTechAndEngiCountForFaction'
warning: ...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua(5564): in function ConsiderActionToAssign' warning: ...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua(6420): in function HaveActionToAssign'
warning: ...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua(6681): in function ConsiderCoreBaseLandZoneEngineerAssignment' warning: ...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua(11595): in function ConsiderLandOrWaterZoneEngineerAssignment'
warning: ...mander forged alliance\mods\m28ai\lua\ai\m28land.lua(5857): in function <...mander forged alliance\mods\m28ai\lua\ai\m28land.lua:5349>
warning: false M28Warning: Count=1: GameTime 1956: No plateau group for iSegmentX=336; iSegmentZ=249; Plateau group of segment midpoint=2751; Plateau Group of tPosition=2751; This can happen on larger maps with long cliffs with narrow paths that small units can travel partially along. Enable logs in the function GetUnitPlateauAndLandZoneOverride for more details
warning: stack traceback:
warning: [C]: in function assert' warning: [C]: in function pcall'
warning: ...r forged alliance\mods\m28ai\lua\ai\m28utilities.lua(55): in function ErrorHandler' warning: ...mmander forged alliance\mods\m28ai\lua\ai\m28map.lua(571): in function GetPlateauAndLandZoneReferenceFromPosition'
warning: ...mander forged alliance\mods\m28ai\lua\ai\m28land.lua(317): in function UpdateUnitPositionsAndLandZone' warning: ...mander forged alliance\mods\m28ai\lua\ai\m28land.lua(6067): in function <...mander forged alliance\mods\m28ai\lua\ai\m28land.lua:6007> warning: false M28Warning: Count=8192: GameTime 1959: We are unable to build at a predefined shield location, will check for small blocking M28 buildings and destroy them warning: stack traceback: warning: [C]: in function assert'
warning: [C]: in function pcall' warning: ...r forged alliance\mods\m28ai\lua\ai\m28utilities.lua(55): in function ErrorHandler'
warning: ...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua(4648): in function ConstructNewShield' warning: ...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua(4748): in function <...er forged alliance\mods\m28ai\lua\ai\m28engineer.lua:4406> warning: false M28Warning: Count=1: GameTime 1959: Adding unit ID brot3bt2 to table of temp other units - either the unit is a land unit really close to water that we incorrectly think is in water, or it is an amphibious/hover unit that has incorrect caterisation; if the unit runs out of orders then will send it to the rally point warning: stack traceback: warning: [C]: in function assert'
warning: [C]: in function pcall' warning: ...r forged alliance\mods\m28ai\lua\ai\m28utilities.lua(55): in function ErrorHandler'
warning: ...mander forged alliance\mods\m28ai\lua\ai\m28navy.lua(1488): in function <...mander forged alliance\mods\m28ai\lua\ai\m28navy.lua:1331>
info: OnDecayed: Time=1961.3000488281; self.UnitId=brnt3perses
debug: Loading module: /projectiles/adfreactoncannon01/adfreactoncannon01_script.lua
debug: Loading module: /projectiles/laserbotterran01/laserbotterran01_script.lua
warning: false M28ERROR Count=81: GameTime 1975: Dont have a valid upgrade ID; UnitID=bsb0002
warning: stack traceback:
warning: [C]: in function assert' warning: [C]: in function pcall'
warning: ...r forged alliance\mods\m28ai\lua\ai\m28utilities.lua(55): in function ErrorHandler' warning: ...der forged alliance\mods\m28ai\lua\ai\m28economy.lua(137): in function UpgradeUnit'
warning: ...mander forged alliance\mods\m28ai\lua\ai\m28team.lua(2659): in function GetAnyMexOrFactoryToUpgrade' warning: ...mander forged alliance\mods\m28ai\lua\ai\m28team.lua(2880): in function ConsiderNormalUpgrades'
warning: ...mander forged alliance\mods\m28ai\lua\ai\m28team.lua(2930): in function <...mander forged alliance\mods\m28ai\lua\ai\m28team.lua:2908>
debug: Loading module: /mods/totalmayhem/projectiles/brot3shbmempproj/brot3shbmempproj_script.lua
info: OnDecayed: Time=1979.8000488281; self.UnitId=xrb9304
debug: Current gametime: 00:33:00
debug: Loading module: /mods/blackopsfaf-unleashed/units/bsb5205/bsb5205_script.lua
debug: Loading module: /mods/totalmayhem/projectiles/brpt1extank2proj/brpt1extank2proj_script.lua
debug: Loading module: /projectiles/tdfplasmaheavy03/tdfplasmaheavy03_script.lua
debug: Loading module: /units/xsl0303/xsl0303_script.lua
debug: Loading module: /mods/blackopsfaf-unleashed/units/bab2306/bab2306_script.lua
debug: Current gametime: 00:33:30
info: Minimized true
info: Minimized false

debug: Current gametime: 00:44:00
info: CWldSession::DoBeat() unknown entity id (0x2ff0029a) supplied in a pose update.
warning: Error running OnNotAdjacentTo script in Entity urb1301 at 6d934808: ...\gamedata\lua.nx2\lua\sim\adjacencybufffunctions.lua(18): attempt to call method DestroyAdjacentEffects' (a nil value) stack traceback: ...\gamedata\lua.nx2\lua\sim\adjacencybufffunctions.lua(18): in function OnBuffRemove'
...gramdata\faforever\gamedata\lua.nx2\lua\sim\buff.lua(687): in function RemoveBuff' ...ver\gamedata\lua.nx2\lua\sim\units\structureunit.lua(810): in function <...ver\gamedata\lua.nx2\lua\sim\units\structureunit.lua:787> [C]: in function Kill'
...gramdata\faforever\gamedata\lua.nx2\lua\sim\unit.lua(1320): in function DoTakeDamage' ...ver\gamedata\lua.nx2\lua\sim\units\structureunit.lua(664): in function DoTakeDamage'
...gramdata\faforever\gamedata\lua.nx2\lua\sim\unit.lua(1286): in function OnDamage' ...gramdata\faforever\gamedata\lua.nx2\lua\sim\unit.lua(5751): in function <...gramdata\faforever\gamedata\lua.nx2\lua\sim\unit.lua:5750> [C]: ? ...ta\faforever\gamedata\lua.nx2\lua\sim\projectile.lua(623): in function DoDamage'
...ta\faforever\gamedata\lua.nx2\lua\sim\projectile.lua(364): in function `OnImpact'
...uclearshell01\tifhetacticalnuclearshell01_script.lua(12): in function <...uclearshell01\tifhetacticalnuclearshell01_script.lua:11>
info: Minimized true
info: Minimized false

@thewatcheral said in M28AI Devlog (v59):

21423105

Thanks, looks like there's an incompatibility with the experimental shield mod that triggered when the experimental shield was killed (and is highly likely to have caused the crash), I'll see if I can resolve in the next release.

v60 Update
Various changes to M28 including a complete rework of how it shields high value targets, the introduction of a very simplified ACU snipe option (don't expect M27 levels of aggression for this), and other improvements that for example mean M28's probably favoured to beat M27 on setons now (one of the few large maps that M27 had left).

• 4 changes from testing M28 against M27 on Burial Mounds (mostly relating to a greater focus on gunships to defend against an experimental in some e.g. low power situations)
• 26 Changes from testing some high mass/AiX modifier scenarios, although I gave up on trying to avoid M28 overflowing mass in its build order at very high modifiers (engineer pathing issues and delays with construction starting and unit roll-off times proved too great a hurdle to overcome)
• 5 changes to improve M28's novax defence logic, including fixing some issues that meant it was only building T3 shields instead of sometimes getting T2 shields
• New shield cycling logic for high value units (e.g. game enders and T3 arti) - prompted from a replay where M28 failed to build 1 game-ender in the time it took the enemy to get 4 T3 arti, despite M28 having double the mass income, which I think was in part caused by almost all of M28's resources being spent on shield assistance. This change involved a lot of work and although I've made a number of adjustments to it from testing, I expect further refinements will be needed over time.
• 5 Changes for misc issues spotted while testing the new shield cycling logic (adjusting bomber attack logic, T2 arti construction logic, and a bug on transfer of certain HQs on an M28 teammate's death)
• 17 Changes from fighting M27 on Setons, including greater ecoing, and various refinements to M28's logic when a paragon is constructed to make greater use of the income (e.g. aircraft carriers will start to build air units)
• 8 Changes for Aeon M5 and M6, including fixing an issue with Aeon M5 where the T3 arti were effectively invisible to M28 (it must have been by chance that it managed to kill them when I tested previously)
• 2 changes for replay 21516542 to reduce the tendancy of retreating MMLs and T3 mobile arti of running into the attacking enemy (if caught out of position)
• 11 Changes for various replays from Radde, including more detailed logic for determining where to build T2 PD (that I hope means it is less likely for T2 PD to be built where they'll just shot into a cliff), increase in land experimental aggression vs ravager, and making it more likely M28 rushes an experimental earlier on
• New simplified ACU snipe logic - it's not really working as well as I'd hoped (and is a far cry from M27's version) but I figured it provides a starting point that I can refine as/when I see it not working properly in future replays
• 5 changes from Sladow replays, mostly relating to early game build order points (more engineers should be built for expansion, land scouts shoudln't be requested for core bases)
• 4 changes from Watheral replay containing a crash, improving compatibility with the experimental shield mod (which could cause a crash due to a flaw in M28's logic), and allowing the Aeon Artemis Satelite Control centre to build and use satellites (previously it was an expensive paperweight)
• 2 changes for a game vs M27 on Open Wonder to improve M28's initial build order/stop it power stalling (M28 still loses to M27, but it's no longer as crushing a loss)
• 5 other changes, including no longer self destructing drones spawned by blackops/similar mods when close to the unit cap, and making M28's ACUs more likely to get the RAS upgrade

Acknowledgements

• Balthazar - Pointing me towards a blank blueprint template for use when tyring to find a custom sized area to build in, and allowing me the use of said blank template (this is used by M28 to identify locations large enough to support its new shield cycling logic)
• Jip - Adding a more detailed 'pathto' function to take into account blocking cliffs (allowing M28 a hopefully less cpu intensive approach to identifying PD placement locations than M27's approach)
• Radde - various replays highlighting flaws with M28 (in addition to the trophy winning replays) and thoughts on improvements
• Zhanghm18 - Identifying an incompatibility with mods that spawn drone units
• Sladow - Several replays and suggestions for improvements to M28's approach to the game
• TheWatheral - Replays containing crashes

Trophy awards

• Radde claims the Sladow trophy for beating v59 M28 at 1.7 AiX (winter duel)
• Radde, Mhad and GhillieWolf claim the Radde trophy for beating M28 with a 1.5 resource mod on mapgen.

Note for future Sladow trophy awards these will only be for 5km or larger maps (i.e. maps that are smaller than 5km x 5km like winter duel won't qualify) - essentially I've had to admit defeat on an AiX 1.5 M28 being able to win against a competent player on this map since unless I devoted a large amount of time to a very map specific build order and approach it is too easy to overcome the resource/build rate deficit. I've also changed the trophy/medal picture in the original post of the thread.

I may well at some point change it to be a 10km only (since that's the size Sladow beat m28 on originally) but I expect there will still be some improvements that can be made on 5km maps more generally even if they're likely to also feature similar issues for M28 as winter duel did.

v61 Mini update
Small update to fix some issues with the special shielding logic added in v60

• 6 Changes mostly relating to making the ACU prioritise attacking enemies instead of reclaiming or building mexes if it has a gun upgrade
• 4 Fixes and redundancies relating to the special shielding logic introduced in v60 to try and reduce the tendency of M28's engineers getting stuck in a loop of building a unit only to reclaim it.

Acknowledgements

• Zhanghm18 - Reproducing a bug where M28 would build and reclaim the same unit

v62 Update
M28 should play noticeably differently on 5km and some 10km maps 1v1 following this update:

• Introduced new 't1 spam' mode, where M28 will try really hard to avoid getting a T2 mex so it can invest the mass into T1 spam or factory upgrades instead. It'll need some refinement but it at least doesn't seem like a significant drop in competitiveness for AI games

18 misc changes, including:

• making 'short range' land units more likely to attack the enemy instead of waiting for the enemy to die to longer range units (which would in some cases lead to a GC staying back so that a lone MML could try and kill the enemy T2 PD)
• Adjusting logic for getting engineers to cancel their orders to reclaim so they'll try and resume their order once done if they were heading towards a mex, and making it less likely an engineer follows up 1 reclaim order with another for reclaim far away
• Improved M28's logic for thinking it has mass to spare in high AiX scenarios (e.g. previously if all the M28 players on a team combined had 4k mass income per second they'd ignore most low mass conditions which could lead to aircraft carriers spamming torp bombers)
• Added basic build logic for the tempest

Acknowledgements

• Zhanghm18 – Testing different scenario’s impact on memory usage
• Sladow – Tips on how to approach a 5km game (suggesting to stay at T1 the entire game)

Trophies
No change to the current holders from their defeat of v59 M28: Radde (Sladow trophy); and Radde, Mhad and GhillieWolf (Radde trophy)

I'm seeing an issue where the AI can't decide what type of shield to build. It will queue up one type and immediately replace it with another type indefinitely. This keeps a number of engineers occupied (50-100?) I watched this happen for at least 10 minutes.

These are both tier 4 shields, but from different mod packs. Nuclear Repulsor Shields, and Wyvern Battlepack. Wyvern is no longer in the vault, but generally works fine.

@bsee Thanks, do you have a replay ID and timestamp? Also do the shields from those mods have a larger footprint than normal shields?