AI Development Guide and M27AI v67 Devlog

Following the 2021 Supreme Computer Cup I was inspired to have a go at developing a basic AI for FAF. As I had zero experience of coding in Lua, I thought at the same time I would write down things as I learnt them to have both a guide for other people interested in getting started in developing an AI, and a dev log for the creation of my AI.

See below for a link to the current AI development guide and devlog (2022-12 update - as people were having issues viewing the original file I've split it out and have separated the main 'devlog' part of M27 development from the more general notes on getting things setup to develop the AI and useful functions)
Devlog extracts: https://docs.google.com/document/d/1uQlEoN-kti7G2MnhwD60uaaHwVevPNYH/edit?usp=sharing&ouid=100973959280546778272&rtpof=true&sd=true
Development guide extracts: https://docs.google.com/document/d/1puzW5hKcOBNE7wu7bXj6qXfTJ60r2f93/edit?usp=sharing&ouid=100973959280546778272&rtpof=true&sd=true

You can also access the mod on Github:
https://github.com/maudlin27/M27AI

I've copied the current overview from the guide below:

This document is intended to do 3 things at once (meaning it does none of them very well!):

• Act as a guide to people new to FAF AI development, including collating any other useful resources out there on AI development
• Be a sort of dev log of my thought process and development steps for creating my first FAF AI, with extracts of code thrown in
• Form my main ‘todo’ list for further developments of my AI
  If you see any errors, or have any comments or suggestions (in particular any AI developers who read this since you’ll probably know better/easier ways of doing things than the methods I’ve chosen) let me know and I can update the guide where relevant.

If you’re here for useful resources and/or details of how to write AI and don’t care about the dev log aspect then just skip to section 2) Setup and background and section 5) Index of useful functions, and/or browse the index for points that may be of interest.

Latest changes (v67 summary)

• 13 bugfixes,
• 25 misc changes
  Most changes are minor, the more noticeable are sharing of mobile shield and stealth technology, and a slightly increased likelihood of building game-enders in team games.

Note: Most of my focus is currently on M28 - the below links to a separate M28 devlog:
https://forum.faforever.com/topic/5331/m28ai-devlog-pre-release

Areas of focus by version (history):
v1-3: Release
v4 - Intel and threat detection
v5 - Use of ACU and hotfix
v7 - Engineers and building upgrades
v9 - Air
v10 - Anti-Turtle
v11-v12 - Pathfinding and misc changes
v13 - Anti-navy
v14 - ACU and mobile shields
v15-v18 - CPU optimisation and hotfixes
v19 - Rally points and air domination strategy
v20 - Late game
v21-23 - Eco management
v24 - Bombers
v25 - Misc
v26 - Nukes
v27 - T3 Arti
v28 - Game options/compatibility
v29 - Optimisation and TMD
v30 - Czar, Soulripper and Defences
v31 - Bombers (again) and TML
v32-33 - Plateaus
v34-35 - Turtling
v36 - Skirmishers
v37-38 - Game enders
v39 - Quantum optics
v40 - Deceiver
v41 - Hives and shield disruptor
v42 - Misc and multi-TML defence
v43 - Misc and UEF experimentals
v44-46 - Teamwork
v47 - Optimisation
v48 - Personality
v49 - Hotfix
v50-51 - Navy
v52 - Desync hotfix
v53 - Shield boats
v54 - Temporary eco strategy
v55 - Experimental defence
v56 - Stealth boat
v57 - Pathfinding (again!)
v58 - Misc (RC tweaks)
v59 - Gunships
v60 - Naval enhancements
v61 - Stats
v62 - Land spam
v63 - T3 arti prioritisation
v64 - Mass fabs
v65-v66 - Skirmisher and T3 arti tweaks
v67 - Misc

v47+ note - the page numbers for sections 4+ will be slightly different in the actual version (since I'm no longer showing the section 3 topics below I wont be updating the below listings often)

1. Overview 35
   1.1) AI goals 35
   1.2) Background 36
   1.3) Approach 36
2. Setup and background 37
   2.1) Lua editors and IDEs 37
   2.1.1) Intellij IDEA 37
   2.1.2) Visual Studio (VS) 38
   2.2) Other guides which include a section on AI: 39
   2.2.1) General information: 39
   2.2.2) How AI in FAF works 39
   2.2.3) AI markers and how AI uses them: 40
   2.2.4) Hooking (e.g. replacing/appending existing game code) 40
   2.2.5) Creating custom scripts for Sorian AI: 40
   2.2.6) Pathfinding 40
   2.2.7) How platoons work 40
   2.2.8) AI Research (non-FAF related) 41
   2.3) Lua coding 41
   2.3.1) How to code in Lua 41
   2.3.2) Stand alone Lua code testing 41
   2.3.3) Lua optimisation 41
   2.3.4) Common causes of errors for non-Lua coders 41
   2.4) Debugging in FAF 42
   2.4.1) Console 42
   2.4.2) Cheats 42
   2.4.3) Log window and error messages 42
   2.4.4) Debug window and shortcuts 43
   2.4.5) Platoon names for debugging 43
   2.4.6) Function hooks and identifying infinite loops 43
   2.4.7) Destroyed objects 44
   2.4.8) Fast speed 44
   2.4.9) In-game tools and information 44
   2.4.10) Other shortcuts and game startup options 46
   2.4.11) Suggested shortcut 47
   2.4.12) Using FAF Develop 48
   2.4.13) Error handling 49
   2.4.14) Optimisation – benchmarks and monitoring 50
   2.4.15) Debugging via replay 51
   2.5) AI development discord 52
   2.6) Core FAF Lua files 53
   2.6.1) Accessing core Lua code 53
   2.6.2) Other built in functions 53
   2.6.3) Unit blueprints 53
   2.7) Testing AI against other AI 54
   2.7.1) Simple offline testing 54
   2.7.2) Mass offline AI testing 55
   2.8) Getting started – using a simple AI template as a starting point 56
   2.9) Downloading and setting up MicroAI 56
   2.10) Giving the AI mod your own name 56
   2.11) General - Hooks, and getting the code to recognise if it’s running your AI 56
   2.11.1) Hooking – Unit callbacks/events – applied example 57
   2.12) Github 61
   2.12.1) Github advantages and Github desktop 61
   2.12.2) Linking Github with a local repo via creating your own init file 61
   2.12.3) Pro Git ebook 61
   2.12.4) Git commit messages 62
   2.12.5) Own notes – contributing to FAF Git 62
   2.13) Uploading your AI to FAF 62
   2.13.1) Pre-upload checklist 62
   2.13.2) FAF rules on uploading mods 62
   2.13.3) Uploading to FAF 63
   2.14) Other AI documentation 63
3. AI changes 63
   3.1) Build order - Changing the initial build order 63
   3.1.1) Adding/removing buildings 63
   3.1.2) Change build order based on map resource positions 64
   3.1.3) Build order conditions 64
   3.2) Build order – ACU builds without moving at start 65
   3.2.1) BaseTemplate 65
   3.3) Build factory and power to benefit from adjacency 66
   3.3.1) Record all mexes on the map, and reference this information 66
   3.3.2) Determine com start position 67
   3.3.3) Identify mexes near to start 68
   3.3.4) Hook onto the core code relating to building a structure at a location 68
   3.3.5) Call build order from custom aiBrain function, to edit how things are built 69
   3.3.6) Adjust build template based on mex location - conditions 70
   3.3.7) Adjust build template based on mex location – build location adjustment 70
   3.3.8) Build initial PGens by existing factory 71
   3.3.9) Adjacency bonus where ACU is away from start location 73
   3.3.10) Referencing unit blueprints for information such as unit size 75
   3.4) Platoons and sending engineers to reclaim 77
   3.4.1) How platoons work 77
   3.4.2) Send the first engineer on reclaim duty 82
   3.4.3) Determine where to send the engineer to reclaim 83
   3.4.4) Identifying reclaim mass value 84
   3.4.5) Find out if a location is accessible by ground: 84
   3.4.6) Mapping reclaim locations and assigning preferred location to an engineer 86
   3.4.7) Don’t send early reclaiming engi if there’s minimal starting reclaim 86
   3.4.8) Issue new reclaim command if engineer becomes idle 87
   3.4.9) Send more engineers to reclaim later (as lower priority) based on the reclaim currently on the map 87
   3.4.10) Send 5 engineers at the start to reclaim and build mexes 88
   3.4.11) Reclaiming trees when no main mass left 89
   3.5) Building and assisting hydro 89
   3.5.1) Check if hydro near start area and have engineer build hydro 89
   3.5.2) Have ACU assist engineer with hydro 90
   3.5.3) Get ACU to continue build order after hydro complete 90
   3.5.4) Subsequent build order changes 92
   3.6) Attack force composition changes and check of AI performance 92
   3.6.1) Build Moar Tanks 92
   3.7) Pre-v1: AI Progress check 92
   3.7.1) Land factory buildings and other build order refinements 93
   3.8) Raiding AI 95
   3.8.1) Raiding behaviour - overview 95
   3.8.2) Queue up move commands for mexes 95
   3.8.3) Identify mexes to target 96
   3.8.4) Locating mexes near to an existing pathing route 98
   3.8.5) Determine when raiders need new targets 99
   3.8.6) Handling complications (stuck units, move orders built on, platoon units dying) 99
   3.8.7) Change factory rally point so units are assigned orders sooner 101
   3.8.8) Making use of platoon names for debugging 103
   3.8.9) Ensure 1 land scout included in each platoon 104
   3.8.10) Use microbots before switching to tanks 104
   3.8.11) Have scouts stay with the platoon 106
   3.8.12) AI scouting, threat detection, and identifying nearby enemy units 107
   3.8.13) Attack engineers en-route: Overview 110
   3.8.14) Tracking when have reached movement points 110
   3.8.15) Determining the platoon minimum range 110
   3.8.16) Identifying nearby enemy units and assessing their threat 112
   3.8.17) Actions based on enemy threat 114
   3.8.18) Avoid attacking enemy mexes for small platoons 115
   3.8.19) Other issues – cheating, walls and min range 116
   3.8.20) Larger raiding platoons 117
   3.9) Large attack wave 117
   3.9.1) Checking when to form a large attack wave 117
   3.9.2) Generalising raider AI 118
   3.9.3) Creating separate platoon AI for the large attack force 118
   3.9.4) Run nearer to base when outnumbered 119
   3.9.5) Recalling nearby raiders to join attack force 119
   3.9.6) Updating threat assessment for ACU health and upgrades 121
   3.9.7) Don’t chase small numbers of faster units 122
   3.9.8) Switch to hunt and kill enemies once reached destination 124
   3.10) Pre-v1: AI Progress check 124
   (3.11 onwards - left out due to post size limitations)
4. Future possible changes 512
5. Index of useful functions 592
   5.1) Map, game setup and general aiBrain information 594
   5.1.1) aiBrain - general 594
   5.1.2) AI Brain – Economy 595
   5.1.3) Map Markers and start positions 596
   5.1.4) Map size and playable area 597
   5.1.5) Terrain info 597
   5.2) Getting a list of units by category, location, or other method 598
   5.2.1) All units that meet a particular category 598
   5.2.2) Units at/near a location 599
   5.3) Unit information 600
   5.3.1) Units on radar 600
   5.3.2) Blueprints and unit categories 600
   5.3.3) Unit upgrades (enhancements) 602
   5.3.4) See 5.4.10) Other unit commands 602
   5.3.5) Location and pathfinding 602
   5.3.6) Unit’s ai Brain owner 602
   5.3.7) Unit’s current commands and whether it’s alive 603
   5.3.8) Unit bones 603
   5.3.9) Shields 603
   5.3.10) Units with missiles/anti-nukes 603
   5.3.11) Unit resource usage 604
   5.4) Unit and platoon commands 604
   5.4.1) Building 604
   5.4.2) Assisting 605
   5.4.3) Pausing 605
   5.4.4) Reclaim 605
   5.4.5) Factory specific commands 605
   5.4.6) Movement and attack-move 606
   5.4.7) Transport 606
   5.4.8) Combat related 607
   5.4.9) Stopping 607
   5.4.10) Other unit commands 607
   5.4.11) Upgrades and enhancements 607
   5.4.12) Current commands and targets 608
   5.4.13) Unit status 608
   5.4.14) Change a unit’s name 610
   5.5) Platoon additional functions 610
   5.5.1) Units platoon handle 610
   5.5.2) Get AIBrain 610
   5.5.3) Get units in a platoon 610
   5.5.4) Disband 610
   5.5.5) Platoon exists 610
   5.5.6) Platoon position 610
   5.5.7) Get all platoons 611
   5.5.8) Platoon ai plan name 611
   5.5.9) Add units to a platoon 611
   5.5.10) Army pool and removing units from a platoon 611
   5.5.11) Create a platoon 611
   5.5.12) Change or reset an AI plan 612
   5.5.13) Platoon movement formation 612
   5.6) Weapon information 612
   5.6.1) Get a unit’s weapon 612
   5.6.2) Get information on that weapon 612
   5.7) Time and thread related 612
   5.7.1) Game time 612
   5.7.2) System time 613
   5.7.3) Wait/delay the code 613
   5.7.4) Fork thread (run independently) 613
   5.8) Events 614
   5.8.1) Creating custom events 616
   5.9) Changing the game itself 616
   5.9.1) Spawning units 616
   5.10) Other 616
   5.10.1) Visual changes – Draw Circles and Lines 616
   5.10.2) On Intel Change 617
   5.10.3) ScenarioInfo.Options 617

Not exactly an AI comment, but you wanna have things right, right?

"Lua" (pronounced LOO-ah) means "Moon" in Portuguese. As such, it is neither an acronym nor an abbreviation, but a noun. More specifically, "Lua" is a name, the name of the Earth's moon and the name of the language. Like most names, it should be written in lower case with an initial capital, that is, "Lua". Please do not write it as "LUA", which is both ugly and confusing, because then it becomes an acronym with different meanings for different people. So, please, write "Lua" right!

Thanks, I'll reflect the correct case usage of Lua in the next version of the guide (I'll probably update it whenever I release an AI update so the two are in sync)

So I decided I might actually post something more relevant

Full stops vs colons – continuing the above example, oUnit.GetUnitId() will return an error, but oUnit:GetUnitId() won’t.

It's maybe worth mentioning why that happens. oUnit:GetUnitId() is syntactic sugar for oUnit.GetUnitId(self). So if you wanna you the version with . you have to pass self as first arg.

F9 opens up the debug window (you may want to change the game resolution to windowed to more easily access this).

That's a log window, not debug. You can open the debugger with Ctrl+F9 or /debug and that's way better for debugging then trying to print things into the log. Since it works as any debugger you are used to, pauses the execution on the line you seleceted and lets you see whats inside global and local variables.

The below link also appears to give a list of core/global functions that can be used (I’ve not tested these to see if they all work, or even if they’re for FAF rather than SupCom2):
https://supcom.fandom.com/wiki/LUADOC_1.5.3599

It is for FA, some of these functions are documented here https://github.com/FAForever/fa/tree/deploy/fafdevelop/engine

I’m not using Github since I’m working on this on my own, and since I doubt I can get the same ease of use I currently have with changing something offline and instantly seeing the results in-game after a Ctrl+F10 restart. However, Github is used by a number of the AI developers so there’s likely some significant benefits to it.

Git is very useful even if you're working alone, as you have history of changes and you can easily switch to previous versions if something breaks. The game can use files from any location, so for my development, it uses files from my local git repo. It's just a matter of setting your own init file https://github.com/FAForever/fa#running-the-game-with-your-changes

To keep things simple (as I couldn’t get more complex builderconditions{} with multiple variables to work, despite seeing it being used in other AIs people have done)

So build conditions are defined in a table

BuildConditions = {
    {BMBC, 'BaseEngineersEnabled', {self.BaseName}},
    {BMBC, 'BaseBuildingEngineers', {self.BaseName}},
    {BMBC, 'HighestFactoryLevel', {i, self.BaseName}},
    {BMBC, 'FactoryCountAndNeed', {i, j, pType, self.BaseName}},
    {BMBC, 'BaseActive', {self.BaseName}},
},

so here, you have 5 build conditions, each in it's own table.

local BMBC = '/lua/editor/BaseManagerBuildConditions.lua'

first value ( BMBC) in the table is thr path the file you wanna use
second one ('BaseEngineersEnabled') is name of the function in that file to use
third ({self.BaseName}) is table of arguments to pass into that function
AI brain is always passed as first argument, and then all other that you specify in the table

So you could rewrite your example to this:

-- Inside the builder
BuilderConditions = {
    {MIBC, 'Is1NearbyMex', {11}}
},


-- Inside MiscBuildConditions.lua
function Is1NearbyMex(aiBrain, distance)
    local iNearbyMex = MapInfo.RecordMexNearStartPosition(Utilities.GetAIBrainArmyNumber(aiBrain), distance, true)
    if iNearbyMex == 1 then
        return true
    end
    return false
end

This way you can already specify the distance for chehcking the nearby mexes.

Determine com start position

This function should do the trick
brain:GetArmyStartPos()

To help with this, I first wanted to understand what the various variables feeding into it were, so made use of the log function for each variable in turn; where it returned an error due to being a table value that I was trying to put into the log

Best would be to use the debugger to see whats what, as you can open the tables.
But if you really wanna dump the table into the log you have to use LOG(repr(table))

@speed2
Thanks, in particular for the debug window - I expect using that would've drastically cut my debugging time! I'll update for your comments in the latest version.

Unfortunately I can't get the debug window to work for me (I just get an error message) and some other people have had the same problem. Asking in the AI discord Jip thought that you need to do an install from the cd version of the game not steam but that you may know the exact procedure?

Error message below in case it helps:

unpack this https://mega.nz/file/VlRCXajY#UMsAx9IR1dIOqX72bPpOYytR6FS3zO1DGVPT2AhSPAY
into \THQ\Gas Powered Games\Supreme Commander - Forged Alliance\coderes\Engine

@speed2
Don't think I've got that folder with the steam install, I've got the following:
C:\Program Files (x86)\Steam\steamapps\common\Supreme Commander Forged Alliance
(this doesn't have a coderecs folder, and adding one in doesnt resolve the issue)

I've also got the FAF location:
C:\ProgramData\FAForever
same issue as with the steamapps folder

(just in case I also tried the \Documents\My Games\Gas Powered Games\Supreme Commander Forged Alliance folder but no luck).

Is there anywhere else I may have missed for a steam version?

Although I'd hoped I'd got the CDs for both as well, checking I've only got the CD for the original supcom so cant try a fresh install via disc.

It belong to the game instalation folders, and fast few ppl having this issue resolved it by creating those folders

@maudlin27

I am not finished reading your guide, but excellent work so far!

v4 changes (in reality v2, but when I uploaded the mod on its first release I had to rush out a couple of hotfixes where I'd forgotten to remove debug logic and accidently introduced a bug that affected all games!)

• Fixed various bugs (see 3.14.3)
• Slightly improved pathfinding performance (see 3.15)
• New AI logic which will detect enemy threats and send units to deal with them (see 3.16)
• New AI logic which attempts to maintain intel over the map to stop surprise attacks (see 3.17)

I've also updated the guide to reflect speed2's comments (as well as a couple of other minor points/sections, such as a bit on uploading a mod and some tests that I now run before uploading)

I am stunned.

Really good work, maudlin27!

v5 changes - these were mostly about the ACU, making it more aggressive and less suicidal

• Fixed various bugs and improved some error handling (see 3.18.4)
  ACU improvements (see 3.19):
• ACU will try to expand to high reclaim or mex locations, while building on mexes it comes across and reclaiming
• ACU will have scout and MAA support
• ACU tries to kite enemies it comes across
• ACU uses manual overcharge to overcharge optimal targets

Also updated the guide slightly to add details of the AI development discord channel in FAF (separate to the main AI development discord), and a couple of common hard crash errors

v6 Hotfix - My AI broke with the recent FAF patch, so this is primarily to fix that. I was in the middle of more extensive changes so my AI is less stable than I'd like but I figured it's better than a broken one! I've also held off on updating the dev log for this release.

Main changes made:

• ACU performs better underwater (reclaims units, tries to move to nearby land when it cant shoot)
• Improvements to ACU overcharge behaviour (it should OC when retreating, and also positions better against enemy PD)
• Switch to new error handling approach (orange messages now appear in the log with more detail; this also happens when any ACU dies)

Issues currently present I'm aware of (which shouldnt have a noticeably detrimental impact on performance, but will be noticeable to anyone looking at the log):

• 'MergeLocations' error message on certain maps - should just mean larger platoons dont regroup at a midpoint before attacking
• 'MexRaider' warning about the final waypoint - in most cases just affects which mexes are targetted (in rare cases might cause a delay in the platoon doing anything)
• 'Stuttering'/delays in performance on certain maps (e.g. open palms)

V7 Update - T2 is now here! My changes were focused on creating my own engineer and platoon formation functionality (which took far far longer than I was expecting due to various difficult bugs - a lot of work for not much benefit), but at the end I threw in some basic upgrade logic so my AI now gets T2.

Various bug fixes. More notable ones are that the AI should work better on adaptive maps that 'hide' some of their mexes, and should be less likely to try and build on unpathable mexes
Custom platoon formation and engineer functionality. A couple of more notable features of this are that engineers now work together to build 1 building (e.g. power) rather than independently try and build 5 at once, and factor in what mexes theyre already planning to build on when choosing where to expand
Building upgrade logic - AI will now upgrade mexes and land factories (meaning for the first time T2 units can hit the battlefield!)
Load spreading of functionality to help counter the CPU performance hit of these changes
I've also updated the guide for a couple of points (e.g. including a section on events)

As before, much more detail on the changes/my thought process as I went about implementing them is in the dev log.

v8 Hotfix - Fixed a game crash that could happen if the AI lost an upgrading building (thanks to relent0r for for flagging there was a crash), along with a couple of other minor bug fixes.

v9 Reintroduced air units:

• Air scouts are built to periodically scout locations of interest
• Interceptors are used to help defend against enemy air attacks
• Bombers should target enemy units of interest.
• Air factory should be built early if there's enough power (and by a hydro if one is available)
• Various bug fixes and small tweaks to behaviour - refer to 3.27.6 - 3.27.7 of the guide

I admire the guide / developers log that you are keeping

V10 - Implemented anti-turtle logic:

• Use of T2+T3 MMLs and mobile arti to break firebases
• Ecoing (not very well, but better than before)
• Reduced turtle feeding

Misc changes:

• Large bomber attack for if we end up with lots of spare bombers
• T3 air factories can be built/upgraded
• Support factories will be built/upgraded instead of HQs
• Higher tier power is built sooner
• Engineers can request guards
• Various bug fixes (see section 3.30.2 of the guide)
• Updated the guide with details on how to run in FAF Develop offline

My AI also has a new icon

For anyone that's interested in the background to the last 5 versions/2 months - back at v5 I was crushed by gunner1069 (I usually get them to play a match every couple of weeks against the latest AI version) with a simple turtle strategy on Theta Passage - they didnt bother getting any mexes and instead just rushed T2 PD with their ACU and my AI had no answer despite having 4x the mass income. My primary focus in all the versions since then has been coming up with the logic that will give my AI a chance in that scenario.

Unfortunately, a relatively simple plan for a human (dont send units single-file into PD, upgrade mexes and then attack with MMLs) required a bunch of different bits of logic:

• Checking for PD and running away from it (so the enemy isn't fed kills/mass), without running away when we're already in range of it and able to beat it (I had some frustrating to watch games where my AI would charge in at a firebase, overwhelm the shields, be about to destroy everything, before deciding to run and turning a winning position into a loss)
• Realising there's PD when it kills our land scouts without being picked up by their radar
• Making use of air scouts to reveal 'hidden' PD to avoid suiciding units and the ACU into the PD
• Quickly expanding to uncontested mexes (including on enemy side of the map)
• Focusing on maintaining map control and ecoing if faced with a turtle rather than trying to break them immediately, which in turn requires various logic for identifying if we're against a turtle, upgrading mexes, building T2+ buildings, upgrading factories, making use of support factories, etc. (I've still not got basic things done like capping mexes with mass storage)
• Using indirect fire units to attack PD/long range enemy units while both protecting them from enemy mobile attacks, and not suiciding our own units trying to protect them

These changes also mean my AI will be worse in other situations - e.g. against a non-turtle it might choose to eco when it would be able to finish the game by pressing the attack, and it's also still poor at the mid-T2+ stage so extending the game via ecoing means it's more likely to snatch defeat from the jaws of victory. Long term though it needed to happen as a 'T1 or die' strategy only has a decent chance on 5v5 maps.

@maudlin27 Nice list of g
Changes of updates for you Ai mod, I'd like to extend you the offer it have this in a more of a cleaner formate over on the new faf wiki, if this is somtbing your interested in please send me a pm here or on the faf discord and I can set you up with a page