Monthly AI Tourney Series
-
@Softles can I enter my AI?
-
@Femboy sure - but what would I do with a banner?
@Dragun101 no plans to include SCTA every week, but it would definitely make a good feature for one of the weeks
-
3rd December 2021 - Edition 2 - 2v2 double trouble
This week the AIs played 2v2s on the <1000 rating TMM pool, with full share active. A similar format was followed to last week, except that 2 AIs were eliminated each round (in order to reduce the total number of games that had to be run). 6 AIs were entered this week:- Adaptive AI
- Sorian AI Adaptive
- Dilli AI - last update: 2018/06/10
- DilliDalli AI - last update: 2021/07/27
- M27AI - last update: 2021/11/19
- RNG AI -
As before, any game that reached 45 mins was ended and called as a draw.
The scores in each round were calculated as 3 points per win and 1 point per draw, with the round results as follows:
AI Round 1 Round 2 Round 3 RNG 103 53 13 M27 98 53 13 DilliDalli 110 46 X Dilli 73 20 X Adaptive 24 X X Sorian Adaptive 18 X X Giving us another tied tournament, with #1 spot shared between M27AI and RNG AI!
These results masked some interesting individual strengths and weaknesses for each of the AIs, including our two winners:
M27 AI:
- Great ACU duelling, superb t3 land rushes, and effectively used gifted bases if teammates died.
- Struggled on water maps (as well as serenity due to a bug), and risky plays with the ACU left it vulnerable.
RNG AI:
- Great all round, particularly outclassing other AIs on the water maps. Solid base construction made it hard to beat.
- Struggled for map control on land focused maps, and doesn't yet make use of gifted allied bases on full share.
Now for some extra stats for your viewing pleasure..
AI head to head results across all rounds (cells are from the perspective of the row AI):RNG M27AI DilliDalli Dilli Adaptive Sorian Adaptive RNG ------ 10W, 9D, 11L 5W, 3D, 12L 17W, 1D, 2L 10W, 0D, 0L 10W, 0D, 0L M27AI 11W, 9D, 10L ------ 12W, 3D, 5L 14W, 2D, 4L 4W, 5D, 1L 7W, 1D, 2L DilliDalli 12W, 3D, 5L 5W, 3D, 12L ------ 14W, 2D, 4L 8W, 1D, 1L 10W, 0D, 0L Dilli 2W, 1D, 17L 4W, 2D, 14L 4W, 2D, 14L ------ 9W, 1D, 0L 10W, 0D, 0L Adaptive 0W, 0D, 10L 1W, 5D, 4L 1W, 1D, 8L 0W, 1D, 9L ------ 1W, 8D, 1L Sorian Adaptive 0W, 0D, 10L 2W, 1D, 7L 0W, 0D, 10L 0W, 0D, 10L 1W, 8D, 1L ------ Special credit to Dilli and DilliDalli for perfect records against Sorian Adaptive, as well as RNG for perfect records against both Adaptive and Sorian Adaptive.
Map stats for each AI:
Map RNG M27AI DilliDalli Dilli Adaptive Sorian Adaptive Adaptive Meadow 6W, 2D, 1L 4W, 2D, 3L 3W, 1D, 4L 5W, 0D, 3L 0W, 2D, 3L 0W, 1D, 4L Angel Lagoon 7W, 2D, 0L 2W, 4D, 3L 3W, 2D, 3L 1W, 3D, 4L 0W, 3D, 2L 2W, 0D, 3L Charity 5W, 1D, 3L 6W, 1D, 2L 6W, 1D, 1L 2W, 0D, 6L 0W, 2D, 3L 0W, 1D, 4L Desert Planet II v2 5W, 1D, 3L 7W, 2D, 0L 5W, 1D, 2L 2W, 0D, 6L 0W, 1D, 4L 0W, 1D, 4L Nomadiah 4W, 1D, 4L 8W, 1D, 0L 5W, 1D, 2L 2W, 0D, 6L 0W, 2D, 3L 0W, 1D, 4L Pelagial v2 7W, 2D, 0L 2W, 4D, 3L 1W, 2D, 5L 3W, 3D, 2L 1W, 2D, 2L 1W, 1D, 3L Strife of Titan 3W, 2D, 4L 5W, 2D, 2L 7W, 0D, 1L 4W, 0D, 4L 0W, 1D, 4L 0W, 1D, 4L Serenity Desert 7W, 0D, 2L 0W, 1D, 8L 8W, 0D, 0L 4W, 0D, 4L 2W, 0D, 3L 0W, 1D, 4L Syrtis Major 4W, 0D, 5L 8W, 0D, 1L 5W, 0D, 3L 4W, 0D, 4L 0W, 1D, 4L 0W, 1D, 4L Turtle Rocks 4W, 2D, 3L 6W, 3D, 0L 6W, 1D, 1L 2W, 0D, 6L 0W, 1D, 4L 0W, 1D, 4L Special credit to DilliDalli for the 100% record on Serenity Desert.
As an extra special treat, I also recorded performance over time stats for each of the games, and averaged over all their games to get an idea of each AI's in game performance (i.e. how quickly does it run in game). This stat isn't perfect, since it doesn't control for how performant opponents were in their games, but it gives a rough idea.
(perf here is measured as #real seconds per 10 game seconds while speed is set to +10, with full +10 roughly meaning a perf of 1.5 and +0 roughly meaning a perf of 13)Hope people find this interesting - tune in next week for 4v4s, 5v5s and 6v6s on classic team-game maps. As ever feel free to drop in ideas for future weeks or anything else
-
Also let me know how you want results tables formatted in the future - should I stick with {Wins, Draws, Losses}, swap to {points} or go back to {win %} in the head to head and map results tables? (or something else entirely??)
-
Points or win %, w/d/l is impossible to compare across a table like that.
-
I would definitely like for your continued insights each week on the different AI's strengths and weaknesses.
Very Interesting Stuff!
Thanks for testing, and posting these updates -
11th December 2021 - Edition 3 - 4v4+: you can't spell teaim without AI.
This week the AIs were playing on a selection of team game maps to find out who you should bring to back you up in a com fight:
- Round 1: Tabula Rasa v3 (4v4) and The Pyramid 5v5 (5v5)
- Round 2: Hilly Plateau (4v4) and Diversity (4v4)
- Round 3: Canis 4v4 spezial edition (4v4) and Adaptive Wonder Open (8v8)
Rules this week were share until death and a 2 hour game limit (which none of the games reached). I'll just include the win % in the tables below since there were no draws this week.
Each AI played every other AI on every map once in round 1, twice in round 2, and 4 times in round 3. Without further ado, here were the results:
AI Round 1 Round 2 Round 3 RNG Standard 60% 58% 87% DilliDalli 90% 75% 13% M27AI 70% 33% X Dilli 60% 33% X Sorian Edit Adaptive 10% X X Swarm 10% X X Congrats to RNG Standard for the first outright win of the series!
The head to head and per map results were as follows:
RNG Standard DilliDalli M27AI Dilli Sorian Edit Adaptive Swarm RNG Standard - 64% 33% 83% 100% 100% DilliDalli 36% - 100% 67% 100% 100% M27AI 67% 0% - 50% 100% 100% Dilli 17% 33% 50% - 100% 100% Sorian Edit Adaptive 0% 0% 0% 0% - 50% Swarm 0% 0% 0% 0% 50% - RNG Standard DilliDalli M27AI Dilli Sorian Edit Adaptive Swarm Tabula Rasa 60% 80% 60% 80% 0% 20% The Pyramid 60% 100% 80% 40% 20% 0% Hilly Plateau 33% 100% 67% 0% X X Diversity 83% 50% 0% 67% X X Canis 100% 0% X X X X Wonder 75% 25% X X X X This week I also pulled performance stats for each AI against itself in the two round 1 maps, which produced the results below:
Again for reference, a performance of X => X/10 real seconds per game second (trying to run at +10 speed). Results plotted against the number of units on the map.Congrats again to RNG Standard for winning, and remember tune in next week for some Free For All chaos!
-
Great series of tournaments, thanks for conducting them
-
22nd January 2022 - Edition 4 - 1v1s: Cry me a river
After a festive break we are back with a bumper edition of the AI tourney - 11 AIs entered, fighting it out across 5 rounds (round robin) of 1v1s on the following maps:
- Twin Rivers (round 1)
- Twin Rivers (round 2)
- Twin Rivers (round 3)
- Twin Rivers (round 4)
- Twin Rivers (round 5)
The AIs entered for this week are:
Adaptive AI, Dalli AI, Dilli AI, DilliDalli AI, M27 AI, RNG Standard AI, SCTA Arm, SCTA Core, Sorian Adaptive AI, Sorian Edit Adaptive AI, Swarm Terror AI.Format
The twist is that each round AIs are granted bonus cheat multipliers based on the previous round's results. In brief, an AI gets a +0.1 boost to resources and build rate for every position down the leaderboard it was in the last round - and accumulates these bonuses as the tournament goes on.To stop these boosts getting out of hand in the matches, the boosts applied in game are scaled down based on the lower of the boosts between two AIs. For example if AI Alpha with boost 2.0 is playing AI Bravo with boost 2.4, then the applied settings would be a 1.0x multiplier for Alpha (2.0/2.0) and a 1.2x multiplier for Bravo (2.4/2.0). Applied boosts are rounded to the nearest 0.1 due to UI limitations.
The aim is that as the rounds progress, we get a better and better idea of what boosts each AI needs to be an equal match for every other AI (kinda like the AI's handicap). The AI with the lowest boost at the end of the tourney wins!
Matches:
M: Accumulated Modifier going into the round
W: Wins that round
P: End of round placement (* joint)Round 1 Round 2 Round 3 Round 4 Round 5 AI M W P M W P M W P M W P M W P Adaptive 1.0 3 8 1.7 7 3* 1.9 0 11 2.9 6 2* 3.0 2 9* Dalli 1.0 9 1* 1.0 2 10 1.9 9 2 2.0 5 7* 2.6 9 2 Dilli 1.0 8 3 1.2 3 8* 1.9 8 3 2.1 6 2* 2.2 2 9* DilliDalli 1.0 6 4* 2.3 7 3* 1.5 4 6* 2.3 2 11 2.9 10 1 M27 1.0 5 6* 1.5 9 1 1.5 3 9 2.3 5 7* 2.9 6 4* RNG Standard 1.0 9 1* 1.0 0 11 2.0 10 1 2.0 6 2* 2.1 4 6* SCTA Arm 1.0 2 9 1.8 5 6* 2.3 4 6* 2.8 3 9* 3.6 7 3 SCTA Core 1.0 1 10* 2.9 8 2 2.0 2 10 2.9 6 2* 3.0 4 6* Sorian Adaptive 1.0 1 10* 1.9 6 5 2.3 6 4 2.6 3 9* 3.4 6 4* Sorian Edit 1.0 6 4* 1.3 3 8* 2.0 5 5 2.4 6 2* 2.5 3 8 Swarm Terror 1.0 5 6* 1.5 5 6* 2.0 4 6* 2.5 7 1 2.5 2 9* Results:
For a final set of modifiers at (raw, normalised to nearest 0.1):
2.6, 1.0: RNG Standard
2.7, 1.0: Dalli
3.0, 1.2: Dilli, DilliDalli
3.2, 1.2: M27, Sorian Edit
3.3, 1.3: Swarm Terror
3.5, 1.3: SCTA Core
3.7, 1.4: Sorian Adaptive
3.8, 1.5: Adaptive, SCTA ArmCongrats to RNG Standard AI for winning, commiserations to everyone else!
For fun, here's a view of the normalised modifiers (normalised compared to leading AI that round) each AI had as the tourney progressed:
I think the scores would have continued to settle down a little bit if I'd kept doing more rounds, but after manually running 275 games in a row I didn't feel like anymore...
--
The particularly sharp amongst you may have noticed that the weekly tourney hasn't exactly been weekly lately - and going forwards this will become a less regular thing (maybe monthly?) so that I can focus on developing AIs to enter.
See you all in the next one!
-
@softles Thanks again for running these! Re moving to once a month Iām happy to run one once a month to share the load if you wanted to have it as a biweekly tourney?
-
AI biweekly tourney - AstromAInia
The humans had their turn on this map in a recent tourney, now it's the AI's time to show how the map should really be played!Format
This time there were 12 entrants - the 11 from the last tourney, and a new addition of the base game's AI: Turtle on what is arguably a turtler's dream map.The setup of the tourney is as follows:
All matches: Random AI faction with a random start position
- Group stage: The 12 AI are split into 4 groups of 3, based on their performance in the last tourney (twin rivers). Each AI plays the other AI in its group, with the worst AI being eliminated
- Quarter finals: The winner from each group plays the 2nd best AI from another group, in a best of 3
- Semi finals: The quarter final winners are paired up to fight in a best of 3, similar to the quarters
- Finals: The two finalists duke it out in a best of 5
The results
Group stagesGroup AI Wins Group 1 RNG Standard 1 Group 1 Swarm Terror 2 Group 1 SCTA Arm 0 Group 2 Dalli 2 Group 2 SCTA Core 0 Group 2 AI: Turtle 1 Group 3 Dilli 1 Group 3 M27 2 Group 3 Sorian Adaptive 0 Group 4 DilliDalli 2 Group 4 Sorian Edit 1 Group 4 Adaptive 0 Quarter finals
AI 1 (group stage winner) AI1 wins AI2 (group stage runner up) AI2 wins Swarm Terror 3 Dilli 0 Dalli 3 Sorian Edit 0 M27 3 RNG Standard 0 DilliDalli 1 AI: Turtle 2 Semi finals
AI 1 AI1 wins AI2 AI2 wins Swarm Terror 2 Dalli 1 M27 3 AI: Turtle 0 Finals
AI 1 AI1 wins AI2 AI2 wins Swarm Terror 0 M27 5 Winner: M27
Overall results
AI Wins Losses Win percent M27 13 0 100% Swarm Terror 7 6 54% Dalli 6 2 75% AI: Turtle 3 5 38% DilliDalli 3 2 60% Sorian Edit 1 4 20% Dilli 1 4 20% RNG Standard 1 4 20% SCTA Arm 0 2 0% SCTA Core 0 2 0% Sorian Adaptive 0 2 0% Adaptive 0 2 0% Other random observations
- First blood - Sorian Edit took just 8m09 to wrap up its winning game; in contrast, Swarm terror averaged 52m53 for the games it won
- Sneak attack - RNG was the only AI to make use of the rear entrance to the base, sending a galactic colossus through it in one game
- Best recovery - In one game Swarm lost 1/3 of its 18 base mexes, yet managed to repel the attack and eventually claim a win
- In a case of the AI results imitating the human tourney, the winning AI's last update was created only a short while before the tourney started.
-
I'll probably do another mini-AI tourney in 1-2 weeks - does anyone have any particular scenarios they're interested in seeing and/or are people still interested in these?
I've noted some scenario examples below; whether I pick a scenario will depend in part on if it won't take too long to manually run (i.e. the number of rounds and games that would be required), and my preference is probably towards a variety of maps (since the tourneys up until now have shown how differently the same AIs can perform based on the map chosen).
Tourney ideas:
- 1v1 5km / 10km / 20km Tourney - selection of just 5km, or just 10km or just 20km maps, based on some of the maps that have featured in recent 1v1 map pools
- Matchmaker based tourney - e.g. <700 rank map pool for 1v1; 2v2; or 4v4 full share
- Mimicking other recent human tourneys (e.g. the FFA tourney; King of Setons; King of Dual Gap)
-
This makes me depressive
-
Would love to see 20km map games with all 3 theaters. But I know we don't have enough coverage in the AI's for naval. But thats an area where we are weakest.
-
~-~-~
-
AI biweekly/monthly tourney: 20km Combined arms
The theme of this map was 20km maps that allow use of land, navy and air. The maplist chosen was:- Crossfire Canal - FAF version
- Norfair
- The Ditch - FAF version
- Setons Clutch - FAF version
- The Bermuda Locket - FAF version
Setup
Setup is similar to the last tourney - 12 AIs are split into 4 groups. The weakest 4 are eliminated in the first round, with the winners from one group fighting the runners up from the other in the quarter-finals in 3 matches.
As there were 5 maps, maps were chosen randomly for the group and quarters, and then for the semis and finals each map was played.
Starting positions are the default first and second positions set by the map (e.g. on Setons this means a 'mid vs mid' battle)Changes to AI
5 of the featured AI have released new versions since the last tourney: M27, Swarm Terror, SCTA Arm, Sorian Edit, SCTA CoreThe results
Group stagesGroup AI name Wins 1 M27 4 1 DilliDalli 2 1 SCTA Core 0 2 Swarm Terror 4 2 Sorian Adaptive 2 2 Sorian Edit 0 3 Dalli 3 3 Dilli 2 3 Adaptive 0 4 RNG Standard 4 4 SCTA Arm 2 4 AI: Turtle 0 Quarter finals
AI Name Wins AI Name Wins M27 3 Dilli 0 Swarm Terror 3 SCTA Arm 0 DilliDalli 3 Dalli 0 RNG Standard 3 Sorian Adaptive 0 Semi finals
AI Name Wins AI Name Wins M27 5 Swarm Terror 0 DilliDalli 5 RNG Standard 0 Finals
AI Name Wins AI Name Wins M27 5 DilliDalli 0 Total wins by AI
AI Name Wins Win % M27 17 100% DilliDalli 10 59% RNG Standard 7 58% Swarm Terror 7 58% Dalli 3 43% Dilli 2 29% Sorian Adaptive 2 29% SCTA Arm 2 29% Sorian Edit 0 0% SCTA Core 0 0% Adaptive 0 0% AI: Turtle 0 0% Detailed results
The full results can be seen in the 'DetailedResults' worksheet of the following file:
https://docs.google.com/spreadsheets/d/1i2ObsZ49S9gUTgWw0E68qZ8VrOkaP_Jk/edit?usp=sharing&ouid=100973959280546778272&rtpof=true&sd=trueOther Observations
- While some AI attempted to build navy, in all cases it proved ineffective with the mass invested in navy far exceeding the mass killed. Neither of the finalists built a single naval unit the entire tourney.
- The finals were closer than the scoreline suggests, with DilliDalli dominating land on most of the maps and winning map control (e.g. holding 5 out of the 6 islands at one point on Crossfire Canals, and effectively raiding and even claiming mexes behind M27's main base on Setons). The finals ended up being an air vs land contest on most maps.
- SCTA Arm managed to have 2 T3 tanks solo a GC (GC's shot was blocked by cliffs) but it wasn't enough to let it claim a win
- Heroic unit of the tourney: DilliDalli's lone striker vs M27 on Setons which managed 19 kills and 1,720 mass kills raiding mexes
- Notable changes in fortunes: Swarm managed a turnaround in speed, going from the slowest AI to claim wins in the last tourney to getting the second fastest win (11m21, beaten only by Dilli who got a win in 10m48)
-
AI biweekly/monthly tourney: MapGen
The idea of this tourney was to use MapGen to create a map for the AI to fight against for 5km, 7.5km, 10km, 15km and 20km. Due to the AI tourney still being run manually a single map was used for the map size for the tourney (rather than generating a new random map every time). The maplist chosen was:- neroxis_map_generator_1.8.5_vgf5tqx2uxrcu_aicae_
- neroxis_map_generator_1.8.5_uydyqpk4utuuu_aidae_
- neroxis_map_generator_1.8.5_xaghpcvo2lnrk_aieae_
- neroxis_map_generator_1.8.5_bvwpdifwrols4_aigae_
- neroxis_map_generator_1.8.5_44sxnthefcek6_aiiae_
Setup
Setup is similar to the last tourney - 12 AIs are split into 4 groups. The weakest 4 are eliminated in the first round, with the winners from one group fighting the runners up from the other in the quarter-finals in 3 matches.
As there were 5 maps, maps were chosen randomly for the group and quarters, and then for the semis and finals each map was played.Changes to AI
3 of the featured AI have released new versions since the last tourney: M27, SCTA Arm, SCTA CoreThe results
Group stagesGroup AI name Wins 3 DilliDalli 4 3 Sorian Edit 3 3 SCTA Arm 0 2 M27 4 4 SCTA Core 0 4 Dalli 2 4 RNG Standard 3 1 Adaptive 1 2 AI: Turtle 2 1 Swarm Terror 4 1 Dilli 2 2 Sorian Adaptive 0 Quarter finals
AI Name Wins AI Name Wins Swarm Terror 2 Sorian Edit 1 M27 3 Dalli 0 Dilli 2 DilliDalli 1 RNG Standard 3 AI: Turtle 0 Semi finals
AI Name Wins AI Name Wins Swarm Terror 0 M27 5 Dilli 1 RNG Standard 4 Finals
AI Name Wins AI Name Wins M27 4 RNG Standard 0 (M27 and RNG drew 1 game)
Total wins by AIAI Name Wins Win % M27 16 94% RNG Standard 10 59% Swarm Terror 6 50% DilliDalli 5 63% Dilli 5 38% Sorian Edit 4 50% Dalli 2 29% AI: Turtle 2 29% Adaptive 1 20% Sorian Adaptive 0 0% SCTA Arm 0 0% SCTA Core 0 0% Detailed results
The full results can be seen in the 'DetailedResults' worksheet of the following file:
https://docs.google.com/spreadsheets/d/1i2ObsZ49S9gUTgWw0E68qZ8VrOkaP_Jk/edit?usp=sharing&ouid=100973959280546778272&rtpof=true&sd=trueOther Observations
- Notable changes in fortunes: In the final, RNG had c.25 mass to M27's 621, yet still managed a draw (M27 attacked RNG's base with its ACU leading to both ACUs dying)
- The 7.5km map featured two land masses separated by water, which caused several AIs issues
- Although SCTA failed to get a win it managed to match Sorian Edit for map control and eco for much of the game, with both games going over an hour
- The group stages required two deciding matches to determine groupings, with an extra Dilli vs Adaptive and Sorian Edit vs DilliDalli match due to them getting the same number of wins as each other.
-
AI biweekly/monthly tourney: Sorian vs the world
The idea for this tournament was to see the highest AiX modifier that each AI could beat Sorian Adaptive at, on a selection of 5km, 10km and 20km maps from the current map pool, hopefully highlighting how much more comeptitive the FAF custom AIs are, as well as giving a better gauge of the distance between different AI than the previous tournaments win rate. The maplist chosen was:- Mirage
- Stickleback Ridge
- Open Palms
- 8 - Badlands_v4
- The Bermuda Locket - FAF version
- Crossfire Canal
Setup
Each AI entered will face off against Sorian Adaptive AiX (with no omni). If they win, then they repeat the fight but with Sorian Adaptive getting +0.1 to its modifier, until the challenging AI no longer wins. This is done for each map separately. This resulted in a mammoth 245 games being run, so I doubt I'll be repeating this format due to the massive amount of time it ended up taking (for comparison, the last tournament took just 51 games)Changes to AI
There were significant changes to the AI this time. In addition to a new version of M27, Sorian Edit, and SCTA, there was a major update for RNG (the first in many months), and we have a new AI entrant with Uveso Rush featuring for the first time!
The results
SummaryAI Name Average AiX Beaten M27 1.63 RNG Standard 1.57 DilliDalli 1.43 Swarm Terror 1.27 Uveso Rush 1.25 SCTA Core 1.15 Adaptive 1.05 Sorian Edit 0.98 Results by map
8 - Badlands_v4 Adaptive 1.0 8 - Badlands_v4 DilliDalli 1.4 8 - Badlands_v4 M27 1.6 8 - Badlands_v4 RNG Standard 1.6 8 - Badlands_v4 SCTA Core 1.6 8 - Badlands_v4 Swarm Terror 1.3 8 - Badlands_v4 Uveso Rush 2.0 8 - Badlands_v4 Sorian Adaptive 0.4 Crossfire Canal Adaptive 1.0 Crossfire Canal DilliDalli 1.1 Crossfire Canal M27 1.3 Crossfire Canal RNG Standard 1.3 Crossfire Canal SCTA Core 0.2 Crossfire Canal Sorian Edit 0.8 Crossfire Canal Swarm Terror 1.1 Crossfire Canal Uveso Rush 0.7 Crossfire Canal Sorian Adaptive 1.0 Mirage Adaptive 1.2 Mirage DilliDalli 1.5 Mirage M27 1.9 Mirage RNG Standard 1.7 Mirage SCTA Core 1.4 Mirage Swarm Terror 1.5 Mirage Uveso Rush 1.2 Mirage Sorian Adaptive 0.4 Open Palms Adaptive 1.1 Open Palms DilliDalli 1.6 Open Palms M27 1.6 Open Palms RNG Standard 1.6 Open Palms SCTA Core 1.3 Open Palms Sorian Edit 1.0 Open Palms Swarm Terror 1.1 Open Palms Uveso Rush 1.1 Open Palms Sorian Adaptive 0.6 Stickleback Ridge Adaptive 1.0 Stickleback Ridge DilliDalli 1.6 Stickleback Ridge M27 2.0 Stickleback Ridge RNG Standard 2.0 Stickleback Ridge SCTA Core 1.3 Stickleback Ridge Sorian Edit 1.1 Stickleback Ridge Swarm Terror 1.2 Stickleback Ridge Uveso Rush 1.5 Stickleback Ridge Sorian Adaptive 0.4 The Bermuda Locket - FAF version Adaptive 1.0 The Bermuda Locket - FAF version DilliDalli 1.4 The Bermuda Locket - FAF version M27 1.4 The Bermuda Locket - FAF version RNG Standard 1.2 The Bermuda Locket - FAF version SCTA Core 1.1 The Bermuda Locket - FAF version Sorian Edit 1.0 The Bermuda Locket - FAF version Swarm Terror 1.4 The Bermuda Locket - FAF version Uveso Rush 1.0 The Bermuda Locket - FAF version Sorian Adaptive 0.7 Detailed results
The full results can be seen in the Data worksheet of the following file:Other Observations
- Uveso was the dominant AI on badlands (due to its ability to punish Sorian Adaptive for sending its ACU to attack on its own early on), managing a win against a 2.2 AiX (it lost against a 2.1 AiX that was running at the same time hence why 2.0 is shown in its results). In contrast as an AiX 1.2 it lost to a 1.0 Sorian Adaptive on Crossfire Canal.
- Crossfire Canal was by far Sorian Adaptive's best map, with 3 of the AI (Sorian Edit, SCTA Core and Uveso) needing AiX modifiers to beat it (although not strictly correct, if an AI needed an AiX modifier to win I've shown each 0.1 modifier as decreasing the Sorian AiX beaten by 0.1, so an AI needing a 1.5 modifier to win would show as 0.5)
- Surprisingly, Adaptive AI was better than Sorian Adaptive, beating it on every map with no modifier, and beating 1.2 on Mirage.
- There were compatibility issues between Sorian Edit and Sorian Adaptive on some maps, so results for these have been omitted
- 3 AI managed the achievement of beating Sorian Adaptive AiX 2.0 or better - M27 and RNG (2.0, Stickleback Ridge) and Uveso (2.2, Badlands)
Edit: Note that some of the AI aren't intended for 20km or water based maps, which will be a reason for poor results on Crossfire (e.g. SCTA isn't meant for such maps). Excluding crossfire from the results would give the following:
-
AI monthly tourney: Survival of the fittest
Suggested by Fearghal, this tournament's theme is to see which AI does best on a survival map. chp2001 helpfully identified the map Final Rush Pro 5 as a potential candidate, since it features 1 team against another (thus avoiding the problem caused by survival maps which lack an enemy brain/base location for the AI to fight against).For those not familiar with it, it's a 4v4 map where teams face off against each other with a central chokepoint area where the AI spawns. The AI spawns will be delivered by transport, but air is disabled (so the transports are harder to shoot down).
The survival spawns are split allocated equally between two different survival 'brains', and wont attack each other, but will attack one (and only one) of the teams. This means that a team can choose to be aggressive by sending their units at the enemy base, to help support the survival AI's attack.
To avoid people exploiting the survival AI, ACUs are prevented from hiding in the water by taking damage over time, and prevented from hiding in the rear corner of the map behind cliffs by a group of galactic colossi. The map has a number of configurable settings, including difficulty, which affects the size of the surivival spawns, and how long it takes before it advances in waves (with a wave starting off with T1 units, before progressively getting harder, e.g. sending in bombers; t2 land; cybran destroyers; t3 land; experimental land; etc.). Players start with 9 hives, an ACU from each faction, and lots of mass points at their base.
Setup
In the first round, each AI will play against itself (i.e. all 8 players will be that AI), and see how long it takes for the last player to die. These times will then be used to give each AI an initial 'seeding', with the last seed playing the second last, the winner of that playing the 3rd last, and so-on until the last ranked AI has lost to the 2nd last ranked AI which has lost to the 3rd last ranked AI and so on.The map settings still present some issues for AI, in particular the logic for preventing exploiting the survival AI by hiding - I turned this off for the initial 'seeding' round over fears most of the AI wouldn't be able to understand the 'ACUs die when entering water' approach or that 'those hostile GC near your base can be ignored' , but it led to its own problems, e.g. fully upgraded ACUs hiding in the water being very slowly whittled down by the torpedoes of a couple of bricks and didnt give a fair reflection of how well some AI did. I therefore settled on a compromise for the main event, where a team would be treated as having lost if all 4 land bases were wiped out (i.e. it had no more mexes or land factories).
The difficulty was also set to easiest, as all AI struggle compared to humans with such a setup, and this way there was more of a chance of seeing waves hit the experimental land stage.
Changes to AI
New versions of M27, RNG, and SCTA were reflected for the tournament.
The results
Initial seedings - Time lasted when fighting itselfAI Name Time survived M27 11m25 AI Turtle 16m28 DilliDalli 17m27 Sorian adaptive 19m20 Adaptive 19m22 RNG Standard 29m06 Sorian Edit 31m41 Uveso Rush 34m02 SCTA Core 35m07 Swarm Terror 37m16 Final rankings and head to heads
AI Name Head to Heads won Head to Heads lost Longest time survived until a team died (in seconds) Sorian Edit 0 6 992 Sorian adaptive 1 3 1122 AI Turtle 2 3 1029 Adaptive 2 3 1080 SCTA Core 1 4 1807 DilliDalli 5 2 1176 RNG Standard 3 2 1629 Uveso Rush 2 2 1907 Swarm Terror 1 1 1940 M27 9 0 1940 For example, DilliDalli was initially seeded 9th (out of 10 AI); it lost to M27 (the 10th seed), but beat the 5 AI that placed worse than it in the final results to reach 5th place. It then lost to RNG (4th place), so didnt advance any further. In contrast, as Swarm was seeded first it only fought the 2nd place initially (which ended up being M27), and after losing that it then fought the then 3rd place (Uveso Rush) which it won, resulting in ending as 2nd place.
Detailed results
The full results can be seen in the Data worksheet of the following file:Other Observations
- There were a wide variety of approaches used by the different AI, for example:
- M27 favoured a firebase appraoch, but chose to place this near the centre of the map meaning it'd get overwhelmed around the time of T3 land or experimentals
- DilliDalli preferred an all-out assault on the enemy team before spawns even started arriving, sticking to T1 mexes but often being able to kill the enemy team before their economic advantage could be brought to bear
- Swarm chose to have a massive swarm of mostly T1 units occupying the central area, meaning they could quickly ambush the survival AI drops as they arrived (up until the late T3 land/early experimental stage)
- Uveso made heavy use of T1 artillery, which when firing from behind T3 shields at the base proved an effective roadblock to the survival AI's T3 land waves
- There were dramatic transformations in fortunes for the AI from their initial seedings. In DilliDalli's case this was because it's aggressive approach would lead to short matches in all its games, while in M27's case it had chain reactions caused by the various starting ACU (16 per team) being near the centre and one of them dying. Sorian Edit's reversal in fortunes was likely caused by the stricter measuring of when a team had lost, as it was usually better at retreating its ACU behind cliffs/into water where the survival AI couldnt reach it
- A number of games came down to the wire with one team losing its final base just before the opposing team lost its final base, including the final between M27 and Swarm, and the semi-final between Uveso Rush and Swarm
- SCTA performed strongly, with a survival time in head to heads of more than 30m (bettered only by Uveso, Swarm and M27), but DilliDalli's early aggression proved its undoing
- It looked like RNG would pull back from the brink in its game against Uveso Rush when it completed a cloaked laser upgrade on its ACU at its final base. Sadly it turns out that the map gives omni vision meaning its ACU still died quickly to the T3 land spam.
-
"This week I also pulled performance stats for each AI against itself in the two round 1 maps, which produced the results below:"
I have been testing for the upcoming RC tourny and have been using "mostly" M27 as the standard to
measure against the other Game specific AI's,
I also start test, at game option speed = Fast, (which is = to +4 sim) to make sure the game speeds are "not likely" to drop below "0" game speed.
Game speed limits the size of the maps I can use -
(if games slow to "-1", after 45 minutes, I limit the AI's to 10x10 maps)
The faster AI's like DilliDalli and RNG are easily able to handle the 20x20 maps.
Since the test games are a combo of the the Game AI and M27 -
I was wondering if with all the recent updates to M27 if it is still as fast as the performance results shown above.I would test vs Uveso, (since it needs to be loaded for all games anyway)
butit seems that after Uveso grows it's base large it also slows game down a little,
so I don't want to run tests with Uveso as the standard, as it might skew the test results.
Update: I just tested Uveso since post and It runs fast.
It was still +4 at end of 37 min game with 6 AI:Uveso and Lots of units!Would it be possible to include another performance chart after you run your next tourny,
with the updated AI versions?
Thanks.
