Weekly AI Tourney Series

11th December 2021 - Edition 3 - 4v4+: you can't spell teaim without AI.

This week the AIs were playing on a selection of team game maps to find out who you should bring to back you up in a com fight:

• Round 1: Tabula Rasa v3 (4v4) and The Pyramid 5v5 (5v5)
• Round 2: Hilly Plateau (4v4) and Diversity (4v4)
• Round 3: Canis 4v4 spezial edition (4v4) and Adaptive Wonder Open (8v8)

Rules this week were share until death and a 2 hour game limit (which none of the games reached). I'll just include the win % in the tables below since there were no draws this week.

Each AI played every other AI on every map once in round 1, twice in round 2, and 4 times in round 3. Without further ado, here were the results:

Congrats to RNG Standard for the first outright win of the series!

The head to head and per map results were as follows:

This week I also pulled performance stats for each AI against itself in the two round 1 maps, which produced the results below:

Again for reference, a performance of X => X/10 real seconds per game second (trying to run at +10 speed). Results plotted against the number of units on the map.

Congrats again to RNG Standard for winning, and remember tune in next week for some Free For All chaos!

Great series of tournaments, thanks for conducting them

22nd January 2022 - Edition 4 - 1v1s: Cry me a river

After a festive break we are back with a bumper edition of the AI tourney - 11 AIs entered, fighting it out across 5 rounds (round robin) of 1v1s on the following maps:

• Twin Rivers (round 1)
• Twin Rivers (round 2)
• Twin Rivers (round 3)
• Twin Rivers (round 4)
• Twin Rivers (round 5)

The AIs entered for this week are:
Adaptive AI, Dalli AI, Dilli AI, DilliDalli AI, M27 AI, RNG Standard AI, SCTA Arm, SCTA Core, Sorian Adaptive AI, Sorian Edit Adaptive AI, Swarm Terror AI.

Format
The twist is that each round AIs are granted bonus cheat multipliers based on the previous round's results. In brief, an AI gets a +0.1 boost to resources and build rate for every position down the leaderboard it was in the last round - and accumulates these bonuses as the tournament goes on.

To stop these boosts getting out of hand in the matches, the boosts applied in game are scaled down based on the lower of the boosts between two AIs. For example if AI Alpha with boost 2.0 is playing AI Bravo with boost 2.4, then the applied settings would be a 1.0x multiplier for Alpha (2.0/2.0) and a 1.2x multiplier for Bravo (2.4/2.0). Applied boosts are rounded to the nearest 0.1 due to UI limitations.

The aim is that as the rounds progress, we get a better and better idea of what boosts each AI needs to be an equal match for every other AI (kinda like the AI's handicap). The AI with the lowest boost at the end of the tourney wins!

Matches:
M: Accumulated Modifier going into the round
W: Wins that round
P: End of round placement (* joint)

Results:
For a final set of modifiers at (raw, normalised to nearest 0.1):
2.6, 1.0: RNG Standard
2.7, 1.0: Dalli
3.0, 1.2: Dilli, DilliDalli
3.2, 1.2: M27, Sorian Edit
3.3, 1.3: Swarm Terror
3.5, 1.3: SCTA Core
3.7, 1.4: Sorian Adaptive
3.8, 1.5: Adaptive, SCTA Arm

Congrats to RNG Standard AI for winning, commiserations to everyone else!

For fun, here's a view of the normalised modifiers (normalised compared to leading AI that round) each AI had as the tourney progressed:

I think the scores would have continued to settle down a little bit if I'd kept doing more rounds, but after manually running 275 games in a row I didn't feel like anymore...

--

The particularly sharp amongst you may have noticed that the weekly tourney hasn't exactly been weekly lately - and going forwards this will become a less regular thing (maybe monthly?) so that I can focus on developing AIs to enter.

See you all in the next one!

@softles Thanks again for running these! Re moving to once a month I’m happy to run one once a month to share the load if you wanted to have it as a biweekly tourney?

AI biweekly tourney - AstromAInia
The humans had their turn on this map in a recent tourney, now it's the AI's time to show how the map should really be played!

Format
This time there were 12 entrants - the 11 from the last tourney, and a new addition of the base game's AI: Turtle on what is arguably a turtler's dream map.

The setup of the tourney is as follows:

All matches: Random AI faction with a random start position

• Group stage: The 12 AI are split into 4 groups of 3, based on their performance in the last tourney (twin rivers). Each AI plays the other AI in its group, with the worst AI being eliminated
• Quarter finals: The winner from each group plays the 2nd best AI from another group, in a best of 3
• Semi finals: The quarter final winners are paired up to fight in a best of 3, similar to the quarters
• Finals: The two finalists duke it out in a best of 5

The results
Group stages

Quarter finals

Semi finals

Finals

Winner: M27

Overall results

Other random observations

• First blood - Sorian Edit took just 8m09 to wrap up its winning game; in contrast, Swarm terror averaged 52m53 for the games it won
• Sneak attack - RNG was the only AI to make use of the rear entrance to the base, sending a galactic colossus through it in one game
• Best recovery - In one game Swarm lost 1/3 of its 18 base mexes, yet managed to repel the attack and eventually claim a win
• In a case of the AI results imitating the human tourney, the winning AI's last update was created only a short while before the tourney started.

I'll probably do another mini-AI tourney in 1-2 weeks - does anyone have any particular scenarios they're interested in seeing and/or are people still interested in these?

I've noted some scenario examples below; whether I pick a scenario will depend in part on if it won't take too long to manually run (i.e. the number of rounds and games that would be required), and my preference is probably towards a variety of maps (since the tourneys up until now have shown how differently the same AIs can perform based on the map chosen).

Tourney ideas:

• 1v1 5km / 10km / 20km Tourney - selection of just 5km, or just 10km or just 20km maps, based on some of the maps that have featured in recent 1v1 map pools
• Matchmaker based tourney - e.g. <700 rank map pool for 1v1; 2v2; or 4v4 full share
• Mimicking other recent human tourneys (e.g. the FFA tourney; King of Setons; King of Dual Gap)

This makes me depressive

Would love to see 20km map games with all 3 theaters. But I know we don't have enough coverage in the AI's for naval. But thats an area where we are weakest.

We could also try an Air battle type game on Air Wars 6v6 map.

AI biweekly/monthly tourney: 20km Combined arms
The theme of this map was 20km maps that allow use of land, navy and air. The maplist chosen was:

• Crossfire Canal - FAF version
• Norfair
• The Ditch - FAF version
• Setons Clutch - FAF version
• The Bermuda Locket - FAF version

Setup
Setup is similar to the last tourney - 12 AIs are split into 4 groups. The weakest 4 are eliminated in the first round, with the winners from one group fighting the runners up from the other in the quarter-finals in 3 matches.
As there were 5 maps, maps were chosen randomly for the group and quarters, and then for the semis and finals each map was played.
Starting positions are the default first and second positions set by the map (e.g. on Setons this means a 'mid vs mid' battle)

Changes to AI
5 of the featured AI have released new versions since the last tourney: M27, Swarm Terror, SCTA Arm, Sorian Edit, SCTA Core

The results
Group stages

Quarter finals

Semi finals

Finals

Total wins by AI

Detailed results
The full results can be seen in the 'DetailedResults' worksheet of the following file:
https://docs.google.com/spreadsheets/d/1i2ObsZ49S9gUTgWw0E68qZ8VrOkaP_Jk/edit?usp=sharing&ouid=100973959280546778272&rtpof=true&sd=true

Other Observations

• While some AI attempted to build navy, in all cases it proved ineffective with the mass invested in navy far exceeding the mass killed. Neither of the finalists built a single naval unit the entire tourney.
• The finals were closer than the scoreline suggests, with DilliDalli dominating land on most of the maps and winning map control (e.g. holding 5 out of the 6 islands at one point on Crossfire Canals, and effectively raiding and even claiming mexes behind M27's main base on Setons). The finals ended up being an air vs land contest on most maps.
• SCTA Arm managed to have 2 T3 tanks solo a GC (GC's shot was blocked by cliffs) but it wasn't enough to let it claim a win
• Heroic unit of the tourney: DilliDalli's lone striker vs M27 on Setons which managed 19 kills and 1,720 mass kills raiding mexes
• Notable changes in fortunes: Swarm managed a turnaround in speed, going from the slowest AI to claim wins in the last tourney to getting the second fastest win (11m21, beaten only by Dilli who got a win in 10m48)

AI biweekly/monthly tourney: MapGen
The idea of this tourney was to use MapGen to create a map for the AI to fight against for 5km, 7.5km, 10km, 15km and 20km. Due to the AI tourney still being run manually a single map was used for the map size for the tourney (rather than generating a new random map every time). The maplist chosen was:

• neroxis_map_generator_1.8.5_vgf5tqx2uxrcu_aicae_
• neroxis_map_generator_1.8.5_uydyqpk4utuuu_aidae_
• neroxis_map_generator_1.8.5_xaghpcvo2lnrk_aieae_
• neroxis_map_generator_1.8.5_bvwpdifwrols4_aigae_
• neroxis_map_generator_1.8.5_44sxnthefcek6_aiiae_

Setup
Setup is similar to the last tourney - 12 AIs are split into 4 groups. The weakest 4 are eliminated in the first round, with the winners from one group fighting the runners up from the other in the quarter-finals in 3 matches.
As there were 5 maps, maps were chosen randomly for the group and quarters, and then for the semis and finals each map was played.

Changes to AI
3 of the featured AI have released new versions since the last tourney: M27, SCTA Arm, SCTA Core

The results
Group stages

Quarter finals

Semi finals

Finals

(M27 and RNG drew 1 game)
Total wins by AI

Detailed results
The full results can be seen in the 'DetailedResults' worksheet of the following file:
https://docs.google.com/spreadsheets/d/1i2ObsZ49S9gUTgWw0E68qZ8VrOkaP_Jk/edit?usp=sharing&ouid=100973959280546778272&rtpof=true&sd=true

Other Observations

• Notable changes in fortunes: In the final, RNG had c.25 mass to M27's 621, yet still managed a draw (M27 attacked RNG's base with its ACU leading to both ACUs dying)
• The 7.5km map featured two land masses separated by water, which caused several AIs issues
• Although SCTA failed to get a win it managed to match Sorian Edit for map control and eco for much of the game, with both games going over an hour
• The group stages required two deciding matches to determine groupings, with an extra Dilli vs Adaptive and Sorian Edit vs DilliDalli match due to them getting the same number of wins as each other.

AI biweekly/monthly tourney: Sorian vs the world
The idea for this tournament was to see the highest AiX modifier that each AI could beat Sorian Adaptive at, on a selection of 5km, 10km and 20km maps from the current map pool, hopefully highlighting how much more comeptitive the FAF custom AIs are, as well as giving a better gauge of the distance between different AI than the previous tournaments win rate. The maplist chosen was:

• Mirage
• Stickleback Ridge
• Open Palms
• 8 - Badlands_v4
• The Bermuda Locket - FAF version
• Crossfire Canal

Setup
Each AI entered will face off against Sorian Adaptive AiX (with no omni). If they win, then they repeat the fight but with Sorian Adaptive getting +0.1 to its modifier, until the challenging AI no longer wins. This is done for each map separately. This resulted in a mammoth 245 games being run, so I doubt I'll be repeating this format due to the massive amount of time it ended up taking (for comparison, the last tournament took just 51 games)

Changes to AI
There were significant changes to the AI this time. In addition to a new version of M27, Sorian Edit, and SCTA, there was a major update for RNG (the first in many months), and we have a new AI entrant with Uveso Rush featuring for the first time!
The results
Summary

Results by map

Detailed results
The full results can be seen in the Data worksheet of the following file:

https://docs.google.com/spreadsheets/d/1i2ObsZ49S9gUTgWw0E68qZ8VrOkaP_Jk/edit?usp=sharing&ouid=100973959280546778272&rtpof=true&sd=true

Other Observations

• Uveso was the dominant AI on badlands (due to its ability to punish Sorian Adaptive for sending its ACU to attack on its own early on), managing a win against a 2.2 AiX (it lost against a 2.1 AiX that was running at the same time hence why 2.0 is shown in its results). In contrast as an AiX 1.2 it lost to a 1.0 Sorian Adaptive on Crossfire Canal.
• Crossfire Canal was by far Sorian Adaptive's best map, with 3 of the AI (Sorian Edit, SCTA Core and Uveso) needing AiX modifiers to beat it (although not strictly correct, if an AI needed an AiX modifier to win I've shown each 0.1 modifier as decreasing the Sorian AiX beaten by 0.1, so an AI needing a 1.5 modifier to win would show as 0.5)
• Surprisingly, Adaptive AI was better than Sorian Adaptive, beating it on every map with no modifier, and beating 1.2 on Mirage.
• There were compatibility issues between Sorian Edit and Sorian Adaptive on some maps, so results for these have been omitted
• 3 AI managed the achievement of beating Sorian Adaptive AiX 2.0 or better - M27 and RNG (2.0, Stickleback Ridge) and Uveso (2.2, Badlands)

Edit: Note that some of the AI aren't intended for 20km or water based maps, which will be a reason for poor results on Crossfire (e.g. SCTA isn't meant for such maps). Excluding crossfire from the results would give the following: