AI MEGATHREAD
-
Here's what I wanna do, categorize the difficulties of AIs so that hopefully one day we have a simple non-confusing drop down for AIs including not only all AI mods but all AI mods sorted by difficulty and not only that, but an auto-matchmake AI option.
AND NOT ONLY THAT, but that same auto-matchmake AI (which is basically vs AI from starcraft where it ramps up or down the difficulty depending on your win or losses against it and you start against the easiest ((it's really really really well done in starcraft)) )
...would be used for the first 15 (or 20?) placement matches in matchmaking.thereby providing at long last a potential solution to the "bar of entry" problem that newer players face with matchmaking AKA ladder.
with good metrics about what AIs are worth we would replace the current :
A seesaw of randomness during fifteen matches where first match you get placed vs Keyser, second match vs some -600, third match you're against Blackheart and that's about when you decide you've had enough.
with this:
first match is vs and AI easy, You actually loose so you get matched vs it again. you win this time, third match is vs an AI normal, you loose, you get placed vs it again, you win....
(at the end of this process the ladder rates you at about 300. insert obligatory "not great, not terrible")
but the experience and end result is clearly way better.
now.
I admit that even vs a uveso AIx (3.0 cheat modifier) most ladder players would do fine and that wont teach useful things to new ladder players but that's the keyword here NEW.I don't think at the level of ladder I'm currently playing at (750), that any of my opponents could beat an uveso AIx (3.0 cheat modifier).
this serves the NEW players first and foremost who need this. And [us thousand plus played games] we're not about to go through placement matches right now, so we should focus on what the experience should be like for NEW players, that we want to see stay in the community and get better at the game.
I'm pretty sure that there are very good value lessons that this kind of AI is dishing out already without being too much of a detriment to fun or too much of a challenge.
To this end I'm asking for your help to categorize AIs, this won't take up much of your time
there are six AI mods worth downloading to my knowledge :
- "AI-Uveso" (a common mod for all AI mods)
- "AI-Swarm"
- "Nutcracker AI"
- "Dilli AI"
- "RNG AI"
- "micro AI"
add these and enable them in lobby (yes, they are inter-compatible)
test at least four of the 44 (yeah...) AI choices you'll have as a result and draft down your observations here :
https://docs.google.com/spreadsheets/d/1GRkWb0x0PcvwdN5kgaXOl83qEOZ5EmUvvAc8Zrttwp0/edit#gid=0
thanks you all! It takes groundwork like this for years later a good mod to emerge. let's make history!
-
You didn't use Uveso-AI with Swarm-AI which is required. Its also Required for RNG-AI, so ur swarm games are completely in-accurate.
-
This post is deleted! -
@Azraeel said in AI MEGATHREAD:
You didn't use Uveso-AI with Swarm-AI which is required. Its also Required for RNG-AI, so ur swarm games are completely in-accurate.
makes sense.
I was getting the impression from the youtube vids that it should be doing much better
I'll redo those.
EDIT : I've redone those and updated the sheet to match with reorder.
with uveso on swarm was indeed hardest. it's also the best non-cheating. -
@tatsu Uveso is pretty scary if he gets up. You start playing survival at that point
-
agreed. and swarm is butt-kicking also.
-
Seems like an ambitious project but seems worth it.
-
Swarm Eternal is brutal indeed, but my problem with Uveso and Swarm is that you are basically playing a survival game most of the time, they just spam T1 continously.
-
hey, idea of rating AI sounds very fun but I'm not so sure about the methodology. I mean rating them on how playing them "feels". There are some ladder players around 1300 that I feel rather comfortable playing and 900 that I wouldn't feel confident playing at all just because of their unusual playstyle and although I would still reliably win with those 900s, I probably couldn't rate them properly just after one game by "how it felt". If you really want to rate those AIs, maybe it would be optimal to also get them play lots of games against each other on multiple maps and see which ones beats the rest(similar way to how we estimate real players strength).
-
yeah, I agree.
that's the core of the method.
I'm super low on time now so that's why I'm opening this up to everyone
-
Setting up AI's to fight each other is ok to get a base number.
Good to have as side information side (another tab on your spreadsheet maybe? its easier to setup an AI game and walk away if your low on time). Small note here, recently the skirmish game mode was made to allow the human to become observer so you could run these on the offline executable without spamming replays to the replay server.But I kinda agree with tatsu in that its not representative of how those same AI's will fair against humans and there needs to be a 'feel' indicator, especially when it comes to how they play with different human play styles, map preferences, cheat multipliers and even ai game settings. One might do ok against a turtle but garbage against a pressure player and vice versa. Its all 'eye of the beholder' sort of thing, but as someone whos stared at AI games far too much its worth something.
-
Currently AI doesn't really do anything smart and barely attacks you, you actually just move up and blast them with your commander followed by your tanks and AI commander sits still in its base and dies. That's my experience.
The problem is that they don't know how to use their commander.
-
Maybe someone should contact OPENAI and see if they're interested in seeing if they can develop a competent AI for the most complex RTS game in the world
-
-
@tatsu I enabled AI Swarm and Uveso and played vs AI Swarm Terror.
https://replay.faforever.com/13437722
https://replay.faforever.com/13437811Second replay I forced map markers as I was confused to why AI didn't get out of his base much. Also second game I spawned a spy plane to see what the hell he was doing and maybe that's why he built an air factory (my bad), still he was unable to do anything with it not even make bombers. I also was a quarter trying to fix my hotkeys so sometimes I did nothing and still managed to utterly roll all over his base as he made no tanks. I'm around 1000 for team games, not really good 1v1.
-
Well I'd say small maps are often not AI's strong suit.
I dunno try with something a tiny bit more spacious like tag rock, eye of the storm, salt rock colony, Syrtis major, the Dark heart, or arcane.
all of which are 10 x 10 but less in your face then cobalt valley.
but yeah that being said AI generally deals pretty poorly with 10x10 commander bum-rush.
try it on 20x20 maps. try seton's clutch they're pretty hard to deal with in this case you may well end up dying.
if not try with AIx and 20x20
-
The replays were ok for me,
To be fair Mylaur anyone above a 300 rank playing against a non cheating AI is going to have an easy time.
For giggles you could try RNGAI (self promotion, just make sure you read the description for setup tips) with a 1.5 cheat/build multiplier on that same map and see if your experience changes.Getting the acu to be remotely in the realm of a human is damn near impossible(have spent days/weeks/months trying), an (almost) single threaded game just doesn't have enough clock cycles to do complex calculations at the speed to capture the data required to react and make decisions, you can do fake micro but like you said the acu doesn't really know 'what' to do.
Example for reference : getting aeon auroras the try and maintain weapon range on a target during combat takes approximately 7 function calls per second per unit plus supporting logic, if there are 50 T1 auroras on the field fighting that's 350 per second and 50 units is a small number.
-
That's a boatload of processing, and you're only talking about a handful of units.
-
I don't doubt the ability of our AI coders, what I doubt is the ability of any coder no matter how proficient to make an AI that is capable within the SC engine of coming even close to even a 1200 rated player.
AI is simply to bulky at the moment code wise and you either end up with an incredibly specific AI for a specific map, or an incredibly generalist AI, that is weak at everything and easily exploited.
All games face this problem. Which is why most strategy games give insane cheats to AI to offset their inefficiency/stupidity.
OpenAI - or Modern AI techniques avoid all of this, because they function on a stastitical model, which can become very accurate through reptition and therefore are uniquely situated in that they are self training, and not overly bulky when it comes to hogging processing power or requiring huge logic trees.
-
the point of this anyhow is ordering them, because right now they're unordered.
And I never suggested AIs on supcom's purpose could eventually be utilized to beat a 1200 human so I dunno why people keep turning the mic back to me on that one O.o
I do think however that once we have a good classification of AI difficulty we can utilize that to train up our roster of -600 to 400 players.
