[The "What"] AIs in ladder - season 2

Fichom

As you might know, @Deribus opened a discord channel to discuss AIs in ladder, in which why we would even want such a thing was discussed [this was added after @Katharsas first message].

There is a lot of back and forth, and it's hard to keep track of who said what and what was said and what wasn't. The essential problem is the discussion very quickly dispersed in all kind of directions, and we all know where that leads to - nowhere (not Netherlands). One thing that is somewhat agreed on is that AI in ladder games (in some form) would be a good possibility.

In lue of the aforementioned, I'm (I know I'm just a worm, but ¯|(ツ)/¯) starting this new thread. Why would we need a new one?

Well, the general problem of this discussion can be very crudely separated into 5 questions:
"Why" do we want to have this in the first place.
"What" do we want the AI in ladder to look like (if at all).
"How" are we going to achieve what we decide on (the technical nitty-gritty).
"If" the thing we decide on can even be done (even more technical nitty-gritty).
"Who" will be doing the actual work of getting this to - work (the omnipresent problem of open source).

The direction I'd like to steer this part of discussion (hence the title) is the 'What'. I'm ready to "hand over the microphone" to anyone who is willing to steer this discussion, and maybe log what things were and weren't said in one post, so people just joining the discussion can read on what has been already said in a summarized way, as reading even 10 posts is a hassle if they are long (like this one, sry). In the meantime, you are stuck with me.

I'm not saying the "If" or "Who" can't affect "What" indirectly by affecting the "How". I understand it can. But we need a baseline, and I feel like little to nothing was achieved in the discord discussion (apart from maybe few people that were maybe diligent enough to read through all of the messages - me NOT included, but I did read/partake in the majority of it).

The podium is open.

To address @Katharsas's message (3rd in this thread): the goal is to enhance the ladder experience by adding AI - to reduce queueing times, make it more approachable to new players, you name it. Keep in mind, this is a very brief summary.

Fichom

[Reserved for logging purposes, unless someone takes the lead in that department]
WIP: (btw, if you plan on commenting, and you already did in discord, you could be patient and wait for me to add it here eventually, so you can just say "I support §17.34." or whatever - but be my guest to write it down again, just know I'm following the stuff I have in my head right now, and it might be a while for me to start integrating new views from the forum posts)

Note that I'm only writing down what I've seen people write. I'm not gonna go into who said what (yet), and I'm not saying these are reflective of my opinion.

In general, the approach I've seen in the discord to the "What" can be split into these 4 broad categories (note they all consider both 1v1 and TMM). Note that I will add details as they come to light (be it pro and con arguments, or the approaches that could be taken, etc.):

§1. Absolutely no AI ladder.

This is quite self-explanatory. There are those who believe that AI should not be integrated into the ladder. Some of the arguments are:

players don't want to play vs AI, that's why they are playing ladder
it could have bad impact on player rating (e.g. players could learn patterns vs AI and get some free juicy rating)

§2. Yes to AI in existing ladder, but with restrictions.

This group argues that AI could be an ok solution, but only in particular use-cases. People aren't necessarily against AI, but are either concerned with players being stuck playing vs AI in times of smaller player base activity, or with how an AI could even function in a ladder matchup (in regards to "player" skill, rating, etc.).

Another point that in a sense covers all of the following is, regardless of the particular implementation, players would like to see number of games played vs AI capped in some way (e.g., 1 in 5 can be vs AI, or only 3 a day). There are two general reasonings for this:

prevent players from being stuck in an unfortunate loop of playing a lot of AI games (say 2 players are active, but the length and offset of their games causes them to both be playing vs AI, instead of each other).
prevents players from "boosting" their rating by intentionally playing vs AI only, assuming there is a way to exploit an AI

Given all of the above, here are the approaches to this general category of view:

§2.1. Allow players to opt-in to play AI.

There are those who believe that each player should be able to decide if he will be matched up against AIs. There are 2 general solutions proposed as to how it might work:

have a setting somewhere that allows the player to opt in or out of AI matching: this approach would mean players don't have to play
have a button that is always there, and if a player is not willing to wait to get matched with another player, they could just press it at any time (no need to opt-in as in previous approach)
The general notion is that not all would be happy with an AI opponent, and might find it a waste of time, so this is a middle ground - allowing for those that would like it to have the ability to use it, without forcing it one those that wouldn't.

§2.2. Restrict AI's impact on rating.

Some consider games played vs AI should not contribute to the score above a certain threshold. Another more particular note was, winning vs AI should not allow for league promotion (if you want to go from silver to gold, you must win vs a player). Some of the arguments for this approach:

most arguments for this idea are very similar to §1.
there are suggestions rating should only be lost vs AI in higher-rated matches
-- some point out that would effectively result in players being hostages in AI games - assuming a player doesn't want to lose rating, they have to waste their time on a game, since they can't gain nothing

§2.3. Restrict AI's matching to only newer/lower rated players.

Some argue that AI should only appear in ladder matches against new or very low rated players. The reasoning behind it is it could be a good approach to facilitate early progress and engagement of newer players, but because of the points raised in §1., it could have negative effects on the mid and high-rated ladder.

§3. Yes to AI in existing ladder, it goes through it just like a player would.

This one is self-explanatory in essence as well. The problem is a lot of people are worried on how this could even work.

§4. Create a separate ladder for "vs AI" games.

This one is also quite self-explanatory. The idea is to leave the existing 1v1 and TMM ladders alone, and create a dedicated PvE ladder that only features player vs AI games. Some of the arguments for it:

there is a population of players that enjoy PvE situations
all the different AIs available in FAF could be confusing and dissuading for players that want to partake in PvE
custom games have a hard time being filled up if they are PvE
given the points above, a dedicated PvE queue could entice players who prefer PvE games to join in on the ladder, and be in general more welcoming to new members of the community
additionally, some of those that do join in the PvE could in theory transfer over to the PvP once they get more confident in their skills and experience (but it's not a given)
-- some point out the opposite could happen, and already relatively small PvP ladder user base could be diluted with addition of PvE ladder

Katharsas

As i argued in the Discord, what is missing here is still the goal of all of this.

If the goal is to accurately determine the skill of an AI, then option 1 and 4 are completely useless

Anybody arguing for 1 should propose an alternaitve solution. Option 4 would not work for that goal because Trueskill rating in a "vs AI" would just suck, so it would not really be a "matchmaking" queue in the sense that we currently have.

If the goal is to allow players to play against AI easier than in custom matches currently, then this is a completely different goal that needs its a spearate discussion.

SpikeyNoob

My opinion is that a separate “queue” that allows u to play ais with a different rating system. Queue being in quotes since for 1v1 it would not be necessary to wait for other players. Ais would be ranked and cheat levels could be applied to let higher rated players play.

arma473

Put me down for #2.

A lot of people are not sophisticated enough to set up "proper" 1v1 games against AIs. They wouldn't know which AI to use or what settings or which maps give the best gameplay against AI. Putting it into the ladder would give them fast access to different AIs and teach them the names of AIs and such.

So adding AIs to the ladder for some players would help them to have a broader gameplay experience. But limit it for example to players under 700 rating. Anyone with over 700 rating should be able to set up their own custom games IF they want to play against an AI.

This would also help to gauge how good the AIs are (constant feedback in terms of what "rating" the AI would have if it was human) and give replays to review to see how to make AIs better

It would give more diverse experiences to lower-rated ladder players

People generally are less stressed out about playing against AI, so even just the possibility that they might draw an AI opponent might make the ladder a little less scary and painful for people without a lot of ladder experience

Making a new queue is bad. That would just complicate things. More options to players is not more better. It's better to improve the options they have, where you can.

"If you don't like playing against AIs, then just git gud" -- I think it's fair to say this. Lower-rated players usually don't even hate playing against AIs. What low-rated player would feel punished if they searched ladder and got an AI opponent?

ThomasHiatt

Option 2/3 seems pretty good since there is pretty much no downside, assuming AI gains and loses rating the same as players do. If the AI are easily abused and farmed then they will lose rating until they are against appropriate opponents, or can't find matches. If the AI is boring or bad enough that a player can recognize it, then they should also be able to beat it easily. Ladder really should be anonymous so you do not know who you are playing against, or if they are AI or human. There's pretty much no chatting or human elements involved in 1v1 anyway.

Sladow-Noob

(ig you alrdy thought about that but first thing that comes into my mind is AIs matching each other pog)

Dunno if I like it. Mostly speaking from trainer-experience but first, we'd need an AI like M27, Uveso or smth similar. The ""normal"" ones FAF has to offer are simply too braindead and only create bad habits.
If implemented, then only for <400 ranked players anyway.
If implemented, then create an option since active players who're just low ranked (e.g. Coca) might not be a fan of playing against AIs over and over again.
Since I'm not in the target-range for non-cheating AIs anymore, can't say if they're a challenge or a waste of time. I honestly don't give any trainee the advice to play against AIs (EXCEPT until they've seen the units in the game. But that's it.).

To sum it up: Might be okay-ish with a modded AI for really new players below 300 or 400, after that.. Not a fan of it at all.
[Trainer perspective, Idc about the potential rank 'manipulation' possibilities AIs will give]

ThomasHiatt

No cheating AI should ever be in ladder and the default AI are very bad and probably should not be in ladder, but low rated players are just as good at creating bad habits as the default AI, so they could be fine too really. Modern FAF AI are pretty good and I don't think any hard rating limit should be implemented, let the rating system do it's job and place them where they belong.

Sladow-Noob

Referring to the 'Modern FAF AI are pretty good'

Problem is that those AIs don't react to certain things. E.g. I can promise you every trainee I got that played actively against AI has the habits of spamming millions of stationary turrets everywhere cuz the normal AI doesn't give a shit. It'll still send the same amount of bombers to the same places.
With M27 afaik those problems are at least fixed, but then everyone has to download the mods.

Fichom

I didn't make this clear (yet) - most of those who are proponents of adding AI, are suggesting it with the core idea being that AI will be only used to fill up the queue if there are no players to match up against.

I'll be going into more details of how (not that it's not a "How" discussion, "How" only related to how it will work under the hood) those could be regulated - a quick note for those pondering it: allow only when there are no appropriately-rated players to fill the queue, and even then, maybe give priority to off-rated players than to AI, depending on the scenario.

The only idea that doesn't fit the above is the idea to "have new ladder players have a couple of first games vs AI", which sounds reasonable.

Also yeah, most people would argue that cheat-modifiers for AIs are a no-no, and that weak AIs are not worth adding (but that is beyond the scope of what we could affect, things like which AI to include and which not to would probably be decided by existing team that take care of the ladder happenings and other game-related technicalities).

I'll also be adding a more detail explanation (in a latter message, as my own point of views, as to not sully the 'Log' post) why technically adding even the most braindead AI wouldn't really matter whatsoever - rating related or "bad habits" related, it doesn't actually matter at all - but again, I want to first complete the Log message (I'm not doing it non-stop, and I ain't paid to do it, so you'll have to be patient).

Tomma

Actually integrating better ais into game is pretty pog, i dont think you need rating system for that, but addidng easy to acces option to play vs like m27/swarm is great for newer players i think. Like currently you have to download ai mod (and some players dont even know they exist), create custom game etc. I think its a bit inaccesible. Adding like vs ai button somewhere wouldnt hurt and will improve new players experience.

Sladow-Noob

I'm with Tomma. If we're talking about those types of AIs.. The bad habits are still there, but not as big and might be outweighted by getting crushed over time due to wrong investments.
Priorisations in the Queue also fixes my other concern.

We'd need a lot of newbies playing 1v1s against those AIs though so we can analyse the games and see where the weakpoints are in general and what ""feelings""/impressions the newbies have

Sladow-Noob

@fichom said in [The "What"] AIs in ladder - season 2:

so you'll have to be patient

weak. I'm procrastinating as well atm, you should do the same and prioritise discussions with randoms on the FAF forum!

arma473

Another reason to add AIs to the ladder is to increase interest in FAF. Part of the message would be: "If you develop your own AI, we might put it into the ladder." That could be a strong incentive to someone with talent to work on FAF AIs.

(Edit: I'm not suggesting that we LACK people with talent. I'm just talking about bringing on MORE people.)

maudlin27

I'd be happy with any of options 2-4 at a high level.

Thoughts on potential options
My preferred solution would probably be non-cheating AI being matched with people who join the 1v1 queue and fail to find a game after a couple of cycles, with an option for people to opt out. Players would only get matched with an AI within a certain rating range (say +/- 300 of that AI's rating), and also would only be matched if their last game (or last x games) was against a human. New players would also fight against an AI as one of their initial 'rating assessment' games (to hopefully give a better new player experience to the 1-sided initial games that can happen with the current system). I'm guessing this falls into option 2.1.

A variation on the options is to also have the AI implementation done temporarily (i.e. for the option 2s, you could have a week each month or quarter where AI gets added to the pool, a bit like how mapgen was featured for a week).

If option 4 was done (separate ladder) then I'd prefer it to have both AI and AiX; if it was done for just 1v1s against the AI then it could be done as a 'AI Challenge' button since a game could be created as soon as you click on it, with the goal of trying to get the highest possible rating. While the rating given from this wouldn't be as useful as under options 2-3 it'd still be of some value.

Potential benefits
As to some of the potential benefits I can think of initially from these options (which will vary depending on the option):

Rating for AIs [options 2-3, and to a lesser extent 4] - this helps people select the appropraite AI for their skill level in custom games, and gives a better sense of their own skill level if they beat said AI. It also helps the AI developer get a sense of how strong their AI is and incentivises them to improve the AI further (in order to improve their AI's rating).
Better assessment of new player ratings [options 2-3] - with appropriately rated AIs, it should be possible to give a better 'new player experience', by having new players fight an AI in one of their initial matches which is likely to provide a gentler introduction than being crushed by a much higher rank
Decreased matchmaker wait times [options 2-3] - allowing AI to play to some extent effectively means more 'players' are playing matchmaker at once, so you wait less time to find a game.
Potentially increased matchmaker popularity [options 2-4] - PvE is very popular on FAF. Depending on how the AI was implemented, it could lead to more people who would normally only play PvE trying out PvP (note that it's also possible for the reverse effect)
Improved AI developer feedback [options 2-4] - AI devs would be able to look at replays against their AI to see how their AI performs and identify improvements to make, resulting in better AI for FAF.
Improved PvE experience [Option 4] - I suspect many players aren't even aware of the various custom AI available in FAF and assume that the best AI we have is the Sorian adaptive AI. A queue that highlights the different AI with appropriate settings could give an easy way for particularly new players to jump in and play a PvE game without needing to decide on an AI or map. Meanwhile having AiX featured would allow PvE players to compete with each other at who can do the best against AI (since each AiX would have a rating, in turn allowing people doing this option to try and gain the highest rating by beating the highest AiX modifiers), and in turn give the AI developers clearer examples of flaws in their AI to be improved.

Potential disadvantages
There are also potential drawbacks from each of the options. The main ones I'd see are:

Time and complexity of implementation - I dont know enough about what would be involved to say which one would be easier/harder. All options would I expect require a significant amount of time though.
Potentially decreased PvP playerbase [more of a risk with option 4] - i.e. with a separate queue there's the risk people who would have queued for PvP 1v1 queue for PvE instead
People being forced to play AI when queuing for PVP [option 2-3, depending on implementation] - if there's no opt in/out option for playing against an AI then you could end up with someone who doesnt want to fight an AI getting a game against an AI (that they then lose rating for if they quit rather than playing)
Increased wait time for people who opt out of AI games [option 2] - e.g. player A joins the queue, and gets matched with an AI. 10 seconds later player B joins the queue. With no AI they'd have been matched with player A, but now they dont get matched. However it's debatable if in aggregate this would result in an increased wait time (if player A would have given up on queuing then the AI could result in more people queuing over time, meaning both players benefit)

Sladow-Noob

I especially like the "AI challenge"-idea.. Might give players a motivation to have a goal instead of just 'getting better', getting crushed by AI also isn't as frustrating as against real players, especially cuz some leave rude comments on the way.
For testing I also fully support the idea with "AI ladder-week".

Deribus

I just want to state that this is in the very early discussion phase. Development and matchmaking teams are still talking about how it would be implemented, and whether it would be a good idea at all.

thecore

I think if the AI was a self learning neural network and playing in ladder games helped it improve then that would be very cool. The google AI for starcraft did something similar in their ladder games.

Katharsas

I have expressed my opinion on this topic in a separate thread, that is hopefully more abstract level (and more about custom games) than this thread:
https://forum.faforever.com/topic/5557/faf-for-casual-players

I generally don't think that queues as we have them for TMM are a good fit for PvE in general. They would have to substantially change to be a good fit, as described in that thread. I think that AIs in queues are a potential good way to asses their rating, that's it, and we should concentrate on that.

A proper coop queue meant for PvE players would at the very least need difficulty selection, so id argue custom games are better suited anyways, unless we just want to implement a way to make finding coop partners easier.

Deribus

@thecore said in [The "What"] AIs in ladder - season 2:

I think if the AI was a self learning neural network and playing in ladder games helped it improve then that would be very cool. The google AI for starcraft did something similar in their ladder games.

I agree it would be very cool. If Google wants to provide us with their bleeding edge AI tech worth millions of dollars as well as the servers to run it all for free I don't think anyone would complain.