Data driven balance with automated replay watching
-
Data driven balance
What units, built when, actually win you the game when you automatically review hundreds of replays with a python CLI program running 5 games at once on a linux server?. See https://forum.faforever.com/topic/7746/unit-logger-mod/2?_=1756499882452
I now have a few gigabytes from 1400 games of 1v1 ranked and sentons of structured data. If you would like this data just PM me. The data format is below:
An index file is created that shows all the replays, this is created from the replay file
replay_id scfa_version replay_version has_mods has_ai has_unit_restrictions cheats_enabled ranked game_start_epoch game_end_epoch no_players no_teams scenario_type victory_condition teams_locked unranked map_folder_name sim_ticks desync_tick share average_rating 21013072 3770 Replay v1.9 False False False False False 1697735541 1697737795 8 2 skirmish demoralization True False SCMP_009 19991 0 FullShare 1883 21013451 3770 Replay v1.9 False False False False True 1697737205 1697738254 2 2 skirmish demoralization True False frithen.v0004 715 0 FullShare 1853 21021738 3770 Replay v1.9 False False False False False 1697830981 1697834029 8 2 skirmish demoralization True False SCMP_009 24578 0 FullShare 977 21029660 3770 Replay v1.9 False False False False True 1697906403 1697906892 2 2 skirmish demoralization True False neroxis_map_generator_1.9.0_aj5umyuf4xuk6_aqea 4512 0 FullShare 1456an index file is created for all the games that have been simulated
replay_id desync end_of_replay_reached sim_time_seconds 21013072 False True 3868 21013451 False True 162 21021738 False True 4460 21029660 False True 558 21029783 False True 6580each game has an armies file generated directly from the replay file
army_name player_id player_name rating_mean rating_deviation faction team start_spot player_color human ai_personality ARMY_2 413862 cZARJc 2048 95 4 3 2 6 True ARMY_4 420129 HorohLoxMato 2123 94 4 3 4 19 True ARMY_8 145 ZLO 2359 71 1 3 8 17 True ARMY_3 393899 GULMO 2242 94 1 2 3 9 True ARMY_7 372109 Terarii 2543 95 1 2 7 7 True ARMY_6 304809 ZmeiGorinich 2099 94 1 3 6 1 True ARMY_5 486930 el_PISTOLERO 2027 97 2 2 5 2 True ARMY_1 278651 ERROR_4O4 1831 94 2 2 1 3 Trueeach game has an armies result file generated after simulation is complete
army_index army_name result 1 ARMY_1 won 2 ARMY_2 defeated 3 ARMY_3 won 4 ARMY_4 defeated 5 ARMY_5 defeated 6 ARMY_6 defeated 7 ARMY_7 won 8 ARMY_8 defeatedEach game has a economy.tsv file generated after simulation is complete
GameTick Army lastUseRequested.MASS lastUseRequested.ENERGY lastUseActual.MASS lastUseActual.ENERGY stored.MASS stored.ENERGY income.MASS income.ENERGY maxStorage.MASS maxStorage.ENERGY reclaimed.MASS reclaimed.ENERGY 13 1 0.00 0.00 0.00 0.00 650.00 3924.00 0.10 2.00 650.00 4000.00 0.00 0.00 20 1 0.00 0.00 0.00 0.00 650.00 3938.00 0.10 2.00 650.00 4000.00 0.00 0.00 27 1 0.00 0.00 0.00 0.00 650.00 3952.00 0.10 2.00 650.00 4000.00 0.00 0.00 34 1 0.00 0.00 0.00 0.00 650.00 3966.00 0.10 2.00 650.00 4000.00 0.00 0.00 46 2 0.00 0.00 0.00 0.00 650.00 3990.00 0.10 2.00 650.00 4000.00 0.00 0.00 52 3 0.00 0.00 0.00 0.00 650.00 4000.00 0.10 2.00 650.00 4000.00 0.00 0.00 59 3 0.00 0.00 0.00 0.00 650.00 4000.00 0.10 2.00 650.00 4000.00 0.00 0.00 66 3 0.00 0.00 0.00 0.00 650.00 4000.00 0.10 2.00 650.00 4000.00 0.00 0.00each game has a units.tsv file. FractionComplete is % built, generated after simulation is complete
GameTick Army EntityId UnitId FractionComplete Position.x Position.y Position.z MaxHealth Health ShieldRatio IsIdle MassKilled 13 1 0 uel0001 1 63.50 25.53 468.50 12000 12000 0 1 0 19 2 1048576 xsl0001 1 448.50 25.53 43.50 11500 11500 0 1 0 67 1 0 uel0001 1 63.50 25.53 468.50 12000 12000 0 1 0 67 1 1 ueb0101 0.04 67.50 25.56 454.50 4000 148 0 1 0 73 2 1048577 xsb0101 0.06 440.50 25.58 46.50 3500 199 0 1 0 73 2 1048576 xsl0001 1 448.50 25.53 43.50 11500 11500 0 1 0 109 1 0 uel0001 1 63.50 25.53 468.50 12000 12000 0 1 0 109 1 1 ueb0101 0.18 67.50 25.56 454.50 4000 708 0 1 0I simulated 1436 sentons and raned 1v1 replays with this tool.
you can determine if the first team to make a stratiegic bomber actually has an increased chance of winning
You can determine the average mass killed of a novax after 10 mins of it being built
you can see if a t1 frigate rush is a good stratergy to win an ocean against players rated 200 less points than the oponent, and you can see what factions are best in the beech position and win that ocean, with unit position data.
You can see how long it takes each game ender, on average to end the game and chance of winning after it has been up for 5 mins.
If you would like the raw unit data please message me, its a few hundred mb. I would like some people to go over it and try and pick out these correlations
-
Hi, that's pretty rad!
I had a similar idea a while back - to settle the Bricks vs Percy/Loyalists vs Titans debate, and analyze the effectiveness of unit/experimental compositions on certain maps.
But I quickly discovered (after painfully patching available FAF replays parsers, yes) that the replays only had some basic stats and chat logs (including experimental build start/end times, and that's it). I had to simulate and log the game itself, which I found cumbersome (even with some outside help with Lua on Discord); and running FAF with log mods on machines are beyond my skill level.
So I gave up and focused on basic analytics, such as drawing a large graph of who-plays-with-who: https://github.com/rampeer/faf-advanced-analytics
. That was not my original plan, but at least I could do it. It looks like I've lost a few last commits, will look into that.You managed to overcome all these difficulties, which I am very excited about. So I am happy to receive the dataset (the sample file seems to contain exactly the unit data I need).
Also, if you are willing to share the settings/scripts for extracting the data (which I had trouble with), I am willing to run it on replays of other maps (dualgap + mapgen) and then share the data, as I don't play seton's much.
Thanks in advance!
-
I would like to see the stats/chances of winning late game for UEF, my hunch would be much higher rate of winning late game with UEF.
-
@rampeer You’re downloading random replays without checking their validity. I can help you with downloading only valid replays (rated). You can additionally filter by game type, league, etc. If you’re interested pm me in Discord
-
Looks like no FAF games for me this week
. Anyways, some time freed for analytics. I decided to start with my little hunch: Bricks and Percies are quite similar in tech/cost/stats/function, but Percies have huge alpha-strike and better DPS (and UEF has T3 transport, but unit-sharing is a thing); so they are much better fit for T3 drops. They are guaranteed to one-shot your T3 mexes if get dropped (even if your army is near).I extracted drops (too-quickly moving units), and found 186 cases. Among them:
Brick: 59
Percival: 127
So my hypothesis looks plausible; will dig into that.Another one was that higher-rated players are more prone to try to drop. This turned out incorrect, see the image. I'll look up the replays manually to check whether there is something off (for example, Bricks/Percies being grabbed by GCs are 'moving too fast' and trigger drop detection system).
Another one: I wanted to visualize brick and percy drops. I could not get map coordinates right, so the lines are scuffed.
I look forward to presenting arguments to nerf Percies (as a Cybran player, yeah
).P.S. about 1/3 of the replays were too short or empty. Looking for an answer from @Nooby about data collection, especially tick registration - maybe we should implement stuff suggested by @Caliber . Thank you guys, that's getting epic!
P.S.S. I am happy to apply my expertise to my favourite game! Suggestions / hypotheses are welcome
-
-
Couple things here
Brick is considered the superior drop unit due to the superior versatility attached to the unit. If there is a swarm of t1 or t2 ready to respond, a brick can easily kite it away while continuing to raid. Percies are strictly there to kill mexes and pgens and that's about it, they aren't going to be surviving much of any response. To reinforce this, dropping a percy on rock is going to be way less successful than a brick or a harb because you won't be able to efficiently deal with the bp available on the island. Your percy will die and it will immediately be reclaimed and the mexes you did manage to kill will be rebuilt. In some situations, it might have been a net loss in value.
Your data here isn't actually supporting just this conclusion either. There are a litany of reasons why percies are dropped more such as UEF being in more games, UEF being predisposed to more late games than Cybran, UEF having to make percies more often because of a lack of viable late game direct fire options and therefore more being available to drop, and so on. It's a POTENTIAL explanation, but I'd disagree with it being the most likely one because it goes against the common wisdom about the two in terms of drops.
I'm unsure if you're serious about this percy nerf, but it's not even a good unit. You shouldn't be relying on percies if you're playing lategame UEF.
-
@FtXCommando said in Data driven balance with automated replay watching:
I'm unsure if you're serious about this percy nerf,
Not very serious
two numbers are not enough to draw any conclusion, hence "plausible" aka "something to think about" (+I'm a Cybran player, okay).Timing is an interesting point; should include that dimension as well. Do UEF spam Percies more than Cybran spam Bricks? If so - what's the optimal timing-rating-eco vs units-per-minute?
You are right, I need a more thorough analysis - that is what I will do.
-
I now have 1400 ranked 1v1 and sentons games simulated of average ratings > 800
https://forum.faforever.com/post/69369
This is well structured dataset with an example of contents shown in above link and I am sure some good balance analysis can be done on some of this data now
Please pm me for the full dataset, and I will continue to grow it
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login