Unit Logger mod

[Nooby]

I have created the "Unit Logger" mod. It is in the mod vault. It is a UI mod.

It logs information on every unit in the game log of the focus army every 10 game ticks, configureable in main.lua
Information logged:
"UL02","GameTick","EntityId","Army","UnitId","FractionComplete","Position.x","Position.y","Position.z","MaxHealth","Health","ShieldRatio","IsIdle","MassKilled"

Log size for long games is normally a few MegaBytes

It works in replays if you select the army of the player you require logging.

It requires UMT / UI Mod Tools / ui-mod-tools-4z0t-v12

Thanks for @Ctrl-K / 4z0t for some of the code that makes this work

This data is not in an compact format. It will compress at around ~ 8>1 ratio

Build order can be extracted from this data - and not just the time units / buildings are started or commands given but when units are finished and where they actually are on the map.

using an external tool unit icons could be plotted onto a map with this data for a kind of "replay prieview" function that can be scrolled fowards and back in time, without having to launch the FAF executable.

Does anyone know about selecting all units while in observer? I would like to convert replay files to this unit position format, and currently I can only run one army at a time.

Example data:

info: UL02	GameTick	EntityId	Army	UnitId	FractionComplete	Position.x	Position.y	Position.z	MaxHealth	Health	ShieldRatio	IsIdle	MassKilled
info: UL02	1350	321	1	uea0303	1	171.915	35.555	717.439	2300	1550	0	0	180
info: UL02	1350	421	1	uea0303	1	231.838	34.695	652.384	2300	300	0	0	482
info: UL02	1350	344	1	uea0303	1	109.883	35.376	720.888	2300	2050	0	0	45
info: UL02	1350	369	1	uea0303	1	142.596	38.140	688.592	2300	1300	0	0	45
info: UL02	1360	736	1	uea0303	1	111.701	36.286	717.892	2300	1550	0	0	270
info: UL02	1360	0	1	ual0001	1	108.207	25.830	920.259	11000	11000	0	1	0
info: UL02	1360	262	1	uea0303	1	98.083	37.232	730.919	2300	1550	0	0	450
info: UL02	1360	280	1	uea0303	1	97.939	36.571	708.237	2300	1050	0	0	90
info: UL02	1360	364	1	uea0303	1	145.077	37.485	686.651	2300	1800	0	0	225
info: UL02	1360	318	1	uea0303	1	142.312	37.525	684.928	2300	1550	0	0	360
info: UL02	1360	438	1	uea0303	1	196.809	37.850	668.522	2300	1550	0	0	433

An example log from a high ranked setons game
replay ID 22654992
from the south air player from 0 mins to 20 mins in
20 mins of a setons game is now only 1.6MB log size with the floating point decimal trimming
test.log

[Nooby]

After creating this mod

I created some python code that runs multiple instances of FAF at once on a linux server with this mod enabled and goers through a folder full of replays at high gamespeed generating this data.

I created a seperate python program to parse out the log into an unitfile displaying
"GameTick,Army,EntityId,UnitId,FractionComplete,Position.x,Position.y,Position.z,MaxHealth,Health,ShieldRatio,IsIdle,MassKilled"

and an economy file displaying
GameTick,Army,lastUseRequested.MASS,lastUseRequested.ENERGY,lastUseActual.MASS,lastUseActual.ENERGY,stored.MASS,stored.ENERGY,income.MASS,income.ENERGY,maxStorage.MASS,maxStorage.ENERGY,reclaimed.MASS,reclaimed.ENERGY

This was done a year ago or so. I have processed 659 replays creating 12GB of CSV files (compressed to ~ 300mb) of unit positions with time, player economies, and if players won or lost. From this data you can absolutley get insight into what units built at what time correlate to a win. If you would like this data please message me.

Example data is attached.

exampledata.zip

[Nooby]

I now have a few gigabytes from 1400 games of 1v1 ranked and sentons of structured data. If you would like this data just PM me. The data format is below:

An index file is created that shows all the replays, this is created from the replay file

replay_id	scfa_version	replay_version	has_mods	has_ai	has_unit_restrictions	cheats_enabled	ranked	game_start_epoch	game_end_epoch	no_players	no_teams	scenario_type	victory_condition	teams_locked	unranked	map_folder_name	sim_ticks	desync_tick	share	average_rating
21013072	3770	Replay v1.9	False	False	False	False	False	1697735541	1697737795	8	2	skirmish	demoralization	True	False	SCMP_009	19991	0	FullShare	1883
21013451	3770	Replay v1.9	False	False	False	False	True	1697737205	1697738254	2	2	skirmish	demoralization	True	False	frithen.v0004	715	0	FullShare	1853
21021738	3770	Replay v1.9	False	False	False	False	False	1697830981	1697834029	8	2	skirmish	demoralization	True	False	SCMP_009	24578	0	FullShare	977
21029660	3770	Replay v1.9	False	False	False	False	True	1697906403	1697906892	2	2	skirmish	demoralization	True	False	neroxis_map_generator_1.9.0_aj5umyuf4xuk6_aqea	4512	0	FullShare	1456

an index file is created for all the games that have been simulated

replay_id	desync	end_of_replay_reached	sim_time_seconds
21013072	False	True	3868
21013451	False	True	162
21021738	False	True	4460
21029660	False	True	558
21029783	False	True	6580

each game has an armies file generated directly from the replay file

army_name	player_id	player_name	rating_mean	rating_deviation	faction	team	start_spot	player_color	human	ai_personality
ARMY_2	413862	cZARJc	2048	95	4	3	2	6	True	
ARMY_4	420129	HorohLoxMato	2123	94	4	3	4	19	True	
ARMY_8	145	ZLO	2359	71	1	3	8	17	True	
ARMY_3	393899	GULMO	2242	94	1	2	3	9	True	
ARMY_7	372109	Terarii	2543	95	1	2	7	7	True	
ARMY_6	304809	ZmeiGorinich	2099	94	1	3	6	1	True	
ARMY_5	486930	el_PISTOLERO	2027	97	2	2	5	2	True	
ARMY_1	278651	ERROR_4O4	1831	94	2	2	1	3	True	

each game has an armies result file generated after simulation is complete

army_index	army_name	result
1	ARMY_1	won
2	ARMY_2	defeated
3	ARMY_3	won
4	ARMY_4	defeated
5	ARMY_5	defeated
6	ARMY_6	defeated
7	ARMY_7	won
8	ARMY_8	defeated

Each game has a economy.tsv file

GameTick	Army	lastUseRequested.MASS	lastUseRequested.ENERGY	lastUseActual.MASS	lastUseActual.ENERGY	stored.MASS	stored.ENERGY	income.MASS	income.ENERGY	maxStorage.MASS	maxStorage.ENERGY	reclaimed.MASS	reclaimed.ENERGY
13	1	0.00	0.00	0.00	0.00	650.00	3924.00	0.10	2.00	650.00	4000.00	0.00	0.00
20	1	0.00	0.00	0.00	0.00	650.00	3938.00	0.10	2.00	650.00	4000.00	0.00	0.00
27	1	0.00	0.00	0.00	0.00	650.00	3952.00	0.10	2.00	650.00	4000.00	0.00	0.00
34	1	0.00	0.00	0.00	0.00	650.00	3966.00	0.10	2.00	650.00	4000.00	0.00	0.00
46	2	0.00	0.00	0.00	0.00	650.00	3990.00	0.10	2.00	650.00	4000.00	0.00	0.00
52	3	0.00	0.00	0.00	0.00	650.00	4000.00	0.10	2.00	650.00	4000.00	0.00	0.00
59	3	0.00	0.00	0.00	0.00	650.00	4000.00	0.10	2.00	650.00	4000.00	0.00	0.00
66	3	0.00	0.00	0.00	0.00	650.00	4000.00	0.10	2.00	650.00	4000.00	0.00	0.00

each game has a units.tsv file. FractionComplete is % built

GameTick	Army	EntityId	UnitId	FractionComplete	Position.x	Position.y	Position.z	MaxHealth	Health	ShieldRatio	IsIdle	MassKilled
13	1	0	uel0001	1	63.50	25.53	468.50	12000	12000	0	1	0
19	2	1048576	xsl0001	1	448.50	25.53	43.50	11500	11500	0	1	0
67	1	0	uel0001	1	63.50	25.53	468.50	12000	12000	0	1	0
67	1	1	ueb0101	0.04	67.50	25.56	454.50	4000	148	0	1	0
73	2	1048577	xsb0101	0.06	440.50	25.58	46.50	3500	199	0	1	0
73	2	1048576	xsl0001	1	448.50	25.53	43.50	11500	11500	0	1	0
109	1	0	uel0001	1	63.50	25.53	468.50	12000	12000	0	1	0
109	1	1	ueb0101	0.18	67.50	25.56	454.50	4000	708	0	1	0