Ladder Curation: An Analysis of Maps

[FtXCommando]

Lately I've been approached by several mappers that were curious about the process utilized to review maps for the ladder. Since this process is going to be extended to team matchmaker maps as that feature gets closer and closer to reality, I figured I might as well as give people an insight into how maps are reviewed so they can both figure out how to get their maps on the matchmaker while also improving their own craft.

I'd also like to say that the process itself isn't some solidly ironed out way of approaching maps. Through the years we have adjusted how we calculate final scores, how we interpret various values in our criteria, and the weight we put on the various factors that we look at when reviewing maps. But overall, I think we've hit a nice groove and we're quite comfortable with what our system allows and what it doesn't allow into the map pools we make.

I will be breaking this post down into the 3 categories we use to look at maps: Aesthetics, Gameplay, and Variance.

Aesthetics:

A lot of people operate under the assumption that the look of a map is irrelevant. All that should matter is if it plays well, and if it looks like a piece of shit then who cares? Well, they're kind of right and kind of wrong in that dissection.

First, aesthetics is not simply the "look" of a map. While the artistic merit of your map is what determines say a 3.5/5 or a 4.5/5 map, it is not what causes a map to be a 1/5. Your map must meet the baseline criteria of having terrain that is both clear and concise. If a map lies to the player, by, for example, having things that look like plateaus that cannot actually be dropped, then it is a bad map aesthetically. These are the things that stay in the mind of a player and quickly make them dislike the map as it can easily cause a game loss through no real fault of the player. If your map cannot meet this bar, then your map could expect to be a 1.5 or a 2 out of 5 even if it makes me cry at the beauty of your decal work.

Beyond that area, we begin to enter the more subjective area we tend to define through the competency of your design and the effort put into it. How do we define both? Well, we have several people on the mapping team (biass, petric, farm, morax) who have made several solid maps aesthetically and have the eye to review whether the decal and strata work put into a map reflects serious effort. Generally when it comes to reviewing this side of aesthetics, we leave reviewing maps up to our circle of highly competent mappers.

So why did I say that the critique that map aesthetics don't matter is kind of right? Well, because we actually do agree with that. We weigh aesthetics to account for 25% of a map's overall score. This is below the expected proportion of 33% if we weighed it equal to gameplay and variance. Your map could look entirely generic and still be a 2/5 so long as it doesn't lie or frustrate the player. This shouldn't harm your overall score (and therefore entry into the ladder) assuming the gameplay and variance are still up to par. I mean, look at badlands. It looks terrible, but it's still a pretty common map in pools because it plays pretty well.

General Guidelines:
0 - Eye gouging; super badly scaled textures/decals, terrain makes no sense, color scheme makes the eyes bleed
1 - Quite ugly; maybe not actually painful on the eyes but extremely bland undetailed and wacky, badly indicated terrain
2 - So-so; a playable configuration even if uninspiring or amateurish. Perhaps not very obvious heightmap in some places. Nothing disgusting tho
3 - Fine; A solid average-looking map, won't win any prize for looks but can't find much fault with it
4 - Good; A very appealing map, maybe because of high detail, pleasant color scheme and lighting, maybe something else
5 - Great; One of the best looking maps in your opinion. Something just makes it stand out and above others

Gameplay:

Now we enter the classic social science experience of redefining common terms so your explanation makes sense. Gameplay has a very specific meaning for us and it's critical to understand it before commenting on it. If you have a tough time with it don't worry, it had to be explained like a dozen times to select people on the ladder team itself.

Here is my go-to explanation for gameplay. Imagine I had some sort of hypothetical vault. If I went into said hypothetical map vault which has 2000 high level, competent games of purely your map and then proceeded to use a random number generator to pick one of these 2000 games, how high quality would that singular game be? So essentially, "gameplay" is a rating of the "meta quality" of your map.

Now of course we don't have 2000 high level, competent games on your map. There's maybe two dozen maps on FAF that could even think of approaching such a criteria. And that's precisely why the ladder team itself exists! It's a pool of 2k+ players that have the ability to determine how a map is likely to play out based on little data. Is it perfect? Certainly not, but since we lack the neural network AI that can play a map ten thousand times to show us the peak efficient play on it, it's the best we got.

Ultimately this part of maps is also graded on a curve. For example, we can't just call Theta Passage a garbage map because it results in t1 spam and a com battle in every game while Roanoke's Abyss let's you do frigate rush or air abuse. You very clearly need to judge every map size in its own category, and so while Theta might be a relatively "simple" map, it does what it does very well and so it is still a highly rated 5x5.

So how do we decide between a low, middle, and high gameplay score? Well this is quite subjective, as you might imagine, but we attempt to balance it out by synthesizing a general score out of the opinions of everyone on the team. In general a map is considered to have a high quality meta if it involves gameplay that requires some level of thought and interaction with the opponent. Maps can be all t1 spam and still be fairly decent on gameplay so long as the t1 spam meta still requires you to maneuver around your enemy and wins you strategic control of various contested areas. If the map is just a flat, boring map where you have a mex every 500 pixels in every direction and so spam can go anywhere and be efficient, well that's just boring and tedious to play.

As you might imagine, gameplay is the most important factor for us and we weigh it as 45% of a map's overall score. The gameplay is the foundation that everything else is built upon.

General Guidelines:
0 - Gameplay makes you pull your hair out
1 - Gameplay is uninspiring and boring or highly annoying
2 - You can live with the gameplay but you still rather avoid this map
3 - Average game experience for your taste
4 - You find gameplay to be interesting and stand above the majority of other maps
5 - This might as well be your favorite map, you love the way it plays, you wouldn't mind to host custom 1v1s on it whole day

Variance:

Finally, the Siamese twin of gameplay. People often have a hard time figuring out where one begins and the other ends, and I sometimes wonder about it myself too.

Ultimately variance is your reaction to gameplay. Basically, it is the measure of how viable and interesting the "rogue" or "off-meta" tactics on your map will be. Maps that have a strictly dominate meta will be able to manage a strong gameplay score while having a terrible variance score (Theta for example). Likewise, some maps are able to have anything and everything as an option but all those options are terrible and not fun to play with because it results in a minute 2 rock, paper, scissors match (Voi Vittu for example).

The best maps are those that have a strong general meta that is consistent and able to lead to long games where a large variety of potential seams could be found. In return, those seams are often the off-meta tactics that allow you to throw your opponent off balance and reward innovation in player gameplay.

We currently give this category a weight of 30%. It used to be higher, but ultimately we weren't fans of the kinds of maps it was putting into pools and so we adjusted the ratio.

General Guidelines:
0 - One viable com path, one viable build, one viable win plan
1 - Very confined gameplay but maybe can squeeze a suprise build or cheese
2 - Some strategy is probably safer and easier than others but can make a different approach work, likely can send com in a few different directions
3 - Generic middle of the line. Has multiple builds, acu paths and strategies to pick from.
4 - A versatile map, various builds, strategies and ideas are all usable. Can win with something completely unexpected.
5 - A map where imagination is the limit. Land focus, air focus, naval focus, 1st land or 1st bomber. Acu or no acu, eco or no eco, anything is viable.

[MadMax]

thanks for sharing hopefully this will start a meaningful discussion about map design, and what mappers can do to improve the 'score' of their maps, which will in turn maybe break the astro cycle..... (doubtful). But anything that help mappers understand what makes for interesting and varied games on their maps, is good in my books and i think will be good for the community at large.