Outsourcing new CPU scores for lobbies

0

I get that it is really to hard to make a proper benchmark. But this kind reads like you just created some code for the benchmark more or less by chance and the numbers lined up on 4 PCs and thats it. We have no idea what this benchmark is really measuring, and the spikes by people that seem to have normal RAM tell me that his benchmark is neither measuring RAM bandwidth nor latency.

Maybe Supcom is really dependent on caches and maybe the code is measuring L2 and MAYBE thats a good thing, but it seems to break down for a bunch of systems obviously. If those systems are not actually as slow as the benchmark indicates, we should tweak it, and making sure it actually is liekly to hit RAM seems like a good tweak to me.

Even if each table in the actual game fits in cache, the game will use all of them per tick (causing cache pressure), so we cannot use a single table that has typical game size if the game uses a whole bunch of them.

Maybe i can write up an idea later.

0

I have written something that we could maybe use as a base for a better memory benchmark:
https://gist.github.com/Katharsas/89b0e12b3cc751a51d0c278421d6599a

I have tried to not run into any of the problems described in my first post. I have taken care not to call the random function inside hot loops in case that this is a slow function.

It consists of 2 benchmark functions operating on a previously filled table. Based on the assumptions about Lua object sizes, it should have a minimum memory size of about 50MB. The Lua interpreter that i used in VSCode will take up 220MB after the table is filled with objects by the setup functions.

I have adjusted the iterations so that each benchmark runs in about 2 seconds in my machine. Now, i don't know how Lua works in the game, so im not sure if this could be ported to the FAF Lua benchmark. It would obviously require normalization / resscaling so that we get numbers that are in a usefull range. However i cannot really see in the FAF source code how the number that we see at the end is calculated and how the time is measured.

0

I've noticed many laptop CPU's stay in their power saving mode when running the newer test (more than before). Would explain some of the spikes.

0

I usually force high-performance mode before I start, tried to run many kinds of benchmarks before CPU test.
Now I randomly got a 260 rating and I'm not doing any new CPU tests 😄
Just to note: before that I had stable 250 rating even in power-saving and balanced modes, and gameplay was normal until late-game, if I accidentally put it in power-saving.

0

Old: 210-220
New: 300-310!
8GB of ram
I5-4288U
2.60GHz

0

I think we can conclude that one reason for this test to trigger higher scores is because it doesn't cause laptops to go into high performance mode. To me, that sounds like a good thing - as typically a laptop can't sustain that for too long when the GPU is involved.

A work of art is never finished, merely abandoned

2

It isn't really a good thing, in all cases I've seen the power saving mode the laptops are in do not reflect the true performance of the laptop in-game, as in the power saving is much lower clock speed even compared to a potentially throttling laptop. Scores between 350 and even 1750, when the laptop true performance would've been somewhere around 250ish or so.

0

Well I just made a few runs ​with my laptop and surprisingly it's actually scoring better than previously.
Old score was jumping around 185-187, new one seems to be landing at average at 174 mark though depending on run it goes as low as 168 , with the worst run scoring 178.(*funnily enough it was the first run)
That was while having the laptop run on half empty battery in performance mode. There's also little bit more bloat compared to the runs done on the old benchmark but it seems like the new one doesn't care about it.

I also agree with Gieb that it's clearly not a good thing if the benchmark can't reliably tell you the expected performance from the players PC.

0

I wasn't referring to power saving mode - I was referring to not being in high performance mode. There is a middle ground, and that is exactly what we want to benchmark in my opinion. At least for my own laptop, it hits high performance mode for the first 15 minutes or so and then it throttles back into 'regular mode'. A typical laptop can not reliably perform at its peak performance - it isn't uncommon that people complain about laptop users.

Lenovo y-50
16GB ram
i7-9750H

before: 160 (regular mode, as supreme commander in the lobby doesn't trigger it)
after: 141 with high performance mode because of a single-core background task, 225 on 'regular mode'.

Lets gather more information at least - keep the scores coming.

A work of art is never finished, merely abandoned

0

I went from 181ish to 140 on my labtop. Will look for additional data in a moment on my labtop.

I’m a shitty 1k Global. Any balance or gameplay suggestions should be understood or taken as such.

Project Head and current Owner/Manager of SCTA Project

0

I've been playing this game for a few months now and never had any issues. However from patch 3723 my cpu score went from 180 to 359.
It is hard for me now to join any games, most of the times i get kicked because of this.
intel i7 5820 , 16 Gb DDr4 , nvidia GTX970
Please restore the old cpu score!
cpu-z 3.jpg cpu-z 2.jpg cpu-z 1.jpg

0

Just to ask, did you run the laptop in performance/high power mode when running the benchmark?
@triple-x1

0

That doesn't look like a laptop, full "K" CPU. Could still be some power saving thing going on?

0

@giebmasse Oh yes, you are right. My sleepy mind was like, this numbers in cpu name are looking like the laptop SKU's so it must be a laptop.
It totally slipped my mind that the earlier I7 tended to have this kind on nomenclature compared to the current more streamlined line-up.

1

Looks like new CPU score is overcorrelated to the cache size. I have similar to Jip's CPU by performance (https://cpu.userbenchmark.com/Compare/Intel-Core-i7-9750H-vs-Intel-Core-i7-7700/m766364vs3887 ) but CPU scoring utulity is thinking that my CPU slower two times. I have checked around 1-15 online replays today (Dual gaps usually ) and never replay slow down to 0, sometimes to +1 . But I am kicked from lobby regularly with verdict bad CPU.

1

I think we've got enough data to make to ratify some form of change. I'm not sure what yet, but please keep more data coming.

A work of art is never finished, merely abandoned

0

Hello,
My test steps are :

  1. Install lua from https://github.com/rjpcomputing/luaforwindows/releases
  2. Use a short adoptation of the faf code from : https://pastebin.com/GUTBXF7a
    I've add GetSystemTimeSeconds() + move out string with yield
    and verify that code shows a strongly relation at my IMac 1492 and running time 1.5 sec (around)
    CPU intel-core-i7-4980hq 16 DDR3 RAM
    Windows machine has 1202 i7-7700

I have a question how is time is calculated in the lobby ?![My CPU](CPUZ.png image url)

0

Another one result (Sorry i have tested it on Linux box):

model name : AMD FX(tm)-6350 Six-Core Processor
serg@debianserver:~# lua CPUscore.lua
test
1295
1295
serg@debianserver:~#

As I see 6 cores AMD CPU was produced in 2013 is a little slower that 4 cores Intel was produced 2017 ! Phenom has few running docker images and this activity is slowing it down !
Lets imagine if we can use only one core :
Intel i7-7770 time = 1202 * 4 = 4808
Intel i7-4980hq = 14924 = 5968
AMD Phenon 6350 time = 1295 * 6 = 7700
After this I've used pages from
https://cpu.userbenchmark.com/Compare/Intel-Core-i7-7700-vs-Intel-Core-i7-4960HQ/3887vsm9629
https://cpu.userbenchmark.com/Compare/Intel-Core-i7-7700-vs-AMD-FX-6350-Six-Core/3887vsm713 and compared all my CPUs
4960 is slower 20 % single core (no 4980 in list) = 4808
1.2= 5889.6
AMD Phenom is slower 65% single core = 4802*1.65 = 7933

Looks Like new lua can use more that one core and it is not good for single core (2 cores) game.
I'm proposing to multiply the current CPU score by CPU count and divide it by 2.

0

@meatontable

Using a benchmark list was my first attempt to compare the speed results.
The same CPU in a different PC has different results.

Just as example, the Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz tested with Prime:
over 700 different results:
https://www.mersenne.org/report_benchmarks/?exp_date=2016-01-01&64bit=1&exover=1&exbad=1&exv25=1&exv26=1&specific_cpu=4379147

0

@meatontable said in Outsourcing new CPU scores for lobbies:

I'm proposing to multiply the current CPU score by CPU count and divide it by 2.

Wouldn't this skew results of CPU's with many cores even though they are high performance? Like new Ryzen chips, even when they have many cores their single thread performance is also excellent for supcom.