Outsourcing new CPU scores for lobbies

TheVVheelboy

Well I just made a few runs with my laptop and surprisingly it's actually scoring better than previously.
Old score was jumping around 185-187, new one seems to be landing at average at 174 mark though depending on run it goes as low as 168 , with the worst run scoring 178.(*funnily enough it was the first run)
That was while having the laptop run on half empty battery in performance mode. There's also little bit more bloat compared to the runs done on the old benchmark but it seems like the new one doesn't care about it.

I also agree with Gieb that it's clearly not a good thing if the benchmark can't reliably tell you the expected performance from the players PC.

Jip

I wasn't referring to power saving mode - I was referring to not being in high performance mode. There is a middle ground, and that is exactly what we want to benchmark in my opinion. At least for my own laptop, it hits high performance mode for the first 15 minutes or so and then it throttles back into 'regular mode'. A typical laptop can not reliably perform at its peak performance - it isn't uncommon that people complain about laptop users.

Lenovo y-50
16GB ram
i7-9750H

before: 160 (regular mode, as supreme commander in the lobby doesn't trigger it)
after: 141 with high performance mode because of a single-core background task, 225 on 'regular mode'.

Lets gather more information at least - keep the scores coming.

Dragun101

I went from 181ish to 140 on my labtop. Will look for additional data in a moment on my labtop.

triple-x1

I've been playing this game for a few months now and never had any issues. However from patch 3723 my cpu score went from 180 to 359.
It is hard for me now to join any games, most of the times i get kicked because of this.
intel i7 5820 , 16 Gb DDr4 , nvidia GTX970
Please restore the old cpu score!
cpu-z 3.jpg cpu-z 2.jpg cpu-z 1.jpg

TheVVheelboy

Just to ask, did you run the laptop in performance/high power mode when running the benchmark?
@triple-x1

Giebmasse

That doesn't look like a laptop, full "K" CPU. Could still be some power saving thing going on?

TheVVheelboy

@giebmasse Oh yes, you are right. My sleepy mind was like, this numbers in cpu name are looking like the laptop SKU's so it must be a laptop.
It totally slipped my mind that the earlier I7 tended to have this kind on nomenclature compared to the current more streamlined line-up.

meatontable

Looks like new CPU score is overcorrelated to the cache size. I have similar to Jip's CPU by performance (https://cpu.userbenchmark.com/Compare/Intel-Core-i7-9750H-vs-Intel-Core-i7-7700/m766364vs3887 ) but CPU scoring utulity is thinking that my CPU slower two times. I have checked around 1-15 online replays today (Dual gaps usually ) and never replay slow down to 0, sometimes to +1 . But I am kicked from lobby regularly with verdict bad CPU.

Jip

I think we've got enough data to make to ratify some form of change. I'm not sure what yet, but please keep more data coming.

meatontable

Hello,
My test steps are :

Install lua from https://github.com/rjpcomputing/luaforwindows/releases
Use a short adoptation of the faf code from : https://pastebin.com/GUTBXF7a
I've add GetSystemTimeSeconds() + move out string with yield
and verify that code shows a strongly relation at my IMac 1492 and running time 1.5 sec (around)
CPU intel-core-i7-4980hq 16 DDR3 RAM
Windows machine has 1202 i7-7700

I have a question how is time is calculated in the lobby ?![My CPU]( image url)

meatontable

Another one result (Sorry i have tested it on Linux box):

model name : AMD FX(tm)-6350 Six-Core Processor
serg@debianserver:~# lua CPUscore.lua
test
1295
1295
serg@debianserver:~#

As I see 6 cores AMD CPU was produced in 2013 is a little slower that 4 cores Intel was produced 2017 ! Phenom has few running docker images and this activity is slowing it down !
Lets imagine if we can use only one core :
Intel i7-7770 time = 1202 * 4 = 4808
Intel i7-4980hq = 14924 = 5968
AMD Phenon 6350 time = 1295 * 6 = 7700
After this I've used pages from
https://cpu.userbenchmark.com/Compare/Intel-Core-i7-7700-vs-Intel-Core-i7-4960HQ/3887vsm9629
https://cpu.userbenchmark.com/Compare/Intel-Core-i7-7700-vs-AMD-FX-6350-Six-Core/3887vsm713 and compared all my CPUs
4960 is slower 20 % single core (no 4980 in list) = 48081.2= 5889.6
AMD Phenom is slower 65% single core = 4802*1.65 = 7933

Looks Like new lua can use more that one core and it is not good for single core (2 cores) game.
I'm proposing to multiply the current CPU score by CPU count and divide it by 2.

Uveso

@meatontable

Using a benchmark list was my first attempt to compare the speed results.
The same CPU in a different PC has different results.

Just as example, the Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz tested with Prime:
over 700 different results:
https://www.mersenne.org/report_benchmarks/?exp_date=2016-01-01&64bit=1&exover=1&exbad=1&exv25=1&exv26=1&specific_cpu=4379147

Giebmasse

@meatontable said in Outsourcing new CPU scores for lobbies:

I'm proposing to multiply the current CPU score by CPU count and divide it by 2.

Wouldn't this skew results of CPU's with many cores even though they are high performance? Like new Ryzen chips, even when they have many cores their single thread performance is also excellent for supcom.

meatontable

Ok . I would like to clarify. Supcom is really old game and it uses 2 cores only. In this case it is no sense to keep in mind how many cores are. Our application uses only 1-2 of them.
I definitely sure that I 'm saying nothing bad about new Ryzen chips.
In our CPUscore Ryzen 9 5900x (3.7HZ)12 cores bits Ryzen 7 5800x 8 cores(3.8GHZ) (I assume that Ryzen 9 has 1.5 twice better value ) but for SUPCom both CPU are near equal and Ryzen7 is better due to better Hz.
CPUscore should show the right information about the slow CPU for the game purpose.
I'm not happy that definitely slow CPU from 2013 year bits more better CPU from 2017 due to the count of cores....

Giebmasse

@meatontable said in Outsourcing new CPU scores for lobbies:

I'm not happy that definitely slow CPU from 2013 year bits more better CPU from 2017 due to the count of cores....

I don't think you have the right picture of how the CPU test in-game is working. The test that is ran in-game is single threaded, don't know why you are thinking about core counts in this way. A slow older gen 6-core AMD CPU will show up as a slow CPU with the test as it should, it does not gain anything from having a larger core count when compared to e.g. a faster 2-4 core Intel CPU from the same era. You can't compare the CPU score ran manually in a newer lua version to the one used in-game.

Also in case it wasn't clear, lower is better in the FAF CPU test.

meatontable

@giebmasse said in Outsourcing new CPU scores for lobbies:

@meatontable said in Outsourcing new CPU scores for lobbies:

I'm not happy that definitely slow CPU from 2013 year bits more better CPU from 2017 due to the count of cores....

I don't think you have the right picture of how the CPU test in-game is working. The test that is ran in-game is single threaded, don't know why you are thinking about core counts in this way. A slow older gen 6-core AMD CPU will show up as a slow CPU with the test as it should, it does not gain anything from having a larger core count when compared to e.g. a faster 2-4 core Intel CPU from the same era. You can't compare the CPU score ran manually in a newer lua version to the one used in-game.

Also in case it wasn't clear, lower is better in the FAF CPU test.

I have take a function from https://github.com/FAForever/fa/blob/741febf45a165e257db972fc2104484a51dd799d/lua/ui/lobby/lobby.lua#L5228
I do nothing with this function except the adding time check before and after checking CPU score. Original FAF score function is related to the CPU's count but game is a single core application.

meatontable

I'm proposing to do next ;

Inform about CPU and your current CPU score.
Run replay with max possible speed.
I hope you have 6 or 8 cores CPU.
I also replay the same game and we can compare the running time.
If CPU score is working correctly then CPUscore * running time in sec should be near to equal for everyone.

ZOB

except faf is not a single threaded software
pause a game and have a look in the task manager
you will see up to about 22 threads spread on up to 3 cores
the sim thread runs by itself on 1 core, usually maxed to 100%, this thread/core is the limiting factor
the other main threads such as rendering, networking and audio run on other cores

meatontable

agree. I can not see Single core CPU at the market. At least 2 CPU + hypertheding and we have no problem with other parts. Only one process is a bottleneck. Ive noticed few times that some guys have better CPU score but are aking -1 -2 when I can support +2

Giebmasse

Again, if I'd disable 4 of my cores I'd get the same CPU score result (4 vs 8 cores).
Yes, the game itself is not purely single threaded, but single thread performance is the most vital measurement we can do for CPU's as it is most often one thread that is the bottleneck. (If anyone is interested some info about supcom and threads can be found here: http://www.slideshare.net/guest40fc7cd/threading-successes-02-supreme-commander)

So I simply don't understand what you are after with the core count in this case. The score is not perfect and doesn't 100% compare with actual game performance, but that is a given as it is a very simple synthetic test.

About a performance test replay, I believe Marlo worked on a "standardized" replay that could be used for this, does anyone know if it got finished?