The Vault sort by rating system is bad and needs to be changed
-
Why is it bad?
The system is heavily biased towards content having a perfect score of all ratings being 5/5 stars, a mod with a single 1 star review and a single 5 star review will be very very far back in the sorting system while a mod with 2 5 star reviews gets placed near the top.
In the above example you can see how a map with 6; 5/5 ratings is sorted above a map with 10; 5/5 ratings and a single 4/5 review. Not only is this very far from being proper sorting the map sorted above is, objectively by reviews and from a personal review of the content, just terrible.Now you might say the first 15 maps seem to be properly sorted since they are quite good and popular -> just check the second page, theres an astro clone on there which was even rejected by the astro community since it barely has 220 plays.
It gets worse
Switching to the mod vault
There are some great mods that show up only on the 4th page of highest rated, it is sad to see great mods that need to fight for popularity in a niche market get sorted in such a terrible fashion, mods that deserve more attention get sorted far back while mods like "T1 UNIT HEALTH X100" get sorted higher. See for yourself:
All of these mods are sorted below the following mods:
goes on and on:
How do they sort it?
They use a very complex mathematical equasion called: Binomial proportion confidence interval. This isnt a bad way to sort but the devs did a terrible implementation. A simpler system like (stars * amount of reviews) would work way better.
Have a great day!
-
For those who care here are explanations of the rating algorithm. Any rating system would have its biases since it is an estimation problem. And it is just a trade off of what is considered a 'good' bias. This system is still entirely based on reviews that people leave.
Wikipedia: https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval
More simplified: https://www.evanmiller.org/how-not-to-sort-by-average-rating.html
-
Always great to hear we do something terrible and then come up with a proposal that makes the situation even worse.
Your proposal would make a 6x 1 star rated mod appear above a 1x 5 star rated mod. -
@Brutus5000 Which is still good, it means that that mod has some form of popularity and allow for changes to be made.
And besides its way better than a 2x5 star rated mod being sorted above a 35xMixed star.
Point still stands, thx for your input
Edit: Total amount of stars - (amount of 1 star reviews + amount of 2 star reviews * 2)
-
If there is no good algorithm, the solution is to have multiple bad algorithms. On Monday, we do the bad algorithm that over-emphasizes having a perfect rating. On Tuesday we do the bad algorithm that over-emphasizes the number of 5-star reviews. Etc.
-
This post is deleted! -
This post is deleted! -
@Degulum said in The Vault sort by rating system is bad and needs to be changed:
@Brutus5000 Which is still good, it means that that mod has some form of popularity and allow for changes to be made.
so the people that rate a mod 1/5 stars in an attempt to warn people that its a bad mod are actually unintentionally boosting its popularity. A great system indeed
-
@TheWheelie Almost all 1 star reviews on my mod are bug reports, im certain that this is also the case with most 1 star reviews on other mods, I have provided an alternative where 1 star and 2 star reviews can yield negative sorting score. I still believe that 1 star reviews can represent general interest in a mod and since theyre mostly "not working" or "game freezes" etc. I think it could either allow the original author to fix it or have another author provide a better comparable mod.
My biggest issue with the current system is that it doesnt make popularity valueable it sorts mods with 2 5 star ratings way infront of big mods people put a lot of work into that have a higher chance of having issues or bugs which may yield bad reviews throughout its development.
-
Reviews should just be removed, they are pretty pointless as they are now.
-
There is a separate category for most popular mods by the way
-
@BlackYps No there are 3 categories for mods, Highest rated, Recommended Ui mods and recent uploads/updates
-
Isn't recommended based on download number? It could definietly have a better name, though.
-
@BlackYps I Dont know what its based on, but whatever it is is much better than what is used for highest rated.
The problem is that its recommended UI mods and not mods
-
The recommended UI mods uses the exact same sorting criteria as highest rated just restricted to just the UI mods.
-
@Sheikah Oh yeah, I see its just because only the first 3 pages actually have mods with any ratings at all and the second page is already at single ratings
-
Project is open source, API includes all required information for you to do the realization of your own algorithm i dont see any problem in having 3-4 type of sort of different approaches.
It is very easy to push devs forward for making new changes, you may notice how productive @MarcSpector is on repo
As long as you come with good ideas, it will be implemented, if the dev is interested in it.For example, i would like to see all dependencies and conflicts of any mod, so i can see which is not compatible with
