I may give this a try..https://stackoverflow.com/questions/57950030/wow-rounding-to-two-decimal-places-in-lua
This is also good: https://wiki.multitheftauto.com/wiki/Math.round
I may give this a try..https://stackoverflow.com/questions/57950030/wow-rounding-to-two-decimal-places-in-lua
This is also good: https://wiki.multitheftauto.com/wiki/Math.round
You don't need to convert it to a string. If you provide a bit more context then I may be able to help you more
A work of art is never finished, merely abandoned
For a given number such as... 19.123456
I need to cut off the last digits specified. Hence with 3 digit precision, the output should be "19.123". Wanting to be able to feed in the decimal place and get only that, no zeros trailing afterwards.
Finally after much research and experimentation, this (sorta) works...
function GetTruncatedNumber(number, precision)
local fmtStr = string.format('%%0.%sf',precision)
number = string.format(fmtStr,number)
return number
end
Output...
Input 18.4453125
Output one decimal: 18.4
Output two decimal: 18.45
Output three decimal: 18.445
@jip said in Can someone tell me what going wrong here?:
If you want it to not go past three decimal points then you can multiply it by a 100, floor it and then multiply it with 0.01.
I feel like I've already mentioned how to do that, have you tried this?
A work of art is never finished, merely abandoned
@jip I'm trying to create a function where all that i need to add is the number and the decimal places wanted. I'm close, yet the above function does do a small bit of rounding.
Input 18.3671875
one: 18.4
two: 18.37 (Rounded!!!)
three: 18.367
The functions where math.floor and ceil are used can produce some rather wonky results. Which is why I'm experimenting.
Tried your method...
function GetTruncatedNumber02(num, dp)
local pwr = math.pow(10, dp)
local inverse = math.pow(10, -dp)
WARN(' pwr: ', pwr,' inverse: ',inverse)
return math.floor(num * pwr) * inverse
end
Results
WARNING: Input to math Truncate: 18.404296875
WARNING: pwr: 10 inverse: 0.10000000149012
Output one: 18.39999961853
WARNING: pwr: 100 inverse: 0.0099999997764826
Output two: 18.39999961853
WARNING: pwr: 1000 inverse: 0.0010000000474975
Output three: 18.404001235962
Still no dice.... Why is it so bloody hard to truncate a number in lua?
It is hard to truncate a number because all numbers in lua are represented as double floating pointvalues which will have the precision issues you are seeing. If you are just looking to see if two numbers are the same why not add in some tolerance? so just a-b<epsilon
where epsilon is some value like 1e-4
After hours of tinkering... Success!
function GetTruncatedNumber02(num, dp)
local pwr = math.pow(10, dp)
num = math.floor(num * pwr)
WARN(' pwr: ', pwr,' num', num,' num / pwr: ', num / pwr)
local result = num / pwr
local fmtStr = string.format('%%0.%sf',dp)
number = string.format(fmtStr, result)
return number
end
Results
Math Truncate: 18.3671875 Decimals given, 1, 2, 3
Decimal: 1 pwr: 10 num 183 num / pwr: 18.299999237061
Output one after string filter: 18.3
Decimal: 2 pwr: 100 num 1836 num / pwr: 18.360000610352
Output two after string filter: 18.36
Decimal: 3 pwr: 1000 num 18367 num / pwr: 18.367000579834
Output three after string filter: 18.367
Ended up combining a few methods to get this result. Not sure how it would behave with negative values but thats for tomorrow. (tired)
Resin