Can someone tell me what going wrong here?

@jip well the problem I'm seeing is the exponent, isn't being seen as such when a direct comparison is made. In this instance the script is comparing the units current location vs the surface height. Should the units elevation be lower than the surface the unit is warped to the surface.

If you look and those end values of the above WARN outputs, the exponents are pretty small values, but show up as whole numbers.

Rather than -6.103515625e-005, the script sees "-6". Causing the unit the warp over and over again.

What's needed here is a way to limit the results to just 2 or 3 decimal places.

You don't need to convert it to a string. If you provide a bit more context then I may be able to help you more

A work of art is never finished, merely abandoned

This post is deleted!

@jip

For a given number such as... 19.123456

I need to cut off the last digits specified. Hence with 3 digit precision, the output should be "19.123". Wanting to be able to feed in the decimal place and get only that, no zeros trailing afterwards.

Finally after much research and experimentation, this (sorta) works...

``````function GetTruncatedNumber(number, precision)
local fmtStr = string.format('%%0.%sf',precision)
number = string.format(fmtStr,number)
return number
end
``````

Output...

``````Input 18.4453125
Output one decimal:   18.4
Output two decimal:   18.45
Output three decimal:   18.445
``````

If you want it to not go past three decimal points then you can multiply it by a 100, floor it and then multiply it with 0.01.

I feel like I've already mentioned how to do that, have you tried this?

A work of art is never finished, merely abandoned

@jip I'm trying to create a function where all that i need to add is the number and the decimal places wanted. I'm close, yet the above function does do a small bit of rounding.

``````Input 18.3671875
one:	18.4
two: 	18.37 (Rounded!!!)
three: 	18.367
``````

The functions where math.floor and ceil are used can produce some rather wonky results. Which is why I'm experimenting.

@jip

``````function GetTruncatedNumber02(num, dp)
local pwr = math.pow(10, dp)
local inverse = math.pow(10, -dp)
WARN('	pwr: ', pwr,'	inverse: ',inverse)
return math.floor(num * pwr) * inverse
end
``````

Results

``````WARNING:        Input to  math Truncate:	18.404296875

WARNING:         pwr: 	10	 inverse: 	0.10000000149012
Output one:	18.39999961853

WARNING:         pwr: 	100	  inverse: 	0.0099999997764826
Output two: 	18.39999961853

WARNING:         pwr: 	1000	   inverse: 	0.0010000000474975
Output three: 	18.404001235962
``````

Still no dice.... Why is it so bloody hard to truncate a number in lua?

It is hard to truncate a number because all numbers in lua are represented as double floating pointvalues which will have the precision issues you are seeing. If you are just looking to see if two numbers are the same why not add in some tolerance? so just `a-b<epsilon` where epsilon is some value like `1e-4`

@sheikah

After hours of tinkering... Success!

``````function GetTruncatedNumber02(num, dp)
local pwr = math.pow(10, dp)
num = math.floor(num * pwr)
WARN('	pwr: ', pwr,'	num', num,'	num / pwr: ', num / pwr)
local result = num / pwr

local fmtStr = string.format('%%0.%sf',dp)
number = string.format(fmtStr, result)
return number
end
``````

Results

``````Math Truncate:	18.3671875    Decimals given, 1,  2,  3

Decimal: 1         pwr: 	10	 num	183	  num / pwr: 	18.299999237061
Output one after string filter:	18.3

Decimal: 2         pwr: 	100	  num	1836	     num / pwr: 	18.360000610352
Output two after string filter: 	18.36

Decimal: 3         pwr: 	1000	   num	18367	        num / pwr: 	18.367000579834
Output three after string filter: 	18.367
``````

Ended up combining a few methods to get this result. Not sure how it would behave with negative values but thats for tomorrow. (tired)

Resin