1 hour ago · Tech · hide · 0 comments

So, speaking of Lua, I wanted to round some fractional coordinates to the nearest whole pixel before drawing, to reduce skrungliness. (That's a technical term.) Lua doesn't have a rounding function in its standard library, which surprised me, but it's a pretty petite stdlib, so ok. And then I was like wait... how DO you round a float? Like, legitimately? The standard thing you see goes something like math.floor(n + 0.5). For negative numbers, floor increases the absolute value instead of decreasing it, but adding a positive half decreases the absolute value, so it cancels out and works out to a well-behaved "ties round away from zero" rounding scheme. Or at least it would, if we were in the real numbers (or even a limited-but-consistent precision regime). But we are not. We're in motherfucking float country, where the space between adjacent numbers gets wider as their magnitude increases. Which means a "debatably correct, but good enough that you can't really catch it behaving badly"…

No comments yet. Log in to reply on the Fediverse. Comments will appear here.