Here’s a pitfall in C# that keeps coming up. C# has a constant double.Epsilon that programmers coming from C naturally assume is the same as C’s DBL_EPSILON. It’s not. In fact, the former is hundreds of orders of magnitude smaller.
C#’s double.Epsilon is the closest floating point number to 0. C’s DBL_EPSILON is the distance between 1 and the closest floating point number greater than 1. Said another way, DBL_EPSILON is the smallest positive floating point number x such that 1 + x != 1, often called “machine epsilon.”
Typically double.Epsilon is on the order of 10^-324 and DBL_EPSILON is on the order of 10^-16. (These values could potentially change depending on the platform, but they hardly ever do.)
C# has no constant corresponding to DBL_EPSILON. This is unfortunate, since this constant appears frequently in numerical software. Why? Because it tells you, for example, when to stop adding series.
If DBL_EPSILON is on the order of 10^-16, that means that if you add two numbers that differ by more than 16 orders of magnitude, the sum doesn’t change. If you’re summing a decreasing series of numbers, say in order to evaluate a Taylor approximation, you might as well stop once the next term is 16 orders of magnitude smaller than the sum. If you keep going past that point, you’ll burn CPU cycles but you won’t change your answer.
DBL_EPSILON is almost always about 10^-16. But by giving it a name, you avoid having 10^-16 as a mysterious constant throughout code. And if your code should ever move to an environment with different floating point resolution, your code will correctly adjust to the new platform.
Do you need to have an explicit constant in your Taylor example? Why can’t you compare the old sum with the new sum and break from the loop when they’re equal?
Marius: In that example, yes, you could just add up terms until your additions make no difference. But in more complex situations, you need to know ahead of time what won’t make a difference. You might need to compute some parameter as a function of DBL_EPSILON.
Another issue with double.Epsilon/Double.Epsilon (and float.Epsilon/Single.Epsilon) is that it’s a denorm; usage of denormalized floating point numbers can impair performance quite dramatically.
This is also related:
http://www.johndcook.com/blog/2010/06/08/c-math-gotchas/
So what’s the best solution? Hardcode DBL_EPSILON as a constant in your code, or estimate the correct value at runtime?
DBL_EPSILON is the largest relative separation between two consecutive floating-point numbers. That’s why it’s so useful.
Janne: I don’t know C#, but the best solution is probably to note that:
DBL_EPSILON = DBL_RADIX ^ (1 – DBL_MANT_DIG)
(Of course, ^ is raising to the power of, not xor.)
Hi!
A more concrete example of the usefulness of DBL_EPSILON: you can calculate a numerical derivative with the formula
df(x) = (f(x+h) – f(x-h)) / (2*h)
but you have to choose h small enough to minimize truncation error, and yet big enough to minimize roundoff error. If f and its derivatives are of order one, it can be shown that the optimal value for h is something like
h = pow(DBL_EPSILON, 1/3.0)
Regards,
Edgar.