Minimizing Roundoff#
Consider subtracting the square of two numbers—taking the difference of two very close-in-value numbers is a prime place where roundoff can come into play.
Instead of doing:
we can instead do:
by factoring this, we are subtracting more reasonably sized numbers, reducing the roundoff.
We can see this directly by doing this with single precision (float) and comparing to an answer computed via double precious (double)
Here’s an example:
#include <iostream>
int main() {
float x{1.00001e15};
float y{1.0000e15};
std::cout << "x^2 - y^2 : " << x*x - y*y << std::endl;
std::cout << "(x - y)(x + y) : " << (x - y) * (x + y) << std::endl;
double m{1.00001e15};
double n{1.0000e15};
std::cout << "double precision value: " << (m - n) * (m + n) << std::endl;
}
As another example, consider computing [1]:
We can alternately rewrite this to avoid the subtraction of two close numbers:
Again we’ll compare a single-precision calculation using each of these methods
to a double precision “correct answer”. To ensure that we use the
single-precision version of the std::sqrt() function, we will use single
precision literal suffix, e.g., 1.0f tells the compiler that this is a
single-precision constant.
#include <iostream>
#include <iomanip>
#include <cmath>
int main() {
float x{201010.0f};
std::cout << std::setprecision(10);
float r1 = std::sqrt(x + 1.0f) - std::sqrt(x);
float r2 = 1.0f / (std::sqrt(x + 1.0f) + std::sqrt(x));
std::cout << "r1 = " << r1 << " r2 = " << r2 << std::endl;
double xd{201010.0};
double d = std::sqrt(xd + 1.0) - std::sqrt(xd);
std::cout << "d = " << d << std::endl;
}
Notice that we get several more significant digits correct when we compute it with the second formulation compared to the original form.
Summation algorithms#
Summing a sequence of numbers is a common place where roundoff error comes into play, especially if the numbers all vary in magnitude and you do not attempt to add them in a sorted fashion. There are a number of different summation algorithms that keep track of the loss due to roundoff and compensate the sum, for example the Kahan summation algorithm.