Skip to content
Snippets Groups Projects
Commit 03b8bb7e authored by Damien George's avatar Damien George
Browse files

py/formatfloat: Fix case of float format where leading digit was "10".

When taking the logarithm of the float to determine the exponent, there
are some edge cases that finish the log loop too large.  Eg for an
input value of 1e32-epsilon, this is actually less than 1e32 from the
log-loop table and finishes as 10.0e31 when it should be 1.0e32.  It
is thus rendered as :e32 (: comes after 9 in ascii).

There was the same problem with numbers less than 1.
parent d88250c0
No related branches found
No related tags found
No related merge requests found
......@@ -198,7 +198,7 @@ int mp_format_float(FPTYPE f, char *buf, size_t buf_size, char fmt, int prec, ch
if (e == 0) {
e_sign_char = '+';
}
} else {
} else if (fp_isless1(f)) {
e++;
f *= FPCONST(10.0);
}
......@@ -250,6 +250,12 @@ int mp_format_float(FPTYPE f, char *buf, size_t buf_size, char fmt, int prec, ch
}
}
// It can be that f was right on the edge of an entry in pos_pow needs to be reduced
if (f >= FPCONST(10.0)) {
e += 1;
f *= FPCONST(0.1);
}
// If the user specified fixed format (fmt == 'f') and e makes the
// number too big to fit into the available buffer, then we'll
// switch to the 'e' format.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment