Skip to content
Snippets Groups Projects
Commit 7a72c0db authored by Damien George's avatar Damien George
Browse files

py: Reduce str/repr precision of float numbers when floats are 30-bit.

With 30-bit floats there aren't enough bits to faithfully print 7 decimal
digits, so reduce the precision to 6 digits.
parent 58467709
No related branches found
No related tags found
No related merge requests found
......@@ -50,7 +50,11 @@ STATIC void complex_print(const mp_print_t *print, mp_obj_t o_in, mp_print_kind_
mp_obj_complex_t *o = MP_OBJ_TO_PTR(o_in);
#if MICROPY_FLOAT_IMPL == MICROPY_FLOAT_IMPL_FLOAT
char buf[16];
#if MICROPY_OBJ_REPR == MICROPY_OBJ_REPR_C
const int precision = 6;
#else
const int precision = 7;
#endif
#else
char buf[32];
const int precision = 16;
......
......@@ -113,7 +113,11 @@ STATIC void float_print(const mp_print_t *print, mp_obj_t o_in, mp_print_kind_t
mp_float_t o_val = mp_obj_float_get(o_in);
#if MICROPY_FLOAT_IMPL == MICROPY_FLOAT_IMPL_FLOAT
char buf[16];
#if MICROPY_OBJ_REPR == MICROPY_OBJ_REPR_C
const int precision = 6;
#else
const int precision = 7;
#endif
#else
char buf[32];
const int precision = 16;
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment