Debian Codegraph @

debian/patches/fix_float_precision

raw
1 From: Gustavo Romero <[email protected]>
2 Subject: ppc64el float handling fix
3
4 I dug a bit further and it looks like that sam_parse1() is actually
5 generating a different value for the floats in question, so when they are
6 loaded back for the comparison they are already screwed up
7
8 because strtod() is used in float_to_le() and so also in u32_to_le(), which are
9 inlined and float_to_le() takes a float and not a double as the first argument
10 the use strtof() instead of strtod() in there looks the best, avoiding a
11 truncation from double to float, which might cause precision issues, specially
12 between different archs (like casts) plus optimization. So the following
13 change fixed the issue for me.
14
15 Index: htslib/sam.c
16 ===================================================================
17 --- htslib.orig/sam.c
18 +++ htslib/sam.c
19 @@ -2570,7 +2570,7 @@ static char *sam_parse_Bf_vals(bam1_t *b
20 if (grow_B_array(b, nalloc, 4) < 0)
21 return NULL;
22 }
23 - float_to_le(strtod(q + 1, &q), b->data + b->l_data);
24 + float_to_le(strtof(q + 1, &q), b->data + b->l_data);
25 b->l_data += 4;
26 }
27 return q;
28 @@ -2835,7 +2835,7 @@ static inline int aux_parse(char *start,
29 }
30 } else if (type == 'f') {
31 b->data[b->l_data++] = 'f';
32 - float_to_le(strtod(q, &q), b->data + b->l_data);
33 + float_to_le(strtof(q, &q), b->data + b->l_data);
34 b->l_data += sizeof(float);
35 } else if (type == 'd') {
36 b->data[b->l_data++] = 'd';