# Template:Precision

Jump to navigation
Jump to search

4

This template determines the precision (as a count of decimal digits) for any amount, large or negative, using a fast algorithm. It can also handle a trailing decimal point (such as "15." or "-41.") or trailing zeroes (such as "15.34000" having precision as 5 decimal digits). For fractional input it returns the base ten logarithm of the numerator.

### Examples

`{{Precision|1111.123456789}}` |
9 |

`{{Precision|1111.12345678}}` |
8 |

`{{Precision|1111.1234567}}` |
7 |

`{{Precision|1111.123456}}` |
6 |

`{{Precision|1111.12345}}` |
5 |

`{{Precision|1111111111.12345678}}` |
8 |

`{{Precision|1111111111.1234567}}` |
7 |

`{{Precision|1111111111.123456}}` |
6 |

`{{Precision|1111111111.12345}}` |
5 |

`{{Precision|1111111111.1234}}` |
4 |

`{{Precision|1111111111.123}}` |
3 |

`{{Precision|1111111111.12}}` |
2 |

`{{Precision|1111111111.1}}` |
1 |

`{{Precision|1111111111.10}}` |
2 |

`{{Precision|1111111111.100}}` |
3 |

`{{Precision|1111111111.1000}}` |
4 |

`{{Precision|1111111111.10000}}` |
5 |

`{{Precision|1111111111}}` |
0 |

`{{Precision|1111111110}}` |
-1 |

`{{Precision|1111111100}}` |
-2 |

`{{Precision|1111111000}}` |
-3 |

`{{Precision|1111110000}}` |
-4 |

`{{Precision|1111100000}}` |
-5 |

`{{Precision|1111000000}}` |
-6 |

`{{Precision|1110000000}}` |
-7 |

`{{Precision|1100000000}}` |
-8 |

`{{Precision|0}}` |
1 |

`{{Precision|1}}` |
0 |

`{{Precision|22.45}}` |
3 |

`{{Precision|22.12345}}` |
6 |

`{{Precision|22}}` |
0 |

`{{Precision|22000}}` |
-3 |

`{{Precision|-15.275}}` |
4 |

`{{Precision|-15.2500}}` |
5 |

`{{Precision|23000222000111.432}}` |
3 |

`{{Precision|-15.123}}` |
4 |

`{{Precision|0.09}}` |
2 |

`{{Precision|0.88}}` |
2 |

`{{Precision|880000}}` |
-4 |

`{{Precision|90000000}}` |
-7 |

### Known bugs

- For numbers in scientific notation, the precision is typically returned as too low by 1 decimal place. Example: {{precision |7.1234E+06}} → -2 (should be precision as 4 decimal digits, not 3).
- Large numbers are limited to 11 trailing zeroes, so even larger numbers still report precision as being -11, such as 9 trillion: {{precision|9000000000000}} → -12 (should be: -12).

### Technical notes

- NOTE A1: This template determines the precision of decimals by counting the length of the numeric string (in a #switch comparing lengths of padded strings), then subtracting integer length, minus the decimal point, and minus 1 if negative. For integers, 1 place is subtracted for each trailing 0 on the integer. For fractions, any prior count is cleared x 0, then returns the base ten logarithm of denominator: (..prior...)*0 + (ln denom / ln 10).
- NOTE D2: The check, for whole integers, compares the amount versus appending "0" at the end: when the amount is a decimal, then the value is unchanged by appending 0 at the end: so 5.23 = 5.230 is true, whereas for whole integers, it would be: 5 = 50 as false, due to values becoming n*10 for integer n. So, for integer n, the check rejects: n = n0 as false; hence n is integer.
- NOTE M3: The magnitude of the integer portion is calculated by logarithm of the floor of absolute value (divided by natural logarithm of 10 to adjust for e=2.71828*), as: ln (floor( abs(-0.050067) )+0.99 )/ln10 Function floor(x) trims the decimal part, to leave the whole count: 0-9 yield 0, 10-19 as 1, 1000-1999 as 3. The abs(x) avoids floor of negatives, floor(-0.1)= -1, hence using abs(x) ensures -0.1 floors to 0 not -1. Near zero, the +0.99 avoids invalid log of 0, but does not round-up any decimals, already floored as nnn.00. Complexity is 6 operations: floor of abs( {1} ) +0.99 then log
_{10}*x*(ln*x*÷ ln10), then floor that logarithm ratio. Decimals -1 < x < 1 yield -1, avoiding log 0.001 = -3. - NOTE N4: Nesting of if-else and nested templates is kept to a minimum, due to the MediaWiki 1.6 limit of 40 levels of if-logic for all nested templates used together. Template {ordomag} was omitted to avoid 2 more levels of nested templates. Template {Precision} had 8 levels, and this template was trimmed to only 5 levels.
- NOTE S5: The #switch is run with "x" prepended in front of the amount, otherwise a #switch will compare as numeric where "2" would match "2.0" even though "2" is length 1 so "x2" no longer matches with "x2.0" as non-numeric. The #switch will exit on the first match, so smaller lengths are compared first, to avoid extra comparisons for more rare, longer numeric strings up to 41 long.
- NOTE W6: The check for integers with whole end-zeroes uses typical n=n/10*10, for each power of 10, where whole millions match: {{#ifexpr: {1}=floor( {1}/1E6 )*1E6| }} Previously, {Precision} had tried to use "round" to detect end-zeroes but "round" loses precision at -5, so, n00000 round -5 differs from n00000 slightly, and comparisons to exact rounded amounts failed to match some numbers when 6 or more zeroes "n000000".
- NOTE Z7: The check on zero for any .00000 compares adding 1 to the amount, versus appending "1" at the end: if the amount is a decimal, then adding 1 will be larger than appending 1 at the end: 0.00 + 1 > 0.001, whereas for whole zero, it would be: 0+1 > 01 as false, due to the value being the same. So, for integer 0, the check rejects: 0+1 > 01 as false; hence whole 0 is integer.