PHPFixing
  • Privacy Policy
  • TOS
  • Ask Question
  • Contact Us
  • Home
  • PHP
  • Programming
  • SQL Injection
  • Web3.0

Thursday, August 11, 2022

[FIXED] why does sum of decimals still have floating point errors?

 August 11, 2022     decimal, floating-point, python-3.x     No comments   

Issue

Im using the decimals module to try to avoid floating point errors. From the decimal module's documentary it says:

Decimal numbers can be represented exactly. In contrast, numbers like 1.1 and 2.2 do not have exact representations in binary floating point. End users typically would not expect 1.1 + 2.2 to display as 3.3000000000000003 as it does with binary floating point.

But when I try sum with decimals, I still get those floating point errors.

decimal.Decimal(4.04)+decimal.Decimal(4.04)
>>Decimal('8.080000000000000071054273576')

Why is this?


Solution

Try putting strings around your float literals, like so:

decimal.Decimal('4.04')+decimal.Decimal('4.04')

In the code in your question, the raw binary (base 2) "float" type is passed to Decimal. When you use strings to represent the number 4.04 for example, Decimal represents '4.04' precisely in base 10.



Answered By - Jerfov2
Answer Checked By - Cary Denson (PHPFixing Admin)
  • Share This:  
  •  Facebook
  •  Twitter
  •  Stumble
  •  Digg
Newer Post Older Post Home

0 Comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Total Pageviews

Featured Post

Why Learn PHP Programming

Why Learn PHP Programming A widely-used open source scripting language PHP is one of the most popular programming languages in the world. It...

Subscribe To

Posts
Atom
Posts
Comments
Atom
Comments

Copyright © PHPFixing