PHPFixing
  • Privacy Policy
  • TOS
  • Ask Question
  • Contact Us
  • Home
  • PHP
  • Programming
  • SQL Injection
  • Web3.0

Sunday, August 7, 2022

[FIXED] How to set the precision of a decimal context for a Class (Python using the decimal module)

 August 07, 2022     decimal, floating-point, precision, python-3.x     No comments   

Issue

I am using the decimal module and want to set the context of a Class to a specific precision.

import decimal
from decimal import Decimal

ctx = decimal.getcontext()
ctx.prec = 6

class MyClass:
    def __init__(self, id: str, int_val: int, float_val: float):
        self.id = id
        self.int_val = int_val
        self.float_val = Decimal(float_val)

    def __str__(self) -> str:
           return f"ID: {self.id}, Integer value: {self.int_val}, Float value: {self.float_val}\n"


if __name__ == "__main__":
    obj = MyClass('001', 42, 304.00006)
    print(obj)

The above gives:

ID: 001, Integer value: 42, Float value: 304.00006000000001904481905512511730194091796875

Whereas, I would expect to get

ID: 001, Integer value: 42, Float value: 304.000 # i.e. precision of 6


Solution

After reading the decimal module documentation more carefully where it states that:

prec is an integer in the range [1, MAX_PREC] that sets the precision for arithmetic operations in the context.

In other words,the prec attribute of the context controls the precision maintained for new values created as a result of arithmetic. Literal values are maintained as described.

Hence, changing the constructor to initialise the float_val as a result of a simple arithmetic did the trick:

self.float_val = Decimal(float_val) * 1

which enforces the precision and returns the desired result:

ID: 001, Integer value: 42, Float value: 304.000



Answered By - Yannis
Answer Checked By - Mary Flores (PHPFixing Volunteer)
  • Share This:  
  •  Facebook
  •  Twitter
  •  Stumble
  •  Digg
Newer Post Older Post Home

0 Comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Total Pageviews

Featured Post

Why Learn PHP Programming

Why Learn PHP Programming A widely-used open source scripting language PHP is one of the most popular programming languages in the world. It...

Subscribe To

Posts
Atom
Posts
Comments
Atom
Comments

Copyright © PHPFixing