PHPFixing
  • Privacy Policy
  • TOS
  • Ask Question
  • Contact Us
  • Home
  • PHP
  • Programming
  • SQL Injection
  • Web3.0

Tuesday, August 9, 2022

[FIXED] How can exact same value differ in math result only by changing type from int to decimal

 August 09, 2022     c#, decimal, integer     No comments   

Issue

I was working on some project and need to do some math :

decimal X = (Value / 881) * (item.Type ? 130: 130 * 2);

the param "Value" equals 3000 for example.

if "Value" is of type int the result is 390.. if "Value" is of type decimal the result is 442.67

how is this possible ??

.NET Fiddle


Solution

Because of Decimal value. If you calculate your formula step by step you will understand this difference is due to values which are coming after decimal point when you are using decimal as a type

When you divide 3000(integer) by 881:

int Value = 3000
//Output is 3. Output is in integer
decimal X = (Value / 881);  //When int is divided by int then result is in int

When you divide 3000(decimal) by 881:

decimal Value = 3000
//Output is 3.4052213393870601589103291714. Output is in decimal.
decimal X = (Value / 881);  //When decimal is divided by int then result is in decimal

.Net fiddle

I hope .net fiddle will give you a better idea of my answer



Answered By - Prasad Telkikar
Answer Checked By - Clifford M. (PHPFixing Volunteer)
  • Share This:  
  •  Facebook
  •  Twitter
  •  Stumble
  •  Digg
Newer Post Older Post Home

0 Comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Total Pageviews

Featured Post

Why Learn PHP Programming

Why Learn PHP Programming A widely-used open source scripting language PHP is one of the most popular programming languages in the world. It...

Subscribe To

Posts
Atom
Posts
Comments
Atom
Comments

Copyright © PHPFixing