Thursday, August 11, 2022

[FIXED] Why 0/0 is NaN but 0/0.00 isn't

Issue

Using DataTable.Compute, and have built some cases to test:

dt.Compute("0/0", null); //returns NaN when converted to double

dt.Compute("0/0.00", null); //results in DivideByZero exception

I have changed my code to handle both. But curious to know what's going on here?


Solution

I guess, that it happens because literals with decimal points are threated as System.Decimal and cause the DivideByZeroException

According to DataColumn.Expression

Integer literals [+-]?[0-9]+ are treated as System.Int32, System.Int64 or System.Double

Real literals without scientific notation, but with a decimal point, are treated as System.Decimal. If the number exceeds the maximum or minimum values supported by System.Decimal, then it is parsed as a System.Double.

Accroding to DivideByZeroException

The exception that is thrown when there is an attempt to divide an integral or Decimal value by zero.

For System.Double it returns Nan, because if the operation is a division and the constants are integers it changes to a double result type, according to reference source (Thanks @steve16351 for nice found)



Answered By - Pavel Anikhouski
Answer Checked By - Dawn Plyler (PHPFixing Volunteer)

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.