Quantcast
Channel: All LabVIEW posts
Viewing all articles
Browse latest Browse all 202933

Re: Expecting a result of zero when subtracting, but getting 138.778E-18

$
0
0

Hi altenback,

 

Thank you for your suggestions in how to clean up my code.  There may be a more elegant way, but I am satisfied.  In my attachment, I provide a VI that compares code that gives me the floating point rounding errors that I was experiencing and code that works as I expected.  I provide some comments in the code to explain the problem I was having.

 

The comparison VI includes the two SampleCode VIs.

 

 

 


Viewing all articles
Browse latest Browse all 202933

Trending Articles