I have a program that gives mysterious results when the sizes of the integers get to be above a certain threshold.
Finding where the overflow occurs is proving to be very elusive, since no trap is generated.
Is this something that's a integral part of the INTEL processor?
For example, could that be checked at the machine code level?
Example: check if A*B is correct - - (Fortran source)
c=a*b
if(a>0 .and. b<0 .and. c <0) .or. &
(a<0 .and. b<0 .and. c <0) .or. &
(a>0 .and. b<0 .and. c >0) .or. &
(a<0 .and. b>0 .and. c >0) then
etc. etc.
But this is extremely awkward putting that everywhere in the code.
And it does not always catch a bad result. You can have a wrong answer with the correct sign.
another approach is do write a special routine to do basic add, multiply, sub, etc.
Notice that using higher precision integers does NOT guarantee you will get accurate results -
If A and B are both int8, then most of the time the result will be wrong.
If A and B are both int4, then you have to FORCE it to use INT8 arithmetic, or
the result will almost always be wrong.
Example: C=int8(a)*b
The problem is far worse with the ** operator, for example if A and B are just one byte integers,
and you say c=int8(a)**b then the result will almost always be wrong.
So, other than going to machine code, is there a way around this?
What about the C++ approach? Well, it doesn;t have as many choices for integer size -