oav163 Posted October 27, 2008 Report Posted October 27, 2008 Dear all, I know this is very easy for many of you but I'm trying to get the bank angle from fsuipc (057c) but when I follow the instrunctions * 360 / (65536 * 65536) I get an overflow (programming in VB6). Any idea? Even when I go to intermidiate window and I paste the calculation I got again overflow so.... Thank you very much Spyros Demas
Pete Dowson Posted October 27, 2008 Report Posted October 27, 2008 I know this is very easy for many of you but I'm trying to get the bank angle from fsuipc (057c) but when I follow the instrunctions * 360 / (65536 * 65536) I get an overflow (programming in VB6). It sounds like you are using an integer variable. Obviously the end value is going to be small, with many fractional parts, which your integer cannot hold. It would be okay (without the overflow) if you divided by the 65536*65536 before multiplying by 360, but then you'd always get zero (of course)! For anything like this ALWAYS copy the value you are reading into a floating point variable first. Only with floating point will you retain all the accuracy of the original in a useful form. Do the conversion after you have it safe in floating point. Pete
BAW019 Posted October 27, 2008 Report Posted October 27, 2008 A useful trick is to tell VB to treat the constants as double precision variables. It gets around the Overflow problem and you can do this by adding # symbols after their values eg X = Y * 360# / (65536# * 65536#) If your receving variable is an Integer or a Long Integer you'll loose the precision and since VB doesn't 'round', it truncates, you may find odd results. If possible, receive the data into a single or double precision variable if the decimal bit are important! Ian
oav163 Posted October 27, 2008 Author Report Posted October 27, 2008 Pete I could not understand how to handle the 65535*65535 but after lan's message now it's clear for me. The # worked perfect for me. REALY thank you guys Spyros Demas
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now