David_Wallis

Im going brain dead today..

im trying to read a byte from a file...

say the value I want to read out is 71 (hex)

this is 113 dec.

I then want to do the following conversions

(((113 * 8) - 750)/750.6) * 14.5

So I have tried the following...


Dim fstream As FileStream = New FileStream("c:\file.bin", FileMode.Open)

Dim breader As BinaryReader = New BinaryReader(fstream)

fstream.Seek("35136", SeekOrigin.Begin)
Dim u As Integer
u = System.Convert.ToSingle(breader.ReadByte)
u = u * 8
u = ((u - 750) / 750.6) * 14.5
MSFlexGrid2.set_TextMatrix(row, col, u)
' close files
breader.Close()
fstream.Close()

obviously Ive snipped a bit of code here and there.. so row and col arent important...

which works ok execpt it returns 3. not 2.97 as I expect..

Im sure its the convert to single.. as the help says

Converts the value of the specified 8-bit unsigned integer to the equivalent single-precision floating point number.

so I presume its the single precision bit...

what should I be doing???

Help.. otherwise im hitting the bottle

David

Fosters

I've not done much .Net yet, but am I correct in thinking that the convert to single actually converts 'u' to a Single. If it does I'm stuck, but if not that's why you're getting the 3.

Don't need to use a format statement on it do you?

(I'll get me coat )

nigelward

Dim u As Integer

Shouldn't that be:

Dim u As Single

Nigel

David_Wallis

tried that dim u as single.

still gives 3 and not 2.7 or whatever...

David

David_Wallis

ok tried that again and it worked.