Printable View
Just for info, different results in a simple code.
[VBA]
Sub Test1()
Dim rs As Single
Dim i As Long
Do While rs <= 0.5
i = i + 1
Cells(i, 1) = rs
rs = rs + 0.05
Loop
Test2
End Sub
Private Sub Test2()
Dim rs As Double
Dim i As Long
Do While rs <= 0.5
i = i + 1
Cells(i, 2) = rs
rs = rs + 0.05
Loop
End Sub
[/VBA]
Hi Malcom,
I did not realize that a Single would get inaccurate so quickly. I noticed that the first cell with a value has 0.0500000007450581 in the formula bar; by my count, 16 digits (I thought the limit was 15?).
Anyways, I tried this in Test1, just to see what gets read back from the cell.
[vba]
Sub Test1()
Dim rs As Single
Dim dblResult As Double
Dim i As Long
Do While rs <= 0.5
i = i + 1
Cells(i, 1) = rs
Debug.Print rs
rs = rs + 0.05
Loop
rs = Cells(2, 1).Value
Debug.Print rs '<--- results in: 0.05
dblResult = Cells(2, 1).Value
Debug.Print dblResult '<---Results in: 5.00000007450581E-02
Test2
End Sub
[/vba]
Chip has a nice write up on Double / Single, although Single is pretty much just the last sentence
http://www.cpearson.com/excel/rounding.htm
Paul