Single or Double

[mdmackillop]

Just for info, different results in a simple code.


Sub Test1()
Dim rs As Single
Dim i As Long
Do While rs <= 0.5
i = i + 1
Cells(i, 1) = rs
rs = rs + 0.05
Loop
Test2
End Sub

Private Sub Test2()
Dim rs As Double
Dim i As Long
Do While rs <= 0.5
i = i + 1
Cells(i, 2) = rs
rs = rs + 0.05
Loop
End Sub

[GTO]

Hi Malcom,

I did not realize that a Single would get inaccurate so quickly. I noticed that the first cell with a value has 0.0500000007450581 in the formula bar; by my count, 16 digits (I thought the limit was 15?).

Anyways, I tried this in Test1, just to see what gets read back from the cell.

Sub Test1()
Dim rs As Single
Dim dblResult As Double

Dim i As Long
Do While rs <= 0.5
i = i + 1
Cells(i, 1) = rs
Debug.Print rs
rs = rs + 0.05
Loop

rs = Cells(2, 1).Value
Debug.Print rs '<--- results in: 0.05
dblResult = Cells(2, 1).Value
Debug.Print dblResult '<---Results in: 5.00000007450581E-02

Test2
End Sub

[Paul_Hossler]

Chip has a nice write up on Double / Single, although Single is pretty much just the last sentence

http://www.cpearson.com/excel/rounding.htm

Paul