RE: select 9000000000000.10*19.80 wrong result 178200000000001.97 - George Law - com.mysql.lists.mysql

8 messages in com.mysql.lists.mysqlRE: select 9000000000000.10*19.80 wro...

From	Sent On	Attachments
wang shuming	29 May 2006 20:05
Jeremy Cole	30 May 2006 13:12
Paul DuBois	30 May 2006 13:21
Chris W	30 May 2006 13:38
George Law	30 May 2006 13:42
Jeremy Cole	30 May 2006 13:46
Chris W	30 May 2006 13:55
Paul DuBois	30 May 2006 14:14

Subject:	RE: select 9000000000000.10*19.80 wrong result 178200000000001.97
From:	George Law (gl...@IONOSPHERE.net)
Date:	05/30/2006 01:42:10 PM
List:	com.mysql.lists.mysql

You don't need to take it to 16 digits :

mysql 4 :

select 1/666; +-------+ | 1/666 | +-------+ | 0.00 | +-------+

mysql 5.0.18: select 1/666; +--------+ | 1/666 | +--------+ | 0.0015 | +--------+

-----Original Message----- From: Chris W [mailto:2wsx...@cox.net] Sent: Tuesday, May 30, 2006 4:39 PM To: wang...@xtech.com.cn Cc: mys...@lists.mysql.com Subject: Re: select 9000000000000.10*19.80 wrong result 178200000000001.97

wang shuming wrote:

Hi, select 9000000000000.10*19.80 wrong result 178200000000001.97 if <= 900000000000.10*19.80 right result 17820000000001.98

This is very typical floating point error. Computers don't have infinite precision or it would take infinite time to compute the answer. I have to ask why you need such precise number. In the real world if you can measure or control something to 4 significant digits you are doing really good. If you can do 6 you are doing great. If you can do 8 you are one step from a miracle worker. Measuring or controlling something to a level of precision of 16 significant digits is completely impossible.

Gift Giving Made Easy Get the gifts you want & give the gifts they want One stop wish list for any gift, from anywhere, for any occasion! http://thewishzone.com