I have a table that should tell me if an IP is a unique one.
the fields are: IP, unixTime, flag (index on Ip,flag)
I need to clean this table often because I get more than 1,500,000 new inset a
day and with time I will fill up my HD.
the idea here is to allow counting unique-ip by setting the flag if the time
unixTime field is 'too old' (when 'too old' can change from 1 hour to 72 hours,
depend on external management control)
Is update on the flag every hour will be a good idea, or a delete every hour
where unixTime is 'too old' will be better.
If I will just to the delete I don't need the flag field at all.
but a delete every hours 1500000/24 = 62500 , in my system takes about
62500 x 0.027 = 1687.5sec (28.125 min.) and I can't insert new record, so it
just not working due to the delete blocking. Is update will be faster, or better
?
If using update, I will need to delete later many records so it will take real
long time (let's say a week file can take to delete 78.7 hours!)
How you/people are solving this log-table problems?
Thanks,
Shai
please CC to me at sh...@nbase.com