7 messages in ru.sysoev.nginxRe: DHT upstream module + nginx patches| From | Sent On | Attachments |
|---|---|---|
| Peter Schüller | Oct 9, 2009 1:46 am |  .gz, .diff |
| W-Mark Kubacki | Oct 9, 2009 2:28 am | |
| Peter Schüller | Oct 9, 2009 3:09 am | |
| bit bull | Oct 9, 2009 7:59 pm | |
| Peter Schüller | Oct 27, 2009 3:13 am | |
| Chieu | Nov 1, 2009 12:45 am | |
| Peter Schüller | Nov 3, 2009 9:45 am |
| Subject: | Re: DHT upstream module + nginx patches | |
|---|---|---|
| From: | W-Mark Kubacki (wmar...@hurrikane.de) | |
| Date: | Oct 9, 2009 2:28:37 am | |
| List: | ru.sysoev.nginx | |
Am 9. Oktober 2009 10:46 schrieb Peter Schüller <sco...@spotify.com>:
[...]
I am attaching two things; a module (spdht) which implements DHT based routing of requests to multiple upstream servers, and a patchset for nginx itself (against 0.7.61) that are needed, in part, in order to use the module.
Thank you for sharing! At least, I am interested.
[...] In addition the DHT ring is configured through DNS. [...]
In a project of mine in academia this turned out to be a very good approach - if you cached DNS responses in the configuration reader, though.
* Add support for SHA1 in the caching module. [...]
No need to use cryptographic hashes in such an application. They're slower than those I will mention below and you don't really need the "from a given output bit you cannot judge to an input bit". It is sufficient that every output bit will be toggled at a probability of ~50%, thus of a almost perfect dispersion.
For faster hashing try Murmurhash64, FNV1A and friends: [1] http://murmurhash.googlepages.com/ [2] http://isthe.com/chongo/tech/comp/fnv/ [3] http://burtleburtle.net/bob/hash/doobs.html (see the very bottom of that page)
BTW, replacing MD5, SHA1 etc. by one of those you can accelerate a lot of Key/Value storages out there.
-- W-Mark Kubacki http://mark.ossdl.de/