atom feed10 messages in com.googlegroups.memcachedRe: HELP ! How Memcached objects are ...
FromSent OnAttachments
Nicolas MartinezNov 14, 2015 5:49 am 
Nicolas MartinezNov 14, 2015 6:14 am 
dormandoNov 14, 2015 2:54 pm 
Nicolas MartinezNov 15, 2015 3:59 am 
dormandoNov 15, 2015 3:30 pm 
Nicolas MartinezNov 16, 2015 6:37 am 
Nicolas MartinezNov 16, 2015 10:01 am 
Nicolas MartinezNov 16, 2015 11:08 am 
dormandoNov 16, 2015 2:54 pm 
Nicolas MartinezNov 17, 2015 1:23 am 
Subject:Re: HELP ! How Memcached objects are stored in the good SLAB
From:dormando (dorm@rydia.net)
Date:Nov 14, 2015 2:54:53 pm
List:com.googlegroups.memcached

The mysql docs don't speak for the main tree... that's their own thing.

the "sizes" binary that comes with the source tree tells you how many bytes an item will use (though I intend to add this output to the 'stats' output somewhere). With CAS this is 56 bytes.

56 + 2 + 30 == 88. Class 1 by default (in 1.4.24) is 96 bytes, but the item still ends up in class 2.

Why is this? (unfortunately?) because memcached pre-renders part of the text protocol into the item header:

*nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", flags, nbytes - 2); return sizeof(item) + nkey + *nsuffix + nbytes;

so the flags + length are getting flattened + \r\n added to the end. Together that's just enough to push it over the edge. It'd also be nice to add a highly optimized numerics printf so I could twiddle options to save a few bytes of memory in objects, but don't get your hopes up for that happening soon :)

On Sat, 14 Nov 2015, Nicolas Martinez wrote:

Add: Memcached version : 1.4.4 (RedHat)

Le samedi 14 novembre 2015 14:49:37 UTC+1, Nicolas Martinez a écrit : Hi, few days i'm reading Memcached documentation and blogs... and i don't
understand how objects are stored.

My test

3 slabs : 

* 96.0 Bytes * 120.0 Bytes * 240.0 Bytes Everywhere, it's told : * if data is < 96 Bytes, it will be stored in Slabs1 (96B) * if data > 96B and < 120B, it will be stored in Slabs2 (120B) * if data > 120B, it will be stored in Slabs3 (240B) * etc. BUT, for example, when i'm creating an 30B object, it's stored in Slab2 (120B),
and NOT in Slab1 (96B).

External sources: For example, the default size for the smallest block is 88 bytes (40 bytes
of value, and the default 48 bytes for the key and flag data). If the size of
the first item you store into the cache is less than 40 bytes, then a slab with
a block size of 88 bytes is created and the value stored. =>
https://dev.mysql.com/doc/mysql-ha-scalability/en/ha-memcached-using-memory.html

WRONG

A slab class is a collection of pages divided into same sized chunks. Each
slab class is referenced to by its chunk size. So we’ll have Slab class 80kb,
Slab class 100kb and so on. When an object needs to be stored, its size
determines where it gets stored. So if the object is larger than 80kb but less
than 100kb, it gets stored into Slab class 100kb.  => http://returnfoo.com/2012/02/memcached-memory-allocation-and-optimization-2/

WRONG

How i create an object:

data=$(pwgen 30 -c 1) echo -e 'set 30 0 3600 30\r\n'$data'\r'| nc ${memcached_server} 11211

So, when 30B object is creating :  * key name : 30 = 2 bytes * value: 30 characters = 30 bytes * tags : 0 = 1 bytes => All = 33 bytes If i add the default 48b as explained on Mysql website : 33 + 48 = 81B ... so <
Slab1 (91B)... but always stored in Slab2 (120B)

So, the size used to store object in the good Slab is not: * object value size * sum of KEY, VALUE and TAGS in bytes KEY size : 1 character = 1 B VALUE size : 1 character = 1 B TAGS size : 1 character = 1 B

... as read everywhere

So, It seems that: (SUM of KEY+VALUE+TAGS ) * For slab1 96.0 Bytes, data stored if <= 31 B (SUM of 2+28+1 ) * For slab2 120.0 Bytes, data stored if <= 55 B (SUM of 2+52+1 ) * For slab3 152.0 Bytes, data stored if <= 87 B (SUM of 2+84+1 ) * For slab4 192.0 Bytes, data stored if <= 126 B (SUM of 3+122+1 ) * For slab5 240.0 Bytes, data stored if <= 174 B (SUM of 3+170+1 ) * etc.

My configuration : * Chunk Size : 48 * Chunk Growth Factore: 1,25 * Max Bytes: 64.0 MBytes 

So, someone could explain me how the data is stored in the right Slabs??? How calculate it??? 

Thank you