atom feed25 messages in ru.sysoev.nginxRe: mod_mogilefs for nginx was (A har...
FromSent OnAttachments
mikeApr 27, 2008 12:36 am 
Dave CheneyApr 27, 2008 1:21 am 
mikeApr 27, 2008 1:58 am 
Dave CheneyApr 27, 2008 2:11 am 
mikeApr 27, 2008 1:34 pm 
Janko HauserApr 27, 2008 1:44 pm 
Igor SysoevApr 27, 2008 9:34 pm 
mikeApr 27, 2008 10:52 pm 
Igor SysoevApr 27, 2008 11:05 pm 
Chavelle VincentApr 28, 2008 3:16 am 
Igor SysoevApr 28, 2008 7:10 am 
mikeApr 28, 2008 11:45 am 
kinglerApr 28, 2008 12:03 pm 
mikeApr 28, 2008 12:26 pm 
Ezra ZygmuntowiczApr 29, 2008 3:11 pm 
mikeApr 29, 2008 4:55 pm 
mikeApr 29, 2008 7:25 pm 
mikeApr 29, 2008 9:11 pm 
Manlio PerilloApr 30, 2008 1:11 am 
mikeApr 30, 2008 2:09 am 
Manlio PerilloMay 1, 2008 2:47 am 
mikeMay 1, 2008 8:17 am 
mikeMay 1, 2008 3:11 pm 
Eden LiMay 1, 2008 6:31 pm 
mikeMay 1, 2008 7:03 pm 
Subject:Re: mod_mogilefs for nginx was (A hardware question)
From:mike (mike@public.gmane.org)
Date:Apr 29, 2008 7:25:52 pm
List:ru.sysoev.nginx

I'm thinking it would be quite easy, once the key hashing style was determined.

i.e. how will it create the mogilefs key, and then how to emulate it in the application when storing the file.

getting the file would be like: get http://foo.com/somepath/foo.jpg

how to translate that to a: domain + key in mogilefs

and when the file is being stored in mogilefs, how to create that key so http://foo.com/somepath/foo.jpg matches up with it.

I'm going to see if I can crosspost to the mogilefs mailing list and see if anyone there is interested in this and has any ideas.

I don't think there is any reason this mod_mogilefs would need any write support. I'd see application level stuff needing to be the place where files are added to the mogilefs cluster. I see nginx interfacing just to get the files being the main purpose.

I believe when the request comes in, nginx will check to see if that location is mogilefs on; - if it is, then it does the key "hashing" mechanism (TBD) there. Then it connects to the mogilefs tracker using a simple socket connection and issues a "get_paths $key" which returns a list of storage node(s) with the file. Then nginx would basically mimic a standard GET through an HTTP upstream proxy to one of those servers (or perhaps add them all in a list and try them in order, just like it does today with proxy_next_upstream)

Since mogstoreds are just normal webservers with DAV support (can even use nginx for that) it's just a simple HTTP GET to get the file. No scripting level language needed...

I think this would be a -very- simple thing to do. If one knows C, that is. The biggest thing is how to hash the URL or URI in a consistent fashion that can be done in PHP/Perl/Python/whatever and later retrieved the same way through nginx.

On 4/29/08, Ezra Zygmuntowicz <ezmo@public.gmane.org>
wrote:

If some enterprising C hacker wanted to work on this Engine Yard would pay a bounty to make it happen as long as it is released open source. Contact me if you are interested in working on this project.

On Apr 28, 2008, at 12:27 PM, mike wrote:

I think it would be something worthwhile myself.

Just needs parameters - the tracker database info, and the domain right? Then you just use the domain + key to get the file. Maybe domain isn't even required.

However, wouldn't you need some sort of logic to determine how to create the key?

Some people use an md5 of the URI I guess, so it would be something like

foo.com/media/abc72849482bc8398af98effd9g843

/media/ would be mapped to something like

option #1 would be nginx do the mysql connection itself. option #2 would be nginx just connects to the tracker (this is probably best) since it's a simple socket connection

location /media/ { mogilefs on; mogilefs_domain images;

option #1 mogilefs_tracker_username mogilefs; mogilefs_tracker_password foo; mogilefs_tracker_host 192.168.1.203; mogilefs_tracker_database mogilefs;

option #2 mogilefs_tracker_host 192.168.1.202:7001;

and perhaps something like mogilefs_key_style hash; ? or something... (used to determine how the keys are actually setup) }

I think the biggest missing piece of the puzzle is how to determine the key structure. Otherwise I am sure it's quite simple, and it uses dav/HTTP to retrieve the files anyway, so the C code needed would be:

option #1: 1) mysql client 2) something which can do efficient HTTP GETs with offset support (probably already in nginx)

option #2: 1) something that can do a simple socket connect, ask the tracker for info, and then pass it on to step 2 (probably already in nginx) 2) something which can do efficient HTTP GETs with offset support (probably already in nginx)

If I knew anything worthwhile in C I would try it myself. I am sure someone even with basic C skills could hack it up really quick. It just needs one socket to talk to the tracker, then depending on the reply, open a second connection to grab the file...

On 4/28/08, kingler <king@public.gmane.org> wrote:

I know there are MogileFS client libraries for Perl/PHP/Ruby, but if someone can code a C library for MogileFS and it is then possible to make a Nginx module to support. I guess this will be more efficient than going down the Perl/PHP/Ruby route.

Does it make sense at all? Would the benefits outweigh the extra efforts to code the nginx module?

On Mon, Apr 28, 2008 at 11:46 AM, mike
<mike@public.gmane.org> wrote:

I am actually thinking about making a MogileFS passthrough

(I posted an email about it, looks like someone in Japan figured it out)

Would allow for minimal PHP/perl/python/whatever language to be involved, just enough to look up where a file is. Then pass that back to nginx with "file is on $server" and then use X-Accel-Redirect to bounce to that server...

Haven't done it yet, but I think it would work quite amazing!

On 4/28/08, Igor Sysoev <is-G@public.gmane.org> wrote:

On Mon, Apr 28, 2008 at 12:16:59PM +0200, Chavelle Vincent wrote:

Igor Sysoev wrote:

We use the proxying in this case instead of NFS:

client > nginx (1) > nginx

On nginx (1) it's better to set "proxy_max_temp_file_size 0" for the proxied location.

You recommend to use proxying for mass file serving. I am in this case, I use nginx and a specific module that I have coded. (to simplify things, it control access files and redirect client to ngx_http_internal_redirect)

I have think to use NFS for serve the files. I don't understand how to use proxying instead of that.

Set up nginx on NFS server and proxy to it instead of getting files via NFS:

location /files/ { proxy_pass http://nfs_server_nginx; proxy_max_temp_file_size 0; }