atom feed8 messages in ru.sysoev.nginxRe: Setting cache parameters via if d...
FromSent OnAttachments
Jeff MitchellJan 27, 2011 9:07 pm 
Ryan MalayterJan 28, 2011 7:55 am 
Jeff MitchellJan 29, 2011 12:19 am 
Jeff MitchellJan 29, 2011 8:05 am 
Guzman BrasoJan 29, 2011 3:57 pm 
agentzhFeb 1, 2011 12:46 am 
agentzhFeb 1, 2011 12:53 am 
Ryan MalayterFeb 1, 2011 7:45 am 
Subject:Re: Setting cache parameters via if directives
From:Guzman Braso (guzm@gmail.com)
Date:Jan 29, 2011 3:57:23 pm
List:ru.sysoev.nginx

Hello Jeff...

On Fri, Jan 28, 2011 at 3:08 AM, Jeff Mitchell <je@jefferai.org> wrote:

Hello,

We have a particular application (gitweb) that performs a particular, extraordinarily slow function when the home page is loaded. As the number of repositories has increased, this has grown to take several *minutes* per page view (yes, ugh).

To combat this, we tried setting up cache so that this only occurs once an hour, but it's still causing too much of a problem...both because it doesn't always last an hour (sometimes we see the delay happen again sooner) and because when it does expire, it simply takes too long (and if multiple clients connect at that same time making that request, it gets even worse).

I think you look the wrong way at the problem you have. You should fix these issues you mention which are NOT the expected behavior before trying to fix your situation with your proposed patch of allowing a different time to a special IP and leave a bot requesting the page.

First problem: early renewal of page.

This is not a mistery why it happens: Either your server it's returning a header that tells nginx to refresh the cache, or you have a problem inside your configuration. If you tell nginx to update each 60 min, it will always respect that except backend server tells nginx to refresh the cache.

Second problem: multiple clientes renewing cache.

This can easily be fixed by using proxy_cache_stale, you can set it up to return users the last cached page while the backend ends updating the first request that expired. This means that after a requests pass to the backend, while that request does not complete, nginx will return the last cached (which already expired) version to any new request for that page.

If you fix those both problems then I think you won't need all that workaround you want to do.

I'm not nginx guru and maybe it's me looking the wrong way at your problem but in case it help this is how I dealt before with the same problem you described.

Good Luck!