I am using Amazon EC2 with nginx primarily for load balancing and proxying
requests using proxy_pass. In this EC2 environment upstream servers can fail
without warning.

It would be a fantastic edition to nginx to have it run a backend shell script
when it detects that it needs to skip over an upstream server because it is not
responding in accordance with proxy_next_upstream.

For instance, if I've set this:

proxy_connect_timeout 3; proxy_send_timeout 3; proxy_read_timeout 3; proxy_next_upstream error timeout http_500 http_503;

Than at any point when nginx decides it must skip a particular upstream server
(based on the above config), then I'd want it to run my bash script
/etc/scripts/upstream_down. Likewise anytime it detects that an upstream server
previously being skipped is now back and available, I'd want it to call another
script /etc/scripts/upstream_isback. I would only want it to call my
upstream_down the first time it goes down and not each time it retries (unless
nginx detects it has come back up, and then later goes down again).

Or perhaps it can be simplified so that there is one command it will execute
when an upstream server changes from down to up or vice-versa, and nginix then
passes to it parameters to indicate whether the status has changed to down or
up.

In either case I'd like to get parameters that tell me what the server name or
IP is (as defined in the upstream config following the "server" statement) and
perhaps what the reason is (i.e. if the upstream server was down due to
"http_500" vs "timeout" etc).

With this feature added, I would set up my script so that it would:

a) send me an email notification warning me that upstream server xyz just
went down (or that xyz just came back up) b) and, in the case where a box went down, my script may then automatically
use the appropriate EC2 command to immediately launch a new ec2 replacement
instance!

I am interested to hear from others if they would find it useful to call a
script upon status change of upstream servers. Likewise I am curious whether
this would be a relatively easy feature to implement. At first I was going to
suggest having the email notification built into nginx as a feature, but then
thought it would be much more useful just to have it kick off a script where we
could send the email ourselves and do other actions as well (such as launch
replacement instances).

Thanks!!