Le vendredi 02 mai 2008 à 10:52 +0200, Manlio Perillo a écrit :

I will not call a sem_wait() but a sem_trywait() of course! If Nginx cannot write because a script hold the semaphore then the script will read the old values, I don't see an issue.

Stub_status works but it's not the cleanest code in Nginx and there's no simple way to extend the variables watched since you need to modify specifically other modules using blocks of conditionnal compilation. If you modify stub_status you potentially break Collectd and Nagios plugins. A file system interface is universal and means it will be easy to use whatever tool you want. A monitoring agent written in C will be happy to read a file, a little bit less happy if a www library or executing wget is needed to fetch data.

That's why I propose two things:

1) A generic interface for monitoring agents

The easiest one: a file-like and a list of key:value, even if the monitoring agent doesn't know the semantic of the key it can report back the value and a graph can be made. Of course it's possible to modify stub_status (and to break compatibility) to do the same things but it will be of no help for point 2. Don't know today if it will be a shared memory or a regular file mmaped (file locking on unices is a complete mess :( ).

2) An API

An API for other modules to help providing variables for monitoring. At the cost of an indirection it may be possible at runtime to choose if this variable is monitored and then to do an atomic_t operation or not. If a module offer some variables for monitoring the user can choose or not to monitor. That's value for the software *and* for the user.

The API could be as simple as:

ngx_monitoring_value_t * ngx_register_monitoring_value (ngx_str_t * name, ngx_str_t * command_name, ngx_int_t option) ;

void ngx_monitoring_value_add (ngx_monitoring_value_t * value, ngx_int_t nbr) ;

void ngx_monitoring_value_set (ngx_monitoring_value_t * value, ngx_int_t new_value) ;

For example, in the case of the upstream server round robin module, code would be like this (pseudo code):

init: servers = array of ngx_monitoring_value_t * [nbr_servers] for each upstream server servers [i] = ngx_register_monitoring_value ("upstream-status-"+server_name [i], "upstream_server_status",0) ; ...

run: if (event == down) { ...  ngx_monitoring_value_set (server [i], 0) ; } else if (event == up) { ... ngx_monitoring_value_set (server [i], 1) ; }

Just put "monitor upstream_server_status ;" in nginx.conf and my module will do all the atomic_t stuff else it will use normal operations. Cost at runtime: one function call and a conditionnal per variable...

OK, gzip ratio was not the best real life example ;-) but imagine you have MRTG graphs and important values in the log, you ran stress tests for 24 h, 1.4 10^9 requests later, the error log is 100 MB long, looks like a nightmare to exploit the log to correlate with load for example, isnt it?

Just a different example where I want logging *and* monitoring. I've a special Nginx module with a circular buffer used to communicate with threads. If there's a buffer overflow, I log it, but it would be nice (for me) to have a circular buffer overflow error counter included in the monitoring watch set. Of course I can "hack" stub_status and collectd plugin, but it's better to propose a more general solution without breaking anything and with no significant performance hit.

Thank you very much for your time and your input Manlio, even if we don't agree on some points, it is very stimulating for me to have a contradictor such as you.

Best regards.