Luke Kanies wrote: [snip]

Howdy.

The idea at this point is: * implement an abstract terminus for queuing, to be used as the caching terminus for the catalog. So, terminus_class => :queue * implement a simple queue client registry to back the abstract terminus, such that in an executable's configuration, we can specify what type of messaging queue client we want to use. * provide some message queue client wrappers for use; at present, we're looking at Stomp::Client for starters, and possibly a wrapper for the memcached client as well; each client wrapper gets registered with a simple symbolic name, much like the termini themselves (:stomp, for instance) * implement a simple new daemon (we're tentatively calling it puppetqd for working discussion) that: subscribes to the relevant message queue for the catalog class; receives catalog objects to store (as a message, which it deserializes) from the queue; stores the catalog objects to the database.

This workflow fits into the Puppet indirector idiom pretty well. We're trying to design the interaction between the queue terminus and the underlying message broker (the middleware that provides the queuing services) to maximize extensibility, so we can over time support any number of different messaging solutions.

The message queue topic is very well-covered in free and non-free software ecosystems. It was not my intent to implement a custom queuing solution for Puppet itself; there are too many simple, proven, effective options out there, and we're collectively better-served if sysadmins in charge of deployments can choose for themselves the messaging solution that best meets their needs.

Our initial work, using Stomp::Client as the messaging client (and therefore using the Stomp protocol), focuses on ActiveMQ for the messaging piece.

Stomp::Client: http://rubyforge.org/projects/stomp/ The Stomp protocol: http://stomp.codehaus.org/Protocol ActiveMQ: http://activemq.apache.org/

ActiveMQ is pretty popular and is quite easy to get running with a basic configuration. These were both factors in the ActiveMQ choice for our initial focus. Critically, you can configure it to be backed by any number of standard RDBMSes for message persistence, which was a concern for the client in question.

http://activemq.apache.org/jdbc-support.html

Stomp::Client and ActiveMQ is just a start. We can potentially use Ruby's StompServer, which is pretty lightweight and has some persistence options, but appears to be possibly stalled (not much activity since late 2007).

Furthermore, Stomp should be usable by other traditional JMS solutions.

On the lighter-weight side: we can set up a client for the memcachced protocol-based solutions like Starling and Sparrow. That should be relatively easy and give sysadmins more options in their deployments.

Ultimately, the resulting workflow should be (Luke may correct me here): * in the client, the catalog class uses :rest indirection, meaning it performs a find() RESTfully through puppetmasterd * puppetmasterd indirects the same class to the :compiler terminus, so the internal find() call in puppetmasterd goes to the parser to do the catalog compilation * when the catalog object is compiled and is ready for returning to the client, the caching step kicks in and will indirect to the :queue terminus. * the :queue terminus serializes the object to YAML, gets the right message queue client object, and sends the object YAML as a message to a queue named for the indirection itself; at this point, puppetmasterd's work is done and it returns to the client with no further delay * puppetqd runs as a separate process, and is listening to the relevant queue (though the degree to which it uses the indirection system natively for queue subscription needs to be worked out a bit more); as messages come in, it serializes them and stores them to the database, using a new database indirection in development for a different thread.

I hope this makes sense and seems reasonable to folks. Fire away!

Thanks. - Ethan