atom feed1 message in org.openstack.lists.openstack-dev[openstack-dev] a little tracing hack
FromSent OnAttachments
Timothy DalyApr 24, 2013 8:30 pm 
Subject:[openstack-dev] a little tracing hack
From:Timothy Daly (tim@yahoo-inc.com)
Date:Apr 24, 2013 8:30:21 pm
List:org.openstack.lists.openstack-dev

I've been working on adding metrics and tracing to OpenStack for use here at
Yahoo!. I got clearance to open source the work and share it with you guys.
For those of you that were at the summit, you may have seen Josh Harlow's
demonstration of it [1].

The most interesting piece is zipkin integration. Zipkin is a nifty distributed
tracing tool that Twitter open sourced.

Here is an example trace from a nova boot:

http://imgur.com/U2uxZJB,aLVEgIg#0

and a detail of one of the mysql queries in that trace:

http://imgur.com/U2uxZJB,aLVEgIg#1

It can also derive statsd metrics from the tracing information, or just log it.

The code is here:

https://github.com/timjr/tomograph

There is a README with instructions for getting started with OpenStack here:

https://github.com/timjr/tomograph/tree/master/doc/openstack-patches

I have a lot of improvements planned, but I guess it's good enough to be worth
trying out as it is. For me, seeing the traces so clearly laid out has been eye
opening. It's so much easier to see what OpenStack is _doing_.

I would love to hear any feedback, positive, negative, or whatever. I'd like to
eventually get this into shape to submit the patches to the various OpenStack
projects, if there's sufficient interest.

Cheers, Tim