For the last few months I've been working on a new project: a web site for interacting with email archives. It makes heavy use of XML. Each email is stored internally as an XML document and accessed using XQuery. All searches, faceted navigation, analytic calculations, and HTML page renderings are performed on a single MarkLogic Server machine.

We launched the site about three weeks ago, hosting all the public emails from the Apache Software Foundation. That's about 500 lists and 4,000,000 messages. We've now started adding other lists of interest. Last week I loaded xml-dev:

http://xml-dev.markmail.org

As you'll see with the chart on the home page, one of our goals with the site has been to focus heavily on analytics. We have lots of graphs and counts. Every query you write gets its own histogram chart.

http://xml-dev.markmail.org/search/from:"michael+kay"+type:announcements

Another goal has been interactivity. Every search result screen gives you lots of ways to refine your search (by sender, list, attachment type, etc). Plus we did a lot with keyboard shortcuts. You can hit "n" and "p" to move to the next and previous result and "j" and "k" to move up and down the thread view. There's a lot of little things like this. Plus if your result message includes Office or PDF files they're in-line interactive too.

http://xml-dev.markmail.org/search/ext:ppt+xml

The subdomain you use implicitly limits the messages you search. Thus http://xml-dev.markmail.org searches only lists with "xml-dev" in their name (a single list).

You can search all Apache lists at http://apache.markmail.org, all Apache Axis lists at http://axis.markmail.org, or across all lists at http://markmail.org. You can always limit your search view using "list:axis" in your query, but using the domain handles that a bit more elegantly.

I hope you all find this useful!

Notes on using the site:

* Search using keywords as well as from:, subject:, extension:, and list: constraints

* The GUI doesn't yet expose it, but you can negate any search item, like -subject:soap

* You can sort results by date by adding order:date-forward or order:date-backward to your query

* Remember to use "n" and "p" keyboard shortcuts to navigate the search results

* You're going to want JavaScript enabled

-jh-

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS to support XML implementation and development. To minimize spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/ Or unsubscribe: xml-...@lists.xml.org subscribe: xml-...@lists.xml.org List archive: http://lists.xml.org/archives/xml-dev/ List Guidelines: http://www.oasis-open.org/maillists/guidelines.php