8 messages in org.apache.activemq.usersRe: SMTP Server (Apache James) spooli...
FromSent OnAttachments
Stefano BagnaraMay 8, 2008 1:55 am 
James StrachanMay 8, 2008 11:55 pm 
Stefano BagnaraMay 9, 2008 1:20 am 
James StrachanMay 9, 2008 2:27 am 
Stefano BagnaraMay 9, 2008 4:25 am 
James StrachanMay 9, 2008 5:55 am 
James StrachanMay 9, 2008 6:08 am 
Stefano BagnaraMay 9, 2008 9:17 am 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:Re: SMTP Server (Apache James) spooling hintsActions...
From:Stefano Bagnara (apa@bago.org)
Date:May 9, 2008 1:20:34 am
List:org.apache.activemq.users

James Strachan ha scritto:

2008/5/8 Stefano Bagnara <apa@bago.org>:

I'm starting analysis on how to replace our default spool with ActiveMQ and [...] in James we currently have a single "message store" and we can "lock on a message" (so no other thread will take it) "retrieve it", "update and unlock it" (alter its state or state+content) or "remove it". How would you manage this with ActiveMQ?

With ActiveMQ you'd use a queue per state/maillet, remove it from the queue, do something with it then put it on some other queue(s) (either changed or the same message). The simple JMS/MOM model of sending to a queue or consuming from a queue turns out to be very fast; allowing a highly SEDA based asynchronous model to go really fast since there's no locking or leasing required - and messages can flow very asynchronously to boost throughput.

What does it happen under the hood when I use so many queues? Is the message fully written to disk each time I move it from a queue to another or does it simply update a reference when it belongs to the same store?

If you do find you wanna grab - edit - put back type thing alot you could look at using JavaSpaces (or Entity Bean :). But I think for JAMES then messaging could work well as it sounds to me (as a newbie JAMES person) like what you're doing processing mail is kinda a pipes and filters type model... http://activemq.apache.org/camel/pipes-and-filters.html

which maps very well to messaging and queues.

Cool! This is very interesting and I never read about it. So I'm going to study it and to play with it a bit.

For more background see : http://activemq.apache.org/camel/enterprise-integration-patterns.html

3) Outgoing spool: The outgoing spool in JAMES is a spool like the main spool, with the difference that a message delivery could fail and there is a retry schedule. [...] Any suggestions on how to do this with ActiveMQ?

It sounds like you could use the delayer pattern... http://activemq.apache.org/camel/delayer.html

Then have separate queues for '30 mins later', '1 hour later', '2 hours later'. [...]

I looked at the website and found an error in this page: http://activemq.apache.org/camel/spring-xml-extensions.html "An error occurred: Connection refused. The system administrator has been notified." I looked at the CWIKI sources (http://cwiki.removeme_apache.org/confluence/display/CAMEL/Spring+XML+Extensions) and I see this: {snippet:id=e3|lang=xml|url=activemq/camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/builder/spring_route_builder_test.xml} Not sure but maybe you have to add svn.apache.org/repos/asf/ in front of it?

I understand this is a lot of questions, but I would really appreciate any hint, even partial. I'm collecting ideas :-)

:)

Thank you! Your answers are even more than what I expected! You're suggestion seems to be very very useful and I think you saved me weeks of thoughts! I'll start with your hints and I'll come back with more questions as soon as I'll have rode the camel! ;-)

PS: we are also evaluating using JCR for inboxes if you was wondering, but this is another story, for another list ;-)

You could store the mail in JCR and use messaging for the process flow. e.g. the JMS messages could just contain a reference (URL?) to the message payload.

How often is the payload of the message mutated as it goes through maillets? If it remains kinda static and its more the headers, states & mailets that change mostly, it could be worth putting the payload in some file system / REST resource / JCR and just referring to the payload for large messages (say over 1-10MB)?

This really depends on custom configurations. We provides many mailets that will alter the payload and many that simply run checks and route the message. I guess an estimation of a generic use case could be this: - 100% of messages we spool will have some of their header changed. - 30% of messages will have their body changed a couple of times.

Very much appreciated, thank you again, Stefano