-----Original Message----- From: cour...@lists.sourceforge.net [mailto:cour...@lists.sourceforge.net]On Behalf Of Nathan Ollerenshaw Sent: Monday, February 12, 2001 10:17 AM To: cour...@lists.sourceforge.net Subject: [courier-users] mail solution

Hi there,

I've been tasked with finding a large mail solution for between 500,000 and 1,500,000 users, and I've been looking at a bunch of commercial products, namely iPlanet Messaging Server, Intermail KX/MX and Sendmail Pro. I've also been looking at fully open solutions, and would like to have some feedback from people who have used this software day-to-day in large environments.

The open solution I have foremost in my mind is namely:

Exim 3.20 Courier IMAP/POP3 server with Maildirs

This would be linked to a proprietary Oracle database which contains all our user data for authentication. The actual configuration I'm envisioning is something like this:

a) two Sun E3500 NFS Servers running Sun Cluster 2.2 or 3 for high availability. Gigabit ethernet for the NFS network, FCAL attached storage with a crapload of spindles and running Veritas.

This would basically export /var/mail which would be a tree of Maildirs for each user mailbox that would be created by the MTA on the fly as needed. The format would probably be something along the lines of /var/mail/c/h/r/chrome/Maildir/. There would also need to be quota checks by the MTA to limit to 20MB.

b) Two SunFire 280r's with about 4GB of ram each.

These would be running the Exim, and I'd have something like an Alteon load balancer in front to distribute incoming SMTP connections over the machines, as well as detect if a machine failed so as to push all the connections to the remaining machine if needed. The local disks would probably be A1000s or T3s for the local spool, striped and mirrored for performance over as many spindles as I can cram in.

Incoming mail would be checked to see if it is a local recipient, and if so, I'd like to make a call to a custom library which would connect to our Oracle proxy and ask the proxy if the account exists. If it does, try to deliver the mail into the /var/mail/whatever Maildir.

c) three SunFire 280r's with 4GB ram for the POP3 and optional IMAP access.

Initially, we wouldn't offer IMAP access, as this is something we'd want to charge extra for (for good reason) but we'd use the Courier-IMAP distribution's POP3 server for the POP3 access. I'd like to have Alteons in front of these machines as well, so I can do load-balancing and fail-over.

The machine would mount /var/mail, obviously, from the NFS server.

The authentication would be done via a command written by us that would do the Oracle check, similar to the authcustom command. The only reservation I have with this is the fork & execute for every POP3 connection this would incur, so I'd like to have a custom authentication library function (ideally the same used by exim) that the Courier IMAP and POP3 daemons call.

Obviously one of the choke points here is going to be the authentication, but making that run fast is my problem :)

Hopefully, this gives me a fast mailserver solution where I don't need to worry about locking over NFS, and I don't have to worry about any machine failing. I should theoretically be able to keep adding machines until the point the NFS breaks, at which point I'd split the data over several NFS mail stores and put all the /var/mail/a-m/* on one cluster and /var/mail/n-z/* on the other cluster (or whatever was an even split). I'd then either get the pop boxes to mount both filesystems over different gigabit ethernet interfaces.

Yes, at some point this is going to break, but I should be able to scale this up massively until it does. At least, thats the theory.

Has anyone on this list done anything remotely similar to this, and can tell me what their mileage is?

Is there any major holes in what I'm trying to do here, anything that I'm missing because I don't know the Courier-imap stuff well enough or Exim well enough? Would I be better to bugger off and stop trying to save money? :)

Currently we effectively have one single (very loaded) E3500 holding the mail for all our users and running sendmail & qpopper, surely what I've described here will do better than that :)

Anyway, any and all feedback will be greatly appreciated.

Regards,