11 messages in net.sourceforge.lists.courier-users[courier-users] Saturation DDoS
FromSent OnAttachments
Zenon PanoussisOct 22, 2007 5:50 am 
Enda CronnollyOct 22, 2007 6:25 am 
Leigh S. JonesOct 22, 2007 7:29 am 
Gordon MessmerOct 22, 2007 7:48 am 
Zenon PanoussisOct 22, 2007 8:25 am 
Gordon MessmerOct 22, 2007 9:50 am 
Zenon PanoussisOct 22, 2007 10:34 am 
Gordon MessmerOct 22, 2007 12:48 pm 
Sam VarshavchikOct 22, 2007 3:36 pm 
Zenon PanoussisOct 23, 2007 12:27 am 
DanielOct 23, 2007 11:29 am 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:[courier-users] Saturation DDoSActions...
From:Zenon Panoussis (ora@provocation.net)
Date:Oct 22, 2007 5:50:55 am
List:net.sourceforge.lists.courier-users

For weeks on end now I am being subjected to what I could call a reverse spam DDoS attack for lack of better term. Some asshole is sending out zillions of messages to non-existent users at legitimate domains, using clearly non-existent sender addresses @myhosteddomain. It seems he is specifically targetting backup MXs and spam filtering services because the messages are first accepted for transport, then bounced. The bounces create a storm of connections to my MX, which in turn causes courier (0.55.1) to choke and stop receiving mail at all.

This is what the log can look like immediately after a restart:

Oct 22 13:56:16 courierd: Waiting. shutdown time=none, wakeup time=Mon Oct 22
14:07:18 2007, queuedelivering=5, inprogress=1 Oct 22 13:56:16 courieresmtpd: started,ip=[::ffff:195.25.12.12] Oct 22 13:56:17 courieresmtpd: started,ip=[::ffff:66.173.214.66] Oct 22 13:56:17 courieresmtpd: started,ip=[::ffff:208.41.143.163] Oct 22 13:56:17 courieresmtpd: started,ip=[::ffff:72.248.85.228] Oct 22 13:56:19 courieresmtpd: started,ip=[::ffff:213.255.87.136] Oct 22 13:56:19 courieresmtpd: started,ip=[::ffff:58.211.213.223] Oct 22 13:56:19 courieresmtpd: started,ip=[::ffff:216.70.235.117] Oct 22 13:56:19 courieresmtpd: started,ip=[::ffff:217.41.15.123] Oct 22 13:56:20 courieresmtpd: started,ip=[::ffff:12.183.242.88] Oct 22 13:56:20 courieresmtpd: started,ip=[::ffff:199.2.119.53] Oct 22 13:56:21 courieresmtpd: started,ip=[::ffff:66.9.136.67] Oct 22 13:56:22 courieresmtpd: started,ip=[::ffff:68.164.193.20] Oct 22 13:56:24 courieresmtpd: started,ip=[::ffff:61.36.155.66] Oct 22 13:56:24 courieresmtpd: started,ip=[::ffff:66.142.164.118] Oct 22 13:56:25 courieresmtpd:
error,relay=::ffff:195.25.12.12,from=<>,to=<Abeg@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:25 courieresmtpd: error,relay=::ffff:195.25.12.12,msg="502 ESMTP
command error",cmd: DATA Oct 22 13:56:25 courieresmtpd:
error,relay=::ffff:66.173.214.66,from=<>,to=<Ferd@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:25 courieresmtpd:
error,relay=::ffff:208.41.143.163,from=<>,to=<Marg@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:26 courieresmtpd:
error,relay=::ffff:72.248.85.228,from=<>,to=<Alde@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:28 courieresmtpd:
error,relay=::ffff:217.41.15.123,from=<>,to=<Efre@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:28 courieresmtpd: started,ip=[::ffff:82.119.204.237] Oct 22 13:56:28 courieresmtpd:
error,relay=::ffff:216.70.235.117,from=<>,to=<Gile@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:29 courieresmtpd: started,ip=[::ffff:193.168.140.69] Oct 22 13:56:29 courieresmtpd:
error,relay=::ffff:12.183.242.88,from=<>,to=<Arnu@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:30 courieresmtpd:
error,relay=::ffff:66.9.136.67,from=<>,to=<Antw@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:30 courieresmtpd:
error,relay=::ffff:199.2.119.53,from=<>,to=<Ferd@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:31 courieresmtpd:
error,relay=::ffff:58.211.213.223,from=<>,to=<Marq@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:31 courieresmtpd:
error,relay=::ffff:68.164.193.20,from=<>,to=<Rubi@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:31 courieresmtpd: started,ip=[::ffff:207.229.32.131] Oct 22 13:56:33 courieresmtpd: started,ip=[::ffff:213.246.40.46] Oct 22 13:56:33 courieresmtpd:
error,relay=::ffff:66.142.164.118,from=<>,to=<Thad@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:34 courieresmtpd:
error,relay=::ffff:61.36.155.66,from=<>,to=<Ferd@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:35 courieresmtpd: started,ip=[::ffff:216.167.161.4] Oct 22 13:56:35 courieresmtpd: started,ip=[::ffff:66.49.172.69] Oct 22 13:56:36 courieresmtpd: started,ip=[::ffff:66.225.112.70] Oct 22 13:56:36 courieresmtpd:
error,relay=::ffff:82.119.204.237,from=<>,to=<Luci@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:36 courieresmtpd: error,relay=::ffff:82.119.204.237,msg="502 ESMTP
command error",cmd: DATA Oct 22 13:56:37 courieresmtpd:
error,relay=::ffff:193.168.140.69,from=<>,to=<Vale@myhosteddomain.org>:
550 User unknown. Oct 22 13:56:37 courieresmtpd: error,relay=::ffff:193.168.140.69,msg="502 ESMTP
command error",cmd: DATA Oct 22 13:56:38 courieresmtpd: started,ip=[::ffff:68.162.95.62] Oct 22 13:56:38 courieresmtpd: started,ip=[::ffff:207.86.183.2]

After a full restart, courier accepts the first two or three dozen connections within a few seconds, then stops accepting connections altogether. The logs of other servers trying to connect to this one say

status=deferred (delivery temporarily suspended: connect to [my courier]:
Connection refused)

So something somewhere gets saturated and simply stops working. This situation persists forever unless courier is restarted, so the effect is a full 100% denial of service to legitimate users. Increasing the number of daemons in authlib/authdaemonrc (tried 5, 10 and 20) doesn't change courier's behaviour. bofh says 'opt BOFHSUPPRESSBACKSCATTER=none'.

As things are, I don't even know where to start looking for the cause, let alone what to look for. Any ideas?

Z