From dev.null at multiartstudio.com Mon May 3 04:52:42 2004 From: dev.null at multiartstudio.com (Development - multi.art.studio) Date: Sun May 2 18:53:42 2004 Subject: [SA-exim] exim 4.3.2 and sa-exim local-scan-patch *solved* i must be stupid ;) In-Reply-To: <40905B98.7090600@multiartstudio.com> References: <40905B98.7090600@multiartstudio.com> Message-ID: <4095A5EA.8040604@multiartstudio.com> Hello, i tried around, and now i have applied the patch, as well as copy local_scan.c and reading some more on the fine docus , i noticed there where some changes ;-) sorry for that inconvenience. yours volker Development - multi.art.studio wrote: > hi everyone, > i need some help with sa-exim. > > i updated my exim from 4.1 to 4.3.2 and applied the latest sa-exim > patch for localscan. > now the mails are not scanned anymore, i use the same config as > before, and the header for sa-exim is missing in my mails. > > i like to use spamassassin with sa-exim-localscan-patch on exim 4.3.2 > with preferences from db. > almost, also if i sent mails from or to addresses all over the world > which are on the white_list > or whatever preference, no sa-exim-header is applied to the mails i send. > also with config from file, it creates no header, before there was a > header in every mail. > > maybe is there something missing? enable sa-exim variable or something > else?? > > thanks in advance for any hints. > yours sincerely > volker > > > _______________________________________________ > SA-Exim mailing list > SA-Exim@lists.merlins.org > http://lists.merlins.org/lists/listinfo/sa-exim > From dev.null at multiartstudio.com Mon May 3 05:04:29 2004 From: dev.null at multiartstudio.com (Development - multi.art.studio) Date: Sun May 2 19:05:20 2004 Subject: [SA-exim] question about sa-exim (local_scan patch) and user for spamassassin sql Message-ID: <4095A8AD.6090600@multiartstudio.com> Hello again, i got another (maybe somewhat stupid) question about sa-exim. i use exim 4.3.2 now, with local_scan patch successfully applied and working. now i edited my sa-exim.conf alias spamassassin.conf and user_prefs files to fit my needs. i put my spamassassin dsn-connection info for database in .spamassassin/user_prefs in the homedirectory of exim, because spamd is called and running as exim and exim has the local_scan patch, running as user exim also. *which user is queried from database-table from dsn-connection, if sa-exim is applied as a local_scan patch and running as exim? does it make use of the domainname and local_part? or local username? (for table column username) *where have i exactly put the dsn-info for use with SA-exim local_scan? is user_prefs ok? this is what exim uses for its SA preferences, e.g. score, the subject line etc. thanks in advance, yours sincerely volker From marc at merlins.org Wed May 5 10:19:51 2004 From: marc at merlins.org (Marc MERLIN) Date: Wed May 5 09:19:53 2004 Subject: [SA-exim] Route all SPAM to an Exchange email account In-Reply-To: <217D777D3789FC4591199BA41FB0617AB0D6@nemo.scriptthis.net> References: <217D777D3789FC4591199BA41FB0617AB0D6@nemo.scriptthis.net> Message-ID: <20040505161951.GI12071@merlins.org> On Thu, Apr 29, 2004 at 10:02:34AM -0400, Jerry Rasmussen wrote: > I have sa-exim setup as a gateway mailserver. It relays mail to my > exchange server. Is there a way with sa-exim to have say all email that > have a score of 10 or more forward the email to an exchange account (i.e. > spam@domain.com). /etc/procmailrc for one solution Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From marc_soft at merlins.org Wed May 5 20:21:53 2004 From: marc_soft at merlins.org (Marc MERLIN) Date: Wed May 5 20:17:13 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: <20040430170631.GA13836@freshdot.net> References: <20040430170631.GA13836@freshdot.net> Message-ID: <20040506022153.GB1337@merlins.org> On Fri, Apr 30, 2004 at 07:06:31PM +0200, Sander Smeenk wrote: > > I wrote a little shell script to track the spamc options and data that > > SA-Exim was using. I discovered that the top Recieved: header (the one > > that is added by Exim on my machine) is not being sent to spamc. I'm > > not sure if this header has even been created yet or not. > > Well, I remember something similar being reported recently about Exiscan > on the exim4 mailinglist. Hmm. It could be... I got a report that Exim 4.32 was modified to not send the last received line to local_scan. I have no idea why and it really sucks as far as plugins are concerned. I'm behind on Email, so I haven't had the time to find out why yet, or study workarounds, but as far as I'm concerned right now, it's an exim flaw (not a bug, since it's apparently intentional, although I fail to see why), and I'm really not certain how sa-exim can work around it without patching exim to revert this change. I'll try to research this further when I can. Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From marc at merlins.org Wed May 5 21:19:28 2004 From: marc at merlins.org (Marc MERLIN) Date: Wed May 5 20:19:38 2004 Subject: [SA-exim] sa-exim with Exim4 + Debian (Sarge) In-Reply-To: <20040430095623.GI16486@freshdot.net> References: <1DBA7B491604E94BBCCE5133069A5BB20DE62C@mail.keyschool.org> <20040429201659.GG16486@freshdot.net> <20040429205224.GE2089@wheat.boylan.org> <20040430095623.GI16486@freshdot.net> Message-ID: <20040506031928.GE1337@merlins.org> On Fri, Apr 30, 2004 at 11:56:23AM +0200, Sander Smeenk wrote: > It does *NOT* check if the system where it was installed has a > monolithic configuration file, so it does *NOT* warn about manually > having to add the configuration line. *Yet*. A warn would be nice, but if people chose the monolithic format, that's too bad for them, they should have to deal with their choice for a less flexible option (no, I'm not biased :) > Also, either Marc will try to move important information from the > INSTALL file to the README file, or I will include the INSTALL file in > the Debian package, and give it a different name. In CVS now: RCS file: /cvsroot/sa-exim/sa-exim/README,v retrieving revision 1.17 diff -u -u -r1.17 README --- README 10 Mar 2004 17:12:56 -0000 1.17 +++ README 6 May 2004 03:04:27 -0000 @@ -12,6 +12,16 @@ See the file named INSTALL for installations instructions (either compiled in exim, or as a stand-alone shared library) +If you got sa-exim prepackaged (like on debian), you have to make sure that +your exim supports a dynamically loadable local_scan (which is true on debian +and probalby on other distros too if they shipped sa-exim as a package), and +that your exim4.conf file contains the following: +local_scan_path = /usr/lib/exim4/local_scan/sa-exim.so +If you are using the split configuration file on debian with the sa-exim deb +package, you'll be fine. If you're using the monolithic file, you are on your +own until/unless the sa-exim packages try to do an in place edit (i.e. you have +to add the above configuration line yourself) + Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From lists at timj.co.uk Thu May 6 09:01:47 2004 From: lists at timj.co.uk (Tim Jackson) Date: Thu May 6 00:01:56 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: <20040506022153.GB1337@merlins.org> References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> Message-ID: Hi Marc, on Wed, 5 May 2004 19:21:53 -0700 you wrote: > On Fri, Apr 30, 2004 at 07:06:31PM +0200, Sander Smeenk wrote: > > Someone else wrote: > > > I discovered that the top Recieved: header (the one that is added by > > > Exim on my machine) is not being sent to spamc. > > Well, I remember something similar being reported recently about > > Exiscan on the exim4 mailinglist. Hmm. It could be... > I got a report that Exim 4.32 was modified to not send the last received > line to local_scan. > I have no idea why and it really sucks as far as plugins are concerned. The relevant ChangeLog entry (4.31) which explains the reasoning is: "66. The generation of the Received: header has been moved from the time that a message starts to be received, to the time that it finishes. The timestamp in the Received: header should now be very close to that of the <= log line. There are two side-effects of this change: (a) If a message is rejected by a DATA or non-SMTP ACL or local_scan(), the logged header lines no longer include the local Received: line, because it has not yet been created. The same applies to a copy of the message that is returned to a non-SMTP sender when a message is rejected. (b) When a filter file is tested using -bf, no additional Received: header is added to the test message. After some thought, I decided that this is a bug fix. This change does not affect the value of $received_for. It is still set after address rewriting, but before local_scan() is called." I can see the reasoning here (though that's not to say it was necessarily a good change), but it's unfortunate that the knock-on effect on Exiscan/SA-Exim wasn't predicted. Tom gets round it in Exiscan by generating his own temporary "fake" Received header before passing the mail to SA. Tim From marc at merlins.org Thu May 6 10:25:13 2004 From: marc at merlins.org (Marc MERLIN) Date: Thu May 6 09:25:15 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> Message-ID: <20040506162512.GC21750@merlins.org> [Cc Philip] On Thu, May 06, 2004 at 08:01:47AM +0100, Tim Jackson wrote: > Hi Marc, on Wed, 5 May 2004 19:21:53 -0700 you wrote: > > > On Fri, Apr 30, 2004 at 07:06:31PM +0200, Sander Smeenk wrote: > > > Someone else wrote: > > > > I discovered that the top Recieved: header (the one that is added by > > > > Exim on my machine) is not being sent to spamc. > > > Well, I remember something similar being reported recently about > > > Exiscan on the exim4 mailinglist. Hmm. It could be... > > I got a report that Exim 4.32 was modified to not send the last received > > line to local_scan. > > I have no idea why and it really sucks as far as plugins are concerned. > > The relevant ChangeLog entry (4.31) which explains the reasoning is: > > "66. The generation of the Received: header has been moved from the time > that a message starts to be received, to the time that it finishes. The > timestamp in the Received: header should now be very close to that of the > <= log line. There are two side-effects of this change: Thanks, I hadn't had the time to go through those yet. > I can see the reasoning here (though that's not to say it was necessarily > a good change), but it's unfortunate that the knock-on effect on > Exiscan/SA-Exim wasn't predicted. > > Tom gets round it in Exiscan by generating his own temporary "fake" > Received header before passing the mail to SA. Yep, I now remember a mail from Nigel warning me of this. This is annoying in Tom's case, but he can at least work around it because he's directly inside the exim code. As a local_scan plugin, I don't have the same access to the exim internals. The worst part is that SA-Exim would now have to duplicate the exim Received line generation code (and track it), but only run it if it's dealing with a version of Exim that doesn't generate the last line in time for local_scan so it'd have to somehow bypass the API and query Exim directly to find the version number (I'm guessing it's possible, but...) All this to say that I'd much rather that this unfortunate exim API change be reverted, which in real life means that it becomes an option so that local_scan plugins can get the original (wanted) behaviour again. Philip, does that sound like a reasonable request? In the meantime, I'm going to recommend that sa-exim users do not use any version of exim past 4.30 until this gets figured out somehow Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From Nigel.Metheringham at dev.InTechnology.co.uk Thu May 6 18:33:04 2004 From: Nigel.Metheringham at dev.InTechnology.co.uk (Nigel Metheringham) Date: Thu May 6 09:33:13 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: <20040506162512.GC21750@merlins.org> References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> Message-ID: <1083861184.5275.43.camel@angua.localnet> On Thu, 2004-05-06 at 17:25, Marc MERLIN wrote: > [Cc Philip] > The worst part is that SA-Exim would now have to duplicate the exim Received > line generation code (and track it), but only run it if it's dealing with > a version of Exim that doesn't generate the last line in time for local_scan > so it'd have to somehow bypass the API and query Exim directly to find the > version number (I'm guessing it's possible, but...) > > All this to say that I'd much rather that this unfortunate exim API change > be reverted, which in real life means that it becomes an option so that > local_scan plugins can get the original (wanted) behaviour again. Why not do something a little different.... Currently, as I understand it, a placeholder for the received header is generated, but the actual header is left until the end of reception. Why not generate a received header right at the start, just as it used to. Then at the end of reception overwrite this header with a new copy of the received header, so getting the new timing and features. Nigel. -- [ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ] From RossBoylan at stanfordalumni.org Thu May 6 11:53:58 2004 From: RossBoylan at stanfordalumni.org (Ross Boylan) Date: Thu May 6 10:54:10 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: <20040506162512.GC21750@merlins.org> References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> Message-ID: <20040506175358.GB6196@wheat.boylan.org> On Thu, May 06, 2004 at 09:25:13AM -0700, Marc MERLIN wrote: > [Cc Philip] > > On Thu, May 06, 2004 at 08:01:47AM +0100, Tim Jackson wrote: > > Hi Marc, on Wed, 5 May 2004 19:21:53 -0700 you wrote: > > > > > On Fri, Apr 30, 2004 at 07:06:31PM +0200, Sander Smeenk wrote: > > > > Someone else wrote: > > > > > I discovered that the top Recieved: header (the one that is added by > > > > > Exim on my machine) is not being sent to spamc. > > > > Well, I remember something similar being reported recently about > > > > Exiscan on the exim4 mailinglist. Hmm. It could be... > > > I got a report that Exim 4.32 was modified to not send the last received > > > line to local_scan. > > > I have no idea why and it really sucks as far as plugins are concerned. > > > > The relevant ChangeLog entry (4.31) which explains the reasoning is: > > > > "66. The generation of the Received: header has been moved from the time > > that a message starts to be received, to the time that it finishes. The > > timestamp in the Received: header should now be very close to that of the > > <= log line. There are two side-effects of this change: > > Thanks, I hadn't had the time to go through those yet. > > > I can see the reasoning here (though that's not to say it was necessarily > > a good change), but it's unfortunate that the knock-on effect on > > Exiscan/SA-Exim wasn't predicted. > > > > Tom gets round it in Exiscan by generating his own temporary "fake" > > Received header before passing the mail to SA. > > Yep, I now remember a mail from Nigel warning me of this. > > This is annoying in Tom's case, but he can at least work around it because > he's directly inside the exim code. > As a local_scan plugin, I don't have the same access to the exim internals. > > The worst part is that SA-Exim would now have to duplicate the exim Received > line generation code (and track it), but only run it if it's dealing with > a version of Exim that doesn't generate the last line in time for local_scan > so it'd have to somehow bypass the API and query Exim directly to find the > version number (I'm guessing it's possible, but...) > > All this to say that I'd much rather that this unfortunate exim API change > be reverted, which in real life means that it becomes an option so that > local_scan plugins can get the original (wanted) behaviour again. > > Philip, does that sound like a reasonable request? > > In the meantime, I'm going to recommend that sa-exim users do not use > any version of exim past 4.30 until this gets figured out somehow > > Marc Just to add another layer to this: all my mail from outside comes through another system. For that reason (among others) I haven't been using any RBL tests. I just realized that the behavior of omitting the first received line actually means I could do RBL on the IP that sent the message to the forwarding system. I think. I guess this means that SA-Exim RBL tests would be effective, but exim ACL's would not. This perhaps suggests that, in general, the ability to recognize the first "outside" address might be useful (though if that could be spoofed, a bit risky). Even if it's a good idea in principle, it may be excessively baroque. Just a thought. I know in a perfect world the other system receiving my mail would take care of everything. But it's running sendmail, so I can't expect too much :) From ph10 at cus.cam.ac.uk Fri May 7 10:19:10 2004 From: ph10 at cus.cam.ac.uk (Philip Hazel) Date: Fri May 7 08:30:31 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: <1083861184.5275.43.camel@angua.localnet> References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> <1083861184.5275.43.camel@angua.localnet> Message-ID: On Thu, 6 May 2004, Nigel Metheringham wrote: > > All this to say that I'd much rather that this unfortunate exim API change > > be reverted, which in real life means that it becomes an option so that > > local_scan plugins can get the original (wanted) behaviour again. Having thought about this, it seemed to me to be "correct" that the time in the Received: header really should be the time of final acceptance, not the time the message started to be received. So I would not want to revert to the previous behaviour, and I think an option would just be another complication that could be confusing. > Why not generate a received header right at the start, just as it used > to. > Then at the end of reception overwrite this header with a new copy of > the received header, so getting the new timing and features. That is a bright idea, Nigel. Thanks for suggesting it. I will look into it, but I don't know when, because I'm going to be away a lot in the next couple of months. Philip -- Philip Hazel University of Cambridge Computing Service, ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714. From marc at merlins.org Fri May 7 09:52:30 2004 From: marc at merlins.org (Marc MERLIN) Date: Fri May 7 08:52:32 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> <1083861184.5275.43.camel@angua.localnet> Message-ID: <20040507155230.GJ21750@merlins.org> On Fri, May 07, 2004 at 09:19:10AM +0100, Philip Hazel wrote: > > Why not generate a received header right at the start, just as it used > > to. > > Then at the end of reception overwrite this header with a new copy of > > the received header, so getting the new timing and features. > > That is a bright idea, Nigel. Thanks for suggesting it. > > I will look into it, but I don't know when, because I'm going to be away > a lot in the next couple of months. Fair enough. Let me ask: now all sa-exim users are pretty much unable to uprade past exim 4.30, and with the remote exploit in check_headers, that's not really a great thing. Until the former behavior of exim get introduced, could you, with minimal effort point me to a patch that would revert exim to its original behavior and could be applied to 4.33? (i.e. sa-exim in local_scan is impacted enough that it's questionable to use it with the new behavior since it can now junk non spam mail as a result, and that's usually not acceptable for people :) Thanks Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From ph10 at cus.cam.ac.uk Fri May 7 18:05:18 2004 From: ph10 at cus.cam.ac.uk (Philip Hazel) Date: Fri May 7 09:05:27 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: <20040507155230.GJ21750@merlins.org> References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> <1083861184.5275.43.camel@angua.localnet> <20040507155230.GJ21750@merlins.org> Message-ID: On Fri, 7 May 2004, Marc MERLIN wrote: > Until the former behavior of exim get introduced, could you, with minimal > effort point me to a patch that would revert exim to its original behavior > and could be applied to 4.33? All I can suggest is that you look at an older release, and retrofit what it did for creating the Received: header. The source module is receive.c. I *may* get round to looking at this before I go away next Thursday, but I equally well may not. Sorry about that. -- Philip Hazel University of Cambridge Computing Service, ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714. From ssmeenk at freshdot.net Fri May 7 19:13:55 2004 From: ssmeenk at freshdot.net (Sander Smeenk) Date: Fri May 7 09:14:02 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> <1083861184.5275.43.camel@angua.localnet> <20040507155230.GJ21750@merlins.org> Message-ID: <20040507161355.GB32240@freshdot.net> Quoting Philip Hazel (ph10@cus.cam.ac.uk): > I *may* get round to looking at this before I go away next Thursday, but > I equally well may not. Sorry about that. >From what I read about this issue, I concluded that there's no (real) chance that legit mail is being rejected. The missing received header makes spam score less high, instead of legit mails scoring higher. I noticed this because I am receiving tons of spams in my inbox, since I upgraded exim to 4.30+ Furthermore I have a 'rejectlog' mailed to me every night, and I can't see legit mail being rejected (yet). Although I *do* think it's good to check on things while you're running exim 4.31 or higher, i don't think systems running 4.31+ with sa-exim active will drop legit mail. Regards, Sander. -- | What do sheep count when they want to fall asleep? | 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D From dlotina at netglobalis.net Fri May 7 13:31:15 2004 From: dlotina at netglobalis.net (Danilo Lotina F.) Date: Fri May 7 09:31:33 2004 Subject: [SA-exim] Feature Message-ID: Hi, I'm new in this list, however I use sa-exim from many months in five relay servers with over one million mails procesing per day each one. I like to suggest sa-exim option in order to specify spamd IP address (and port number). Regards! Danilo Lotina F. Gerente T?cnico NetGlobalis S.A From marc at merlins.org Fri May 7 10:52:52 2004 From: marc at merlins.org (Marc MERLIN) Date: Fri May 7 09:52:55 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: <20040507161355.GB32240@freshdot.net> References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> <1083861184.5275.43.camel@angua.localnet> <20040507155230.GJ21750@merlins.org> <20040507161355.GB32240@freshdot.net> Message-ID: <20040507165252.GL21750@merlins.org> On Fri, May 07, 2004 at 06:13:55PM +0200, Sander Smeenk wrote: > > I *may* get round to looking at this before I go away next Thursday, but > > I equally well may not. Sorry about that. > > >From what I read about this issue, I concluded that there's no (real) > chance that legit mail is being rejected. The missing received header > makes spam score less high, instead of legit mails scoring higher. Not really actually. If you're missing the last received line, it will for instance show that a dialup IP is in the last (and only) received line, which looks as is a dialup IP connected to my MX directly instead of going through a proper relay On my setup, that's a SA score of 4 right there, almost a kiss of death. header RCVD_IN_DUL rbleval:check_rbl('dialup', 'dialups.mail-abuse.org.') describe RCVD_IN_DUL Received from dialup, see http://www.mail-abuse.org/dul/ score RCVD_IN_DUL 4 header X_RCVD_IN_DUL_FH rbleval:check_rbl('dialup-firsthop', 'dialups.mail-abuse.org.') describe X_RCVD_IN_DUL_FH Received from first hop dialup, see http://www.mail-abuse.org/dul/ score X_RCVD_IN_DUL_FH -3.5 Then again, not many people may know about this and use it, as I'm the one who wrote the dialup vs dialup-firsthop code in SpamAssassin :) (it's properly documented in the conf docs though :) > I noticed this because I am receiving tons of spams in my inbox, > since I upgraded exim to 4.30+ Exim 4.30 is fine, it 4.31 and above that aren't. (I'm running 4.30 myself) > Furthermore I have a 'rejectlog' mailed to me every night, and I can't > see legit mail being rejected (yet). Although I *do* think it's good to > check on things while you're running exim 4.31 or higher, i don't think > systems running 4.31+ with sa-exim active will drop legit mail. They can, depends on your config. Upgrading to 4.31 on mine would really overscore a lot of otherwise legit mail. Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From rick at linuxmafia.com Fri May 7 17:38:37 2004 From: rick at linuxmafia.com (Rick Moen) Date: Fri May 7 16:38:44 2004 Subject: [SA-exim] SA-exim suddenly started ignoring "X-Spam-Status: Yes" In-Reply-To: <20040424212938.GB16038@freshdot.net> References: <20040424201447.GI3951@linuxmafia.com> <20040424212938.GB16038@freshdot.net> Message-ID: <20040507233836.GW12221@linuxmafia.com> Quoting Sander Smeenk (ssmeenk@freshdot.net): > The permissions on /var/spool/sa-exim are wrong, if you ask me. They > indeed should have been Debian-exim:Debian-exim. But this should not be > the show-stopper for spam detection, afaik. Concur. For the record, those would have been created by the sa-exim .deb that I downloaded from Marc's site, when I converted from Exim3, last year. Having done the necessary recursive chown operations on the /var/spool/sa-exim/ tree, suddenly /var/log/exim4/paniclog became glad tidings, i.e., null contents. [On the main matter, denoted by the subject header:] > > Any clues as to how I figure out what has suddenly broken? > > Logfiles. It never hurts to give this advice. ;-> A surprising number of people are oddly unaware that that's the first place you look. That's about the second thing I learned about Unix systems, back in the day. The first was "98% of all Unix problems turn out, upon examination, to involve ownership or permissions." I'm posting in order to reveal the explanation, which turned out to be that I'd at some point chgrp'd /home/rick/inboxes/junkmail to mail:rick instead of rick:rick, and _procmail_, my local delivery agent (not Exim4, not spamassassin) was unhappy. Almost certainly, I managed to carelessly do this somehow when I cp'd /home/rick/inboxes/junkmail to /tmp in order to su to my sa-exim user and run sa-learn on it. At that time, I'm in the habit of truncating the /home/rick/inboxes/junkmail file to zero length, and must have screwed that step up. So, it fundamentally wasn't an Exim4 problem, and not an sa-exim one. -- Cheers, "A raccoon tangled with a 23,000 volt line, today. The results Rick Moen blacked out 1400 homes and, of course, one raccoon." rick@linuxmafia.com -- Steel City News From marc at merlins.org Fri May 7 19:03:55 2004 From: marc at merlins.org (Marc MERLIN) Date: Fri May 7 18:03:56 2004 Subject: [SA-exim] SA-exim suddenly started ignoring "X-Spam-Status: Yes" In-Reply-To: <20040507233836.GW12221@linuxmafia.com> References: <20040424201447.GI3951@linuxmafia.com> <20040424212938.GB16038@freshdot.net> <20040507233836.GW12221@linuxmafia.com> Message-ID: <20040508010355.GG13275@merlins.org> On Fri, May 07, 2004 at 04:38:37PM -0700, Rick Moen wrote: > Quoting Sander Smeenk (ssmeenk@freshdot.net): > > > The permissions on /var/spool/sa-exim are wrong, if you ask me. They > > indeed should have been Debian-exim:Debian-exim. But this should not be > > the show-stopper for spam detection, afaik. > > Concur. For the record, those would have been created by the sa-exim > .deb that I downloaded from Marc's site, when I converted from Exim3, > last year. The problem is that the debian exim package changed the primary exim user half way, becoming incompatible with the older versions of sa-exim. The current deb has the matching permissions Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From dlotina at netglobalis.net Sun May 9 18:01:17 2004 From: dlotina at netglobalis.net (Danilo Lotina F.) Date: Sun May 9 14:01:28 2004 Subject: [SA-exim] spamd on remote host - was Feature In-Reply-To: Message-ID: Hi, I've modified a couple of lines adding SAspamcHost option in sa-exim.conf: Example: SAspamcHost: 200.14.80.82 It works fine ! :-) --- sa-exim.c.orig Wed Mar 10 03:19:20 2004 +++ sa-exim.c Sun May 9 16:34:36 2004 @@ -527,6 +527,7 @@ /* Options we read from /etc/exim4/sa-exim.conf */ static char *SAspamcpath=SPAMC_LOCATION; static char *SAspamcSockPath=NULL; + static char *SAspamcHost="127.0.0.1"; static char *SAEximRunCond="0"; static char *SAEximRejCond="1"; static int SAmaxbody=250*1024; @@ -667,6 +668,7 @@ M_CHECKFORVAR(SAEximDebug, "%d"); M_CHECKFORSTR(SAspamcpath); M_CHECKFORSTR(SAspamcSockPath); + M_CHECKFORSTR(SAspamcHost); M_CHECKFORSTR(SAEximRunCond); M_CHECKFORSTR(SAEximRejCond); M_CHECKFORVAR(SAmaxbody, "%d"); @@ -868,9 +870,10 @@ } else { - ret=execl(SAspamcpath, "spamc", "-s", string_sprintf("%d", SAmaxbody+16384), NULL); + ret=execl(SAspamcpath, "spamc", "-s", string_sprintf("%d", SAmaxbody+16384), "-d", SAspamcHost, NULL); CHECKERR(ret,string_sprintf("exec %s", SAspamcpath),__LINE__); } + } if (SAEximDebug > 8) > -----Mensaje original----- > De: Danilo Lotina F. [mailto:dlotina@netglobalis.net] > Enviado el: Viernes, 07 de Mayo de 2004 12:31 > Para: sa-exim@lists.merlins.org > Asunto: Feature > > > Hi, > > I'm new in this list, however I use sa-exim from many months > in five relay servers with over one million mails procesing per > day each one. > > I like to suggest sa-exim option in order to specify spamd IP > address (and port number). > > > Regards! > > > Danilo Lotina F. > Gerente T?cnico > NetGlobalis S.A > > From marc at merlins.org Sun May 9 15:27:56 2004 From: marc at merlins.org (Marc MERLIN) Date: Sun May 9 14:27:58 2004 Subject: [SA-exim] spamd on remote host - was Feature In-Reply-To: References: Message-ID: <20040509212756.GJ24862@merlins.org> On Sun, May 09, 2004 at 05:01:17PM -0400, Danilo Lotina F. wrote: > Hi, > > I've modified a couple of lines adding SAspamcHost option in sa-exim.conf: > Example: > SAspamcHost: 200.14.80.82 > > It works fine ! :-) Right, that's how to do it for now. I'll put this and a port number option in the next version Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From ph10 at cus.cam.ac.uk Mon May 10 15:56:45 2004 From: ph10 at cus.cam.ac.uk (Philip Hazel) Date: Mon May 10 06:56:52 2004 Subject: [SA-exim] Re: Bug#246715: sa-exim: network checks are failing because headers are incomplete In-Reply-To: References: <20040430170631.GA13836@freshdot.net> <20040506022153.GB1337@merlins.org> <20040506162512.GC21750@merlins.org> <1083861184.5275.43.camel@angua.localnet> <20040507155230.GJ21750@merlins.org> Message-ID: On Fri, 7 May 2004, Philip Hazel wrote: > All I can suggest is that you look at an older release, and retrofit > what it did for creating the Received: header. The source module is > receive.c. > > I *may* get round to looking at this before I go away next Thursday, but > I equally well may not. Sorry about that. I did. I've just put out 4.34 because of the security fix. I also changed the Received: stuff. I hope I did it right. Philip -- Philip Hazel University of Cambridge Computing Service, ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714. From tor at slett.net Mon May 10 16:43:41 2004 From: tor at slett.net (Tor Slettnes) Date: Mon May 10 15:43:52 2004 Subject: [SA-exim] Re: [Exim] EXIM 4.31, courier-imap, Clamd, exiscan, spamassassinLoad problems In-Reply-To: <000101c436dc$e6c917a0$0405010a@efastfunding.com> References: <000101c436dc$e6c917a0$0405010a@efastfunding.com> Message-ID: <7C607DC1-A2D3-11D8-9777-0030656CF512@slett.net> On May 10, 2004, at 15:19, Doug Block wrote: > Well I am using a AMD XP200+ yes I know it's not a server chip but it > was > what I had free at the time. I have Sa-scan handling the Spamassassin > and > exiscan handling the clamd. You probably mean "SA-Exim", not "SA-Scan". SA-Exim has problems with Exim v4.31 through 4.33, due to the Received: header change. Either stay with 4.30, or upgrade to 4.34, or you will get wildly inaccurate SA scores (specifically, SA-Exim may trust forged Received: headers, or else treat messages sent through a smarthost as if they came directly from a dialup host). That, though, probably has nothing to do with your load issues. > I have about 80 power users that receive anywhere between 4k to 10k of > msgs > per day. The box I currently have for them is 1.5 gig 1024 meg RH9 > box > with a Ide OS drive for RH9 with all of the mail related items stored > on a > hardware mirrored 73 gig scsi320 drives (/home, /mail /var/spool, > etc). So > far have migrated about 7 gig of mail in maildir format over for about > 30 > users and I have notice that my proc is spiking (which is normal) but > my > users are noticing a delay in Imap response which I think is due to the > spike to 100% for spamd and clamd when a ton of mail comes thru. Well, for starters, Courier is not exactly the fastest IMAP server out there. Something like Cyrus (or even DoveCot) will definitely give faster access, especially for large mailboxes and many users. That said, IMAP access to a newly Courier folder will speed up after the first time. Initially, Exim (and other 3rd party tools that drop mails in Maildir/ hierarchies) place stuff in the "new/" subfolder, unindexed. Once Courier has had a chance to move the new mail into "cur/" and to index it, you will see that the mailbox can be opened faster. In your case, a bigger problem may be SpamAssassin itself. Don't allow it to scan very large mails (e.g. >256k). I believe SA-Exim's configuration file has a maximum message size setting; correct me if I am wrong. Or else, in Exim you can set an ACL condition on $message_size. -tor From tor at slett.net Mon May 10 18:50:45 2004 From: tor at slett.net (Tor Slettnes) Date: Mon May 10 17:50:51 2004 Subject: [SA-exim] Re: [Exim] EXIM 4.31, courier-imap, Clamd, exiscan, spamassassinLoad problems In-Reply-To: <000401c436e2$3f4bfd70$0405010a@efastfunding.com> References: <000401c436e2$3f4bfd70$0405010a@efastfunding.com> Message-ID: <3C5FE168-A2E5-11D8-9777-0030656CF512@slett.net> [Top-quoted, non-attributed message corrected. See http://www.netmeister.org/news/learn2quote.html] On May 10, 2004, at 15:58, Doug Block wrote: > Tor Slettnes wrote: >> You probably mean "SA-Exim", not "SA-Scan". SA-Exim has problems >> with Exim >> v4.31 through 4.33, due to the Received: header change. Either stay >> with >> 4.30, or upgrade to 4.34, or you will get wildly inaccurate SA scores >> (specifically, SA-Exim may trust forged Received: headers, or else >> treat >> messages sent through a smarthost as if they came directly from a >> dialup >> host). > > Yes your right!!! On the spam and the courier > Oh the sa-exim is working great so far on the scores. I have had a > 75% drop > in spam and I have been using spam pit to check the scores which so > far have > been correct on both real mail and spam. I may upgrade to 4.34 once I > get > some time since it came out today. Keep in mind that your success is not only reflected in how much spam you block, but more importantly, in how much legitimate mail you let through. With that in mind, running SA-Exim with Exim 4.31 - 4.33 is at best ignorant - at worst, irresponsible. Especially this is the case if you are hosting mail for other people. (Sure, blocking 75% of spam is good; but with a properly configured SA, you ought to be able to catch well above 90%, while not impacting legitimate mail at all). > I have a older (half the speed) exim 4 box that handled the load for > up 150 users with courier-imap with out this problem but it did not > have clamd, sa-exim, exiscan, and spamassassin on it. This box worked > fine for 18 months+ but the virus's and spam where a big problem. I > have spam set to under 256k but I have been watching this while > writing this email and I notice 8-12 messages at a time coming in and > eating 100% of the cpu for about 5 secs while they get spamd and > clamd. You could start "spamd" with "--nicelevel 15" or so, essentially lowering its priority if there are other (presumably more important) tasks going on. (On a Debian machine, look in /etc/default/spamassassin). You could disable the SA network tests (which, while using Exim 4.31 to 4.33, probably do more harm than good): skip_rbl_checks 1 use_dcc 0 use_pyzor 0 use_razor2 0 Also, if you are using the teergrube facilities of SA-Exim, beware that the 8-12 simultaneously running Exim processes don't neccessarily compete for CPU time - they may be just sitting there doing what they do best: stalling, waiting, stalling, waiting... If so, you may additionally need to remove the limit of simultaneously running Exim processes in Exim's main section (or else, it only takes about 20 spams to perform a DoS on your machine): # Don't set a limit on incoming SMTP connections smtp_accept_max = 0 # ... unless the system load is above 10. smtp_load_reserve = 10 -tor From lists at efastfunding.com Mon May 10 18:58:02 2004 From: lists at efastfunding.com (Doug Block) Date: Mon May 10 21:25:04 2004 Subject: [SA-exim] RE: [Exim] EXIM 4.31, courier-imap, Clamd, exiscan, spamassassinLoad problems In-Reply-To: <7C607DC1-A2D3-11D8-9777-0030656CF512@slett.net> Message-ID: <000401c436e2$3f4bfd70$0405010a@efastfunding.com> Yes your right!!! On the spam and the courier Oh the sa-exim is working great so far on the scores. I have had a 75% drop in spam and I have been using spam pit to check the scores which so far have been correct on both real mail and spam. I may upgrade to 4.34 once I get some time since it came out today. I have a older (half the speed) exim 4 box that handled the load for up 150 users with courier-imap with out this problem but it did not have clamd, sa-exim, exiscan, and spamassassin on it. This box worked fine for 18 months+ but the virus's and spam where a big problem. I have spam set to under 256k but I have been watching this while writing this email and I notice 8-12 messages at a time coming in and eating 100% of the cpu for about 5 secs while they get spamd and clamd. Is this box to small for this load or should I start buying beer for people to audit my configs. I have no problem getting a multi-proc box if that will solve the problem it maybe extream over kill. On May 10, 2004, at 15:19, Doug Block wrote: > Well I am using a AMD XP200+ yes I know it's not a server chip but it > was what I had free at the time. I have Sa-scan handling the > Spamassassin and > exiscan handling the clamd. You probably mean "SA-Exim", not "SA-Scan". SA-Exim has problems with Exim v4.31 through 4.33, due to the Received: header change. Either stay with 4.30, or upgrade to 4.34, or you will get wildly inaccurate SA scores (specifically, SA-Exim may trust forged Received: headers, or else treat messages sent through a smarthost as if they came directly from a dialup host). That, though, probably has nothing to do with your load issues. <...snip..> Well, for starters, Courier is not exactly the fastest IMAP server out there. Something like Cyrus (or even DoveCot) will definitely give faster access, especially for large mailboxes and many users. That said, IMAP access to a newly Courier folder will speed up after the first time. Initially, Exim (and other 3rd party tools that drop mails in Maildir/ hierarchies) place stuff in the "new/" subfolder, unindexed. Once Courier has had a chance to move the new mail into "cur/" and to index it, you will see that the mailbox can be opened faster. In your case, a bigger problem may be SpamAssassin itself. Don't allow it to scan very large mails (e.g. >256k). I believe SA-Exim's configuration file has a maximum message size setting; correct me if I am wrong. Or else, in Exim you can set an ACL condition on $message_size. -tor -- ## List details at http://www.exim.org/mailman/listinfo/exim-users Exim details at http://www.exim.org/ ## From nmw at ion.le.ac.uk Tue May 11 11:04:17 2004 From: nmw at ion.le.ac.uk (Nigel Wade) Date: Tue May 11 08:17:45 2004 Subject: [SA-exim] Re: [Exim] EXIM 4.31, courier-imap, Clamd, exiscan, spamassassinLoad problems In-Reply-To: <000401c436e2$3f4bfd70$0405010a@efastfunding.com> References: <000401c436e2$3f4bfd70$0405010a@efastfunding.com> Message-ID: <40A09711.1000906@ion.le.ac.uk> Doug Block wrote: > Yes your right!!! On the spam and the courier > Oh the sa-exim is working great so far on the scores. I have had a 75% drop > in spam and I have been using spam pit to check the scores which so far have > been correct on both real mail and spam. I may upgrade to 4.34 once I get > some time since it came out today. > > I have a older (half the speed) exim 4 box that handled the load for up 150 > users with courier-imap with out this problem but it did not have clamd, > sa-exim, exiscan, and spamassassin on it. This box worked fine for 18 > months+ but the virus's and spam where a big problem. I have spam set to > under 256k but I have been watching this while writing this email and I > notice 8-12 messages at a time coming in and eating 100% of the cpu for > about 5 secs while they get spamd and clamd. > > Is this box to small for this load or should I start buying beer for people > to audit my configs. I have no problem getting a multi-proc box if that > will solve the problem it maybe extream over kill. > > A multi-processor box certainly isn't overkill - especially if it makes your mail system work. I have a dual-cpu mail server here for a much lower load. What might be simpler in the short term is to utilize your old exim box as a spam/virus scanner. I don't know about clamd, but spamd can certainly be run on a different system to that which runs exim. Either spamc or exiscan can be setup to talk to spamd on a different host. If necessary, as a simple upgrade, put in the fastest processor that old box will take. Also, do the "power" users receive most of their mail from trusted sources? If so, it may be possible to configure the system to not scan those messages. -- Nigel Wade, System Administrator, Space Plasma Physics Group, University of Leicester, Leicester, LE1 7RH, UK E-mail : nmw@ion.le.ac.uk Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555 From bpatel at tum.net Tue May 11 14:44:32 2004 From: bpatel at tum.net (Bikesh Patel) Date: Tue May 11 10:44:43 2004 Subject: [SA-exim] wont't detect spam correctly Message-ID: Folks, I'm running SA-Exim as local_scan: somehow a good amount is making through when running as localscan and the same email is tagged spam correctly when running as transport The following is NOT tagged spam, when it is in SA-Exim ------------------------------------------- Return-path: Reply-To: "zula heredia" From: "zula heredia" X-SA-Exim-Connect-IP: 67.66.107.43 X-SA-Exim-Mail-From: drake_elsie@cox.net Subject: Fwd: V^|cod|n & v|@grA ` Va+l+ium & XAN@X So|m|a ' Pnte.r.min umodhrnsojsa X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on hostname X-Spam-Level: *** X-Spam-Status: No, hits=3.8 required=5.0 tests=BAYES_90,BIZ_TLD,HTML_70_80, HTML_MESSAGE,MIME_HTML_ONLY,MIME_HTML_ONLY_MULTI autolearn=no version=2.63 X-SA-Exim-Version: 4.0 (built Sun Apr 18 20:21:57 EDT 2004) X-SA-Exim-Scanned: Yes (on hostname) ----------------------------------------------------------------------------------- now running the same email saved with all headers spamassassin -t < spam ------------------------------------------------------------------ Reply-To: "zula heredia" From: "zula heredia" To: "Lisa" Date: Tue, 11 May 2004 21:06:45 +0600 MIME-Version: 1.0 (produced by spurgecensorious 5.9) Subject: Fwd: V^|cod|n & v|@grA ` Va+l+ium & XAN@X So|m|a ' Pnte.r.min umodhrnsojsa X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on hostname X-Spam-Level: ******* X-Spam-Status: Yes, hits=7.1 required=5.0 tests=BAYES_99,BIZ_TLD,HTML_70_80, HTML_MESSAGE,MIME_HTML_ONLY,MIME_HTML_ONLY_MULTI autolearn=no version=2.63 X-Spam-Report: * 0.1 HTML_MESSAGE BODY: HTML included in message * 5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100% * [score: 0.9993] * 0.1 HTML_70_80 BODY: Message is 70% to 80% HTML * 0.3 MIME_HTML_ONLY BODY: Message only has text/html MIME parts * 0.1 BIZ_TLD URI: Contains a URL in the BIZ top-level domain * 1.1 MIME_HTML_ONLY_MULTI Multipart message only has text/html MIME parts ... Spam detection software, running on the system "", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or block similar future email. If you have any questions, see spam_report@hostname for details. Content preview: livinghealthy.biz A Hi Lisa, [...] Content analysis details: (7.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.1 HTML_MESSAGE BODY: HTML included in message 5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100% [score: 0.9993] 0.1 HTML_70_80 BODY: Message is 70% to 80% HTML 0.3 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.1 BIZ_TLD URI: Contains a URL in the BIZ top-level domain 1.1 MIME_HTML_ONLY_MULTI Multipart message only has text/html MIME parts Thanks Bikesh From marc at merlins.org Tue May 11 13:13:15 2004 From: marc at merlins.org (Marc MERLIN) Date: Tue May 11 12:13:17 2004 Subject: [SA-exim] wont't detect spam correctly In-Reply-To: References: Message-ID: <20040511191315.GS19848@merlins.org> On Tue, May 11, 2004 at 01:44:32PM -0400, Bikesh Patel wrote: > > Folks, > I'm running SA-Exim as local_scan: > somehow a good amount is making through when running as localscan > and the same email is tagged spam correctly when running as transport Right. You need to fix your permissions or settings so that spamc/spamd has permissions to access and write to the baysian filtering database. > 5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100% > [score: 0.9993] Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From bikesh at tum.net Tue May 11 16:16:12 2004 From: bikesh at tum.net (The UNIX Mighty!) Date: Tue May 11 12:17:01 2004 Subject: [SA-exim] wont't detect spam correctly In-Reply-To: <20040511191315.GS19848@merlins.org> References: <20040511191315.GS19848@merlins.org> Message-ID: Everything is running as the as user and the files are owned by user mail mail 15076 14791 3 15:15:01 ? 0:01 /usr/local/bin/perl -T /usr/local/bin/spamd -u mail -a -d mail 14791 1 0 15:08:32 ? 0:03 /usr/local/bin/perl -T /usr/local/bin/spamd -u mail -a -d mail 15075 15071 0 15:15:01 ? 0:00 spamc -s 272384 Thanks Bikesh On Tue, 11 May 2004, Marc MERLIN wrote: > On Tue, May 11, 2004 at 01:44:32PM -0400, Bikesh Patel wrote: > > > > Folks, > > I'm running SA-Exim as local_scan: > > somehow a good amount is making through when running as localscan > > and the same email is tagged spam correctly when running as transport > > Right. > You need to fix your permissions or settings so that spamc/spamd has > permissions to access and write to the baysian filtering database. > > > 5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100% > > [score: 0.9993] > > Marc > -- > "A mouse is a device used to point at the xterm you want to type in" - A.S.R. > Microsoft is to operating systems & security .... > .... what McDonalds is to gourmet cooking > Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key > From marc at merlins.org Tue May 11 13:20:14 2004 From: marc at merlins.org (Marc MERLIN) Date: Tue May 11 12:20:16 2004 Subject: [SA-exim] wont't detect spam correctly In-Reply-To: References: <20040511191315.GS19848@merlins.org> Message-ID: <20040511192014.GT19848@merlins.org> On Tue, May 11, 2004 at 03:16:12PM -0400, The UNIX Mighty! wrote: > Everything is running as the as user and the files are owned by user mail > > mail 15076 14791 3 15:15:01 ? 0:01 /usr/local/bin/perl -T /usr/local/bin/spamd -u mail -a -d > mail 14791 1 0 15:08:32 ? 0:03 /usr/local/bin/perl -T /usr/local/bin/spamd -u mail -a -d > mail 15075 15071 0 15:15:01 ? 0:00 spamc -s 272384 I can't easily debug your setup from where I am. All I can give you is my setup: local.cf: auto_whitelist_path /var/spool/spamassassin/auto-whitelist auto_whitelist_file_mode 0666 bayes_path /var/spool/spamassassin/bayes bayes_file_mode 0666 magic:~# l /var/spool/spamassassin/ total 10628 drwxr-xr-x 3 nobody root 4096 May 11 12:17 ./ drwxr-xr-x 10 root root 4096 Nov 10 2003 ../ -rw-rw-rw- 1 nobody nogroup 102349 May 11 12:18 bayes_journal -rw-rw-rw- 1 nobody nogroup 5259264 May 11 12:17 bayes_seen -rw-rw-rw- 1 nobody nogroup 10682368 May 11 12:17 bayes_toks drwxrwxrwx 2 nobody nogroup 4096 Aug 2 2003 .pyzor/ magic:~# psg spamd nobody 7199 0.0 1.0 26408 10924 ? S Apr28 1:26 /usr/sbin/spamd -u nobody -m 64 -H /var/spool/spamassassin -d --pidfile=/var/run/spamd.pid Hopefully you'll be able to figure it out on your side Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From ross at biostat.ucsf.edu Tue May 11 13:21:23 2004 From: ross at biostat.ucsf.edu (Ross Boylan) Date: Tue May 11 12:21:48 2004 Subject: [SA-exim] Re: [Exim] EXIM 4.31, courier-imap, Clamd, exiscan, spamassassinLoad problems In-Reply-To: <3C5FE168-A2E5-11D8-9777-0030656CF512@slett.net> References: <000401c436e2$3f4bfd70$0405010a@efastfunding.com> <3C5FE168-A2E5-11D8-9777-0030656CF512@slett.net> Message-ID: <1084303283.4596.3.camel@iron.libaux.ucsf.edu> On Mon, 2004-05-10 at 17:50, Tor Slettnes wrote: > [Top-quoted, non-attributed message corrected. > See http://www.netmeister.org/news/learn2quote.html] > > On May 10, 2004, at 15:58, Doug Block wrote: > > Tor Slettnes wrote: > >> You probably mean "SA-Exim", not "SA-Scan". SA-Exim has problems > >> with Exim > >> v4.31 through 4.33, due to the Received: header change. Either stay > >> with > >> 4.30, or upgrade to 4.34, or you will get wildly inaccurate SA scores > >> (specifically, SA-Exim may trust forged Received: headers, or else > >> treat > >> messages sent through a smarthost as if they came directly from a > >> dialup > >> host). > > > > Yes your right!!! On the spam and the courier > > Oh the sa-exim is working great so far on the scores. I have had a > > 75% drop > > in spam and I have been using spam pit to check the scores which so > > far have > > been correct on both real mail and spam. I may upgrade to 4.34 once I > > get > > some time since it came out today. > > Keep in mind that your success is not only reflected in how much spam > you block, but more importantly, in how much legitimate mail you let > through. With that in mind, running SA-Exim with Exim 4.31 - 4.33 is > at best ignorant - at worst, irresponsible. Especially this is the > case if you are hosting mail for other people. > > (Sure, blocking 75% of spam is good; but with a properly configured SA, > you ought to be able to catch well above 90%, while not impacting > legitimate mail at all). > > > > I have a older (half the speed) exim 4 box that handled the load for > > up 150 users with courier-imap with out this problem but it did not > > have clamd, sa-exim, exiscan, and spamassassin on it. This box worked > > fine for 18 months+ but the virus's and spam where a big problem. I > > have spam set to under 256k but I have been watching this while > > writing this email and I notice 8-12 messages at a time coming in and > > eating 100% of the cpu for about 5 secs while they get spamd and > > clamd. > > You could start "spamd" with "--nicelevel 15" or so, essentially > lowering its priority if there are other (presumably more important) > tasks going on. (On a Debian machine, look in > /etc/default/spamassassin). > On my Debian machine (testing level) the following settings go in /etc/spamassassin/local.cf, I think. > You could disable the SA network tests (which, while using Exim 4.31 to > 4.33, probably do more harm than good): > skip_rbl_checks 1 > use_dcc 0 > use_pyzor 0 > use_razor2 0 Does anyone know if the use_xxx settings are necessary if skip_rbl_checks is 1? Also, the manpage lists use_bayes in the same section (Network Test Options). This seems odd. Should it be there? Is it just doing RBL tests? From LHendricks at austin.rr.com Wed May 12 00:46:45 2004 From: LHendricks at austin.rr.com (Lucas Hendricks) Date: Tue May 11 21:40:06 2004 Subject: [SA-exim] Can't get past make on SA-Exim 4.0 Message-ID: <001701c437dc$21800f70$e946fea9@lucas> Ok, I am taking a crash course in "linux" here (if you consider Cygwin linux) and having a really hard time of it. Could not even get through make on the exim that came WITH cygwin til I just shotgun-downloaded a bunch of libraries and got lucky. I thought I'd try this list before completely giving up on this. I am running the latest Cygwin on a w2k server (with netsec OFF that was a pain to have on). SpamAssassin is installed and working. I can install exim 4.32-1 fine, and the localscan_dlopen_exim_4.20_or_better.patch seems to run fine, but when I go to start building sa-exim it spams a bunch of undefined reference to: errors and then gives an error: ld returned. Specifically I get the below output: Building sa-exim-4.0.so gcc -I/usr/src/exim-4.32-1/src -I./eximinc -DDLOPEN_LOCAL_SCAN -DSPAMASSASSI N_CONF=\"/etc/sa-exim.conf\" -DSPAMC_LOCATION=\"/bin/spamc\" -O2 -Wall -shar ed -o sa-exim-4.0.so sa-exim.c sa-exim.c:492:2: warning: #warning you should not worry about the "might be clobbered by longjmp", see source /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x1bd):sa-exi m.c: undefined reference to `_log_write' /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x239):sa-exi m.c: undefined reference to `_string_copyn' /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x274):sa-exi m.c: undefined reference to `_log_write' /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x67b):sa-exi m.c: undefined reference to `_expand_string' ...(pages and pages of the above) /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x7b08):sa-ex im.c: undefined reference to `_sender_address' /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x7b11):sa-ex im.c: undefined reference to `_string_sprintf' /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x7b2f):sa-ex im.c: undefined reference to `_string_sprintf' /cygdrive/c/DOCUME~1/LHENDR~1/LOCALS~1/Temp/1/ccPMLXIz.o(.text+0x7b5b):sa-ex im.c: undefined reference to `_log_write' collect2: ld returned 1 exit status make: *** [sa-exim-4.0.so] Error 1 I'm not sure where it's getting that temp directory from or if it is related to the problem. Previously I had troubles because I had not installed all the correct libraries, but when I searched for all the includes in sa-exim.c they all seemed to be present so I'm pretty lost as to what might be causing the undefined refeference errors or even if they're related to the final failure. I tried running make -k to see if it was just the sa-exim.so and the access.so fails similarly (with only 1 of the above errors). I have not found anything about this on google groups or in the mail list archives so I'm either missing something obvious or its some goofy problem unique to my setup. I tried installing exiscan before this one and could not get past the patch (kept getting hunk failures). Thanks for any help or advice, I might look into sendmail instead, or just use the solution where SA is installed as a transport agent for exim if this doesnt work. Lucas From brian at enchanter.net Thu May 13 21:56:41 2004 From: brian at enchanter.net (Brian Kendig) Date: Thu May 13 17:56:47 2004 Subject: [SA-exim] local_scan is crashing on big messages Message-ID: <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> My uncle is trying to send me two emails with vacation pictures, and it's crashing sa-exim. I'm running Exim 4.32, SA-Exim 4.0, and SpamAssassin 2.63. I have sa-exim.c compiled directly into my Exim mail server; I'm not using it as a dynamic module. Exim's 'mainlog' shows these messages: 2004-05-13 03:42:18 HXN6Q2-000I42-2K local_scan() function crashed with signal 11 - message temporarily rejected (size 2402302) 2004-05-13 04:06:39 HXN7UG-000I46-P6 local_scan() function crashed with signal 11 - message temporarily rejected (size 3363716) 2004-05-13 04:45:02 HXN9MG-000I4D-K3 local_scan() function crashed with signal 11 - message temporarily rejected (size 3363716) 2004-05-13 04:54:47 HXNA2V-000I4H-FT local_scan() function crashed with signal 11 - message temporarily rejected (size 2402302) 2004-05-13 05:28:57 HXNBNT-000I4K-KI local_scan() function crashed with signal 11 - message temporarily rejected (size 2402302) 2004-05-13 06:06:33 HXNDEB-000I58-BC local_scan() function crashed with signal 11 - message temporarily rejected (size 3363716) 2004-05-13 06:32:06 HXNEL2-000I5B-1V local_scan() function crashed with signal 11 - message temporarily rejected (size 2402302) 2004-05-13 07:07:13 HXNG7G-000I5G-15 local_scan() function crashed with signal 11 - message temporarily rejected (size 3363716) 2004-05-13 07:33:54 HXNHG2-000I5M-NF local_scan() function crashed with signal 11 - message temporarily rejected (size 2402302) And so on and so forth. I'm guessing that 2402302 and 3363716 are the sizes of the two emails. Is this a known problem, a config issue, or can I provide any additional debugging info? - B From merlins.org at paulm.com Fri May 14 18:33:51 2004 From: merlins.org at paulm.com (Paul Makepeace) Date: Fri May 14 09:32:52 2004 Subject: [SA-exim] greylisting, exim-4.34 status Message-ID: <20040514163351.GJ20330@mythix.realprogrammers.com> Hi, I'm about to attempt an upgrade from my venerable and heavily hacked exim-4.05-VA-mm1 to the newer exim4 package with its split config. This is going to be fairly un-fun I suspect so in the interests of un-fun-ness reduction... Is a exim.deb/sa-exim.deb release imminent now that exim 4.34 has had the Received: header code backed out? If so I might hold off a while. Also, how have people found greylisting? Any thoughts on thresholds etc having used it for a while now? Thanks, Paul -- Paul Makepeace ................................ http://paulm.com/ecademy "If cars could fly, then jam will taste of carbon fibre." -- http://paulm.com/toys/surrealism/ From ssmeenk at freshdot.net Fri May 14 20:26:18 2004 From: ssmeenk at freshdot.net (Sander Smeenk) Date: Fri May 14 10:26:25 2004 Subject: [SA-exim] greylisting, exim-4.34 status In-Reply-To: <20040514163351.GJ20330@mythix.realprogrammers.com> References: <20040514163351.GJ20330@mythix.realprogrammers.com> Message-ID: <20040514172617.GA11781@freshdot.net> Quoting Paul Makepeace (merlins.org@paulm.com): > Is a exim.deb/sa-exim.deb release imminent now that exim 4.34 has had > the Received: header code backed out? If so I might hold off a while. I'm not planning on releasing a new sa-exim deb yet. The current version will work just right again with Exim 4.34. The change with Received: headers in Exim brake SpamAssassin's scoring, not sa-exim itself. > Also, how have people found greylisting? Any thoughts on thresholds etc > having used it for a while now? I still haven't tried. That's because Greylisting wasn't designed to work with two MX servers that don't share the same configuration files. I haven't had the time to set something up with sfs or so, so both can use the same Bayes and Greylisting information. (Or convince myself that only the primary MX server will do just fine). Regards, Sander. -- | Why does a kamikazepilot wear a helmet? | 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D From ssmeenk at freshdot.net Fri May 14 20:34:34 2004 From: ssmeenk at freshdot.net (Sander Smeenk) Date: Fri May 14 10:34:38 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> References: <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> Message-ID: <20040514173434.GB11781@freshdot.net> Quoting Brian Kendig (brian@enchanter.net): > I'm running Exim 4.32, SA-Exim 4.0, and SpamAssassin 2.63. You should consider updating to 4.34 because 4.3{2,3} has a problem with Received: headers that are missing when the mail is fed to SpamAssassin. This has nothing to do with your current problem, afaik, but it is important because it might cause legit mail to be rejected. > I have sa-exim.c compiled directly into my Exim mail server; I'm not > using it as a dynamic module. Any real good reason not to use the dynamic module? It's much easier to use the dynamic module, you can enable / disable it at will, and it doesn't matter in performance. > Exim's 'mainlog' shows these messages: > 2004-05-13 03:42:18 HXN6Q2-000I42-2K local_scan() function crashed with > signal 11 - message temporarily rejected (size 2402302) Is there anything related to these ID's in the rejectlog and/or paniclog? I'm not sure what distribution you are using, but spamc/spamd might also log messages to /var/log/{syslog,messages,mail.log}. > And so on and so forth. I'm guessing that 2402302 and 3363716 are the > sizes of the two emails. They most probably are. I haven't checked, but I can't think of any other size to display in such an error message ;) > Is this a known problem, a config issue, or can I provide any > additional debugging info? Tried sending a large message to yourself? Updating exim, spamassassin and/or sa-exim? Tried running spamc by hand on a very large file? Try running exim4 in debug mode, strace -f the process, etc, etc. :) Sander. -- | Daylight savings time - why are they saving it and where do they keep it? | 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D From brian at enchanter.net Fri May 14 15:09:49 2004 From: brian at enchanter.net (Brian Kendig) Date: Fri May 14 11:09:55 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: <20040514173434.GB11781@freshdot.net> References: <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> Message-ID: > You should consider updating to 4.34 because 4.3{2,3} has a problem > with Received: headers that are missing when the mail is fed to > SpamAssassin. Thanks for the tip. I found 4.34 on the download site, but the main Exim page still says that 4.32 is the latest version - is the Exim home page wrong, or is 4.34 a pre-release version? > Any real good reason not to use the dynamic module? It's much easier to > use the dynamic module, you can enable / disable it at will, and it > doesn't matter in performance. I'm using Mac OS X, and a year or so ago when I tried to use the SA-Exim dynamic module, it wouldn't work. I seem to remember there was a library missing, and Marc said that Mac OS X didn't appear to support dynamic modules yet. I don't remember the details. I'll try it again after I get this problem fixed. >> Exim's 'mainlog' shows these messages: >> 2004-05-13 03:42:18 HXN6Q2-000I42-2K local_scan() function crashed >> with >> signal 11 - message temporarily rejected (size 2402302) > > Is there anything related to these ID's in the rejectlog and/or > paniclog? > I'm not sure what distribution you are using, but spamc/spamd might > also > log messages to /var/log/{syslog,messages,mail.log}. The same exact lines appear in both the mainlog and the rejectlog, but nothing relevant appears in the paniclog: # grep HXP86L-000KXM-2Y mainlog rejectlog paniclog mainlog:2004-05-14 06:08:58 HXP86L-000KXM-2Y local_scan() function crashed with signal 11 - message temporarily rejected (size 2402302) rejectlog:2004-05-14 06:08:58 HXP86L-000KXM-2Y local_scan() function crashed with signal 11 - message temporarily rejected (size 2402302) > Tried sending a large message to yourself? Updating exim, spamassassin > and/or sa-exim? Sending a large message to myself also fails in the same way with local_scan() crashing. I'm using the latest releases of SpamAssassin and SA-Exim, and what I thought was the latest release of Exim; I could move to the pre-release SpamAssassin 3.0, but I'd rather not be on the bleeding edge. > Tried running spamc by hand on a very large file? That just sends the file back to me via stdout. If I give spamc the -R option, it reports "0/0", which I believe means an error occurred, but I think this is because I don't know the right format of the message to feed to spamc. At any rate, spamc isn't crashing. > Try running exim4 in debug mode, strace -f the process, etc, etc. :) Not yet, but I'll give that a try. Thanks for the tips. From brian at enchanter.net Fri May 14 18:06:57 2004 From: brian at enchanter.net (Brian Kendig) Date: Fri May 14 14:07:04 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: <20040514173434.GB11781@freshdot.net> References: <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> Message-ID: Okay, I've upgraded to Exim 4.34, still with SA-Exim 4.0 compiled in, still running against SpamAssassin 2.63. When I send a large message to myself (I tried a 3.4MB file attachment), local_scan still crashes. I ran exim with the -d option, sent the large message, and captured the debug output. It's too long to copy here, so I put the whole thing on my web server at 'http://www.enchanter.net/eximdebug.txt', but here's the relevant excerpt: 28653 calling local_scan(); timeout=300 28653 LOG: MAIN REJECT 28653 local_scan() function crashed with signal 11 - message temporarily rejected (size 4892599) 28653 SMTP>> 421 enchanter.net local verification problem - closing connection. 28653 search_tidyup called 28653 >>>>>>>>>>>>>>>> Exim pid=28653 terminating with rc=1 >>>>>>>>>>>>>>>> 28651 child 28653 ended: status=0x100 28651 0 SMTP accept processes now running 28651 Listening... Mac OS X doesn't have strace, but it does have ktrace, which is similar: # ktrace -p `cat /var/spool/exim/exim-daemon.pid` -d (at this point I sent the test message, then turned tracing off and looked at the log) # ktrace -C # kdump -f ktrace.out 28691 exim-4.34-1 RET select 1 28691 exim-4.34-1 CALL accept(0,0xbffff3a0,0xbffff478) 28691 exim-4.34-1 RET accept 2 28691 exim-4.34-1 CALL getrlimit(0x8,0xbffff1f0) 28691 exim-4.34-1 RET getrlimit 0 28691 exim-4.34-1 CALL fcntl(0x2,0x3,0) 28691 exim-4.34-1 RET fcntl 2 28691 exim-4.34-1 CALL dup(0x2) 28691 exim-4.34-1 RET dup 4 28691 exim-4.34-1 CALL fcntl(0x4,0x3,0) 28691 exim-4.34-1 RET fcntl 2 28691 exim-4.34-1 CALL getsockname(0x2,0xbffff2c0,0xbffff2f0) 28691 exim-4.34-1 RET getsockname 0 28691 exim-4.34-1 CALL fork 28691 exim-4.34-1 RET fork 28701/0x701d 28691 exim-4.34-1 CALL close(0x2) 28691 exim-4.34-1 RET close 0 28691 exim-4.34-1 CALL close(0x4) 28691 exim-4.34-1 RET close 0 28691 exim-4.34-1 CALL wait4(0xffffffff,0xbffff47c,0x1,0) 28691 exim-4.34-1 RET wait4 0 28691 exim-4.34-1 CALL select(0x1,0xbffff3b0,0,0,0) 28691 exim-4.34-1 RET select -1 errno 4 Interrupted system call 28691 exim-4.34-1 PSIG SIGCHLD caught handler=0x5dfc mask=0x0 code=0x0 28691 exim-4.34-1 CALL sigaction(0x14,0xbfffec60,0xbfffecd0) 28691 exim-4.34-1 RET sigaction 0 28691 exim-4.34-1 CALL #184(0xbfffee58,0x1) 28691 exim-4.34-1 RET #184 JUSTRETURN 28691 exim-4.34-1 CALL wait4(0xffffffff,0xbffff47c,0x1,0) 28691 exim-4.34-1 RET wait4 28701/0x701d 28691 exim-4.34-1 CALL wait4(0xffffffff,0xbffff47c,0x1,0) 28691 exim-4.34-1 RET wait4 -1 errno 10 No child processes 28691 exim-4.34-1 CALL sigaction(0x14,0xbffff2d0,0xbffff340) 28691 exim-4.34-1 RET sigaction 0 28691 exim-4.34-1 CALL select(0x1,0xbffff3b0,0,0,0) The mainlog only logs that one "local_scan() function crashed" line, but I found out that rejectlog logs more, though it's not particularly useful: 2004-05-14 17:01:29 HXQ2EE-000M59-7U local_scan() function crashed with signal 11 - message temporarily rejected (size 4892721) Envelope-from: Envelope-to: P Received: from calypso.enchanter.net ([10.0.1.14]) by enchanter.net with esmtp (Exim 4.34) id HXQ2EE-000M59-7U for brian@enchanter.net; Fri, 14 May 2004 17:01:28 -0400 Mime-Version: 1.0 (Apple Message framework v613) T To: Brian Kendig I Message-Id: Content-Type: multipart/mixed; boundary=Apple-Mail-4-1034587599 F From: Brian Kendig Subject: test Date: Fri, 14 May 2004 17:01:26 -0400 X-Mailer: Apple Mail (2.613) X-SA-Exim-Connect-IP: 10.0.1.14 X-SA-Exim-Rcpt-To: brian@enchanter.net X-SA-Exim-Mail-From: brian@enchanter.net If you have any other ideas for things I could try, please let me know! - B From ssmeenk at freshdot.net Sat May 15 15:16:27 2004 From: ssmeenk at freshdot.net (Sander Smeenk) Date: Sat May 15 05:16:36 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: References: <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> Message-ID: <20040515121627.GA9093@freshdot.net> Quoting Brian Kendig (brian@enchanter.net): > 28653 calling local_scan(); timeout=300 > 28653 LOG: MAIN REJECT > 28653 local_scan() function crashed with signal 11 - message > temporarily rejected (size 4892599) Nothig special here :( > # kdump -f ktrace.out > 28691 exim-4.34-1 RET select 1 > 28691 exim-4.34-1 CALL select(0x1,0xbffff3b0,0,0,0) Too bad I can't see anything that indicates a SIGSEGV in this trace. I bet you know strace, it's easy to point out where the program segfaulted. > The mainlog only logs that one "local_scan() function crashed" line, > but I found out that rejectlog logs more, though it's not particularly > useful: rejectlog always logs the full headers of the message it was processing when it gets rejected. > If you have any other ideas for things I could try, please let me know! I'm begining to wonder if the local_scan patch you used still works that good with Exim 4.34. IIRC it was developed for 4.20, and thereafter never updated. I'm not sure, Marc could jump in here. Regards, Sander. -- | It is easier to get forgiveness than permission. | 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D From marc at merlins.org Sat May 15 11:44:11 2004 From: marc at merlins.org (Marc MERLIN) Date: Sat May 15 10:44:13 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: <20040515121627.GA9093@freshdot.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> References: <20040515121627.GA9093@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> Message-ID: <20040515174411.GH12624@merlins.org> (back from a quick trip and home with a good cold, bear with me if the temperature makes me say stupid things :) On Thu, May 13, 2004 at 08:56:41PM -0400, Brian Kendig wrote: > My uncle is trying to send me two emails with vacation pictures, and > it's crashing sa-exim. > > I'm running Exim 4.32, SA-Exim 4.0, and SpamAssassin 2.63. I have > sa-exim.c compiled directly into my Exim mail server; I'm not using it > as a dynamic module. Which is fine. The dynamic module is more flexible, but patching exim directly is less complex. > Is this a known problem, a config issue, or can I provide any > additional debugging info? Yep, set SAEximDebug: to 10 in sa-exim. On Fri, May 14, 2004 at 02:09:49PM -0400, Brian Kendig wrote: > I'm using Mac OS X, and a year or so ago when I tried to use the > SA-Exim dynamic module, it wouldn't work. I seem to remember there was > a library missing, and Marc said that Mac OS X didn't appear to support > dynamic modules yet. I don't remember the details. I'll try it again > after I get this problem fixed. More specifically, I must have said that I didn't have the OS dependent knowledge to help debug dynamic linking on other unixes. I'm sure it works on MacOS X, but it might require a patch to the makefile or something. > >Tried sending a large message to yourself? Updating exim, spamassassin > >and/or sa-exim? > > Sending a large message to myself also fails in the same way with > local_scan() crashing. I'm using the latest releases of SpamAssassin Ok, that's great news, at least you can reproduce at will, it will help a lot. See what you get with a debuglevel of 10 > and SA-Exim, and what I thought was the latest release of Exim; I could > move to the pre-release SpamAssassin 3.0, but I'd rather not be on the > bleeding edge. It doesn't matter sa-exim should never crash, regardless of what SA you are using. Also, please send me your sa-exim.conf by private mail to see if somehow I can reproduce on my side (since I haven't had sa-exim crash on my system on more than a year, regardless of the headers, or huge messages that people might send me) On Fri, May 14, 2004 at 05:06:57PM -0400, Brian Kendig wrote: > Mac OS X doesn't have strace, but it does have ktrace, which is similar: > > # ktrace -p `cat /var/spool/exim/exim-daemon.pid` -d > > (at this point I sent the test message, then turned tracing off and > looked at the log) Yeah, that's not extremely useful. Hopefully debuglevel will help On Sat, May 15, 2004 at 02:16:27PM +0200, Sander Smeenk wrote: > > If you have any other ideas for things I could try, please let me know! > > I'm begining to wonder if the local_scan patch you used still works that > good with Exim 4.34. IIRC it was developed for 4.20, and thereafter > never updated. I'm not sure, Marc could jump in here. It should work, the API is supposed to be stable, but eh, something could have changed that I don't know about yet. Also, if it were a change in the local_scan API 1) it would most likely crash on all messages 2) it should also crash on the dynamically loadable local_scan Marc PS: I know I need to put out a new sa-exim since I have a few patches outstanding. Hopefully I'll get to do that after sorting this issue out -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From marc at merlins.org Sat May 15 12:22:57 2004 From: marc at merlins.org (Marc MERLIN) Date: Sat May 15 11:23:00 2004 Subject: [SA-exim] greylisting, exim-4.34 status In-Reply-To: <20040514172617.GA11781@freshdot.net> <20040514163351.GJ20330@mythix.realprogrammers.com> References: <20040514163351.GJ20330@mythix.realprogrammers.com> <20040514172617.GA11781@freshdot.net> <20040514163351.GJ20330@mythix.realprogrammers.com> Message-ID: <20040515182257.GI12624@merlins.org> On Fri, May 14, 2004 at 05:33:51PM +0100, Paul Makepeace wrote: > Hi, > > I'm about to attempt an upgrade from my venerable and heavily hacked > exim-4.05-VA-mm1 to the newer exim4 package with its split config. Whao, I didn't know anyone was still running that :) > Is a exim.deb/sa-exim.deb release imminent now that exim 4.34 has had > the Received: header code backed out? If so I might hold off a while. You don't need a new release, the current code should work fine. For that matter, now that exim 4.34 is in unstable, I just upgraded and things look fine. I also doublechecked that Philip's fix in 4.34 works fine, I indeed receive the last received line. I'm now running exim 4.34 and the current sa-exim on my main mail server without any problems. > Also, how have people found greylisting? Any thoughts on thresholds etc > having used it for a while now? The one I give in the docs really work great for me: tempreject at 3, permreject at 11, and greylisting lowers the score by 8 between the temp lower and the SA greylisted score On Fri, May 14, 2004 at 07:26:18PM +0200, Sander Smeenk wrote: > That's because Greylisting wasn't designed to work with two MX servers > that don't share the same configuration files. I haven't had the time Actually greylisting would work there too. > to set something up with sfs or so, so both can use the same Bayes and > Greylisting information. (Or convince myself that only the primary MX > server will do just fine). Yeah, things will work fine. 1) most mail will go through just fine 2) greylisted mail that is spam will go to your secondary MX, unless it was already sent there to start with 3) secondary MX will also greylist sender 4) if sender resends a 2nd or 3rd time depending on above, the mail will be accepted 5) ... 6) profit (*) Mmmh, actually not quite, I should build a greylist hack so that if /var/spool/sa-exim/tuplets/12/174/92/all or /var/spool/sa-exim/tuplets/12/174/92/all/all exit, then the mail is whitelisted automatically This would allow you to accept all mail from your secondary MX without greylisting it one more time (things work as is mind you, but in some cases you would needlessly delay mail by one extra hour) In the meantime, you can also setup a spamassassin header rule that matches the IP of your secondary MX and lowers the SA score enough to allow the mail through Marc (*) sorry, gotta watch South Park to get that one :) -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From marc at merlins.org Sat May 15 12:44:17 2004 From: marc at merlins.org (Marc MERLIN) Date: Sat May 15 11:44:18 2004 Subject: [SA-exim] SA run cond question In-Reply-To: <40746B01.10500@sc.younglife.org> References: <40746B01.10500@sc.younglife.org> Message-ID: <20040515184417.GM12624@merlins.org> On Wed, Apr 07, 2004 at 02:56:33PM -0600, Tim Sexton wrote: > Hello All, > I'm having some trouble getting sa-exim to quit scanning mail from a > trusted network. I have tried toying with both the spamassassin > trusted_networks parameter in the local.cf and changing the sa-exim run > condition. Neither parameter seems to make any difference. > > here is my sa-exim run condition: SAEximRunCond: ${if and > {{def:sender_host_address} {!eq {$sender_host_address}{172.16.206.143}} > {!eq {$sender_host_address}{127.0.0. > 1}}{!eq {$h_X-SA-Do-Not-Run:}{Yes}} } {1}{0}}SAEximRunCond: ${if and > {{def:sender_host_address} {!eq {$sender_host_address}{172.16.206.143}} > {!eq {$sender_host_address}{127.0.0. > 1}}{!eq {$h_X-SA-Do-Not-Run:}{Yes}} } {1}{0}} That looks like a double paste, but basically, it should work. Outside of not having restarted exim, I'm not sure why it wouldn't work for > Does anyone have any idea why my sa-exim run condition isn't picking up > the 172.16.206.143 address? Honestly, no Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From Nigel.Metheringham at dev.InTechnology.co.uk Mon May 17 13:46:28 2004 From: Nigel.Metheringham at dev.InTechnology.co.uk (Nigel Metheringham) Date: Mon May 17 04:46:39 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: References: <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> Message-ID: <1084794388.8077.56.camel@angua.localnet> On Fri, 2004-05-14 at 19:09, Brian Kendig wrote: > > You should consider updating to 4.34 because 4.3{2,3} has a problem > > with Received: headers that are missing when the mail is fed to > > SpamAssassin. > > Thanks for the tip. I found 4.34 on the download site, but the main > Exim page still says that 4.32 is the latest version - is the Exim home > page wrong, or is 4.34 a pre-release version? Sourceforge appear to have broken their CVS stuff and the exim WWW pages were kept in SF CVS and updated to the webserver that way. In any case the website is separately maintained from exim itself and is done on a best efforts basis - so if I am busy on other stuff it can lag for a few weeks. This will almost certainly change when we transfer to the new infrastructure at cambridge. Nigel. -- [ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ] From brian at enchanter.net Mon May 17 09:37:50 2004 From: brian at enchanter.net (Brian Kendig) Date: Mon May 17 05:37:58 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: <20040515174411.GH12624@merlins.org> References: <20040515121627.GA9093@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040515174411.GH12624@merlins.org> Message-ID: <025C8B5C-A7FF-11D8-A8EB-000A9595347C@enchanter.net> On May 15, 2004, at 1:44 PM, Marc MERLIN wrote: > Ok, that's great news, at least you can reproduce at will, it will > help a > lot. > See what you get with a debuglevel of 10 Here's what I get in the mainlog with SAEximDebug set to 10: http://www.enchanter.net/sa-exim.debug The relevant lines appear to be these; I'll bet it shouldn't be reporting a message body of zero bytes: SA: Debug4: Message body is about 0 bytes and the initial offset is 4892260 SA: Debug: SATruncBodyCond expand returned: '0' local_scan() function crashed with signal 11 - message temporarily rejected (size 4892599) And here's my config file (after I set SAEximDebug back to 0): http://www.enchanter.net/sa-exim.conf Thanks for looking into this! Let me know if there's any more info I can provide. - B From merlins.org at paulm.com Tue May 18 02:32:33 2004 From: merlins.org at paulm.com (Paul Makepeace) Date: Mon May 17 17:31:28 2004 Subject: [SA-exim] greylisting, exim-4.34 status In-Reply-To: <20040515182257.GI12624@merlins.org> References: <20040514163351.GJ20330@mythix.realprogrammers.com> <20040514172617.GA11781@freshdot.net> <20040514163351.GJ20330@mythix.realprogrammers.com> <20040515182257.GI12624@merlins.org> Message-ID: <20040518003233.GA20733@mythix.realprogrammers.com> Je 2004-05-15 19:22:57 +0100, Marc MERLIN skribis: > Yeah, things will work fine. > 1) most mail will go through just fine > 2) greylisted mail that is spam will go to your secondary MX, unless it was > already sent there to start with > 3) secondary MX will also greylist sender > 4) if sender resends a 2nd or 3rd time depending on above, the mail will > be accepted > 5) ... > 6) profit (*) > > Mmmh, actually not quite, I should build a greylist hack so that if > /var/spool/sa-exim/tuplets/12/174/92/all > or > /var/spool/sa-exim/tuplets/12/174/92/all/all > exit, then the mail is whitelisted automatically Does SAEximRunCond not enable you to bypass SA when taking connections from your MXen? (I have for example, {!eq {$sender_host_address}{217.207.14.60}}in the 'and' condition.) > This would allow you to accept all mail from your secondary MX without > greylisting it one more time (things work as is mind you, but in some cases > you would needlessly delay mail by one extra hour) So SAEximRunCond could solve exactly this, working as a whitelist for the greylisting? Paul -- Paul Makepeace ................................ http://paulm.com/ecademy "What is Mr Oh's mission? A melding of two souls." -- http://paulm.com/toys/surrealism/ From marc at merlins.org Mon May 17 18:36:53 2004 From: marc at merlins.org (Marc MERLIN) Date: Mon May 17 17:36:55 2004 Subject: [SA-exim] local_scan is crashing on big messages In-Reply-To: <025C8B5C-A7FF-11D8-A8EB-000A9595347C@enchanter.net> References: <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040514173434.GB11781@freshdot.net> <8FFC3692-A541-11D8-98AE-000A9595347C@enchanter.net> <20040515174411.GH12624@merlins.org> <025C8B5C-A7FF-11D8-A8EB-000A9595347C@enchanter.net> Message-ID: <20040518003652.GH30824@merlins.org> On Mon, May 17, 2004 at 08:37:50AM -0400, Brian Kendig wrote: > On May 15, 2004, at 1:44 PM, Marc MERLIN wrote: > >Ok, that's great news, at least you can reproduce at will, it will > >help a > >lot. > >See what you get with a debuglevel of 10 > > Here's what I get in the mainlog with SAEximDebug set to 10: > > http://www.enchanter.net/sa-exim.debug > > The relevant lines appear to be these; I'll bet it shouldn't be > reporting a message body of zero bytes: > > SA: Debug4: Message body is about 0 bytes and the initial offset is > 4892260 > SA: Debug: SATruncBodyCond expand returned: '0' > local_scan() function crashed with signal 11 - message temporarily > rejected (size 4892599) Mmmh, interesting, so the code is this: > if (SATruncBodyCond[0] != '1' || SATruncBodyCond[1] != 0) > { > expand=expand_string(SATruncBodyCond); > if (expand == NULL) > { > PANIC(string_sprintf("SATruncBodyCond expansion failure on %s", SATruncBodyCond)); > } > > if (SAEximDebug) > { > log_write(0, LOG_MAIN, "SA: Debug: SATruncBodyCond expand returned: '%s'", expand); > } it worked up to here, but add this line here: log_write(0, LOG_MAIN, "SA: Not Dead 0"); > if (expand[0] == 0 || (expand[0] == '0' && expand[1] == 0)) this most likely worked, but the new log_writes will tell would the following have crashed? Try adding some debugs to check: > { log_write(0, LOG_MAIN, "SA: Not Dead 1"); log_write(0, LOG_MAIN, "SA: size: %d", fdsize-18); log_write(0, LOG_MAIN, "SA: mesgid: %s", safemesgid); log_write(0, LOG_MAIN, "SA: mailinfo: %s", mailinfo); log_write(0, LOG_MAIN, "SA: Not Dead 2"); > log_write(0, LOG_MAIN, "SA: Action: check skipped due to message size (%d bytes) and SATruncBodyCond expanded to false (Message-Id: %s). %s", fdsize-18, safemesgid, mailinfo); log_write(0, LOG_MAIN, "SA: Not Dead 3"); > header_add(' ', "X-SA-Exim-Scanned: No (on %s); Message bigger than SAmaxbody (%d)\n", primary_hostname, SAmaxbody); log_write(0, LOG_MAIN, "SA: Not Dead 4"); > return LOCAL_SCAN_ACCEPT; > } > } But by just looking at the code, I'm not quite sure why it'd crash there. If you don't see "SA: Not Dead 0" and "...1", then I'll be very confused. Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From marc at merlins.org Mon May 17 18:38:10 2004 From: marc at merlins.org (Marc MERLIN) Date: Mon May 17 17:38:12 2004 Subject: [SA-exim] greylisting, exim-4.34 status In-Reply-To: <20040518003233.GA20733@mythix.realprogrammers.com> References: <20040514163351.GJ20330@mythix.realprogrammers.com> <20040514172617.GA11781@freshdot.net> <20040514163351.GJ20330@mythix.realprogrammers.com> <20040515182257.GI12624@merlins.org> <20040518003233.GA20733@mythix.realprogrammers.com> Message-ID: <20040518003810.GI30824@merlins.org> On Tue, May 18, 2004 at 01:32:33AM +0100, Paul Makepeace wrote: > Je 2004-05-15 19:22:57 +0100, Marc MERLIN skribis: > > Yeah, things will work fine. > > 1) most mail will go through just fine > > 2) greylisted mail that is spam will go to your secondary MX, unless it was > > already sent there to start with > > 3) secondary MX will also greylist sender > > 4) if sender resends a 2nd or 3rd time depending on above, the mail will > > be accepted > > 5) ... > > 6) profit (*) > > > > Mmmh, actually not quite, I should build a greylist hack so that if > > /var/spool/sa-exim/tuplets/12/174/92/all > > or > > /var/spool/sa-exim/tuplets/12/174/92/all/all > > exit, then the mail is whitelisted automatically > > Does SAEximRunCond not enable you to bypass SA when taking connections > from your MXen? (I have for example, {!eq > {$sender_host_address}{217.207.14.60}}in the 'and' condition.) Sorry, my fever was making me say stupid things. Yes, of course, you shouldn't run SA when receiving mail from your secondary MXes, which solves the greylisting problem too. > So SAEximRunCond could solve exactly this, working as a whitelist for > the greylisting? Right Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From merlins.org at paulm.com Tue May 18 02:45:19 2004 From: merlins.org at paulm.com (Paul Makepeace) Date: Mon May 17 17:44:12 2004 Subject: [SA-exim] greylisting, exim-4.34 status In-Reply-To: <20040518003810.GI30824@merlins.org> References: <20040514163351.GJ20330@mythix.realprogrammers.com> <20040514172617.GA11781@freshdot.net> <20040514163351.GJ20330@mythix.realprogrammers.com> <20040515182257.GI12624@merlins.org> <20040518003233.GA20733@mythix.realprogrammers.com> <20040518003810.GI30824@merlins.org> Message-ID: <20040518004519.GB20733@mythix.realprogrammers.com> Je 2004-05-18 01:38:10 +0100, Marc MERLIN skribis: > On Tue, May 18, 2004 at 01:32:33AM +0100, Paul Makepeace wrote: > > Je 2004-05-15 19:22:57 +0100, Marc MERLIN skribis: > > > Yeah, things will work fine. > > > 1) most mail will go through just fine > > > 2) greylisted mail that is spam will go to your secondary MX, unless it was > > > already sent there to start with > > > 3) secondary MX will also greylist sender > > > 4) if sender resends a 2nd or 3rd time depending on above, the mail will > > > be accepted > > > 5) ... > > > 6) profit (*) > > > > > > Mmmh, actually not quite, I should build a greylist hack so that if > > > /var/spool/sa-exim/tuplets/12/174/92/all > > > or > > > /var/spool/sa-exim/tuplets/12/174/92/all/all > > > exit, then the mail is whitelisted automatically > > > > Does SAEximRunCond not enable you to bypass SA when taking connections > > from your MXen? (I have for example, {!eq > > {$sender_host_address}{217.207.14.60}}in the 'and' condition.) > > Sorry, my fever was making me say stupid things. Yes, of course, you > shouldn't run SA when receiving mail from your secondary MXes, which solves > the greylisting problem too. If it's any consolation, it took me several days before this thought popped into my head... Might be worth mentioning it in the README? "If you want to whitelist some hosts from being greylisted..." Those expansions are performed in the context of the executing exim, right? It would be nice to be able to create a hostlist and check $sender_host_address against them. I can never quite remember exim syntax long enough to do this kind of thing myself ;-) Anyone? Paul -- Paul Makepeace ................................ http://paulm.com/ecademy "What is position if there ain't no contortin'? The longer you can you can't." -- http://paulm.com/toys/surrealism/ From marc at merlins.org Tue May 18 14:58:21 2004 From: marc at merlins.org (Marc MERLIN) Date: Tue May 18 13:58:24 2004 Subject: [SA-exim] greylisting, exim-4.34 status In-Reply-To: <40AA7801.6030001@lithvall.se> References: <20040514163351.GJ20330@mythix.realprogrammers.com> <20040514172617.GA11781@freshdot.net> <20040514163351.GJ20330@mythix.realprogrammers.com> <20040515182257.GI12624@merlins.org> <20040518003233.GA20733@mythix.realprogrammers.com> <20040518003810.GI30824@merlins.org> <40AA7801.6030001@lithvall.se> Message-ID: <20040518205821.GS30824@merlins.org> On Tue, May 18, 2004 at 10:54:25PM +0200, Richard Lithvall wrote: > On 2004-05-18 02:38, Marc MERLIN wrote: > > > you shouldn't run SA when receiving mail from your secondary MXes > > Ofcourse you should, but you may not want to reject any mail from your > secondary MXes. Sorry, I forgot to say that I was assuming your secondary MX was running sa-exim too. If all your MXes aren't running sa-exim with grelylisting, then you can't really use greylisting (the discussion at hand) Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From marc at merlins.org Thu May 20 14:05:45 2004 From: marc at merlins.org (Marc MERLIN) Date: Thu May 20 13:05:49 2004 Subject: [SA-exim] greylisting, exim-4.34 status In-Reply-To: <20040518004519.GB20733@mythix.realprogrammers.com> References: <20040514163351.GJ20330@mythix.realprogrammers.com> <20040514172617.GA11781@freshdot.net> <20040514163351.GJ20330@mythix.realprogrammers.com> <20040515182257.GI12624@merlins.org> <20040518003233.GA20733@mythix.realprogrammers.com> <20040518003810.GI30824@merlins.org> <20040518004519.GB20733@mythix.realprogrammers.com> Message-ID: <20040520200544.GH2924@merlins.org> On Tue, May 18, 2004 at 01:45:19AM +0100, Paul Makepeace wrote: > If it's any consolation, it took me several days before this thought > popped into my head... Might be worth mentioning it in the README? "If > you want to whitelist some hosts from being greylisted..." Yeah, I'll put more details. > Those expansions are performed in the context of the executing exim, > right? It would be nice to be able to create a hostlist and check > $sender_host_address against them. I can never quite remember exim > syntax long enough to do this kind of thing myself ;-) Anyone? I'm not sure sa-exim has access to that from within the process, but maybe it does, I never tried. Let me know how it goes :) Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From merlins.org at paulm.com Fri May 21 13:16:41 2004 From: merlins.org at paulm.com (Paul Makepeace) Date: Fri May 21 04:15:48 2004 Subject: [SA-exim] Greylisting mails with URLs Message-ID: <20040521111641.GZ29101@mythix.realprogrammers.com> I'm going to toss this idea out there without even looking at the code just in case anyone wants to pick this up. Pretty much the only spams getting through here at this point are those that really do not look like spam and contain just a URL ("click here for more www.medserver.b1z"). Now, there are look-ups based on domain name/URI e.g. http://www.surbl.org/ ; (heck I even wrote one, http://sdbl.org/ ) but they take time to get reported. So - I would like to have emails that contain a URL or even (un-whitelisted) domain name in the body of the mail to be deferred for some time during which time one hopes the domain'll end up on spamcop. http://www.spamcop.net/w3m?action=inprogress&type=www which is then fed or in my own list. Roughly: something to tag the email with a "URL/domain of some description whatever detected" and then in the GL code look for that domain and possibly use that as an additional or parallel GL parameter along with envelope and host info. Paul -- Paul Makepeace ................................ http://paulm.com/ecademy "If I caught my jism in my mouth, then I would let your dog fuck me in the ass!" -- http://paulm.com/toys/surrealism/ From merlins.org at paulm.com Sat May 22 17:39:25 2004 From: merlins.org at paulm.com (Paul Makepeace) Date: Sat May 22 08:38:31 2004 Subject: [SA-exim] spamassassin's home dir Message-ID: <20040522153925.GY29101@mythix.realprogrammers.com> I'm sure this is obvious... I've noticed spamassassin is writing its various bits to /var/spool/exim4/.spamassassin which is Debian-exim's home dir. How is it getting this? spamd is started and indeed runs as root via /etc/init.d/spamassassin. So is it picking up the $HOME via spamc called via local_scan in exim? ** To me there is something wrong-feeling about this (probably because it does actually violate FHS :-) so I did: mkdir /var/lib/spamassassin chown -R Debian-exim:Debian-exim /var/lib/spamassassin Changes to /etc/spamassassin/local.cf: bayes_path /var/lib/spamassassin/bayes auto_whitelist_path /var/lib/spamassassin This still leaves ~Debian-exim/.spamassassin/user_prefs being auto- created which can be stopped by taking out the -c option from /etc/default/spamassassin Paul -- Paul Makepeace ................................ http://paulm.com/ecademy "If causes and effects were interchangeable, then the sun would light up your face." -- http://paulm.com/toys/surrealism/ From merlins.org at paulm.com Sat May 22 17:47:44 2004 From: merlins.org at paulm.com (Paul Makepeace) Date: Sat May 22 08:46:39 2004 Subject: [SA-exim] Bayes tweaks Message-ID: <20040522154744.GZ29101@mythix.realprogrammers.com> Actually RTFM I noticed the bayes_ignore_header option: bayes_ignore_header header_name If you receive mail filtered by upstream mail systems, like a spam- filtering ISP or mailing list, and that service adds new headers (as most of them do), these headers may provide inappropriate cues to the Bayesian classifier, allowing it to take a "short cut". To avoid this, list the headers using this setting. Example: bayes_ignore_header X-Upstream-Spamfilter bayes_ignore_header X-Upstream-SomethingElse Which prompted me to wonder about, bayes_ignore_header X-SA-Exim-Connect-IP bayes_ignore_header X-SA-Exim-Mail-From bayes_ignore_header X-SA-Exim-Rcpt-To bayes_ignore_header X-SA-Exim-Scanned bayes_ignore_header X-SA-Exim-Version I can't decide whether this is worthwhile or not; there might actually be bayes-classifiably useful info in there. Any thoughts? HTH, Paul -- Paul Makepeace ................................ http://paulm.com/ecademy "What is my difficulty with Perl? Fourteenth Century German Poetry." -- http://paulm.com/toys/surrealism/ From merlins.org at paulm.com Sat May 22 18:01:41 2004 From: merlins.org at paulm.com (Paul Makepeace) Date: Sat May 22 09:00:37 2004 Subject: [SA-exim] Improving base SA Message-ID: <20040522160141.GA29101@mythix.realprogrammers.com> In the hopes this might spark a thread of "here's stuff I did to get SA-exim to work more effectively" here is a list of some recent changes I've been experimenting with: ** FWIW, I've found the stock SA 2.63 misses sufficiently much spam now that the Bayes classifier is autolearning spam, which then feeds back and lets even more through. To help against that I used, bayes_auto_learn_threshold_nonspam -0.5 ..rather than the default of 0.1. ** The extra rules at the SA Rules Emporium have helped a lot too: http://www.rulesemporium.com/ The ones on http://www.rulesemporium.com/rules.htm I think are pretty much implementable right away. There are some others that need a little care before firing up (e.g. the antidrug.cf stuff if you're a meds provider). ** The SpamCopURI patch to query SURBL is well worth adding as the URL/domain is sometimes about the only thing that identifies a mail as spam. http://sourceforge.net/projects/spamcopuri ** The SA-exim greylisting patch is really great - benefits: * rejecting spam * reducing resource usage: * it's reducing the amount of spam that gets quarantined here so /var/lib/sa-exim requires less space and end-user hassle * false positives are less drowned out, reducing mistakes IMO, if you've been putting off implementing this on your system, definitely consider pushing it up the sysadmin TODO list. http://marc.merlins.org/linux/exim/files/sa-exim-cvs/README.greylisting HTH, Paul -- Paul Makepeace ................................ http://paulm.com/ecademy "If we park in driveways and drive on parkways, then that must make me a goddess." -- http://paulm.com/toys/surrealism/ From ssmeenk at freshdot.net Sat May 22 23:14:42 2004 From: ssmeenk at freshdot.net (Sander Smeenk) Date: Sat May 22 13:15:59 2004 Subject: [FWD] Re: [SA-exim] Greylisting mails with URLs Message-ID: <20040522201442.GA16135@freshdot.net> [hum, did a direct reply instead of a group reply] Quoting Paul Makepeace (merlins.org@paulm.com): You obviously were bored today :) *heheh* > Roughly: something to tag the email with a "URL/domain of some > description whatever detected" and then in the GL code look for that > domain and possibly use that as an additional or parallel GL parameter > along with envelope and host info. I filter out stuff on URL by creating my own set of spamassassin rules. Greylisting as it is now would also filter out this kind of spam, won't it? > Paul ok, next mail... Quoting Paul Makepeace (merlins.org@paulm.com): > I'm sure this is obvious... > I've noticed spamassassin is writing its various bits to > /var/spool/exim4/.spamassassin which is Debian-exim's home dir. True. If not configured to do otherwise. > How is it getting this? spamd is started and indeed runs as root via > /etc/init.d/spamassassin. So is it picking up the $HOME via spamc > called via local_scan in exim? Exim runs as Debian-exim, local_scal (exim that is) calls spamc, spamc runs spamassassin as the calling user, and that is Debian-exim, whose $HOME points to /var/spool/exim4. > mkdir /var/lib/spamassassin > chown -R Debian-exim:Debian-exim /var/lib/spamassassin I'm thinking of doing this in my next sa-exim packages. Especially with greylisting stuff, it's more and more important to have a nice place for spamassassin / sa-exim related cruft. imho. Nice transition scripts, so current setups will get converted... hmm. > Paul next message! Quoting Paul Makepeace (merlins.org@paulm.com): > Actually RTFM I noticed the bayes_ignore_header option: > Which prompted me to wonder about, > bayes_ignore_header X-SA-Exim-Connect-IP > bayes_ignore_header X-SA-Exim-Mail-From > bayes_ignore_header X-SA-Exim-Rcpt-To > bayes_ignore_header X-SA-Exim-Scanned > bayes_ignore_header X-SA-Exim-Version Never cared about those. They always contain the same values, and I don't think they will actually cause spam to score higher or lower. Must admit I haven't tried, and I don't think it would HURT adding these headers in the ignore_header list. > Paul ok, next message! Quoting Paul Makepeace (merlins.org@paulm.com): > In the hopes this might spark a thread of "here's stuff I did to get > SA-exim to work more effectively" here is a list of some recent changes > I've been experimenting with: > FWIW, I've found the stock SA 2.63 misses sufficiently much spam now > that the Bayes classifier is autolearning spam, which then feeds back > and lets even more through. To help against that I used, > bayes_auto_learn_threshold_nonspam -0.5 > ..rather than the default of 0.1. Hmm. I have thought about NOT having SA-exim autolearn ham and spam, since my system runs on my own personal bayes databases. I have configured my spamassassin to read /home/ssmeenk/.spamassassin/bayes_* instead. I have required hits at 4.0, and reject mail at a score of 8.0. A friend of mine has his permreject set to 4.0, and no problems, according to him. I don't want to risk legit mail, since i'm also scanning for some of the other users on my system, using MY bayes db's.. > The extra rules at the SA Rules Emporium have helped a lot too: > http://www.rulesemporium.com/ Hmm. I only have the 'backhair' set, and then a ever growing set of self-made rules in which I put rules that trigger on persistant spam that comes through the normal rules. > (e.g. the antidrug.cf stuff if you're a meds provider). I've looked at the antidrug.cf set, that almost resembles my anti drug set, which filters out a LOT of spam :) > The SpamCopURI patch to query SURBL is well worth adding as the > URL/domain is sometimes about the only thing that identifies a > mail as spam. http://sourceforge.net/projects/spamcopuri I didn't want to use remote checks, because then my mail delivery would depend on someone elses ability to keep a stable server, AND my connection was too slow to handle that back then. I could check it out now, along with sender / recipient callouts. > The SA-exim greylisting patch is really great - benefits: > IMO, if you've been putting off implementing this on your system, > definitely consider pushing it up the sysadmin TODO list. I hate patching spamassassin. Marc, any word on upstream implementing your patches? :) > Paul next message! ah, no next message. okay then. I also drop TONS of mails with my ACL in exim that rejects HELO's with my own IP. I don't know if you already have that, but i'd recommend it. Furthermore I noticed that recently more and more spam came through the filters, but I think this had to do with the Received-header problem, and me still running sa-exim. I fear that my bayes db's might have been "poisoned"... Another question, do you sa-learn? And if so, do you sa-learn ham, spam or both? I only sa-learn spam. This works for me, but people told me I should also sa-learn ham... For now, this is all my text, and i will go fix me some food. :) Regards, Sander. -- | Why is the time of day with the slowest traffic called rush hour? | 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D ----- End forwarded message ----- -- | Why do they call it "chilli" if it's hot? | 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D From marc_news at merlins.org Sun May 23 22:11:37 2004 From: marc_news at merlins.org (Marc MERLIN) Date: Sun May 23 21:11:40 2004 Subject: [Exim] Re: [SA-exim] local_scan is crashing (log_write crash) In-Reply-To: <1085322797.4532.3470.camel@kaa.jungle.aubergine.my-net-space.net> References: <025C8B5C-A7FF-11D8-A8EB-000A9595347C@enchanter.net> <20040518003652.GH30824@merlins.org> <2B258C7A-A9B4-11D8-B0AF-000A9595347C@enchanter.net> <20040520041200.GD1909@merlins.org> <818037B8-AA69-11D8-BCCB-000A9595347C@enchanter.net> <058501c43e82$b78788c0$eb00010a@andromeda> <1E2E0194-AA86-11D8-9EF9-000A9595347C@enchanter.net> <1085076699.1322.11711.camel@kaa.jungle.aubergine.my-net-space.net> <1085322797.4532.3470.camel@kaa.jungle.aubergine.my-net-space.net> Message-ID: <20040524041137.GQ6482@merlins.org> [Reply-To: sa-exim@lists.merlins.org] On Sun, May 23, 2004 at 03:33:17PM +0100, Adam D. Barratt wrote: > > That did the trick, thank you very much! I changed this line, as well > > as two debug lines I found which try to print fdsize-18. > > It should probably be (double)(fdsize - 18) so that it still works on > platforms where sizeof(off_t) != 8 (i.e. most of them *g*). Adam, thanks for finding all this. I can't believe I made that stupid mistake, I should know better. At least, I did define fdstart as an off_t, and not an int, but I completely forgot that (s)printf (which I'm guessing exim uses internally for log_write) doesn't actually convert types (I've one way too much perl in the last years, one gets lazy after that) > > Why does this fix work? I figure "%.0f" means a floating-point value > > with no decimal places, but why does displaying the number as a float > > instead of a decimal solve the problem? > > "%%d" *isn't* (just) "a decimal", it's a synonym for "%i", and thus an > /int/. On many (probably still most) platforms, sizeof(int) == Yeah, that was my mistake. Stuff like that works in higher level languages were %d really does mean decimal. Again, thanks for the analysis, I'll put out a new sa-exim with this and a few other fixes soon. Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From ross at biostat.ucsf.edu Mon May 24 14:24:51 2004 From: ross at biostat.ucsf.edu (Ross Boylan) Date: Mon May 24 13:25:05 2004 Subject: [SA-exim] spamassassin's home dir In-Reply-To: <20040522153925.GY29101@mythix.realprogrammers.com> References: <20040522153925.GY29101@mythix.realprogrammers.com> Message-ID: <1085430291.1562.205.camel@iron.libaux.ucsf.edu> On Sat, 2004-05-22 at 08:39, Paul Makepeace wrote: > I'm sure this is obvious... > > I've noticed spamassassin is writing its various bits to > /var/spool/exim4/.spamassassin which is Debian-exim's home dir. > > How is it getting this? spamd is started and indeed runs as root via > /etc/init.d/spamassassin. So is it picking up the $HOME via spamc > called via local_scan in exim? > The spamd man page says -u username, --username=username Run as the named user. If this option is not set, the default behaviour is to setuid() to the user running "spamc", if "spamd" is running as root. So exim runs spamc with exim's identity, and spamd runs that way as well. From tor at slett.net Tue May 25 17:59:27 2004 From: tor at slett.net (Tor Slettnes) Date: Tue May 25 16:59:39 2004 Subject: [SA-exim] spamassassin's home dir In-Reply-To: <20040522153925.GY29101@mythix.realprogrammers.com> References: <20040522153925.GY29101@mythix.realprogrammers.com> Message-ID: <8E6E8914-AEA7-11D8-85B6-0030656CF512@slett.net> On May 22, 2004, at 08:39, Paul Makepeace wrote: > How is it getting this? spamd is started and indeed runs as root via > /etc/init.d/spamassassin. So is it picking up the $HOME via spamc > called via local_scan in exim? Check the OPTIONS="..." setting in /etc/default/spamassassin. Remove the "-H" flag, and specify the desired username, e.g. "-u mail". "man spamd" for details. -tor From simon at nuit.ca Sat May 29 18:24:31 2004 From: simon at nuit.ca (simon@nuit.ca) Date: Sat May 29 10:25:30 2004 Subject: [SA-exim] faking an accept Message-ID: <20040529172431.GE20614@nuit.ca> hi folks, i'm getting a lot of unnecessary traffic from a host that's forwarding mail here. it keeps trying to send my mail server spam, which i don't want, but need to "accept" it in order to cut down on the traffic. the only way i can see to do both not deliver mail to the account, but still filter it, would be to fake the acceptance of the mail. but i don't want to do this for every host - only for a few select ones, like mailing lists, and this one upstream MTA for instance. how would i do this? comments, opinions, flames welcome. -- @@-----------------------------------------------------------------@@ | ,''`. http://www.debian.org/ | http://www.nuit.ca/ | | : :' : Debian GNU/Linux | http://simonraven.nuit.ca/ | | `. `' | PGP key fingerprint (new one): | | `- | 7C49 FD9C 1054 7300 3B7B | | | 8BF4 6A88 7AE2 711D F097 | @@-----------------------------------------------------------------@@ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 652 bytes Desc: Digital signature Url : http://lists.merlins.org/archives/sa-exim/attachments/20040529/73e67d2b/attachment.bin