5.6.5-B2367 Spam Check Failures

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
Post Reply
mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-18 02:21

I've run 5.6.5-B2367 for a couple days and noticed that it does not perform SPF and HELO spam checks properly and flags things that it should not. Here are examples of each:

Here is an example where HMS should not have flagged a HELO spam failure.


I use the servers at dnsexit.com as my MX and have HMS set to recognize them in the Relays section.




Return-Path: UEPCECP@publix.com
Received: from smtp4.dnsexit.com (smtp4.dnsexit.com [67.214.161.140]) by mpfrench.com
with ESMTP ; Fri, 17 Jun 2016 14:31:16 -0500
Received: from localhost (smtp-rx1 [67.214.161.131]) by smtp4.dnsexit.com (8.13.8/8.13.8)
with ESMTP id u5HJUoET018326 for <xxxx@mpfrench.com>; Fri, 17 Jun 2016
15:30:50 -0400
Received: from smtp2.dnsexit.com ([67.214.161.138]) by localhost (smtp-rx1.dnsexit.com
[67.214.161.131]) (amavisd-new, port 10024) with ESMTP id ljqI_4RfmyrH for
<xxxx@mpfrench.com>; Fri, 17 Jun 2016 15:30:59 -0400 (EDT)
Received: from mail.publix.com (mail.publix.com [12.238.159.23]) by smtp2.dnsexit.com (8.13.8/8.13.8) [THIS IS MY MX]
with ESMTP id u5HJUoLQ028637 for <xxxx@mpfrench.com>; Fri, 17 Jun 2016
15:30:51 -0400
X-Notes-To-Recipient: You received this email forwarded by smtp1.dnsexit.com because you signed up
for DNS Exit Mail Forwarding, Mail Redirection or Mail Backup service for
your domain. No emails ever originated from this server. To report Spam,
make sure you trace to the first IP that originates the email. We will
terminate your service immediately if you report our IP as Spam by
mistake.
Received: from PEPCA01 (10.104.36.36) by A46PHUBA03.corp.ad.publix.com (10.104.68.51) with
Microsoft SMTP Server id 14.3.248.2; Fri, 17 Jun 2016 15:30:35 -0400
MIME-Version: 1.0
From: <CustomerCorrespondence.CustomerCare@publix.com>
To: <xxxx@mpfrench.com>
Date: Fri, 17 Jun 2016 15:30:35 -0400
Subject: [SPAM-HMS] Publix Reply - Case Ref # 1088809
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64
Message-ID: <5f639078-b0b8-485e-9fa7-e68f97e5140d@A46PHUBA03.corp.ad.publix.com>
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: The host name specified in HELO does not match IP address. - (Score: 13)
X-hMailServer-Reason-Score: 13




The SPF spam check is not operating correctly. It flags things that have no problems.

See the string below for more details.

------- Forwarded message follows -------
Ara,
I think you are correct. My mail system is not evaluating SPF correctly.

Your system delivered mail to my MX (smtp1.dnsexit.com) using IP address 209.134.158.11. While your SPF TXT records are complex, they appear to allow 209.134.158.11 from the TXT line ip4:209.134.158.0/24.

Sorry to bother you. I am disabling SPF checks until I solve this problem.

Thanks,
Mike


On 6/17/2016 12:08 PM, Ara Bulbulian wrote:
Michael,

Thanks for bringing to our attention your findings. After taking a look at the appropriate DNS entries (see below). It appears the issue is on the dnsexit side in evaluating our sending IP / domain.

dig mailer158011.service.govdelivery.com +short
209.134.158.11

nslookup -type=txt spf.ep.service.govdelivery.com
spf.ep.service.govdelivery.com text = "v=spf1 ip4:209.134.144.4 ip4:209.134.144.240/28 ip4:209.134.151.0/24 ip4:209.134.158.0/24 -all"


Also,

dig uscms.service.govdelivery.com TXT +short
"v=spf1 a mx include:service.govdelivery.com -all"


What appears odd to me is one of the last headers which has a ‘%i’ in the place where an IP should reside. Possibly this was a variable that didn’t get evaluated properly by a dnsexit service?

X-hMailServer-Reason-1: Blocked by SPF (%i is not a valid mail sender for the GovDelivery Service.)


Let me know how else we can assist you and what you’re able to find out on your end.

thanks

Ara Bulbulian
| Lead Delivery Engineer | GovDelivery, Inc.
p: (651) 726-7309 c: (651) 208-8721 f: (651) 665-0943


--------------------------------------------------------------------------------
From: Michael <xxxx@mpfrench.com>
Sent: Friday, June 17, 2016 09:25
To: cmslists@uscms.service.govdelivery.com; techops
Subject: Re: [SPAM-HMS] eMSN issue resolved

Your Sender Policy Framework is hosed.

See the message header from the message you sent below:

Return-Path: info99@service.govdelivery.com
Received: from smtp3.dnsexit.com (nd196.dnsexit.com [64.182.102.196]) by mpfrench.com
with ESMTP ; Fri, 17 Jun 2016 08:42:13 -0500
Received: from localhost (smtp-rx4 [64.182.101.67]) by smtp3.dnsexit.com (8.13.8/8.13.8)
with ESMTP id u5HDfmUc029473 for <xxxx@mpfrench.com> ; Fri, 17 Jun 2016
09:41:48 -0400
Received: from smtp1.dnsexit.com ([64.182.103.22]) by localhost (smtp-rx4.dnsexit.com [64.182.101.67])
(amavisd-new, port 10024) with ESMTP id MD9kEgsUFkWo for <xxxx@mpfrench.com> ;
Fri, 17 Jun 2016 09:41:41 -0400 (EDT)
Received: from mailer158011.service.govdelivery.com (mailer158011.service.govdelivery.com
[209.134.158.11]) by smtp1.dnsexit.com (8.13.8/8.13.8) with ESMTP id
u5HDfeLk018156 for <xxxx@mpfrench.com> ; Fri, 17 Jun 2016 09:41:40 -0400
X-Notes-To-Recipient: You received this email forwarded by smtp1.dnsexit.com because you signed up
for DNS Exit Mail Forwarding, Mail Redirection or Mail Backup service for
your domain. No emails ever originated from this server. To report Spam,
make sure you trace to the first IP that originates the email. We will
terminate your service immediately if you report our IP as Spam by
mistake.
X-VirtualServer: Default-CMS, mailer158011.service.govdelivery.com, 172.24.0.11
X-VirtualServerGroup: Default-CMS
X-MailingID: 17745100::20160617.60424461::1001::MDB-PRD-BUL-20160617.60424461::xxxx@mpfrench.com::71125_0
X-SMHeaderMap: mid="X-MailingID"
X-Destination-ID: xxxx@mpfrench.com
X-SMFBL: a2FyZW5AbXBmcmVuY2guY29t
Content-Transfer-Encoding: 7bit
Content-Type: multipart/alternative; boundary="----=_NextPart_436_AD11_2CD29ED8.4C335D3B"
x-subscriber: 3.XfDESGhJnMBzhkthhKOgXYHvbvOtzVXQT83NV73Yv7CqxXMGfL8ginFtQJfXg3KtHAEUKjFe2NZ5Rt2uq4nxbRMFN37Za432FXo3y3z1R+6u+xNRcIv/tsljqvtfFnOEB/9IOdCEzhb2vrS/+/okrQ==
X-Accountcode: USCMS
Errors-To: info99@service.govdelivery.com
Reply-To: cmslists@uscms.service.govdelivery.com
MIME-Version: 1.0
Message-ID: <17745100.71125@subscriptions.cms.hhs.gov>
X-ReportingKey: LJJJ2EWJK7822OJJA70RJJ::xxxx@mpfrench.com::xxxx@mpfrench.com
Subject: [SPAM-HMS] eMSN issue resolved
Date: Fri, 17 Jun 2016 08:31:16 -0500
To: xxxx@mpfrench.com
From: "=?US-ASCII?Q?Medicare.gov?=" <cmslists@uscms.service.govdelivery.com>
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Blocked by SPF (%i is not a valid mail sender for the GovDelivery Service.)
- (Score: 6)
X-hMailServer-Reason-Score: 6
Last edited by jimimaseye on 2016-06-18 10:54, edited 1 time in total.
Reason: Disabled interfering smileys

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-18 10:30

Regarding the first HELO problem:


'Internal Relays' says:
hMailServer ....will use the connecting IP address to determine where the message is arriving from. When hMailServer receives an email from a MX backup server, hMailServer ..... will know that messages from this TCP/IP address is being forwarded and that the backup sender is not the original sender. hMailServer will then try to determine the original senders IP address by parsing the Received headers of the email message.
My interpretation of that doesnt say that it excludes the internal relay listed addresses from the RECEIVED HEADER checks, only that it excludes them from a CONNECTING address check. Therefore it seems it is then going down the received headers and checking whatever is listed. And encountering this:

An NSLOOKUP shows

> smtp2.dnsexit.com
Non-authoritative answer:
Name: smtp2.dnsexit.com
Address: 67.214.161.143

where the email shows
Received: from smtp2.dnsexit.com ([67.214.161.138]
(therefore not matching)

Couldnt this be the problem?

(Could you temporarily enable DEBUG and TCPIP logging, arrange another email to come in, and then poist the logs? Hopefully that will show us its cycle of checks and what it is checking).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-18 11:01

Regarding the SPF record:

Looking at the "service.govdelivery.com" spf record (http://mxtoolbox.com/SuperTool.aspx?act ... n=toolpage), shows this:

Code: Select all

v=spf1 include:spf.as.service.govdelivery.com include:spf.ep.service.govdelivery.com include:spf.sp.service.govdelivery.com a mx -all exp=spferror.service.govdelivery.com
This is wrong. It has "exp=spferror.service.govdelivery.com" at the end of the record

a, it shouldnt be at the end of the record, the -ALL should always be at the end of the record and
(b, it actually doesnt equate to anything)

Also, there are too many 'SPF lookups'. (http://mxtoolbox.com/domain/service.govdelivery.com/). According to RFC it must not exceed 10. Again, maybe having too many variables beyond what is normal might lead to this 'out of place' parameter. (Not excusing HMS for giving a %i as it must handle the error better but that said if there are incorrect records then the results can understandably be unpredictable).

I wouldnt be surprised if this incorrect positioning (or the extra lookups) may be the cause (HMS finding a variable (hence it being a "%i") out of range after already processing the '-all' marker. Perhaps you could discuss with the sender and get them to change the spf record (correct positioning) and do another test?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-18 11:47

SPF problem:

CONFIRMED! It is the incorrect position (and in combination of domain specific) of "exp=spferror.service.govdelivery.com" entry.

I did a test by sending in a control email from my domain that has a correct spf record. Then I sent in the SAME email after adding "exp=spferror.service.govdelivery.com" to the end of my spf record (after the "-all")

Lo and behold, I too get the same result
To: Jim <user1@jim.com>
Subject: [SPAM] Re: test1
References: <576513E2.8000005@jim.com>
In-Reply-To: <576513E2.8000005@jim.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-hMailServer-Spam: YES
X-hMailServer-Reason-2: The host name specified in HELO does not match IP address. - (Score: 2)
X-hMailServer-Reason-3: Blocked by SPF (%i is not a valid mail sender for the GovDelivery Service.)
- (Score: 3)
X-hMailServer-Reason-Score: 5
I then MOVED its position to being BEFORE the "-all" (which should always be at the end) and the error then did not happen:
To: Jim <user1@jim.com>
Subject: [SPAM] Re: test1
References: <576513E2.8000005@jim.com>
In-Reply-To: <576513E2.8000005@jim.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-hMailServer-Spam: YES
X-hMailServer-Reason-2: The host name specified in HELO does not match IP address. - (Score: 2)
X-hMailServer-Reason-3: Blocked by SPF () - (Score: 3)
X-hMailServer-Reason-Score: 5
(this is correct)

So, yes, it is the senders incorrect SPF record format.

Its a combination of it being this mix of (possibly too long, or recursive spf failure) the domain 'service.govdelivery.com' because if I use a different non-existent address for the 'exp=' parameter (eg, "exp=non-exist.spferror.mydomain.com") the error doesnt happen (even if I put it after the -all).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 3133
Joined: 2006-08-21 15:38
Location: Denmark

Re: 5.6.5-B2367 Spam Check Failures

Post by SorenR » 2016-06-18 13:31

jimimaseye wrote:Regarding the first HELO problem:


'Internal Relays' says:
hMailServer ....will use the connecting IP address to determine where the message is arriving from. When hMailServer receives an email from a MX backup server, hMailServer ..... will know that messages from this TCP/IP address is being forwarded and that the backup sender is not the original sender. hMailServer will then try to determine the original senders IP address by parsing the Received headers of the email message.
My interpretation of that doesnt say that it excludes the internal relay listed addresses from the RECEIVED HEADER checks, only that it excludes them from a CONNECTING address check. Therefore it seems it is then going down the received headers and checking whatever is listed. And encountering this:

An NSLOOKUP shows

> smtp2.dnsexit.com
Non-authoritative answer:
Name: smtp2.dnsexit.com
Address: 67.214.161.143

where the email shows
Received: from smtp2.dnsexit.com ([67.214.161.138]
(therefore not matching)

Couldnt this be the problem?

(Could you temporarily enable DEBUG and TCPIP logging, arrange another email to come in, and then poist the logs? Hopefully that will show us its cycle of checks and what it is checking).
HELO problems should be diagnosed by looking at the LOG, NOT the headers :!: If you look at the code you will see that NO SMTP DATA has been transferred when the check is done.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-18 13:44

SorenR wrote: HELO problems should be diagnosed by looking at the LOG, NOT the headers :!: If you look at the code you will see that NO SMTP DATA has been transferred when the check is done.
Not in the case of an INTERNAL RELAY in use. If the incoming connection is an INTERNAL RELAY then it delays the HELO check until all 'DATA' has been received (in line with what the documentation says)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-18 13:47

"DEBUG" 2556 "2016-06-18 13:42:05.237" "Creating session 34"
"DEBUG" 2556 "2016-06-18 13:42:05.461" "TCP connection started for session 32"
"SMTPD" 2556 32 "2016-06-18 13:42:05.462" "192.168.0.200" "SENT: 220 jimHMS_Mail"
"SMTPD" 2664 32 "2016-06-18 13:42:05.523" "192.168.0.200" "RECEIVED: HELO mydomain.net"
"SMTPD" 2664 32 "2016-06-18 13:42:05.524" "192.168.0.200" "SENT: 250 Hello."
"SMTPD" 3084 32 "2016-06-18 13:42:05.584" "192.168.0.200" "RECEIVED: MAIL FROM:<sylvester@mydomain.net>"
"SMTPD" 3084 32 "2016-06-18 13:42:05.713" "192.168.0.200" "SENT: 250 OK"
"SMTPD" 2556 32 "2016-06-18 13:42:06.106" "192.168.0.200" "RECEIVED: RCPT TO:<user1@jim.co>"
"SMTPD" 2556 32 "2016-06-18 13:42:06.117" "192.168.0.200" "SENT: 250 OK"
"SMTPD" 2664 32 "2016-06-18 13:42:06.178" "192.168.0.200" "RECEIVED: DATA"
"SMTPD" 2664 32 "2016-06-18 13:42:06.199" "192.168.0.200" "SENT: 354 OK, send."
"DEBUG" 1532 "2016-06-18 13:42:06.771" "Could not retrieve PTR record for IP (false)! 192.168.0.200"
"DEBUG" 1532 "2016-06-18 13:42:06.803" "Adding task AsynchronousTask to work queue Asynchronous task queue"
"DEBUG" 2916 "2016-06-18 13:42:06.817" "Executing task AsynchronousTask in work queue Asynchronous task queue"
"DEBUG" 2916 "2016-06-18 13:42:07.162" "Spam test: SpamTestDNSBlackLists, Score: 0"
"DEBUG" 2916 "2016-06-18 13:42:07.163" "Spam test: SpamTestHeloHost, Score: 0"
"DEBUG" 2916 "2016-06-18 13:42:07.229" "Spam test: SpamTestMXRecords, Score: 0"
"DEBUG" 2916 "2016-06-18 13:42:07.476" "Spam test: SpamTestSPF, Score: 3"
"DEBUG" 2916 "2016-06-18 13:42:07.477" "Total spam score: 3"
"DEBUG" 2916 "2016-06-18 13:42:07.480" "SURBL: Execute"
"DEBUG" 2916 "2016-06-18 13:42:07.499" "SURBL: Match not found"
"DEBUG" 2916 "2016-06-18 13:42:07.499" "Spam test: SpamTestSURBL, Score: 0"
"DEBUG" 2916 "2016-06-18 13:42:07.571" "DKIM: Message passed validation."
"DEBUG" 2916 "2016-06-18 13:42:07.572" "Spam test: SpamTestDKIM, Score: 0"
"DEBUG" 2916 "2016-06-18 13:42:07.597" "Creating session 35"
"DEBUG" 1532 "2016-06-18 13:42:07.600" "TCP connection started for session 35"
"DEBUG" 1532 "2016-06-18 13:42:07.601" "Sending message to SpamAssassin. Session 35, File: C:\Program Files (x86)\hMailServer\Data\{26E6D8DE-3DC7-477D-BDA5-2931D0172F33}.eml"
"DEBUG" 3076 "2016-06-18 13:42:14.111" "Parsing response from SpamAssassin. Session 35"
"DEBUG" 3084 "2016-06-18 13:42:14.186" "SA - Copy+Delete used"
"DEBUG" 3084 "2016-06-18 13:42:14.187" "Ending session 35"
"DEBUG" 2916 "2016-06-18 13:42:14.189" "Spam test: SpamTestSpamAssassin, Score: 0"
"DEBUG" 2916 "2016-06-18 13:42:14.190" "Total spam score: 0"
"DEBUG" 2916 "2016-06-18 13:42:14.235" "Executing event OnAcceptMessage"
"DEBUG" 2916 "2016-06-18 13:42:14.767" "Event completed"
"DEBUG" 2916 "2016-06-18 13:42:14.768" "Saving message: {26E6D8DE-3DC7-477D-BDA5-2931D0172F33}.eml"
(192.168.0.200 is set as an internal relay)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-18 23:00

Here is the HMS log for the HELO spam check failure that I originally posted.

"DEBUG" 6880 "2016-06-17 14:31:16.233" "TCP connection started for session 574"
"SMTPD" 6880 574 "2016-06-17 14:31:16.233" "67.214.161.140" "SENT: 220 mpfrench.com ready"
"SMTPD" 1700 574 "2016-06-17 14:31:16.280" "67.214.161.140" "RECEIVED: EHLO smtp4.dnsexit.com"
"SMTPD" 1700 574 "2016-06-17 14:31:16.280" "67.214.161.140" "SENT: 250-mpfrench.com[nl]250-SIZE[nl]250-AUTH LOGIN PLAIN[nl]250 HELP"
"SMTPD" 5160 574 "2016-06-17 14:31:16.327" "67.214.161.140" "RECEIVED: MAIL From:<UEPCECP@publix.com> SIZE=5763"
"SMTPD" 5160 574 "2016-06-17 14:31:16.342" "67.214.161.140" "SENT: 250 OK"
"SMTPD" 6880 574 "2016-06-17 14:31:16.389" "67.214.161.140" "RECEIVED: RCPT To:<xxxx@mpfrench.com>"
"SMTPD" 6880 574 "2016-06-17 14:31:16.405" "67.214.161.140" "SENT: 250 OK"
"SMTPD" 1700 574 "2016-06-17 14:31:16.451" "67.214.161.140" "RECEIVED: DATA"
"SMTPD" 1700 574 "2016-06-17 14:31:16.451" "67.214.161.140" "SENT: 354 OK, send."
"DEBUG" 4808 "2016-06-17 14:31:16.685" "Adding task AsynchronousTask to work queue Asynchronous task queue"
"DEBUG" 7040 "2016-06-17 14:31:16.685" "Executing task AsynchronousTask in work queue Asynchronous task queue"
"TCPIP" 7040 "2016-06-17 14:31:16.826" "DNS lookup: 131.161.214.67.zen.spamhaus.org, 0 addresses found: (none), Match: False"
"TCPIP" 7040 "2016-06-17 14:31:16.857" "DNS lookup: 131.161.214.67.bl.spamcop.net, 0 addresses found: (none), Match: False"
"TCPIP" 7040 "2016-06-17 14:31:16.904" "DNS lookup: 131.161.214.67.cbl.abuseat.org, 0 addresses found: (none), Match: False"
"DEBUG" 7040 "2016-06-17 14:31:16.904" "Spam test: SpamTestDNSBlackLists, Score: 0"
"DEBUG" 7040 "2016-06-17 14:31:16.904" "Spam test: SpamTestHeloHost, Score: 13"
"DEBUG" 7040 "2016-06-17 14:31:16.935" "Spam test: SpamTestMXRecords, Score: 0"
"DEBUG" 7040 "2016-06-17 14:31:16.935" "Total spam score: 13"

Since I use an off-site MX (dnsexit.com), one cannot see the IP address that connected to my MX using this log. One can only see that connecting IP address from the e-mail header which I posted originally.

If one manually performs an nslookup on the connecting server, the IP address matches what was presented to my MX. So this proves HMS made a mistake.

Regarding the SPF problem, I'm going to gather more data and repost.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-18 23:15

The log doesnt help to evaluate, it only reaffirms that the HELO check has been put off until after the DATA is received (as expected due to using the internal relay).

Your original email (and its headers) was sufficient to see what has happened.

As I originally posted, I dont think there is a problem and that its findings are correct. It reads the RECEIVED headers of the email. And found the mismatch as I highlighted.

But there is a confusion why the dnsexit server "localhost (smtp-rx1.dnsexit.com [67.214.161.131])" added the header:

RECEIVED FROM: smtp2.dnsexit.com ([67.214.161.138].....)

when in fact a real look shows this server to be 67.214.161.143 - and it did this just 8 seconds (at 15:30:59) after that previous server smtp2 originally received the email at 15:30:51. Can it REALLY change its ip address so quickly?)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 19810
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: 5.6.5-B2367 Spam Check Failures

Post by mattg » 2016-06-19 00:27

I think that the two addresses are on the same subnet, and the two mail servers simply shared the mail message, one for incoming and one for outgoing
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-19 17:52

Here is a better example of the SPF problem in HMS. The following is the message header:

Return-Path: JCPenney@e.jcpenney.com
Received: from smtp3.dnsexit.com (nd196.dnsexit.com [64.182.102.196]) by mpfrench.com
with ESMTP ; Sun, 19 Jun 2016 04:21:45 -0500
Received: from localhost (smtp-rx4 [64.182.101.67]) by smtp3.dnsexit.com (8.13.8/8.13.8)
with ESMTP id u5J9Ll1s021176 for <xxxx@mpfrench.com>; Sun, 19 Jun 2016
05:21:47 -0400
Received: from smtp1.dnsexit.com ([64.182.103.22]) by localhost (smtp-rx4.dnsexit.com [64.182.101.67])
(amavisd-new, port 10024) with ESMTP id 1dBVtHdwpkkz for <xxxx@mpfrench.com>;
Sun, 19 Jun 2016 05:21:48 -0400 (EDT)
Received: from omp.e.jcpenney.com (omp.e.jcpenney.com [199.7.206.230]) by smtp1.dnsexit.com
(8.13.8/8.13.8) with ESMTP id u5J9LmPT012614 for <xxxx@mpfrench.com>; Sun,
19 Jun 2016 05:21:48 -0400
X-Notes-To-Recipient: You received this email forwarded by smtp1.dnsexit.com because you signed up
for DNS Exit Mail Forwarding, Mail Redirection or Mail Backup service for
your domain. No emails ever originated from this server. To report Spam,
make sure you trace to the first IP that originates the email. We will
terminate your service immediately if you report our IP as Spam by
mistake.
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=jcpenney; d=e.jcpenney.com; h=MIME-Version:Content-Type:Content-Transfer-Encoding:Date:To:From:Reply-To:Subject:List-Unsubscribe:Message-ID;
i=JCPenney@e.jcpenney.com; bh=xSJalPySS77wbOPfUoCx71juzBE=; b=H5vGaPnhZtHedO8FftjTmFY8ON0Z9RJHGd54LDNh1HpcNkHzv/Tr5aeGFSZsaLOjjw3kS/i69WJ6
7IM5W98m/fPbyryiEk7dQuQSnfKfuDhS5275lQg0BaviWNOzN7QwCuRxg4wW1IhnrcvQxL/pAS+M
XTAuDUyn5t94JKHKPZo=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=jcpenney; d=e.jcpenney.com; b=o6oxybdclGpOAOUSbwGqSNOOolp0kmDAiAQ+908bPVLE77s7QP+dwqDEWQ/8utgOUS7gwNrMOskr
XvcFinqU1OolM+otxGVBAdWKd6vOMMXAy/gyX800ItihvJsL9Lsp6RcQk0SXyvi+6ZmzFi0q9Rrj
SEK37CiKCK4ZW0QRY7Y=;
Received: by omp.e.jcpenney.com id hcpi28162ksh for <xxxx@mpfrench.com>; Sun, 19 Jun
2016 02:21:17 -0700 (envelope-from <JCPenney@e.jcpenney.com>)
X-CSA-Complaints: whitelist-complaints@eco.de
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Date: Sun, 19 Jun 2016 02:21:17 -0700
To: xxxx@mpfrench.com
From: "JCPenney" <JCPenney@e.jcpenney.com>
Reply-To: "JCPenney" <no.reply@e.jcpenney.com>
Subject: [SPAM-HMS]=?UTF-8?Q?=20Happy=20Father=E2=80=99s=20Day=E2=80=94there=E2=80=99s=20sti?=
=?UTF-8?Q?ll=20time=20to=20find=20gifts=20for=20dad!?=
Feedback-ID: 54117:1630345:oraclersys
List-Unsubscribe: <https://e.jcpenney.com/pub/optout/Unsub ... q4hOjpwA8W>,
<mailto:unsubscribe-YQpglLjHJlYQGnWIXpDPOA37Yzf1zefprsgsDTIHATzemeqzejacDzf4zgTbE1JAsBjJdB@imh.rsys5.com?subject=List-Unsubscribe>
X-sgxh1: skohiWxuiMjLgJoQJhu
X-rext: 5.interact5.Eg4b_i9yw6UB5xSVMijknk92ba62VuL-VrHJ9xWF2IGSc6k7rcK7
X-cid: jcpenney.1314445
Message-ID: <0.1.5B.B06.1D1CA0BEEF83256.0@omp.e.jcpenney.com>
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Blocked by SPF () - (Score: 6)
X-hMailServer-Reason-Score: 6



Delivery to my MX is shown on the line --
Received: from omp.e.jcpenney.com (omp.e.jcpenney.com [199.7.206.230]) by smtp1.dnsexit.com
(8.13.8/8.13.8) with ESMTP id u5J9LmPT012614 for <xxxx@mpfrench.com>; Sun,
19 Jun 2016 05:21:48 -0400

Here is the SPF TXT record for e.jcpenney.com:
e.jcpenney.com. 8552 IN TXT "v=spf1 ip4:199.7.200.0/21 ip4:12.130.135.0/24 -all"

This definition allows delivery from the IP range 199.7.200.1 - 199.7.207.254. The delivering IP address was 199.7.206.230 which is within the allowed SPF range. Therefore HMS performed the SPF evaluation incorrectly.
Last edited by jimimaseye on 2016-06-19 18:01, edited 1 time in total.
Reason: Disabled interfering smileys

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-19 18:09

Its nothing to do with jcpenny.com. Its the same problem as the other post: its DNSEXIT.com
mpfrench wrote:Received: from smtp1.dnsexit.com ([64.182.103.22]) by localhost (smtp-rx4.dnsexit.com [64.182.101.67])
However:

SPF for dnsexit,com (http://mxtoolbox.com/SuperTool.aspx?act ... n=toolpage#) is:
v=spf1 ip4:64.182.101.0/24 ip4:67.214.175.64/27 ip4:64.182.102.0/24 ip4:67.214.161.129/27 ip4:67.214.171.65/28 ip4:64.182.105.0/24 ip4:24.172.195.33/28 ip4:66.85.182.216/29 ip4:184.164.135.0/29 ip4:192.219.19.52/28 ip4:192.37.168.253/30 ip4:198.204.224.146/29 ip4:24.172.194.209/28 ip4:65.23.155.0/24 ip4:66.235.177.33/28 ip6:2604:4300:a:25::0/64 ip4:69.30.236.138/29 ip4:142.54.187.122/29 ip4:107.150.47.178/29 ip4:209.160.25.8/24 ip4:209.160.34.101/32 ip4:209.160.24.225/32 ip4:173.208.189.26/29 ip4:192.187.102.10/29 ip4:142.0.45.162/27 ip4:104.219.19.52/28 ip4:192.151.149.82/28 ip4:199.255.159.224/29 ip4:192.37.168.252/30 ip4:104.37.168.253/30 ip4:192.254.74.210/29 ip4:49.213.13.56/32 ip4:66.235.177.222/32 ip4:142.0.41.116/32 ip4:173.208.184.42/29 ip4:63.141.228.202/29 ~all
There is no provision for 64.182.103.xxx as far as I can see. (I might be wrong though as these things are not the easiest things to read for me. But dont think so in this case.).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-19 18:24

Here is what the HMS 5.6 documentation says about Incoming Relays:

"When hMailServer performs anti-spam tests on a message it will use the connecting IP address to determine where the message is arriving from. When hMailServer receives an email from a MX backup server, hMailServer can't use the senders TCP/IP address since this is the IP address of the backup server. If you add the MX backup servers IP address as an incoming relay, hMailServer will know that messages from this TCP/IP address is being forwarded and that the backup sender is not the original sender. hMailServer will then try to determine the original senders IP address by parsing the Received headers of the email message.

Most anti spam methods in hMailServer are performed before the actual message body has been delivered to hMailServer. hMailServer uses the HELO, MAIL FROM and RCPT TO commands to determine whether the message is likely to be spam. However, if a message arrives from a incoming relay, hMailServer has to wait until the entire message has been received before it can perform anti spam. In other words, using incoming relays will make the anti spam mechanisms more expensive, in terms of bandwidth and time.

Please note that incoming relays does not affect grey listing. Grey listing always takes place before the Received headers have been transmitted to hMailServer. You may want to add any incoming relays IP address to the grey listing white list, to prevent incoming relays from being grey listed."


The way I interpret this, the IP address of the server that connects to my MX ( one of dnsexit's servers that are defined as incoming relays within HMS) is supposed to be used by HMS for spam checking. This version of HMS does not actually work this way.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-19 19:02

My interpretation is that it is doing what it says. An INTERNAL RELAY has been defined, and the incoming connecting address is within your internal relay addresses. Therefore it awaits full delivery of the message befire commencing its spam checks from headers (instead of the last connected address).

Admittedly, it would be better for you that where those RECEIVED headers matched your internal relays they would be ignored/not checked. I think this is what you are expecting. However, RECEIVED headers can be spoofed so it would create a loophole of it were to do so. IMO it is doing what the documentation says and by design. (Mind you, if dnsexit.com got their act together and ensured their addresses match what they say they are to be then you wouldnt have a problem. Given the nature of their business you wold expect them to do so).

As a matter of interest, are you suggesting this version of HMS is performing differently from another version under the same circumstances?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-19 19:31

I may have setup the Incoming Relay definition incorrectly previously. I've changed it to match the A records of smtp.dnsexit.com, 64.182.101.45 and 67.214.161.149, which are my MX IP addresses. I'll run this way for a while and report the results after I've double-checked HMS's operation.

The HMS documentation should be a bit clearer on how to set-up Incoming Relays. It is difficult to understand now.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-19 21:19

Mike I dont think you are understanding.

The machine (ie, the Internal Relay) passing the email to your HMS was smtp4.dnsexit.com (smtp4.dnsexit.com [67.214.161.140]). This is what should be included in your Internal Relays. If you change your internal relay to be 64.182.101.45 and 67.214.161.149 (as you just said) then HMS will antispam check the email immdiately based on the incoming connection only (and you will never catch spam - assuming dnsexit never fail spam checks).

You need your internal relays to be whatever the smtp servers that dnsexit use (in this case, including 67.214.161.140) to ensure that it ignores this incoming connection and continues to just check the RECEIVED headers.

From the documentation it goes like this:

with internal relay

C: I have an email
S (your HMS): ok, whats your address?
C: smtp4.dnsexit.com
S: I see you are connecting on 123.45.67.89, Ah, you are down as an internal relay. Therefore I wont spam check you but I will wait for the whole message and spam check all the RECEIVED headers individually to test their validity instead.

without the internal relay

C: I have an email
S: ok, whats your address?
C: smtp4.dnsexit.com
S: I see you are connecting on 123.45.67.89. Hang on, Im checking to see if you are decent enough
a, .........Yep, your clean and all match. Send me the email and Ill deliver it. OR
b, ....err hangon, a lookup of smtp4.dnsexit.com says you should be 222.111.222.111 (HELO). And dnsexit shouldnt be sending from 123.45.67.89 (SPF). Sod off, youre dodgy!
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-20 01:47

I initially had the Incoming Relays defined as all the dnsexit servers that deliver mail to mpfrench.com. This did not allow HMS to work correctly. It makes more sense to me that I need to tell HMS what my MXs IPs are so that it can start the spam checks on the IP address that connects to my MX. I have set HMS Incoming Relays to my MX IP addresses. As I stated, I'll report what I find tomorrow.

The HMS Incoming Relay documentation really needs to be updated to clarify its use.

User avatar
mattg
Moderator
Moderator
Posts: 19810
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: 5.6.5-B2367 Spam Check Failures

Post by mattg » 2016-06-20 02:20

mpfrench wrote:The HMS Incoming Relay documentation really needs to be updated to clarify its use.
Please put together some suggested changes, and I'll see what we can do..
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-21 03:26

I'll be happy to suggest some documentation changes as soon as I convince myself that I really understand how to use the Incoming Relays section in HMS.

I've been running with the HMS Incoming Relays section set to my two MXs [smtp.dnsexit.com], 64.182.101.45, and 67.214.161.149. I have not noticed any HELO spam check errors. However, I have noticed some SPF errors.

SPF Error Example 1:
Return-Path: bounce@bounces.choicehotels.com
Received: from smtp3.dnsexit.com (nd196.dnsexit.com [64.182.102.196]) by mpfrench.com
with ESMTP ; Mon, 20 Jun 2016 15:46:35 -0500
Received: from localhost (smtp-rx4 [64.182.101.67]) by smtp3.dnsexit.com (8.13.8/8.13.8)
with ESMTP id u5KKkMhi029210 for <xxxx@MPFRENCH.COM>; Mon, 20 Jun 2016
16:46:38 -0400
Received: from smtp1.dnsexit.com ([64.182.103.22]) by localhost (smtp-rx4.dnsexit.com [64.182.101.67])
(amavisd-new, port 10024) with ESMTP id T-N3dxPWfPoT for <xxxx@MPFRENCH.COM>;
Mon, 20 Jun 2016 16:46:45 -0400 (EDT)
Received: from sm1.choicehotels.com (sm1.choicehotels.com [50.56.16.8]) by smtp1.dnsexit.com
(8.13.8/8.13.8) with ESMTP id u5KKkial007854 for <xxxx@MPFRENCH.COM>; Mon,
20 Jun 2016 16:46:45 -0400
X-Notes-To-Recipient: You received this email forwarded by smtp1.dnsexit.com because you signed up
for DNS Exit Mail Forwarding, Mail Redirection or Mail Backup service for
your domain. No emails ever originated from this server. To report Spam,
make sure you trace to the first IP that originates the email. We will
terminate your service immediately if you report our IP as Spam by
mistake.
Received: from sm1.choicehotels.com ([10.134.137.204]) by sm1.choicehotels.com (-); Mon,
20 Jun 2016 16:42:30 -0400
X-VirtualServer: loyalty, sm1.choicehotels.com, 172.18.100.8
X-VirtualServerGroup: loyalty
X-MailingID: 16826089::375851::106950::127044::438819892::2189272_928304699_0__2231914
X-SMHeaderMap: mid="X-MailingID"
X-Destination-ID: xxxx@MPFRENCH.COM
X-SMFBL: TVNIT1BATVBGUkVOQ0guQ09N
DKIM-Signature: v=1; a=rsa-sha1; c=simple; d=your.choicehotels.com; s=sm; i=@your.choicehotels.com;
h=Content-Transfer-Encoding: Content-Disposition:Content-Type:MIME-Version:Message-ID:
X-ReportingKey:X-PVIQ:Subject:Date:To:Reply-To:From; bh=X7VU/f2M
MkS757OkGK6K/dmFq0A=; b=nbaTLhI3Wl8mLY+XLz/HvBABnyc6fxgLkamKCzBZ
JurtFL0qunzZnJCoRVqaJS8mPIt0P+ykN9Cd4+qyxyLFsxXMQJOyQsbKTS8AxXtr
CY6pDkdceqEFwjSmFwoPIBYKaNB5nbr9uYEFMT+gKCdwVSVEze0TCTBGEJFl+LQe lJc=
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Content-Type: text/html; charset="UTF-8"
MIME-Version: 1.0
Message-ID: <16826089.2189272@your.choicehotels.com>
X-ReportingKey: MJ2GF2VI9HTFB4_KK727OJJGF38ZJ10DH423KH27BM::xxxx@MPFRENCH.COM::438819892
X-PVIQ: 000320-001168-000000-375851-000000
Subject: [SPAM-HMS] You can Help the Orlando Community
Date: Mon, 20 Jun 2016 16:42:29 -0400
To: xxxx@MPFRENCH.COM
Reply-To: do-not-reply@your.choicehotels.com
From: "=?UTF-8?Q?Choice=20Privileges?=" <email_choiceprivileges@your.choicehotels.com>
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Blocked by SPF () - (Score: 6)
X-hMailServer-Reason-Score: 6



your.choicehotels.com. 10799 IN TXT "v=spf1 ip4:143.192.4.29 ip4:143.192.4.41 ip4:143.192.4.42 ip4:143.192.4.43 ip4:143.192.4.52 ip4:143.192.4.44 ip4:143.192.4.57 ip4:143.192.4.58 ip4:143.192.4.59 ip4:207.7.85.151 ip4:50.56.16.8 ip4:50.56.16.9 ip4:50.56.16.10 ip4:50.56.16.11 ip4:50.56.16.12" "ip4:50.56.16.13 ip4:207.7.84.151 ip4:199.30.90.206 ip4:199.30.94.161 ip4:143.192.4.44 ip4:74.52.181.215 mx ~all"

mail.your.choicehotels.com. 10799 IN A 143.192.4.41


Received: from sm1.choicehotels.com (sm1.choicehotels.com [50.56.16.8]) by smtp1.dnsexit.com
(8.13.8/8.13.8) with ESMTP id u5KKkial007854 for <xxxx@MPFRENCH.COM>; Mon,
20 Jun 2016 16:46:45 -0400

NOTE-SMTP1.DNSEXIT.COM IS ONE MX FOR MPFRENCH.COM

50.56.16.8 is explicitly allowed by SPF. Therefore, HMS made an error.




SPF Error Example 2:
Return-Path: bounce-21178_HTML-180248985-2947886-10142840-173@bounce.homedepotemail.com
Received: from smtp3.dnsexit.com (nd196.dnsexit.com [64.182.102.196]) by mpfrench.com
with ESMTP ; Mon, 20 Jun 2016 15:05:51 -0500
Received: from localhost (smtp-rx4 [64.182.101.67]) by smtp3.dnsexit.com (8.13.8/8.13.8)
with ESMTP id u5KK5sYU023276 for <xxxx@mpfrench.com>; Mon, 20 Jun 2016 16:05:54
-0400
Received: from smtp1.dnsexit.com ([64.182.103.22]) by localhost (smtp-rx4.dnsexit.com [64.182.101.67])
(amavisd-new, port 10024) with ESMTP id LgSZlRSA7i7I for <xxxx@mpfrench.com>;
Mon, 20 Jun 2016 16:05:56 -0400 (EDT)
Received: from mta19.homedepotemail.com (mta19.homedepotemail.com [136.147.142.96]) by
smtp1.dnsexit.com (8.13.8/8.13.8) with ESMTP id u5KK5u0o027003 for <xxxx@mpfrench.com>;
Mon, 20 Jun 2016 16:05:56 -0400
X-Notes-To-Recipient: You received this email forwarded by smtp1.dnsexit.com because you signed up
for DNS Exit Mail Forwarding, Mail Redirection or Mail Backup service for
your domain. No emails ever originated from this server. To report Spam,
make sure you trace to the first IP that originates the email. We will
terminate your service immediately if you report our IP as Spam by
mistake.
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=200608; d=email.homedepot.com; h=From:To:Subject:Date:List-Unsubscribe:MIME-Version:Reply-To:Message-ID:Content-Type;
i=HomeDepotCustomerCare@email.homedepot.com; bh=2hc7uGbhKSkknQm2LGZKkD9XKZo=;
b=BQy4AvNEYGtWDzNd8yfeoXLJrn5N+vs2HgZ68IhkijgfvPbfvGYr6QnO+Q3tfC1eaQN3hcp4msC1
iMXHwmyO6pcVlYAP3p3RICDrliLiuGdaWsgQZ+1G3TaDq99PprMXvdFsMOchC8pko/CvAijR3kUr
O5PRJcRvZg+aMJgz9pg=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=200608; d=email.homedepot.com; b=nS5DzXBkMjsCI93myj5UdZLx90fFSezjDNV6ySZyMPU8h4SPk8TE8nlZNgnVxyQ3Lw26Y0EGrogr
wjLJzoR8Qa3cOEPpeaTUO+9yPvaM6qWeyGTAdnJhl8ZQAMVug+nuFvpmDni+Q3ntzGvenk6kdAAT
SEujNC6wpxK7F1xMiaM=;
Received: by mta19.homedepotemail.com id hd169e163hsl for <xxxx@mpfrench.com>; Mon, 20
Jun 2016 20:04:42 +0000 (envelope-from <bounce-21178_HTML-180248985-2947886-10142840-173@bounce.homedepotemail.com>)
From: "The Home Depot" <HomeDepotCustomerCare@email.homedepot.com>
To: <xxxx@mpfrench.com>
Subject: [SPAM-HMS] We Have What You Are Looking For
Date: Mon, 20 Jun 2016 14:04:41 -0600
List-Unsubscribe: <mailto:leave-fce8167174630c122d502d29-fe221578756600757c1c75-fe83167971630c7573-fe6b1570746006757114-ff64157776@leave.homedepotemail.com>
MIME-Version: 1.0
Reply-To: "The Home Depot" <reply-fe83167971630c7573-21178_HTML-180248985-10142840-173@email.homedepot.com>
x-job: 10142840_2947886
Message-ID: <5e9e02c3-8f67-49b1-b8f7-5c2df0b3c25f@ind1s01mta439.xt.local>
Content-Type: multipart/alternative; boundary="YRIlw9YhZcr1=_?:"
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Blocked by SPF () - (Score: 6)
X-hMailServer-Reason-Score: 6


email.homedepot.com. 3599 IN TXT "v=spf1 include:cust-spf.exacttarget.com -all"

cust-spf.exacttarget.com. 299 IN TXT "v=spf1 ip4:64.132.92.0/24 ip4:64.132.88.0/23 ip4:66.231.80.0/20 ip4:68.232.192.0/20 ip4:199.122.120.0/21 ip4:207.67.38.0/24 " "ip4:207.67.98.192/27 ip4:207.250.68.0/24 ip4:209.43.22.0/28 ip4:198.245.80.0/20 ip4:136.147.128.0/20 ip4:136.147.176.0/20 ip4:13.111.0.0/19 -all"

Received: from mta19.homedepotemail.com (mta19.homedepotemail.com [136.147.142.96]) by
smtp1.dnsexit.com (8.13.8/8.13.8) with ESMTP id u5KK5u0o027003 for <xxxx@mpfrench.com>;
Mon, 20 Jun 2016 16:05:56 -0400

NOTE-SMTP1.DNSEXIT.COM IS ONE MX FOR MPFRENCH.COM

136.147.128.0/20 = 136.147.128.1 through 136.147.143.254; Therefore, HMS made a mistake.


Incidentally, a comment in a previous forum entry stated that nothing can come after the "-all" in an SPF TXT record. This is not true. "The modifiers defined in this document ("redirect" and "exp") MAY appear anywhere in the record, but SHOULD appear at the end, after all mechanisms." Quote from http://www.openspf.org/svn/project/spec ... #modifiers
Last edited by jimimaseye on 2016-06-21 10:05, edited 1 time in total.
Reason: Disabled interfering smileys

User avatar
RvdH
Senior user
Senior user
Posts: 739
Joined: 2008-06-27 14:42
Location: Netherlands

Re: 5.6.5-B2367 Spam Check Failures

Post by RvdH » 2016-06-21 09:57

probably due to concatenated string in the TXT DNS record, eg (in red): "v=spf1 ip4:64.132.92.0/24 ip4:64.132.88.0/23 ip4:66.231.80.0/20 ip4:68.232.192.0/20 ip4:199.122.120.0/21 ip4:207.67.38.0/24 " "ip4:207.67.98.192/27 ip4:207.250.68.0/24 ip4:209.43.22.0/28 ip4:198.245.80.0/20 ip4:136.147.128.0/20 ip4:136.147.176.0/20 ip4:13.111.0.0/19 -all"

This should be OK though...maybe martin can confirm?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-21 10:16

Both of this examples are valid failures. For the same reasons as the previous examples. There is no HMS error. And I would put money on I will be saying the same thing for every example mpfrench gives as long as the Received: from smtp1.dnsexit.com ([64.182.103.22]) header exists.

See it for yourself. Go to: http://www.kitterman.com/spf/validate.html

And in the TEST AN SPF RECORD, input

IP ADDRESS:
64.182.103.22

SPF RECORD (taken from http://mxtoolbox.com/SuperTool.aspx?act ... n=toolpage#).:
v=spf1 ip4:64.182.101.0/24 ip4:67.214.175.64/27 ip4:64.182.102.0/24 ip4:67.214.161.129/27 ip4:67.214.171.65/28 ip4:64.182.105.0/24 ip4:24.172.195.33/28 ip4:66.85.182.216/29 ip4:184.164.135.0/29 ip4:192.219.19.52/28 ip4:192.37.168.253/30 ip4:198.204.224.146/29 ip4:24.172.194.209/28 ip4:65.23.155.0/24 ip4:66.235.177.33/28 ip6:2604:4300:a:25::0/64 ip4:69.30.236.138/29 ip4:142.54.187.122/29 ip4:107.150.47.178/29 ip4:209.160.25.8/24 ip4:209.160.34.101/32 ip4:209.160.24.225/32 ip4:173.208.189.26/29 ip4:192.187.102.10/29 ip4:142.0.45.162/27 ip4:104.219.19.52/28 ip4:192.151.149.82/28 ip4:199.255.159.224/29 ip4:192.37.168.252/30 ip4:104.37.168.253/30 ip4:192.254.74.210/29 ip4:49.213.13.56/32 ip4:66.235.177.222/32 ip4:142.0.41.116/32 ip4:173.208.184.42/29 ip4:63.141.228.202/29 ~all

Like this:
capture.png
Then "TEST"

Results: Results - "softfail domain owner discourages use of this host ". That is not a pass.

Having it in your list of relays doesnt make a difference as it is not the address passing it to your server. Only the machine connecting to your server will be checked on your Internal Relay list. An in this case its the smtp3.dnsexit.com address (64.182.102.196) that is connecting to your server, noted as an Internal Relay and therefore invoking RECEIVED HEADER checks instead.



RvdH wrote:probably due to concatenated string in the TXT DNS record, eg (in red): "v=spf1 ip4:64.132.92.0/24 ip4:64.132.88.0/23 ip4:66.231.80.0/20 ip4:68.232.192.0/20 ip4:199.122.120.0/21 ip4:207.67.38.0/24 " "ip4:207.67.98.192/27 ip4:207.250.68.0/24 ip4:209.43.22.0/28 ip4:198.245.80.0/20 ip4:136.147.128.0/20 ip4:136.147.176.0/20 ip4:13.111.0.0/19 -all"

This should be OK though...maybe martin can confirm?
Thats definitely an invalid record. Empty double quotes - it has no place in an SPF record. http://www.kitterman.com/spf/validate.html
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-21 10:22

mpfrench wrote:Incidentally, a comment in a previous forum entry stated that nothing can come after the "-all" in an SPF TXT record. This is not true. "The modifiers defined in this document ("redirect" and "exp") MAY appear anywhere in the record, but SHOULD appear at the end, after all mechanisms." Quote from http://www.openspf.org/svn/project/spec ... #modifiers
Yep, noted and acknowledged. And in fact when I did some testing with an "exp" modifier (in that previous post you refer to where its placement was questioned) even being at the end after the '-all' it was accepted without question - in that case it was the content of the entry that made it invalid.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 739
Joined: 2008-06-27 14:42
Location: Netherlands

Re: 5.6.5-B2367 Spam Check Failures

Post by RvdH » 2016-06-21 12:45

jimimaseye wrote:
RvdH wrote:probably due to concatenated string in the TXT DNS record, eg (in red): "v=spf1 ip4:64.132.92.0/24 ip4:64.132.88.0/23 ip4:66.231.80.0/20 ip4:68.232.192.0/20 ip4:199.122.120.0/21 ip4:207.67.38.0/24 " "ip4:207.67.98.192/27 ip4:207.250.68.0/24 ip4:209.43.22.0/28 ip4:198.245.80.0/20 ip4:136.147.128.0/20 ip4:136.147.176.0/20 ip4:13.111.0.0/19 -all"

This should be OK though...maybe martin can confirm?
Thats definitely an invalid record. Empty double quotes - it has no place in an SPF record. http://www.kitterman.com/spf/validate.html
String concatenation is allowed as far a i read (RFC1035)
https://kb.isc.org/article/AA-00356/0/C ... cters.html
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-06-21 13:27

Another SPF checker [http://tools.bevhost.com/spf/] that I stumbled upon gives slightly different results. The Choicehotels example yeilds a permerror while the Homedepot example yields a pass.

I don't see a valid reason for the double quotes in the SPF TXT records but I'm not sure that they are technically wrong to be there.

I've been in e-mail contact with Martin on these two examples and will wait for his response before posting anything further. I'm hoping Martin shares more details of how the Incoming Relays section really works.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-21 16:53

RvdH wrote:
jimimaseye wrote:
RvdH wrote:probably due to concatenated string in the TXT DNS record, eg (in red): "v=spf1 ip4:64.132.92.0/24 ip4:64.132.88.0/23 ip4:66.231.80.0/20 ip4:68.232.192.0/20 ip4:199.122.120.0/21 ip4:207.67.38.0/24 " "ip4:207.67.98.192/27 ip4:207.250.68.0/24 ip4:209.43.22.0/28 ip4:198.245.80.0/20 ip4:136.147.128.0/20 ip4:136.147.176.0/20 ip4:13.111.0.0/19 -all"

This should be OK though...maybe martin can confirm?
Thats definitely an invalid record. Empty double quotes - it has no place in an SPF record. http://www.kitterman.com/spf/validate.html
String concatenation is allowed as far a i read (RFC1035)
https://kb.isc.org/article/AA-00356/0/C ... cters.html
Hmmm, Yes, it certainly seems so, doesnt it. http://www.openspf.org/RFC_4408#multiple-strings.

Still, I think it is all a red herring because the problem with all of these SPF failures is with DNSEXIT.COM not having its SPF records correct (and not the original senders.)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 19810
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: 5.6.5-B2367 Spam Check Failures

Post by mattg » 2016-06-22 08:26

jimimaseye wrote:Still, I think it is all a red herring because the problem with all of these SPF failures is with DNSEXIT.COM not having its SPF records correct (and not the original senders.)
Me too
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-06-22 10:13

jimimaseye wrote:SPF problem:

CONFIRMED! It is the incorrect position (and in combination of domain specific) of "exp=spferror.service.govdelivery.com" entry.
X-hMailServer-Reason-3: Blocked by SPF (%i is not a valid mail sender for the GovDelivery Service.)
- (Score: 3)
Ive been doing a little bit more reading about this.

Not only is the SPF lookup a fail, the formatting of their record is incorrect.

Reference: http://www.openspf.org/RFC_4408#mod-exp
The SPF convention says that the EXP modifier can point to other domain TXT records which then 'spell out' whatever message the domain owner (in this case, "GovDelivery" being the domain owner) wants it to say in case of a fail. And those TXT records can include a parameter "%i" which, upon resolving, should interpret the variable in to the original domain. In this case, the EXP modifier points "spferror.service.govdelivery.com" which has a TXT record:
"%i is not a valid mail sender for the GovDelivery Service." (And this is what is being displayed in the SPF failed header.)

The problem is that the calling modifier "exp=spferror.service.govdelivery.com" doesnt include the parameter being passed.

A TEST
Using "exp=spferror.mydomain" modifier and a description of "%i not allowed to send" record for the TXT 'spferror' I recreated the error:
X-hMailServer-Reason-3: Blocked by SPF (%i is not allowed to send.)
However, then adding %{d} to the spf record: "exp=spferror.%{d}" the header (%{d} being equivalent to mydomain) to invoke macro expansion and then it displayed (macro-expanded) correctly:
X-hMailServer-Reason-2: Blocked by SPF (192.168.0.200 is not allowed to send.)
(This is in accordance to the expansion explanation, quote: "For IPv4 addresses, both the "i" and "c" macros expand to the standard dotted-quad format." - was the address responsible and therefore the error is correct. A retest, changing it to %s, does in fact display the 'local-part' of the email address as according to the expansion explanation)

WHAT IS CORRECT?
My only question or doubt is whether it is correct and necessary to have the original %{d} (or other parameter) in the original SPF record to invoke expansion (as it seems in this experiment), or should HMS do the expansion of the exp=spferror.mydomain" dns record regardless. (The implementation guidelines doesnt suggest it should be necessary in my interpretation but I could be wrong)

FIX OR NOT TO FIX
But here's the nub of it all: Why anyone would fanny around with EXP modifier is beyond me - default "SPF FAIL" says enough for me. Other mail servers dont necessarily use these EXP modifier txt records. For example, doing the same test, YAHOO records its own format on the SPF fail report:
Received-SPF: fail (domain of mydomain does not designate 123.45.67.89 as permitted sender)
completely ignoring this EXP modifier text. And im pretty sure that every other spf compliant mail server will also apply its own default text which would give a perfectly adequate explanation. (HMS default is "X-hMailServer-Reason-3: Blocked by SPF () - (Score: 3)" where the blank () would normally contain the user-preferred expanded text. But surely "Blocked by SPF" says enough).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-07-19 02:33

I've been experimenting a bit and found a way to make 5.6.5-B2367 spam checking work correctly with my off-site MX. The lesson I learned was that one must populate the Incoming Relays sections with all the IP addresses that one's off-site MX could use, including its own internal hand-off IPs. Evidently, hMailServer must be told (via the Incoming Relays section) of all the IPs that belong to the external MX and its own network, i.e., every IP that could appear in a mail header that belongs to the external MX system.

Perhaps it would be best to change the hMailServer documentation of the Incoming Relays section to add the following:
"Spam checking starts by examining the IP address that connects to your Mail Exchanger (MX). Unless the Incoming Relays section is populated, hMailServer assumes that it is the MX for your domain.

When hMailServer is not the MX, it must be made aware of those IP addresses that belong to your MX and the MX’s internal hand-off IPs. Set the Incoming Relays section to all the IPs that can appear in a mail header that belong to your MX and its internal hand-offs."

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: 5.6.5-B2367 Spam Check Failures

Post by percepts » 2016-07-19 12:41

or you could just put following in your hmailserver.ini (restart hmailserver for it to take effect)

SPFPolicyOverride=v=spf1 A -all

or set to it to whichever domains,ips,MXs you want to.

It will overide all mail from all sources with whatever you set so be careful with its use. Check hmailserver.ini for other options related to spf

User avatar
mattg
Moderator
Moderator
Posts: 19810
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: 5.6.5-B2367 Spam Check Failures

Post by mattg » 2016-07-19 14:49

mpfrench wrote:Perhaps it would be best to change the hMailServer documentation of the Incoming Relays section to add the following:
https://www.hmailserver.com/documentati ... omingrelay

How does that look?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
SorenR
Senior user
Senior user
Posts: 3133
Joined: 2006-08-21 15:38
Location: Denmark

Re: 5.6.5-B2367 Spam Check Failures

Post by SorenR » 2016-07-19 14:51

percepts wrote:or you could just put following in your hmailserver.ini (restart hmailserver for it to take effect)

SPFPolicyOverride=v=spf1 A -all

or set to it to whichever domains,ips,MXs you want to.

It will overide all mail from all sources with whatever you set so be careful with its use. Check hmailserver.ini for other options related to spf
Are you using one of Bill's old versions?
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

mpfrench
Normal user
Normal user
Posts: 57
Joined: 2007-07-18 11:27

Re: 5.6.5-B2367 Spam Check Failures

Post by mpfrench » 2016-07-19 15:03

Mattg, the wording you proposed is an improvement. However, it is a bit wordy and I'm afraid a user may get lost. The most important fact which must come across to the reader is the fact that all of the possible IP addresses that are associated with an external MX must be defined in the Incoming Relays section.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: 5.6.5-B2367 Spam Check Failures

Post by percepts » 2016-07-19 15:11

SorenR wrote:
percepts wrote:or you could just put following in your hmailserver.ini (restart hmailserver for it to take effect)

SPFPolicyOverride=v=spf1 A -all

or set to it to whichever domains,ips,MXs you want to.

It will overide all mail from all sources with whatever you set so be careful with its use. Check hmailserver.ini for other options related to spf
Are you using one of Bill's old versions?
No but my understanding was that these spf policies were still active in current version. May even have been you that told me so :wink:

User avatar
mattg
Moderator
Moderator
Posts: 19810
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: 5.6.5-B2367 Spam Check Failures

Post by mattg » 2016-07-19 15:14

percepts wrote:
SorenR wrote:
percepts wrote:or you could just put following in your hmailserver.ini (restart hmailserver for it to take effect)

SPFPolicyOverride=v=spf1 A -all

or set to it to whichever domains,ips,MXs you want to.

It will overide all mail from all sources with whatever you set so be careful with its use. Check hmailserver.ini for other options related to spf
Are you using one of Bill's old versions?
No but my understanding was that these spf policies were still active in current version. May even have been you that told me so :wink:
That did work until only a few versions back, but I don't think it does in current release
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: 5.6.5-B2367 Spam Check Failures

Post by percepts » 2016-07-19 15:19

scrub that idea then.

User avatar
SorenR
Senior user
Senior user
Posts: 3133
Joined: 2006-08-21 15:38
Location: Denmark

Re: 5.6.5-B2367 Spam Check Failures

Post by SorenR » 2016-07-19 15:42

mattg wrote:
percepts wrote:
SorenR wrote: Are you using one of Bill's old versions?
No but my understanding was that these spf policies were still active in current version. May even have been you that told me so :wink:
That did work until only a few versions back, but I don't think it does in current release
Just been through 5.4 and up on GitHub... Nothing in hmailserver/hmailserver/source/Server/Common/Application/IniFileSettings.cpp on any version with reference to "SPF" so unless some other code is reading the ini-file, it was only in Bill's version.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: 5.6.5-B2367 Spam Check Failures

Post by percepts » 2016-07-19 17:12

better clean up my ini file then !!!

good job I wasn't trying to use it.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7950
Joined: 2011-09-08 17:48

Re: 5.6.5-B2367 Spam Check Failures

Post by jimimaseye » 2016-07-20 10:59

mattg wrote:
mpfrench wrote:Perhaps it would be best to change the hMailServer documentation of the Incoming Relays section to add the following:
https://www.hmailserver.com/documentati ... omingrelay

How does that look?
Matt the opening line of that documentation is wrong:
Spam checking starts by examining the IP address that connects to your Mail Exchanger (MX). Unless the Incoming Relays section is populated, hMailServer assumes that it is the MX for your domain.
The wording is ambiguous and could be read as implying if no Incoming Relays are listed then the connecting machine is the MX for your domain. It isnt.

Suggestion: Change to read "Unless the Incoming Relays section is populated with a matching address, hMailServer assumes that itself is the MX for your domain."
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 19810
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: 5.6.5-B2367 Spam Check Failures

Post by mattg » 2016-07-20 12:09

OK done
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Post Reply