Caedite eos. Novit enim Dominus qui sunt eius.
I don't work on Viagra but I did miss my very important invitation to the XXX International Conference on Computer Science (i.e., 30th conference). My argument was the danger of content filtering (call it AI if you want), and that it can bite completely unexpectedly (so whitelists are of no use).It seems to me that people legitimately working with Viagra, etc must know that there is an issue with spam and have already worked up solutions
The day when I see a server behaving in this way, greylisting is dead.Someone sends in an email
Server farm attempts delivery from ip a.b.c.d
Hits greylist.
Server reattempts delivery from j.k.l.m
Hits greylist
Server reattempts delivery from w.x.y.z
This confirms my assumption! You see: my technique reduces a million of IPs to two hundred. Worst case is 82, which means that typically their messages would pass after some 40 attempts. Not great, but better than half million attempts as it is now.That's 181 netblocks or 1.328.916 individual IPv4 addresses
(I can't agree more!)What a world. People really have too much time to waste.
Sorry for an off-topic comment... but there is much reason in social networks, actually an interplay of many reasons.There is nothing SOCIAL about modern social networks.
Oops no. With 82 subnets, with probability of 0.5 the message will pass by 10th attempt. With probability of 0.94, it will pass by 20th attempt, i.e., in one day. The probability of not passing by 40th attempt is 0.000005, and by 50th attempt (2 days) is 0.000000001. Given that a decent server should make at least 50 attempts, in non-time-critical applications no special treatment is needed.Worst case is 82, which means that typically their messages would pass after some 40 attempts.
not sure what religion you mean, but common religions - particularly Abrahamic 3 - strictly impose modesty and fairness, forbid egocentrism and illusionism.gelbukh wrote: ↑2018-09-15 06:00Another SOCIAL function of social networks is a substitute for religion: a human being wants to be a part of eternity and to be in the center of the Universe. In the past, religion catered for this need. Now, when it is mostly lost, social networks picked up the function of a supernatural ether to which our souls belong. With this, any worth-for-nothing person having nothing to do with his useless life has an illusion of spiritual fusion with eternity when he posts his stupid selfie in The Cloud (= The Universe) where Somebody surely watches it. Anyway, better than narcotics, and serves the same SOCIAL goal.
I do indeed have a decent server, and my decent server gives up after 3 attempts. Gmail tries for 3 days but a quick (incomplete) Google search did not yield the number of attempts. The number of attempts is up to each administrator, so there's no golden rule to follow. You can't rely on what "a decent server" should do.
So, these services do not use a reasonably limited number of subnets?Amazon and their "mail for hire - amazonses.com" service is all over the map and so is Linkdin.
Yes they do, https://aws.amazon.com/blogs/messaging- ... addresses/
Code: Select all
^(a)[0-9]{1,2}(-)[0-9]{1,3}(\.smtp-out\.([a-z]{2}\-[a-z]{4,5}\-\d{1,2}\.)?amazonses\.com)$Code: Select all
^(mail[a-z]{1}\-[a-z]{2})(\.linkedin\.com)$Code: Select all
Case "amazonses.com"
oRegEx.Pattern= "^54\.240\.([0-9]|[1-5][0-9]|6[0-3])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
"^199\.127\.(2(3[2-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
"^199\.255\.(1(9[2-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$"
case "linkedin.com"
oRegEx.Pattern= "^199\.101\.162\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
"^108\.174\.3\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
"^108\.174\.6\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
"^108\.174\.0\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
"^199\.101\.161\.130$"
This is the only bit I was after, just with various ways to show the allowed IP range
1 line of code? Go ahead and make the changes yourself then... the project is on github for a reasongelbukh wrote: ↑2018-09-16 04:01If greylisting is dead and the delays are the way to go, then it would be better to have the delays as one of the anti-spam mechanisms in HMS itself, to enjoy the same treatment (skip on SPF pass, A or MX, whitelisting) as other anti-spam mechanisms. The minimum modification would be to select the action for greylisting: rejection (as now) or delay. Nearly one line of code changes, in the file greylisting.cpp.
FYI, I think you changed:
Yes, exactly... Not sure what I was looking at when I whote that. My diff's say exactly the same as you wrote.
Code: Select all
#!/usr/bin/perl
$domain=shift @ARGV;
@results=getit($domain);
sub getit {
my $domain=shift;
my @foo=`nslookup -q=TXT $domain`;
my @results=();
foreach (@foo) {
next if not /$domain\ttext/;
s/$domain\ttext = "v=spf1//;
@results=split /\s+/;
foreach (@results) {
next if /-all/;
print "$_\n";
if (/include:/) {
s/include://;
getit($_);
}
}
}
}This works on Windows. Changes are marked by CHANGED.Not 100% sure what to change to make it run on Windows...
Code: Select all
#!/usr/bin/perl
$domain=shift @ARGV;
@results=getit($domain);
sub getit {
my $domain=shift;
my @foo=`nslookup -q=TXT $domain`;
my @results=();
foreach (@foo) {
next if not /^\s*"v=spf1/; # CHANGED
s/^\s*"v=spf1//; # CHANGED
@results=split /\s+/;
foreach (@results) {
next if /-all/ or /^$/; # CHANGED
if (/include:/) {
s/include://;
getit($_);
}
else { # CHANGED
print "$_\n"; # CHANGED
}
}
}
}
Well, can we fix it? Here is an idea:Greylisting was great until the mass email domain hosters started up - now it just doesn't work.
Code: Select all
With CreateObject("WScript.Shell")
iReturn = .Run("""spfverify.exe"" 174.125.82.54 google.com", 0, True)
if iReturn = 0 Then
EventLog.Write("pass")
REM add to the greylistwhitelist
elseif iReturn = 1 Then
EventLog.Write("not pass")
else
EventLog.Write("Command error")
End if
End WithDid you look at this ?RvdH wrote: ↑2018-09-17 13:22@SorenR
Isn't this doing exactly that, walking the SPF?
Downside of this method is the fact the greylistwhitelist could get very long and causing performance issues in hmailserver, and it could hold entries one might never get a connection from
Anyway, besides that i liked the idea off walking the SPF, so i'm working on something in C# to walk the (ip4) spf records and be able to dynamically add entries to the greylistwhitelist
What the program does is basically doing what i did manually first, you still validate HELO/EHLO like before using regexp, then you call the program which first walks the SPF records, adds them to a regexp pattern and verify the connecting ip address matched against this...eliminating the need of predefined SPF IP regexp ranges, inspired by mattg's question if this could be done on the flyCode: Select all
With CreateObject("WScript.Shell") iReturn = .Run("""spfverify.exe"" 174.125.82.54 google.com", 0, True) if iReturn = 0 Then EventLog.Write("pass") REM add to the greylistwhitelist elseif iReturn = 1 Then EventLog.Write("not pass") else EventLog.Write("Command error") End if End With
No, but basically it does the same... ip4 only though, hmailserver does not support ip6 in greywhitelisting to my knowledge
Code: Select all
3416 "2018-09-17 15:45:47.555" "spfverify.exe 40.107.13.82 passed for: EUR01-HE1-obe.outbound.protection.outlook.com"
3408 "2018-09-17 15:48:11.127" "spfverify.exe 209.85.219.172 passed for: mail-yb1-f172.google.com"
Huh
Had to re-read it again...
Code: Select all
Function Lookup(strRegEx, strMatch) : Lookup = False
With CreateObject("VBScript.RegExp")
.Pattern = strRegEx
.Global = False
.MultiLine = True
.IgnoreCase = True
If .Test(strMatch) Then Lookup = True
End With
End Function
Function oLookup(strRegEx, strMatch, bGlobal)
With CreateObject("VBScript.RegExp")
.Pattern = strRegEx
.Global = bGlobal
.MultiLine = True
.IgnoreCase = True
Set oLookup = .Execute(strMatch)
End With
End Function
Function LongIntegerFromIP(p_strIP)
Dim arrTemp, i, lngTemp
arrTemp = Split(p_strIP, ".")
For i = 0 To UBound(arrTemp)
lngTemp = lngTemp + CLng(arrTemp(i)) * (256 ^ (3 - i))
Next
LongIntegerFromIP = lngTemp
End Function
Function isAutoBan(oMessage)
Dim strRegEx, i, a, Match, Matches, m_strIP, m_strLowerIP, m_strUpperIP
Dim oApp : Set oApp = CreateObject("hMailServer.Application")
Call oApp.Authenticate(ADMIN, PASSWORD)
strRegEx = "(?:[0-9]{1,3}\.){3}[0-9]{1,3}"
For i = 0 To oMessage.Headers.Count-1
If oMessage.Headers(i).Name = "Received" Then
If Lookup("by backup-mx.post.tele.dk", oMessage.Headers(i).Value) Then
Set Matches = oLookup(strRegEx, oMessage.Headers(i).Value, False)
If Matches.Count > 0 Then
For Each Match In Matches
m_strIP = LongIntegerFromIP(Match.Value)
For a = 0 To oApp.Settings.SecurityRanges.Count-1
If (oApp.Settings.SecurityRanges.Item(a).Priority = 20) Then
m_strLowerIP = LongIntegerFromIP(oApp.Settings.SecurityRanges.Item(a).LowerIP)
m_strUpperIP = LongIntegerFromIP(oApp.Settings.SecurityRanges.Item(a).UpperIP)
If (m_strIP >= m_strLowerIP) And (m_strIP <= m_strUpperIP) Then
Result.Value = 2
Result.Message = "5.7.1 CODE08 The SMTP service on IP address (" & Match.Value & ") is not welcome here."
Exit Function
End If
End If
Next
Next
Exit Function
End If
End If
End If
Next
End FunctionCode: Select all
m_strLowerIP = LongIntegerFromIP(oApp.Settings.SecurityRanges.Item(a).LowerIP)
m_strUpperIP = LongIntegerFromIP(oApp.Settings.SecurityRanges.Item(a).UpperIP)
If (m_strIP >= m_strLowerIP) And (m_strIP <= m_strUpperIP) Then
Result.Value = 2
Result.Message = "5.7.1 CODE08 The SMTP service on IP address (" & Match.Value & ") is not welcome here."
Exit Function
End IfCode: Select all
Dim oRegEx
Set oRegEx = CreateObject("VBScript.RegExp")
oRegEx.IgnoreCase = True
oRegEx.Global = False
oRegEx.Pattern= "^([a-z]{3}[\d]{2}\-[a-z]{2}[\d]\-)(obe\.outbound\.protection\.outlook\.com)$|" &_
"^(mail\-[a-z]{2}[\d]\-f[\d]{1,3})(\.google\.com)$|" &_
"^(o[\d]{1,2})\.(email\.wetransfer\.com)$|" &_
"^(o[\d]{1,2})\.(email\.airbnb\.com)$|" &_
"^(o[\d]{1,2}\.sg|mail\-[\w]{2,3}[\d]+|mailout\-[\w]{2,3}\-[\w]{2,3})(\.booking\.com)$|" &_
"^(mail[a-z]{1}\-[a-z]{2})(\.linkedin\.com)$|" &_
"^(spruce\-goose\-[a-z]{2}|spring\-chicken\-[a-z]{2})(\.twitter\.com)$|" &_
"^(mx\-out\.facebook\.com)$|" &_
"^(cpsmtpb\-ews[\d]{2,3}|cpsps\-ews[\d]]{2,3}|cpdelvb\-safe[\d]]{2,3})(\.kpnxchange\.com)$|" &_
"^(lb[\d]{1}\-smtp\-cloud[\d]{1})(\.xs4all\.net)$|" &_
"^(marktplaats\.nl)$|" &_
"^(a)[0-9]{1,2}(-)[0-9]{1,3}(\.smtp-out\.([a-z]{2}\-[a-z]{4,5}\-\d{1,2}\.)?amazonses\.com)$|" &_
"^(o[\d]{1,2})(\.em\.spotify\.com)$|" &_
"^(webgrid[a-z]{1}\d{3}\.emsecure\.net)$|" &_
"^(mailrelay|mailsec)\d{3}(\.isp\.belgacom\.be)$"
If oRegEx.Test(oClient.HELO) Then
Call AddGreyList(oClient.IPAddress, oClient.HELO)
Result.Value = 0
Exit Sub
End If
Set oRegEx = Nothing
Code: Select all
Sub AddGreyList(ByVal strIP, ByVal strHELO)
dim iReturn : iReturn = 2
dim hostname : hostname = getDomainName(strHELO)
Dim oApp
Set oApp = CreateObject("hMailServer.Application")
Call oApp.Authenticate("Administrator", sAdminPassword)
With LockFile("C:\Program Files (x86)\hMailServer\Temp\greylistwhite.lck")
On Error Resume Next
oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Refresh
If oApp.Settings.AntiSpam.GreyListingWhiteAddresses.ItemByName(strIP) Is Nothing Then
With CreateObject("WScript.Shell")
iReturn = .Run("""C:\Program Files (x86)\hMailServer\Events\spfverify.exe"" " & strIP & " " & hostname & "", 0, True)
End With
if iReturn = 0 Then
EventLog.Write("spfverify.exe " & strIP & " passed for: " & hostname)
With oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Add
.Description = Date & " Auto-Added '" & strHELO & "'"
.IPAddress = strIP
.Save
End With
ElseIf iReturn = 1 Then
EventLog.Write("spfverify.exe " & strIP & " failed for: " & hostname)
Else
EventLog.Write("spfverify.exe command error, spfverify.exe " & strIP & " failed for: " & hostname)
End if
Else
With oApp.Settings.AntiSpam.GreyListingWhiteAddresses.ItemByName(strIP)
.Description = Date & " Auto-Added '" & strHELO & "'"
.Save
End With
End If
oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Refresh
On Error Goto 0
.Close '// Close LockFile
End With
Set oApp = Nothing
End Sub
Function getDomainName(byVal strHELO)
dim aryDomain, str2ndLevel, strTopLevel
getDomainName = Null
If Len(strHELO) > 0 Then
aryDomain = Split(strHELO,".")
If uBound(aryDomain) >= 1 Then
str2ndLevel = aryDomain(uBound(aryDomain)-1)
strTopLevel = aryDomain(uBound(aryDomain))
getDomainName = str2ndLevel & "." & strTopLevel
End If
End If
End Function
Function LockFile(strPath)
Const Append = 8
Const Unicode = -1
With CreateObject("Scripting.FileSystemObject")
Dim oFile, i
For i = 0 To 30
On Error Resume Next
Set oFile = .OpenTextFile(strPath, Append, True, Unicode)
If (Not Err.Number = 70) Then
Set LockFile = oFile
On Error Goto 0
Exit For
End If
On Error Goto 0
Wait(1)
Next
End With
Set oFile = Nothing
If (Err.Number = 70) Then
EventLog.Write("ERROR: EventHandlers.vbs")
EventLog.Write("File " & strPath & " is locked and timeout was exceeded.")
Err.Clear
ElseIf (Err.Number <> 0) Then
EventLog.Write("ERROR: EventHandlers.vbs : Function LockFile")
EventLog.Write("Error : " & Err.Number)
EventLog.Write("Error (hex) : 0x" & Hex(Err.Number))
EventLog.Write("Source : " & Err.Source)
EventLog.Write("Description : " & Err.Description)
Err.Clear
End If
End Function
Function Wait(sec)
With CreateObject("WScript.Shell")
.Run "timeout /NOBREAK /T " & Int(sec), 0, True
' REM .Run "sleep -m " & Int(sec * 1000), 0, True
' REM .Run "powershell Start-Sleep -Milliseconds " & Int(sec * 1000), 0, True
End With
End Function
Code: Select all
spfverify.exe ipaddress hostname [-verbose]
Code: Select all
spfverify.exe 5.57.20.177 booking.com
IP address matches SPF record(s) ip range for domain (return code = 0)