Automatic whitelisting

This section contains scripts that hMailServer has contributed with. hMailServer 5 is needed to use these.
andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Automatic whitelisting

Post by andyp » 2008-12-17 22:10

This script adds all known external email addresses to the hm spam whitelist.

How does it work?
If an authenticated user sends an email to an external address, this address will be stored in a file for a defined time. You can also add addresses manually to this file. The addresses in the file and all external addresses of all distribution lists of your server will be added to hm spam whitelist. This way, trusted email addresses won't be marked or deleted as spam and won't have to wait the delay time of the greylist until they pass the first time. Since version5 you can also have a separate whitelist file.

Basically this is the same as in v4 section, but you can now add addresses manually and the script should be faster.
It works with v5, but it should also be compatible with v4. However you can use the old emailaddressfile of v4.

Installation manual and script itself is in the zip.

Version2: Separate thread for update of whitelist, which is executed in the eventhandler.
Advantages: The SMTP session is a lot shorter and you can slow the script down to reduce CPU activity. Should be a lot better for large installation or long distribution lists. You can also execute this script manually.

Version3: Increased performance

Version4: Bugfix

Version5: Bugfix - Existing email addresses were not update correctly.
New feature - separate whitelist in a file, e.g. provided by another application.

Version6: Instability fix in write_log function; improved register_emailadress function

Version7: Some functions had been missing in version6

Version8: Now all IPs of the mailservers used by registered email addresses are whitelisted as well. The old emailaddressfile will be converted.

Version9: You can now set, what you like to whitelist or not.
Attachments
awl_v5_old_versions.zip
(41.24 KiB) Downloaded 564 times
awl_v5_version9.zip
(7.25 KiB) Downloaded 960 times
Last edited by andyp on 2009-10-31 12:48, edited 10 times in total.

User avatar
M*I*B
New user
New user
Posts: 28
Joined: 2008-12-30 19:09
Location: Germany
Contact:

Re: Automatic whitelisting

Post by M*I*B » 2008-12-31 17:57

... hmmm ... Maybe I'm tooooo stupid ...

After editing and copying the file to the Folder where the EventHandlers.vbs sleeps and name it AutoWhiteList.vbs and after reload ... must be a file called emailaddresses.txt exist since the 1st eMails send out from me to external receipient?!? Is'nt it?!?

Some of my config:

hMailserver work in Folder C:\Program Files (x86)\hMailServer
I edit the INI-fite to match my own need, so the other Folder are under C:\Documents and Settings\Administrator\My Documents\Mailserver\. I edit the AutoWhiteList also to match the right folder.

1st lines of my hMailServer.INI

Code: Select all

[Directories]
ProgramFolder=C:\Program Files (x86)\hMailServer
DataFolder=C:\Documents and Settings\Administrator\My Documents\Mailserver
DatabaseFolder=C:\Documents and Settings\Administrator\My Documents\Mailserver
LogFolder=C:\Documents and Settings\Administrator\My Documents\Mailserver\~logz~
EventFolder=C:\Documents and Settings\Administrator\My Documents\Mailserver\ereignis
TempFolder=C:\temp
[GUILanguages]
ValidLanguages=english,german,swedish
1st lines of the AutoWhiteList.vbs

Code: Select all

'General
Public obApp
Public Const ipslocalhost = "0.0.0.0#87.230.89.37#127.0.0.1"  'separated by #
Public Const user = "[me myself and I]"
Public Const pw = "[what else *g*]"
Public Const logspath = "C:\Documents and Settings\Administrator\My Documents\Mailserver\~logz~\"
Public Const write_log_active = true

'Automatic whitelisting
Public Const remember_wl_address = 365
Public Const emailaddressesfile = "C:\Documents and Settings\Administrator\My Documents\Mailserver\ereignis\emailaddresses.txt"
So what's wrong?

Will be fine if someone can help...
... with much greetings ...

Micha

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2008-12-31 18:32

> and name it AutoWhiteList.vbs and after reload

You shouldn't do that. Why do you make up your own script file name?

User avatar
M*I*B
New user
New user
Posts: 28
Joined: 2008-12-30 19:09
Location: Germany
Contact:

Re: Automatic whitelisting

Post by M*I*B » 2008-12-31 18:40

... öhem ... Why not? Is not it does'nt matter how the script called?
... with much greetings ...

Micha

User avatar
mattg
Moderator
Moderator
Posts: 20302
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Automatic whitelisting

Post by mattg » 2009-01-01 09:31

Unless it is named 'EventHandlers.vbs' it won't be called by hMailServer scripts.
http://www.hmailserver.com/documentatio ... _scripting
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

westdam
Senior user
Senior user
Posts: 728
Joined: 2006-08-01 21:24
Location: Padova, Italy
Contact:

Re: Automatic whitelisting

Post by westdam » 2009-01-07 13:10

using it on my v.5.321 installation and work like a charm.

great work as the v4 :)

bye
marco

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-08 00:10

Very Nice Script, Thankyou!

However seems I'm having some performance issues. When I try to send at times it gives me a timeout error(but the emails are still sent)

35+domains and whitelist currently 1440.

Any ideas?
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-08 00:58

I think you will see performance problems if the number of white listing records gets too big. It's not really tested with more than 100-150 white listing records. I expect that you will see slow downs if you have more than a few hundreds. :-\

Maybe it's not the issue you see though, since your email messages are still sent. Also, I'm a bit surprised that you see a timeout already after 1500 records. Maybe if you turn on SMTP logging and post the error here the next time it happens I can see what's going on.

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-08 18:46

I had the same problem with large dist lists and am now using a separate script file to update the whitelist. By using a separate thread the smtp session continues within seconds and the long update process is done afterwards. It also doesn't use 100% of CPU capacity any more, because I have added breaks.

Will post it as version2 in the next days. Works fine for 2 weeks...

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-08 22:36

sounds great look forward to trying it.
Thanks again.
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-09 20:34

Posted the new script...

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-09 20:50

Nice work. Just a generic hint which can improve script performance quite a bit. When you do obApp.Domains, a list of domains are fetched from the database ad loaded into the collection. This means that if you execute the following code, the entire domain list is read three times from the database:

Code: Select all

a = obapp.Domains.Item(i) 
b = obapp.Domains.Item(i) 
c= obapp.Domains.Count
Which means that when you execute the following code, the entire domain list is read <number of domains> * 2 times from the database. The same thing applies to dom.DistributionLists and other collections. With your version of the script, if the distribution list contains 100 members, the entire list will be read 200 times from the database.

Code: Select all

Do While i <= obapp.Domains.Count - 1
  Set dom = obapp.Domains.Item(i)
Loop
To get much better performance, you can do

Code: Select all

set obDomains = obApp.Domains
Do While i <= obDomains.Count - 1
Set dom = obDomains.Item(i)
and

Code: Select all

set obDistLists = obApp.DistributionLists
for i = 0 to obDistLists.Count -1

next
and so on. For this specific script, in larger installations, I guess doing this would improve script performance ~10000% or something. :)

It's not obvious if you see it from a "API client" perspective, but when you call obapp.Domains.Item(i) , the script engine will first execute obApp.Domains which will cause the entire domain list to be read from the database. On the object returned by the Domains property, it will then call the Item property. So this single line causes two different items to be created, a Domains collection, and a Domains object.

The benefit of having it this way is that a user of the API doens't need to call Refresh() all the time before trying to access a collection. I've thought about changing this, but at the same time I'm trying to get hMailServer to keep less things in memory (less caching) and more rely on the database...

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-09 22:02

Thanks, I assume this applies to all other collections as well. I guess there will be a version3

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-09 22:23

Yep, pretty much all collections (with maybe some exception).

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 05:44

Thanks for update, it seems to be working fine.

However, I am seeing this error over and over.

Script Error: Source: (null) - Error: 80070002 - Description: (null) - Line: 135 Column: 2 - Code: (null)"

line 135 contains: WshShell.run(vbs_update_wl)
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-11 13:11

I think the variable in
Public const vbs_update_wl = ...
is not properly defined.

Replace line 135 by and you will see if script can find the file

Code: Select all

		if fs.FileExists(vbs_update_wl) then
			WshShell.run(vbs_update_wl)
		else
			write_log ("      Error, cannot find file " & vbs_update_wl)
		end if
Then I would check the WshShell object is properly created. Lines

Code: Select all

	Dim WshShell
	Set WshShell = CreateObject("WScript.Shell")
Just doubleclick the EventHandlers_awl_update.vbs file to see if it works. Update process will start after 2 seconds.

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 16:42

It only gives error when adding a new email to file, otherwise no errors.
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-11 16:55

How do you add the email to the file, could you give me an example.
Does it throw an error when executed manually or executed by hmailserver?

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 17:11

when executed by hMail
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-11 18:20

Can you execute EventHandlers_awl_update.vbs manually without error?

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 18:33

object required: obApp
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-11 18:42

Have you

Public obApp
and
Set obApp = CreateObject("hMailServer.Application")

in EventHandlers_awl_update.vbs

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 19:01

The EventHandlers_awl_update.vbs was missing Set obApp = CreateObject("hMailServer.Application")

I added it and still same error
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-11 19:57

Ups, my mistake, version3 contains an error.
add to main part.

Set obApp = CreateObject("hMailServer.Application")
Call obApp.Authenticate(user, pw)

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 20:05

OK thx
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 20:19

still same error with version4.

There's nothing after execute update in logs. Is second script being called?

"1/11/2009 1:17:02 PM" " execute update"
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 21:16

update.....

It works manual now with v4.

"1/11/2009 2:11:26 PM" " Start updating whitelist"
"1/11/2009 2:12:18 PM" " Cleared whitelist."

but Still errors with hMail running it. (hangs at execute)

"1/11/2009 2:13:48 PM" " Registering recipient addresses."
"1/11/2009 2:13:49 PM" " adding new line with xxxx@gmail.com"
"1/11/2009 2:13:49 PM" " writing emailaddressesfile"
"1/11/2009 2:13:49 PM" " execute update"
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-11 23:17

Have you tried to create a second vbs file which calls the update vbs? If this doesn't work, the error has nothing to do with hmailserver.

Code: Select all

Public Const vbs_update_wl = "C:\hMailServer\Events\EventHandlers_awl_update.vbs"

Dim WshShell
Set WshShell = CreateObject("WScript.Shell")
if fs.FileExists(vbs_update_wl) then
	WshShell.run(vbs_update_wl)
else
	wscript.echo("      Error, cannot find: " & vbs_update_wl)
end if

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 23:23

no, just the scripts that you created.

If i call it with your code i get error for object required: fs

when I set fs: it didn't error.
Last edited by DFitch on 2009-01-11 23:32, edited 1 time in total.
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-11 23:32

Add
Set fs = CreateObject("scripting.filesystemobject")
just after the other line containing CreateObject...

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 23:34

thx martin, just edited that while you were posting, it didn't error then.

However in his main script it has those variables, but errors.
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-11 23:39

So have you enabled logging in the EventHandlers_awl_update.vbs?
Does it output anything?

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-11 23:42

Just whatever logging he has coded already. Seems it never gets past that update part from the first script.

I have no scripting knowledge so no clue what to look for.
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-11 23:59

Try adding
wscript.echo("hello")
just before
WshShell.run(vbs_update_wl)
in your main script.

Then in hMailServer Administrator, refresh script, send a message, see if the "hello" is added to the log.

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-12 00:05

wasn't added
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-12 00:09

Doesn't make any sense. Mayby andyp has some idea. Could be some permission issue in the file system, or something completely different...

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-12 00:15

Tell me about it.

This is from main script.
-----------------------------------------
If upd = True Then
write_log (" execute update")
if fs.FileExists(vbs_update_wl) Then
WshShell.run(vbs_update_wl)
else
write_log (" Error, cannot find: " & vbs_update_wl)
end if
End If

--------------------------------------

Where in logs it writes "execute update" then stops.
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-12 00:32

When i execute another script with just what Andy suggested, it works fine!
------------------------
Public Const vbs_update_wl = "c:\hMailServer\Events\EventHandlers_awl_update.vbs"

Dim WshShell
Set WshShell = CreateObject("WScript.Shell")
Set fs = CreateObject("scripting.filesystemobject")
if fs.FileExists(vbs_update_wl) Then
WshShell.run(vbs_update_wl)
else
wscript.echo(" Error, cannot find: " & vbs_update_wl)
end if
---------------------

I don't see the difference and why it doesn't execute in main script
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-12 00:37

Yes, you said that.

Is the vbs_update_wl variable exactly the same in both scripts?
If so, this is probably a permission issue. Maybe the hMailServer service is unable to access the directory containing the file referenced by vbs_update_wl, or it's unable to read the file. Or something similar to that.

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-12 00:44

yes there the same.
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Automatic whitelisting

Post by martin » 2009-01-12 00:52

There's no error log created, right?

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-12 00:54

yes when hMail executes i get error: Script Error: Source: (null) - Error: 80070002 - Description: (null) - Line: 135 Column: 2 - Code: (null)"
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Re: Automatic whitelisting

Post by DFitch » 2009-01-12 02:07

update...

Ran test using absolute path of directory and the script fired away and worked properly.

As far as paths to logs, etc were fine if they contained spaces. (program files\hmailserver)

However, when trying to execute the second script it didn't like spaces so correcting with progra~1\hmailserver fixed the issues.

Thanks Andy and Martin for your time!
hMailServer 5.3.3: External MySql
Win2k3 Server | eWall 4.0 Anti-Spam Anti-Virus SMTP Proxy {http://sssolutions.net/}
SpamAssassin 3.31 - ClamAV on backend Ubuntu Server 10.04(VMware)

luci
Normal user
Normal user
Posts: 70
Joined: 2008-02-29 14:29
Location: Romania

Re: Automatic whitelisting

Post by luci » 2009-01-20 10:24

How about a version of this script that is not executed each time a message is sent ?
The whitelist can be updated by parsing the SMTP log file.
This can be executed with task scheduller, will require less CPU and will avoid the crashes I experience caused by the VBScript runtime.

This version of the script does not work well with hMailServer 4, as it crashes the server from time to time.
Radical Image Optimization Tool developer
Project Manager at CRIOSWEB

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Automatic whitelisting

Post by ^DooM^ » 2009-01-20 10:36

This version of the script is for hMailServer V5 not V4.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

luci
Normal user
Normal user
Posts: 70
Joined: 2008-02-29 14:29
Location: Romania

Re: Automatic whitelisting

Post by luci » 2009-01-20 10:43

Yes, I know, but the author said:
t works with v5, but it should also be compatible with v4.
I was just reporting a problem with v4
Radical Image Optimization Tool developer
Project Manager at CRIOSWEB

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-20 12:07

By
but it should also be compatible with v4
I ment I didn't test it, because I don't use v4 any more. Either way I tried to keep it compatible to v4. Please post your error message with version v4, in case you like help.

If you want to run a task with the os scheduler, just execute EventHandlers_awl_update.vbs by scheduler and comment the line

Code: Select all

WshShell.run(vbs_update_wl)

luci
Normal user
Normal user
Posts: 70
Joined: 2008-02-29 14:29
Location: Romania

Re: Automatic whitelisting

Post by luci » 2009-01-20 12:25

In v4 I get "Source: SocketConnection::OnDataReceived, Code: HM4377, Description: An error occured while parsing data." and the server crashes after a while. I know this is not a VBScript error, but it seems related to the script, because when disabled there are no more errors. The errors occur once or several times a day. When they are occuring they persist until the service is restarted. Sometimes the machines needs to be rebooted.

I will try running that script manually to test it. Thanks for the tip.
I don't move to v5 until I am sure this is stable enough for production use.
Radical Image Optimization Tool developer
Project Manager at CRIOSWEB

luci
Normal user
Normal user
Posts: 70
Joined: 2008-02-29 14:29
Location: Romania

Re: Automatic whitelisting

Post by luci » 2009-01-20 12:40

So what script I execute manually ? EventHandlers_awl_update.vbs or awl_script ?
Your modification reffers to awl_script. I tried to execute both but the whitelist is not updated.
In fact awl_script relies on OnAcceptMessage to execute.

I need to update the whitelist once at the end of the day based on the SMTP log. Are you sure the script parses the SMTP log ? It does not seem to me.
Radical Image Optimization Tool developer
Project Manager at CRIOSWEB

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-20 15:03

Execute EventHandlers_awl_update.vbs with the os task. Awl_script needs to be integrated in the hm eventhandler.vbs, check instruction in the zip.
I need to update the whitelist once at the end of the day based on the SMTP log. Are you sure the script parses the SMTP log ? It does not seem to me.
No and it is not designed to do that. It is best to write a skript on your own, because 80% will change then.

Error: Which script did you execute? In what line occurs the error?

luci
Normal user
Normal user
Posts: 70
Joined: 2008-02-29 14:29
Location: Romania

Re: Automatic whitelisting

Post by luci » 2009-01-20 16:10

I got errors using awl_script, executed with hmail event handlers. The errors are shown in the hmail log. I am just assuming the script is responsible, even if there are no runtime errors, because when I disable it, the errors go away.
The problem is that I don't have crashes at every script run, but from time to time (from several times in a day to maybe twice a week)
Radical Image Optimization Tool developer
Project Manager at CRIOSWEB

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-20 17:25

Still cannot help you, if you don't post me the line code where the error happens. This information is also in the error log.

If you have a large installation and many scripts are executed at the same time by different smtp session. You will get into trouble if 2 or more scripts try to access the file of whitelist storage.

luci
Normal user
Normal user
Posts: 70
Joined: 2008-02-29 14:29
Location: Romania

Re: Automatic whitelisting

Post by luci » 2009-01-20 17:53

If you have a large installation and many scripts are executed at the same time by different smtp session. You will get into trouble if 2 or more scripts try to access the file of whitelist storage.
I can't give any line number because no line number is shown. The script does not return an error, but the server crashes when using your script from time to time.
Probably that's the thing - concurent opening of the whitelist file. This can explain the impredictible nature of the problem.
I assume the problem persists in hmailserver v5 ?
Radical Image Optimization Tool developer
Project Manager at CRIOSWEB

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-01-20 18:41

Well I don't think it's the access problem, because the error message would say something about "access denied". I have that problem with the log file and am currently trying to avoid that by a lock file.

The error msg indicates the socket connection with the smtp partner might be to long and is disturbed somehow. Afterwards the data cannot be parsed. This could be the result of a long smtp session caused by the script during the update of whitelist. The last version above should has that problem as it executes the update in a separate thread. So it seems to be an internal hm problem. Maybe Martin can help you with that.

User avatar
tymewyse
Normal user
Normal user
Posts: 65
Joined: 2005-07-17 00:50
Location: Canyonville, OR
Contact:

Re: Automatic whitelisting

Post by tymewyse » 2009-03-05 06:22

BTW, Thanks for the script, it works quite well. When this script is being used, does it interfere at all with using the internal hMS white list?


Jeff

hMS v5.0 B326
SpamAssassin 3.25 (internal)
Win2k AS
TymeWyse Internet - Wireless & Dial-up Internet
P.O.Box 84 - 110 Ecklund St., Canyonville, OR 97417
tel: (541) 839-6027 - info@tymewyse.com

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-03-05 12:29

No, it doesn't interfere. But it does delete all entries with zvbs and zlst at the beginning on each update.

chrislees
Normal user
Normal user
Posts: 63
Joined: 2006-11-04 18:30
Location: UK

Re: Automatic whitelisting

Post by chrislees » 2009-03-05 17:30

andyp - This is excellent. THANKS!

Worked first time once I'd stopped rushing and concentrated! Now that I have this I feel that the other users on the box I run will accept grey listing much better. Plan to allow it to build a white list for a while then switch on grey listing.

Slightly off topic:
I understand most concepts (I think) but cannot find any forum discussions or anything in the documentation to describe the white listing tab within grey listing in the hMailServer Administrator interface itself. I realise you are not responsible but reckon you probably know the answer.

Can you direct me to anything or perhaps answer yourself:

I understand that the addresses listed in the 'White listing' section in the hMailServer Administrator interface that stands alone (that your script utilises) are not subject to the setup in Spam protection at all. However, does the grey listing's white list override grey listing or override Spam protection in the same way?

Thanks
hMailServer 5.4 (1931)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-03-05 17:37

The whitelist I am adding the addresses applies to all spam filters. The tab greylisting whitelist only to the greylisting spam protection, at least I would guess that. Never really tried the other whitelist.

chrislees
Normal user
Normal user
Posts: 63
Joined: 2006-11-04 18:30
Location: UK

Re: Automatic whitelisting

Post by chrislees » 2009-03-06 12:31

Having been confident yesterday that I had this all running nicely I have spotted what might be nothing or might just be a small problem.

I had assumed that the 'last email' date in the emailaddresses.txt file would update to reflect the last date an email was sent addressed to a user. Yesterday an outgoing mail wrote the address in successfully as you can see below, but today a mail from the same internal user to the same external user has triggered a "3/6/2009 10:17:57 AM" " writing emailaddressesfile" in the log, but the date has not changed in the emailaddresses.txt file.

The user that was mailed to yesterday AND today is the top line - chris.lees@...

Am I wrong in what I expected to happen or is there a problem with my setup?

emailaddresses.txt is included from just a few minutes ago (ie after the log shows the 'writing' entry

Code: Select all

# email	nr_emails	last_email	int_date
# add new addresses by writing every single address in a separate line

chris.lees@externaldomain.net	1	3/5/2009	39877
stephen.cooper@externaldomain.co.uk	1	3/5/2009	39877
osyjavied@externaldomain.com	1	3/5/2009	39877
m.longden-jones@externaldomain.net	1	3/5/2009	39877
clees@externaldomain.sch.uk	1	3/5/2009	39877
cliffordaleigh@externaldomain.com	1	3/5/2009	39877
sarahcalvey@externaldomain.co.uk	1	3/5/2009	39877
john.gregory@externaldomain.co.uk	1	3/6/2009	39878
hMailServer 5.4 (1931)

andyp
Normal user
Normal user
Posts: 191
Joined: 2008-01-18 21:00

Re: Automatic whitelisting

Post by andyp » 2009-03-06 22:56

It might be possible more hm threads try to access the file simultaneously. Avoiding this has not been implemented, yet. In this case the second thread would throw an error which would be definetly in hm log. Something like "access denied".

Post Reply