hMailCleaner (Compare hMaildatabase with files on disk)

This section contains scripts that hMailServer has contributed with. hMailServer 4 is needed to use these.
Post Reply

Was this script usefull to you?

Yes
13
87%
No
1
7%
Files on harddrive and hmaildatabase match
1
7%
 
Total votes: 15

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

hMailCleaner (Compare hMaildatabase with files on disk)

Post by large » 2007-08-02 09:03

I have been using hMailserver from the early betas and love the job Martin does! Thanks; keep up the good work. This script is how I can give something back to Martin & the community :)

I recently changed the server hardware and copied the %programfiles%\hMailServer\Data\ to the new disk. After checking the filecount against the SQL-database the actual filecount was 20906 against 4655 in the database. Meaning that I had 16251 files taking up space without a reference. That was a waste of space (around 1,5GB).

I dunno how this happened, but I had a faulty RAID controller that made random files writeprotected.

Anyway, here is a Description of the script:
It makes a recursive list of a given directory (usally where you store the *.eml files :)) and then it compares to the filenames that are stored in the hmailserver database. If there is a mismatch the file will be renamed to *.deleteme

Who need this?
If you're running a hmailserver and had a couple of lockups or simular I guess you might have some "leftovers" on the disk.

The script (hMailCleaner.vbs):
Please run the script in a command prompt using cscript.exe!

Code: Select all

' ====================================================
' hMailCleaner by Lars Werner (http://lars.werner.no/)
' ====================================================
' Usage: cscript hmailcleaner.vbs "data-path" adminpassword [/delete]
'
' How does it work?
' -----------------
' The script makes a recursive list of a given directory (the place where you store the *.eml files) and
' then it compares to the filenames that are stored in the hmailserver database.
' If there is a mismatch the file will be renamed to *.deleteme
'
'WARNING: If /delete option is given then all the mismatched files will be permanently deleted!!!
'
' Who need this?
' --------------
' If you're running a hmailserver and had a couple of lockups or simular I guess you might have some "leftovers" on the disk.
' This script marks the files that aren't in the database or even delete them directly!
' Pretty usefull if you have a crappy RAID controller that makes everything writeprotected, like I did ;)
'
'Disclaimer: I use hMailserver in a small scale! This script has only been tested with my server & setup and it might crash or harm your system. So please take backup, and don't come to me crying about it!

Option Explicit

Dim aFiles() 'Array of all the files in the given path
Dim aMailFiles() 'Array of files in the database
Dim iCount, iArrayCount, iPercentage, iLastPercentage
Dim bDelete  'Really delete?
Dim objFSO, objFolder, colFiles, objFile

'===========================
'== SUBS & FUNCTIONS
'= Store each file from a given directory into an array
Function ArrayOfFiles(Folder) 'Returns an object of objFolder (used in recursive mode)
	Dim SubFolder
	For Each Subfolder in Folder.SubFolders
		Set objFolder = objFSO.GetFolder(Subfolder.Path)
		Set colFiles = objFolder.Files
		Redim preserve aFiles(UBound(aFiles) + colFiles.Count) 'Resize the array with the new files we got
		For Each objFile in colFiles
			if StrComp(Right(objFile.Name, 9), ".deleteme") <> 0 then
				aFiles(iCount) = objFolder.Path & "\" & objFile.Name
			end if
			iCount = iCount + 1
		Next
		ArrayOfFiles Subfolder 'Recursive call
	Next
End Function

'===========================
'= Store each db "file" into an array
Sub FillMailArray(sAdminPw)
	Dim oApp
	Set oApp = CreateObject("hMailServer.Application")

	' Authenticate the client.
	Call oApp.Authenticate ("Administrator", sAdminPw)

	Dim iDomains, iAccounts, iMessages
	iArrayCount = 0
	ReDim aMailFiles(iArrayCount)

	for iDomains = 0 to oApp.Domains.Count - 1
		Dim oDomain
		Set oDomain = oApp.Domains.Item(iDomains)
		WScript.Echo "Domain: " & oDomain.Name
	
		for iAccounts = 0 to oDomain.Accounts.Count - 1
			Dim oAccount
			set oAccount = oDomain.Accounts.Item(iAccounts)
			WScript.Echo "	-> " & oAccount.Address
			WScript.Echo "		> Messages: " & oAccount.Messages.Count
			Redim preserve aMailFiles(UBound(aMailFiles) + oAccount.Messages.Count)
			for iMessages = 0 to oAccount.Messages.Count - 1
				Dim oMessage
				set oMessage = oAccount.Messages.Item(iMessages)
				aMailFiles(iArrayCount) = oMessage.Filename
				iArrayCount = iArrayCount + 1
				set oMessage = nothing
			Next 'End iMessages
			set oAccount = nothing
		Next 'End iAccounts
		set oDomain = nothing
	Next 'End iDomains

	set oApp = nothing
End Sub

'===========================
'== MAIN
bDelete = false

'Check if we're using wscript (quit) or cscript (ok)
if InStr(lcase(WScript.Fullname), "wscript") > 0 then
	WScript.Echo "Please do not run this program from windows! Use the command prompt and cscript.exe instead"
	WScript.Quit
end if

'Check how many arguments we have
if WScript.Arguments.Count >=2 then
	Set objFSO = CreateObject("Scripting.FileSystemObject")
	iCount = 0 'Overall counter to know where we are in the file-array
	Set objFolder = objFSO.GetFolder(Wscript.Arguments(0))
	Set colFiles = objFolder.Files
	Redim aFiles(colFiles.Count)
	For Each objFile in colFiles
		'Ignore already "deleted" messages
		if StrComp(Right(objFile.Name, 9), ".deleteme") <> 0 then
			aFiles(iCount) = objFolder.Path & "\" & objFile.Name
		end if
		iCount = iCount + 1
	Next

	'Just wait message for the user
	WScript.Echo "Please wait, creating filelist for: " & Wscript.Arguments(0)

	'Now get an recursive list of files
	ArrayOfFiles objFSO.GetFolder(Wscript.Arguments(0))

	'Now create an array of files from the database
	FillMailArray(Wscript.Arguments(1))

	'Print out the info before we start renaming
	WScript.Echo
	WScript.Echo " Number of physical files on disk: " & UBound(aFiles)
	WScript.Echo "Number of files in hmail database: " & UBound(aMailFiles)
	WScript.Echo

	'Check if aMailFiles is bigger than aFiles and give a warning!
	if UBound(aFiles) < UBound(aMailFiles) then
		WScript.Echo "WARNING: hMaildatabase contains more files than there actually is on the hard drive, this will hMailServer to give ""failed to open"" messages in IMAP"
		WScript.Echo
	end if

	'Check if we're gonna delete & Write warning if we're gonna delete the file
	if WScript.Arguments.Count = 3 then
		if InStr(lcase(WScript.Arguments(2)), "/delete") then
			bDelete = true
			WScript.Echo "WARNING: Mismatched files will be permanently deleted! (To abort press CTRL-C, 3 sec delay)"
			WScript.Echo
			WScript.Sleep 3000
		else
			WScript.Echo "WARNING: Unknown third argument, files will be renamed!"
			WScript.Echo
		end if
	end if

	'Now loop through the files & compare with the database files
	iLastPercentage = 0 'Stores the last procentage so it does not update so much
	WScript.Echo "Please wait while the files are compared..."
	for iCount = 0 to UBound(aFiles) - 1

		'Check if we can find the filename in the database
		Dim bFound, sComp
		bFound = false

		'We only do the check when there is an actual file!
		if Len(aFiles(iCount)) <> 0 then
			'Get the filename we are gonna to compare with (remove the directory)
			sComp = right(lcase(aFiles(iCount)), len(aFiles(iCount)) - len(Wscript.Arguments(0))) 'We only compare the info behind selected dir :)
			if Left(sComp,1) = "\" then sComp = Right(sComp, len(sComp)-1) 'Remove the \ if it exists...

			'Now check with the databaselist
			for iArrayCount = 0 to UBound(aMailFiles)
				'Check if the entry actually is an entry
				if(Len(aMailFiles(iArrayCount))) <> 0 then
					if StrComp(sComp, lcase(Right(aMailFiles(iArrayCount), Len(sComp))), 0) = 0 then
						aMailFiles(iArrayCount) = ""
						bFound = true
						exit for
					end if
				end if
			Next

			'If we haven't found the file we are going to rename/delete it
			if bFound = false then
				if bDelete = false then
					call objFSO.MoveFile(aFiles(iCount), aFiles(iCount) & ".deleteme")
					WScript.Echo "Renamed: " & aFiles(iCount)
				else
					call objFSO.DeleteFile(aFiles(iCount))
					WScript.Echo "Deleted: " & aFiles(iCount)
				end if
			end if
		end if 'Len(aFiles(iCount) <> 0

		'Here we write the progress (5% at the time)
		iPercentage = FormatNumber((iCount * 100) / UBound(aFiles), 0)
		if iPercentage - iLastPercentage = 5 then
			WScript.Echo "Progress: " & iPercentage & "%"
			iLastPercentage = iPercentage
		end if

	Next

	WScript.Echo
	WScript.Echo "Finished!"

	'Cleanup of objects
	Set colFiles = nothing
	Set objFolder = nothing
	set objFSO = nothing
else
	WScript.Echo "===================================================="
	WScript.Echo "hMailCleaner by Lars Werner (http://lars.werner.no/)"
	WScript.Echo "===================================================="
	WScript.Echo
	WScript.Echo "Usage: cscript hmailcleaner.vbs ""data-path"" adminpassword [/delete]"
	WScript.Echo
	WScript.Echo "Example: cscript hmailcleaner.vbs ""C:\Program Files\hMailServer\Data"" AdminPw"
	WScript.Echo "With logfile: cscript hmailcleaner.vbs ""C:\Program Files\hMailServer\Data"" AdminPw >log.txt"
	WScript.Echo "(The log.txt will be placed the current directory that you're running the script from, no output will be shown)"
	WScript.Echo
	WScript.Echo "How does it work?"
	WScript.Echo "-----------------"
	WScript.Echo "The script makes a recursive list of a given directory (the place where you store the *.eml files) and then it compares to the filenames that are stored in the hmailserver database."
	WScript.Echo "If there is a mismatch the file will be renamed to *.deleteme (default behaviour)"
	WScript.Echo
	WScript.Echo "WARNING: If /delete option is given then all the mismatched files will be permanently deleted!!!"
end if
Please give feedback on the poll!

Warning: I use hMailserver in a small scale! This script has only been tested with my server & setup and it might crash or harm your system. So please take backup, and don't come to me crying about it!
Last edited by large on 2007-08-02 10:51, edited 4 times in total.
Lars Werner
http://lars.werner.no
Check out my tools:
http://lars.werner.no/unpacker/ - 100% automated extraction tool
http://lars.werner.no/sizeme/ - Maximize the output on a given media (like CD/DVD ect)

User avatar
bagu
Normal user
Normal user
Posts: 219
Joined: 2005-06-17 03:08
Location: France
Contact:

Post by bagu » 2007-08-02 09:24

Usefull but popups are really anoying

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

Post by large » 2007-08-02 09:27

Please do not run this script directly from Windows (by doubleclicking on it).

Run it from a command prompt with cscript.exe command :)

Edit:
I've updated the script to notify the user if he doubleclicked on the hMailCleaner.vbs file
Lars Werner
http://lars.werner.no
Check out my tools:
http://lars.werner.no/unpacker/ - 100% automated extraction tool
http://lars.werner.no/sizeme/ - Maximize the output on a given media (like CD/DVD ect)

GlenC
Senior user
Senior user
Posts: 680
Joined: 2004-08-17 23:31
Location: Santiago, Chile

Post by GlenC » 2007-08-03 01:53

Hey that was really nice. I always wondered if I was in sync or not (mostly due to a lot of experimentation I've done in the past). For the record, I only had two orphans.. both 0 byte files.

Thanks large... and Martin too for the reliable mailserver. :)

cnsjfruiz
New user
New user
Posts: 1
Joined: 2007-11-15 07:25

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by cnsjfruiz » 2007-11-16 08:41

Could you provide me assistance for using your script to remove the entire directory instead of just removing the eml files? For example if you deleted an account but for some reason the directory did not get deleted you could run this script to remove orphan directories? By the way the script is awesome!! Thank you in advance!

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

Post by large » 2007-11-16 09:40

cnsjfruiz:
Well if you want to delete empty directories on a subuser you should modify the code like this:

Code: Select all

         'If we haven't found the file we are going to rename/delete it
         if bFound = false then
            if bDelete = false then
               call objFSO.MoveFile(aFiles(iCount), aFiles(iCount) & ".deleteme")
               WScript.Echo "Renamed: " & aFiles(iCount)
            else
               call objFSO.DeleteFile(aFiles(iCount))
               RmDir Left(aFiles(iCount), InStrRev(aFiles(iCount), "\")-1) 'Removes the directory if it is empty.
               WScript.Echo "Deleted: " & aFiles(iCount)
            end if
         end if
      end if 'Len(aFiles(iCount) <> 0 
It has not been tested, but it should work.

If you actually delete an user then a simple deltree of that username should be the simplest solution (but thats not what this script is all about)
Lars Werner
http://lars.werner.no
Check out my tools:
http://lars.werner.no/unpacker/ - 100% automated extraction tool
http://lars.werner.no/sizeme/ - Maximize the output on a given media (like CD/DVD ect)

Kaan1983
Senior user
Senior user
Posts: 595
Joined: 2007-01-30 16:26
Location: TÜRKIYE

Post by Kaan1983 » 2007-11-16 10:43

how is it different than hMailSynchronizer?

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

Post by large » 2007-11-16 11:01

I guess you are referring to Data directory synchronizer (DDS): http://www.hmailserver.com/documentatio ... rguide_dds ?

This script does not import any e-mails and it does not alter your database. It just cleans up the mails on the harddrive that don't have a match in the database.

DDS is used when you have had a database error or simular and want to import mail that exists on the harddrive but not in the database.

On my server all the "leftovers" where spam (or files contained virus). I guess a file-lock or similar accrued when external programs checks a file sometimes.

Kaan1983
Senior user
Senior user
Posts: 595
Joined: 2007-01-30 16:26
Location: TÜRKIYE

Post by Kaan1983 » 2007-11-16 14:25

Well I don't have any "leftovers" yet but if I have one day how can I be sure that it's spam. Or even if it's a spam and if I get it to the database with DDS it'll be one spam for one of the users maybe. I mean what's the ratio of these leftovers to the users..

Or maybe instead of deleting em immediately you can send em again to the recipient and hope the anti-spam and virus scanner work fine this time.

Also if you find such an e-mail that none of the spam scanners recognize neither virus scanners and doesn't reach to the recipient then you can use that file for hacking :D

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

Post by large » 2007-11-16 14:51

Some people have leftovers for different reasons. You don't know if it's spam before you take some samples of the leftovers.

The script renames (or deletes based on your choice) the files to <oldfilename>.deleteme, so you can check the content afterwards.

Remember that the DDS import mails under a user-directory without a filter-, spam- or viruscheck. You can risk importing a bunch of unwanted mails into the users account.
If a file is present on the harddrive and does not exists on the database changes are high that the content is unwanted, if you haven't experienced any database problems.



When you look at my stats earlier; 16251 eml-files that are not listed in the database. Cause on my system where;
* Realtime virusscan locking files after clam-win check. hMailserver couldn't delete the file, but it cleaned it from the database of cause)
* Faulty raid-system that sometimes made files locked (unlocked on next reboot). hMailserver cannot delete, but cleans up in the database.

On my account alone it would give me over 2000 spam/virus mails that I actually needed to delete manually if I used DDS (because of the bypass of checks). So this script wasn't made for fun you know. It is a handy tool if you want to check if there are leftovers and you can reuse the array-procedures to actually make a tree-view for instance in an asp/php-page :)

Kaan1983
Senior user
Senior user
Posts: 595
Joined: 2007-01-30 16:26
Location: TÜRKIYE

Post by Kaan1983 » 2007-11-16 19:50

Please don't get me wrong, beyond being unexperienced I didn't even study anything related to computer sciences... I'm trying to learn.

I only have 200 eml files at the total and 0 leftovers :) But, in your case, over 2000 leftovers should be boring.

Btw, I don't wanna pass over my boudaries but the same think could've be done in a more effiecient way by querying the DB.

duke16
Normal user
Normal user
Posts: 40
Joined: 2007-03-20 12:13

Post by duke16 » 2007-12-18 16:09

Is this differerent from the addon that comes with hMailserver? DataDirectorySynchronizer.exe

rodolfor
Senior user
Senior user
Posts: 282
Joined: 2005-06-30 09:05
Location: Gubbio - Italy

Post by rodolfor » 2007-12-25 11:57

Warning:
1. the script check also files in \data and could rename/delete emails also if there are in normal transit (and not inserted in database).
2. if you have a lot of messages, it could take hours to complete, comparing files and database as they are in very different times.

To avoid this during this script hmailserver must be isolated from the users.

I suggest to check only files older more than a day.

thanks

BrandonH75
Normal user
Normal user
Posts: 41
Joined: 2007-06-18 20:46
Location: MN

Post by BrandonH75 » 2007-12-28 00:56

I've been thinking I needed something like this for a while. I'm glad I finally found it. :)

I'm having an issue with it though. First, I don't know anything about VB. When I run the script it gets up to "Please wait while the files are compared...", then I get the error...

"C:\hMailServer\Addons\Utilities\hmailcleaner.vbs(177, 16) Microsoft VBScript runtime error: File not found"

...and the script stops.

The first time I ran it, it found some leftovers in the Data directory and a few in a couple of the accounts folders before it gave that error. I deleted those files, and every run after that it doesn't find anything before it errors out.

There is about a 30k difference in the file count. :P Am I doing something wrong?

Thanks!

redrummy
Senior user
Senior user
Posts: 370
Joined: 2007-06-21 06:52
Location: Alaska

Post by redrummy » 2007-12-28 05:23

Line 177 is in the code that renames or deletes orphaned files:

Code: Select all

         'If we haven't found the file we are going to rename/delete it 
         if bFound = false then 
            if bDelete = false then 
               call objFSO.MoveFile(aFiles(iCount), aFiles(iCount) & ".deleteme") 
               WScript.Echo "Renamed: " & aFiles(iCount) 
            else 
               call objFSO.DeleteFile(aFiles(iCount)) 
               WScript.Echo "Deleted: " & aFiles(iCount) 
            end if 
         end if 
      end if 'Len(aFiles(iCount) <> 0 
I suspect the server contents change before the script completes, thus corrupting the results (see rodolfor's post before yours). This does seem a bit dangerous as hMS has to be running for the script to work. If you have physical/console access to the server maybe use a firewall to temporarily block traffic or disable the NIC while the script runs?

P.S. I don't know anything about VB either, which became painfully obvious once I started trying to write my own scripts. ;)

Greta
Senior user
Senior user
Posts: 306
Joined: 2007-01-02 13:23
Contact:

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by Greta » 2011-04-07 10:00

This script is for version 4. But can I run it also on hmail version 5.3?

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by ^DooM^ » 2011-04-07 12:26

If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Greta
Senior user
Senior user
Posts: 306
Joined: 2007-01-02 13:23
Contact:

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by Greta » 2011-04-07 13:33

That PHP script doesn't work because php is not installed on the server.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by ^DooM^ » 2011-04-07 20:30

Then you will just have to test the script to find out.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

Re: hMailCleaner (Compare hMaildatabase with files on disk) 21.06.2020 version

Post by large » 2020-06-22 11:57

Hi all!
It has been a while since I used this script and I have made a modification to ignore files from today.
Also added a filter on files that are created today, since it *could* be a valid file that hasn't been included into the database yet.
Added a progress while searching for files, it shows a dot (.) for each mail that was found and x for each file that is from today and not added.

While the script is running, please do not add / delete / rename any accounts / domains that will halt the script.

hMailserver has been running for me since 2011 and here are my stats for today
Number of physical files on disk: 381306
Number of files in hMail database: 373613
Martins programming is really stable and my raid-controller is really crappy :)

Here are the updated script, enjoy and only use after a successfully backup:

Code: Select all

' ====================================================
' hMailCleaner by Lars Werner (http://lars.werner.no/)
' ====================================================
' Usage: cscript hmailcleaner.vbs "data-path" adminpassword [/delete]
'
' How does it work?
' -----------------
' The script makes a recursive list of a given directory (the place where you store the *.eml files) and
' then it compares to the filenames that are stored in the hmailserver database.
' If there is a mismatch the file will be renamed to *.deleteme
'
'WARNING: If /delete option is given then all the mismatched files will be permanently deleted!!!
'
' Who need this?
' --------------
' If you're running a hmailserver and had a couple of lockups or simular I guess you might have some "leftovers" on the disk.
' This script marks the files that aren't in the database or even delete them directly!
' Pretty usefull if you have a crappy RAID controller that makes everything writeprotected, like I did ;)
'
' 21.06.2020: 
' Tested on v5.x of hMailserver and it works.
' Added function to ignore files that are from the same day (makes it 


Option Explicit

Dim aFiles() 'Array of all the files in the given path
Dim aMailFiles() 'Array of files in the database
Dim iCount, iCountTotal, iArrayCount, iPercentage, iLastPercentage
Dim bDelete  'Really delete?
Dim objFSO, objFolder, colFiles, objFile, objFiledate, dToday

'===========================
'== SUBS & FUNCTIONS
'= Store each file from a given directory into an array
Function ArrayOfFiles(Folder) 'Returns an object of objFolder (used in recursive mode)
	Dim SubFolder
	For Each Subfolder in Folder.SubFolders
		Set objFolder = objFSO.GetFolder(Subfolder.Path)
		WScript.Echo ""
		WScript.Echo "Directory: " & Folder & "\" & objFolder.Name
		Set colFiles = objFolder.Files
		Redim preserve aFiles(UBound(aFiles) + colFiles.Count) 'Resize the array with the new files we got
		For Each objFile in colFiles
			if StrComp(Right(objFile.Name, 9), ".deleteme") <> 0 then
				if FormatDateTime(objFile.DateCreated,vbShortDate) <> dToday Then 'If it is not from today, we want to check it
					aFiles(iCount) = objFolder.Path & "\" & objFile.Name
					iCount = iCount + 1
					Wscript.Stdout.Write "."
				else
					Wscript.Stdout.Write "x"
				End if		
			end if
		Next
		ArrayOfFiles Subfolder 'Recursive call
	Next
End Function

'===========================
'= Store each db "file" into an array
Sub FillMailArray(sAdminPw)
	Dim oApp
	Set oApp = CreateObject("hMailServer.Application")

	' Authenticate the client.
	Call oApp.Authenticate ("Administrator", sAdminPw)

	Dim iDomains, iAccounts, iMessages
	iArrayCount = 0
	ReDim aMailFiles(iArrayCount)

	for iDomains = 0 to oApp.Domains.Count - 1
		Dim oDomain
		Set oDomain = oApp.Domains.Item(iDomains)
		Log "Domain: " & oDomain.Name
	
		for iAccounts = 0 to oDomain.Accounts.Count - 1
			Dim oAccount
			set oAccount = oDomain.Accounts.Item(iAccounts)
			Log "	-> " & oAccount.Address
			Log "		> Messages: " & oAccount.Messages.Count
			Redim preserve aMailFiles(UBound(aMailFiles) + oAccount.Messages.Count)
			for iMessages = 0 to oAccount.Messages.Count - 1
				Dim oMessage
				set oMessage = oAccount.Messages.Item(iMessages)
				aMailFiles(iArrayCount) = oMessage.Filename
				iArrayCount = iArrayCount + 1
				set oMessage = nothing
			Next 'End iMessages
			set oAccount = nothing
		Next 'End iAccounts
		set oDomain = nothing
	Next 'End iDomains

	set oApp = nothing
End Sub

'===========================
'== MAIN
bDelete = false

'Check if we're using wscript (quit) or cscript (ok)
if InStr(lcase(WScript.Fullname), "wscript") > 0 then
	WScript.Echo "Please do not run this program from windows! Use the command prompt and cscript.exe instead"
	WScript.Quit
end if


'Check how many arguments we have
if WScript.Arguments.Count >=2 then
	dToday = FormatDateTime(Now,vbShortDate)

	Log "hMailCleaner can be runned while hMailserver is running, files from today are ignored"
	Log "====================================================================================="
	Log ""
	Log "Please note that this script takes *alot* of time and you cannot add/delete/whatever on accounts while this is running"
	Log "Error handling are by design programmed to stop while it is collecting data!"
	Log ""
	Log "Get filelist for " & WScript.Arguments(0) & " and all subdirectories"
	
	Set objFSO = CreateObject("Scripting.FileSystemObject")
	iCount = 0 'Overall counter to know where we are in the file-array
	Set objFolder = objFSO.GetFolder(Wscript.Arguments(0))
	Set colFiles = objFolder.Files

	Log "Getting file count..."
	iCountTotal = colFiles.Count
	WScript.Echo "Directory: " & Wscript.Arguments(0)
	
	Redim aFiles(iCountTotal)
	For Each objFile in colFiles
		'Ignore already "deleted" messages
		if StrComp(Right(objFile.Name, 9), ".deleteme") <> 0 then
			if FormatDateTime(objFile.DateCreated,vbShortDate) <> dToday Then 'If it is not from today, we want to check it
				aFiles(iCount) = objFolder.Path & "\" & objFile.Name
				iCount = iCount + 1
				Wscript.Stdout.Write "."
			else
				Wscript.Stdout.Write "x"
			End if
		End if
	Next

	'Now get an recursive list of files
	ArrayOfFiles objFSO.GetFolder(Wscript.Arguments(0))

	WScript.Echo ""
	Log "Getting mail list from hMailserver database"
	
	'Now create an array of files from the database
	FillMailArray(Wscript.Arguments(1))

	'Print out the info before we start renaming
	WScript.Echo
	WScript.Echo " Number of physical files on disk: " & UBound(aFiles)
	WScript.Echo "Number of files in hMail database: " & UBound(aMailFiles)
	WScript.Echo

	'Check if aMailFiles is bigger than aFiles and give a warning!
	if UBound(aFiles) < UBound(aMailFiles) then
		WScript.Echo "WARNING: hMaildatabase contains more files than there actually is on the hard drive, this will hMailServer to give ""failed to open"" messages in IMAP"
		WScript.Echo
	end if

	'Check if we're gonna delete & Write warning if we're gonna delete the file
	if WScript.Arguments.Count = 3 then
		if InStr(lcase(WScript.Arguments(2)), "/delete") then
			bDelete = true
			WScript.Echo "WARNING: Mismatched files will be permanently deleted! (To abort press CTRL-C before 15 seconds have run out)"
			WScript.Echo
			WScript.Sleep 15000
		else
			WScript.Echo "WARNING: Unknown third argument, files will be renamed!"
			WScript.Echo
		end if
	end if
	
	'From this point we want to ignore errors and handle them without stopping the script
	On Error Resume Next
	
	'Now loop through the files & compare with the database files
	iLastPercentage = 0 'Stores the last procentage so it does not update so much
	WScript.Echo "Please wait while the files are compared..."
	for iCount = 0 to UBound(aFiles) - 1

		'Check if we can find the filename in the database
		Dim bFound, sComp
		bFound = false

		'We only do the check when there is an actual file!
		if Len(aFiles(iCount)) <> 0 then
			'Get the filename we are gonna to compare with (remove the directory)
			sComp = right(lcase(aFiles(iCount)), len(aFiles(iCount)) - len(Wscript.Arguments(0))) 'We only compare the info behind selected dir :)
			if Left(sComp,1) = "\" then sComp = Right(sComp, len(sComp)-1) 'Remove the \ if it exists...

			'Now check with the databaselist
			for iArrayCount = 0 to UBound(aMailFiles)
				'Check if the entry actually is an entry
				if(Len(aMailFiles(iArrayCount))) <> 0 then
					if StrComp(sComp, lcase(Right(aMailFiles(iArrayCount), Len(sComp))), 0) = 0 then
						aMailFiles(iArrayCount) = ""
						bFound = true
						exit for
					end if
				end if
			Next

			'If we haven't found the file we are going to rename/delete it
			if bFound = false then
				if bDelete = false then
					Log "Renamed: " & aFiles(iCount)
					call objFSO.MoveFile(aFiles(iCount), aFiles(iCount) & ".deleteme")
				else
					Log "Deleted: " & aFiles(iCount)
					call objFSO.DeleteFile(aFiles(iCount))
				end if
			end if
		end if 'Len(aFiles(iCount) <> 0

		'Here we write the progress (5% at the time)
		iPercentage = FormatNumber((iCount * 100) / UBound(aFiles), 0)
		if iPercentage - iLastPercentage = 5 then
			Log "Progress: " & iPercentage & "%"
			iLastPercentage = iPercentage
		end if

		'Do a error check
		if Err.Number <> 0 then
			Log Err.Description
			Log Err.Source
			Err.Clear
		End if
	Next

	Log ""
	Log "Finished!"

	'Cleanup of objects
	Set colFiles = nothing
	Set objFolder = nothing
	set objFSO = nothing
else
	WScript.Echo "===================================================="
	WScript.Echo " hMailCleaner by Lars Werner http://lars.werner.no/"
	WScript.Echo "===================================================="
	WScript.Echo
	WScript.Echo "Usage: cscript hmailcleaner.vbs ""data-path"" adminpassword [/delete]"
	WScript.Echo
	WScript.Echo "Example: cscript hmailcleaner.vbs ""C:\Program Files\hMailServer\Data"" AdminPw"
	WScript.Echo "With logfile: cscript hmailcleaner.vbs ""C:\Program Files\hMailServer\Data"" AdminPw >log.txt"
	WScript.Echo "(The log.txt will be placed the current directory that you're running the script from, no output will be shown)"
	WScript.Echo
	WScript.Echo "How does it work?"
	WScript.Echo "-----------------"
	WScript.Echo "The script makes a recursive list of a given directory (the place where you store the *.eml files) and then it compares to the filenames that are stored in the hmailserver database."
	WScript.Echo "If there is a mismatch the file will be renamed to *.deleteme (default behaviour)"
	WScript.Echo
	WScript.Echo "WARNING: If /delete option is given then all the mismatched files will be permanently deleted!!!"
end if

'------------------------------------------------------------
'Return filename date to use
Sub Log(sText)
	WScript.Echo FixDate(Now) & " - " & sText
End Sub

'------------------------------------------------------------
'Return filename date to use
Function FixDate(dDate)
	FixDate = Year(dDate) & "-" & LPad(Month(dDate),"0",2) & "-" & LPad(Day(dDate),"0",2) & " " & LPad(Hour(dDate), "0", 2) & "." & LPad(Minute(dDate), "0", 2) & "." & LPad(Second(dDate), "0", 2)
End Function

'------------------------------------------------------------
'Pads a str for better presentation of days with single number
Function LPad (str, pad, length)
	LPad = String(length - Len(str), pad) & str
End Function
Lars Werner
http://lars.werner.no
Check out my tools:
http://lars.werner.no/unpacker/ - 100% automated extraction tool
http://lars.werner.no/sizeme/ - Maximize the output on a given media (like CD/DVD ect)

User avatar
jimimaseye
Moderator
Moderator
Posts: 8577
Joined: 2011-09-08 17:48

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by jimimaseye » 2020-06-22 23:49

Thanks Lars.

Out of interest what is the benefit of this script over the option in data directory syncroniser (to delete if not in the database)?

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by large » 2020-06-23 00:04

Well in my case around 60% of the files where 0 bytes or near.
35% was replies from other servers sayimg user unknown and 5% genuine mails.

So by doing this I avoid 95% crap to be added :-)

The 5% can be added by the dir sync tool.
Lars Werner
http://lars.werner.no
Check out my tools:
http://lars.werner.no/unpacker/ - 100% automated extraction tool
http://lars.werner.no/sizeme/ - Maximize the output on a given media (like CD/DVD ect)

User avatar
jimimaseye
Moderator
Moderator
Posts: 8577
Joined: 2011-09-08 17:48

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by jimimaseye » 2020-06-24 13:56

large wrote:
2020-06-23 00:04
Well in my case around 60% of the files where 0 bytes or near.
35% was replies from other servers sayimg user unknown and 5% genuine mails.

So by doing this I avoid 95% crap to be added :-)

The 5% can be added by the dir sync tool.
I meant what is the difference between using your script to delete orphaned (non-DB) email files and using data directory synchroniser to delete orphaned (non-DB) emails?
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

large
Normal user
Normal user
Posts: 33
Joined: 2004-12-31 12:47
Contact:

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by large » 2020-06-25 16:13

jimimaseye wrote:
2020-06-24 13:56
I meant what is the difference between using your script to delete orphaned (non-DB) email files and using data directory synchroniser to delete orphaned (non-DB) emails?
At the time I made the script I did not know about the database synchroniser.
But I wanted to check the files before deleting so I guess the difference between is that this script renames the files to *.deleteme :)
Lars Werner
http://lars.werner.no
Check out my tools:
http://lars.werner.no/unpacker/ - 100% automated extraction tool
http://lars.werner.no/sizeme/ - Maximize the output on a given media (like CD/DVD ect)

User avatar
jimimaseye
Moderator
Moderator
Posts: 8577
Joined: 2011-09-08 17:48

Re: hMailCleaner (Compare hMaildatabase with files on disk)

Post by jimimaseye » 2020-06-25 22:29

Good answer. 🤓
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

Post Reply