HOW TO: Ready-To-Go Backup and Cleardown script

This section contains user-submitted tutorials.
User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2015-04-28 13:52

Last updated: 27th November 2017

INTRODUCTION

I wrote this backup script and post it here in order to help starters get under way without too much worry. It backs up your data and Zips it to an archive folder, whilst removing previous zip archive files older than a defined period.

It is designed for systems using the INTERNAL database or a MySQL database only. (It is NOT written for any other database (as I do not know the methods for those databases - but feel free to change accordingly to suit. However, a quick google for MSSQL equivalent of dumping the database suggests this: https://www.howtogeek.com/50295/backup- ... mand-line/ - this will be the direct replacement for the full "mysqldump" command as found in the ":4th"of BackupMail.bat below).

It is an 'easy start' script offering requiring just a review of a few tailored variable at the top of the scripts.

It works very well and does the following:

  • • backs up the hmaildata directory
    • backs up the database keeping it inline with the hmaildata files
    • updates spamassassin definitions if Spamassassin for Windows is installed and in use
    • backs up other important custom data directories you wish to include
    • clears down emails from the TRASH folders (or any others you wish to include) - tailorable from Zero to X days old
    • performs and reports checks on the service stop/restarts to ensure continuity of data

* The data+database+customer datadrive goes into a single date-stamped ZIP file with a linked log file (eg "Serverdata_2015-03-10-200000.zip" and "Backup_2015-03-10-200000.log").

* The cleardown script gives you a nice email informing you of what happened detailing the accounts, and the quantities relating to their accounts that have been processed. (Copy and save as emailclearup.vbs - dont forget to change your parameters at implementation.)

* The whole backup routine gives you a FULL run down of what happened and emails you at the end (with variable SUBJECT depending on success/failure) with red highlighting of failures
eg
Subject: !!! Backup completed WITH ERRORS. CHECK LOG FILE FOR DETAILS!!!

Hmailserver service shutdown: Ok
Spamassassin service shutdown: Ok
Spam Assassin Def Update: Failed
Spam Assassin Service startup: Ok
Hmailserver service startup: Ok
Zip: Ok
Email Cleardown Script: Ok

See attached log file below for details.
ADDITIONAL PREREQUISITES

NOTE: The solution uses THREE EXTRA scripts/programs:-

a, You will see in the section called :GoEmail that I use a 3rd party program called BLAT.exe to send the email notifications out - this standalone program needs to be downloaded and installed to use the script. You can change the reference to whatever your choice of emailer program is (you might use a CDO script or an HMS emailing script you already have written) - just replicate the appropriate parameters in the :GoEmail section on the calling command line in the BackupMail.Bat script.

b, It also uses 7zip that is shipped with Hmailserver (in the Bin directory) to perform the compression - again you can use or change to suit as you wish. See notes below.

c, It also utilises Windows Server 'ROBOCOPY' for doing the data copy (mirror). This is standard on most MS Server editions of windows but can easily be obtained for other OS's if you dont already have it.

d, The EMAILCLEARUP.VBS script also needs to be saved (to the same scripts directory) for being called upon.

IMPLEMENTATION

1, Download and save the 2 scripts (BackupMail.Bat and Emailcleardown.vbs) and save them into a SINGLE holding 'script' folder from where they will be called.

2, Download and install BLAT.exe (http://www.blat.net/) as necessary

3, REVIEW AND MODIFY the various variables at the top of both scripts - BE CAREFUL TO GET THEM RIGHT. Also, take a careful read through the BAT script and fill out the extra sections (such as ":7th", ":8th" etc) to add additional copying or processing of other directories in to the script where required.

4, Call the BackupMail.bat daily (with Task Scheduler)

The basis of the backup works by initially taking a MIRROR copy of the current data folder, so the first run could take some time and you will need to be aware of this for disk space purposes ie, you will need another 100% space allocation of the current data on whatever drive you specify under the %BACKUPTEMPdir% variable. Subsequent runs, however, will be a lot quicker as only 'incremental' changes a replicated. On my system it backs up 10.5GB of data and zips it in about 20 minutes in total whist the mail service is only offline for only around 30 seconds. Its important to leave this temporary directory populated for this speed benefit.

NOTE:
There is one thing you should note: the Zip file is currently as a type ZIP compression and not the 7zip native '7z' compression. The reason for this is because it is A LOT quicker than 7Z BUT it is not so compact. To explain better, note that my total unzipped directory starts at 10.5GB but

as a ZIP:
  • time to zip: 17 minutes
    size: 7.17GB
as a 7z:
  • time to zip: 51 minutes
    size: 6.4GB
So you can see there is a trade off between compression and speed. You can choose and if you want the 7Z type for higher compression then make the changes to the script as necessary by modifying the line accordingly in the BackupMail.bat (search for "7za a" for find the line):

Code: Select all

7za a -tzip "%BACKUPdir%\Serverdata_%inDate%" "%BACKUPTEMPdir%\*"
If you change the script to a 7Z archive then remember that you will need a suitable 7Z archive compatible software that is able to read and uncompress 7Z files in the event it needs to be restores. You may choose to use the full 7Zip software (http://www.7-zip.org/download.html).

At least I hope you find it useful as a starting point. You can always add to or remove more or irrelevant sections to suit but it should be sufficient as it is (if you follow the instructions and tailor the variables accordingly) to get you going with the important data.

BackupMail.Bat

Code: Select all

rem   #### CONFIG START ####
rem  *******  FILL OUT VARIABLES BELOW  ***************************
set emailRecipient=admin@yourdomain.com
set emailFrom="Backup Daemon <noreply@yourdomain.com>"

rem ---  SET LOCATIONS (without trailing '\')  ----
Set BACKUPdir=D:\MailserverBackups
Set BACKUPTEMPdir=D:\BackupTemp
set MAILDATAdir=D:\hMailServer\Data
set HMAILSERVERprogdir=C:\Program Files (x86)\hMailServer
set BLATdir=C:\pathto\blat324\full\blat.exe

rem  Spamassassin installed and in use: enter "yes" or "no" (lowercase!). Also set program directory if 'yes'.
set SA_In_Use=yes
set SPAMASSASSINdir=C:\Program Files (x86)\JAM Software\SpamAssassin for Windows

rem  enter "internal" or "mysql" (lowercase!) as the database type in use
set DBtype=internal

rem ** IF DBtype = 'internal' set the following variables
set InternalDATABASEdir=C:\Program Files (x86)\hMailServer\Database

rem ** IF DBtype = MySQL set the following variables
set MYSQLBINdir=C:\Program Files\MySQL\MySQL Server 5.5\bin
set MYSQLDBuser=root
set MYSQLDBpass=yoursqlpassword
set MYSQLDBport=3306

rem ---  HOW MANY recent backup Zips do you want to retain?  ----
set DAYSZIPTOKEEP=7

rem  ****  FILL OUT VARIABLES ABOVE  *******************************
rem   #### CONFIG END ####

Set SCRIPTdir=%~dp0

for /F "usebackq tokens=1,2 delims==" %%i in (`wmic os get LocalDateTime /VALUE 2^>NUL`) do if '.%%i.'=='.LocalDateTime.' set ldt=%%j
set inDate=%ldt:~0,4%-%ldt:~4,2%-%ldt:~6,2%-%ldt:~8,6%
set BackLog="%BACKUPdir%\Backup_%inDate%.log"
set emailer_log="%SCRIPTdir%\emailer.log"
set Failed=false
set FailedScript=Not performed
set FailedZip=Not performed
set FailedSAUpdate=Not performed
set FailedSAService=Not performed
set FailedDNSService=Not performed
set FailedHmailservice=Not performed
set outf="%SCRIPTdir%\body.html"

echo ^<table border='0'^> > %outf%

set colorfill=font color="red"
set fontBlack=!--
set noBold=!--

echo Backup Start: %date% %time% > %BackLog%

REM :: Perform backup to temporary directory

set section=1st

:1st
:maildata
echo %time% Stopping Hmailserver service...>> %BackLog%&net stop hmailserver >> %BackLog%
set FailedHmailservice=Ok
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedHmailservice=Failed to stop. Backup of emails not performed!&set bold=b&set fontcolor=%colorfill%
echo %FailedHmailservice%! >> %BackLog%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Hmailserver service shutdown:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedHmailservice%^</td^>^</tr^> >>%outf%

:2nd
if not "%SA_In_Use%" == "yes" goto 3rd
echo %time% Stopping Spamassassin service...>> %BackLog%&net stop spamassassin >> %BackLog%
set FailedSAService=Ok
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedSAService=Failed to stop&set bold=b&set fontcolor=%colorfill%
echo %FailedSAService%! >> %BackLog%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Spamassassin service shutdown:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedSAService%^</td^>^</tr^> >>%outf%

@rem Update Spamassassin before restarting service
if "%FailedSAService%" == "Failed to stop" goto 3rd
echo %time% Performing Spamassassin Update check...>> %BackLog%
set FailedSAUpdate=Ok
cd /D "%SPAMASSASSINdir%"
sa-update.exe -v --nogpg --channelfile UpdateChannels.txt >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedSAUpdate=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Spam Assassin Def Update:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedSAUpdate%^</td^>^</tr^> >>%outf%

:3rd
set section=3rd
if "%FailedHmailservice:~0,14%" == "Failed to stop" goto 5th
robocopy "%MAILDATAdir%" "%BACKUPTEMPdir%"\hMailData /mir /ndl /r:43200 /np /w:1 >> %BackLog%
goto ROBOerrorcheck

:4th
if "%DBtype%" == "mysql" (
   erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
   "%MYSQLBINdir%"\mysqldump -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
) ELSE (
   robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
   goto ROBOerrorcheck
)

:5th
@rem Restart Spamassassin and Hmailserver services
if not "%FailedSAService%" == "Ok" goto 6th
echo %time% Starting Spamassassin service...>> %BackLog%&net start spamassassin >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedSAService=Failed to restart&set bold=b&set fontcolor=%colorfill%
echo %FailedSAService%! >> %BackLog%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Spam Assassin Service startup: ^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedSAService%^</td^>^</tr^> >>%outf%

:6th
if not "%FailedHmailservice%" == "Ok" set section=7th & goto 7th
echo %time% Starting Hmailserver service... >> %BackLog%&net start hmailserver >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedHmailservice=Failed to restart&set bold=b&set fontcolor=%colorfill%
echo %FailedHmailservice%! Errorlevel=%errorlevel% >> %BackLog%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Hmailserver service startup:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedHmailservice%^</td^>^</tr^> >>%outf%

REM --- Use the following sections 7th to 9th as example of adding other Data directories in the backup (remove the REM's)  ---
:7th
:Sagedata
rem robocopy "D:\SageAccountData" "%BACKUPTEMPdir%"\SageAccountData /mir /ndl /r:43200 /np /w:1 >> %BackLog%
rem goto ROBOerrorcheck

:8th
:Miscdata
rem robocopy "D:\Miscdata" "%BACKUPTEMPdir%"\Miscdata /mir /ndl /r:43200 /np /w:1 >> %BackLog%
rem goto ROBOerrorcheck

:9th
rem robocopy "D:\anotherdir" "%BACKUPTEMPdir%"\anotherdir /mir /ndl /r:43200 /np /w:1 >> %BackLog%
rem goto ROBOerrorcheck

REM --- End of Example sections  ---

:10th
REM::  Clear down existing backup Zips over DAYSZIPTOKEEP
echo. & echo Clearing old archives
forfiles.exe /s /p "%BACKUPdir%" /m *.* /d -%DAYSZIPTOKEEP% /c "cmd /c echo Deleting over %DAYSZIPTOKEEP% days....@path & del /q @path" >> %BackLog%

REM :: Zip temporary directory and email log file
cd /D "%HMAILSERVERprogdir%"\Bin >> %BackLog%
echo %date% %time% 7Zip Creating archive %BACKUPdir%\Serverdata_%inDate%.zip >> %BackLog%
set FailedZip=Ok
7za a -tzip "%BACKUPdir%\Serverdata_%inDate%" "%BACKUPTEMPdir%\*"
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedZip=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Zip:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedZip%^</td^>^</tr^> >>%outf%

REM :: Empty the Trash folder
echo. >> %BackLog%
set FailedScript=Ok
:runcleardown
echo Email clear up started at %time% ************************************ >> %BackLog%
cscript "%SCRIPTdir%"\EmailClearup.vbs //nologo //T:600   >> %BackLog%
echo Email clear up finished at %time% (errorlevel: %errorlevel%) **********************>> %BackLog%
if "%FailedScript%" == "RERUN-Ok" goto skiprun
if errorlevel 1 set FailedScript=RERUN-Ok& echo RERUN being called......***************>> %BackLog% & goto runcleardown
:skiprun
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedScript=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Email Cleardown Script: ^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedScript%^</td^>^</tr^> >>%outf%

:test
set result=%date% %time% Backup procedure FAILED!!! Zip: %FailedZip%, Script: %FailedScript%, SA Update: %FailedSAUpdate%, SA Service: %FailedSAService%, Hmailserver service: %FailedHmailservice%.
set resulttext="!!! Backup completed WITH ERRORS.  CHECK LOG FILE FOR DETAILS!!!"
if not %Failed%==false goto Testend
:success
set result=%date% %time% Backup 7Zip Archive created - no errors encountered.
set resulttext="Backup completed. See attachment log file."
goto Testend

:ROBOerrorcheck
if errorlevel 16 echo ***FATAL ERROR*** >> %BackLog% & goto end
if errorlevel 15 echo OKCOPY + FAIL + MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 14 echo FAIL + MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 13 echo OKCOPY + FAIL + MISMATCHES >> %BackLog% & goto end
if errorlevel 12 echo FAIL + MISMATCHES>> %BackLog% & goto end
if errorlevel 11 echo OKCOPY + FAIL + XTRA >> %BackLog% & goto end
if errorlevel 10 echo FAIL + XTRA >> %BackLog% & goto end
if errorlevel 9 echo OKCOPY + FAIL >> %BackLog% & goto end
if errorlevel 8 echo FAIL >> %BackLog% & goto end
if errorlevel 7 echo OKCOPY + MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 6 echo MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 5 echo OKCOPY + MISMATCHES >> %BackLog% & goto end
if errorlevel 4 echo MISMATCHES >> %BackLog% & goto end
if errorlevel 3 echo OKCOPY + XTRA >> %BackLog% & goto end
if errorlevel 2 echo XTRA >> %BackLog% & goto end
if errorlevel 1 echo OKCOPY  >> %BackLog% & goto end
if errorlevel 0 echo No Change >> %BackLog% & goto end
:end
if errorlevel 8 set Failed=true
if %section%==9th set section=10th
if %section%==8th set section=9th
if %section%==7th set section=8th
if %section%==6th set section=7th
if %section%==5th set section=6th
if %section%==4th set section=5th
if %section%==3rd set section=4th
if %section%==2nd set section=3rd
if %section%==1st set section=2nd
goto %section%

:Testend
if not %FailedZip%==Failed forfiles /p "%BACKUPdir%" /m Serverdata_%inDate%.zip /d +0 /c "cmd /c echo Created: @path @fsize" >> %BackLog%
echo ^<tr^>^<td^>^<br^>^</td^>^</tr^>^</table^>See attached log file below for details. >>%outf%
echo %result% >> %BackLog%

echo %time% %inDate% backup log.  Emailing.... >> %emailer_log%
:GoEmail
net start | find "hMailServer" > nul
if not %errorlevel%==0 echo Hmailservice not running - email not possible >> %emailer_log%&goto eof

if "%FailedHmailservice%" == "Failed to restart" echo Hmailservice not running - email not possible >> %emailer_log%&goto eof
"%BLATdir%" %outf% -mailfrom %emailFrom% -to %emailRecipient% -subject %resulttext% -server localhost -attachi %BackLog% -log %emailer_log%
:eof
echo ********************************** >> %emailer_log%
del %outf%
:Finish
Emailclearup.vbs

Code: Select all

'  Routine empties the TRASH folders (Zero days retained) and is called by scheduler
'  in the backup script

Option Explicit

'   #### CONFIG START ####
   Const DAYS_TO_KEEP_MESSAGES = "0"            ' Days old to keep mails
   Const MESSAGES_FOLDER = "Trash|Deleted*"     ' Folder to delete from, case insensitive, subfolder delimiter needs to be as
                                                 ' below (.) based on ur delimiter setting in hmailserver, multiple folders can
                                                 ' be specified seprated by | (pipe) and can include asterix wilcard
                                                 ' eg: "spam|trash|deleted*|inbox.sub*"

   Const IMAP_DELIMITER = "."                    ' This needs to be same as what u used above for subfolders based on delimiter
                                                 ' setting in hmailserver
   Const HMSADMINUSER = "Administrator"          ' Admin username
   Const HMSADMINPWD = "secretpassword"          ' Admin password
   Const HMSSERVER = "127.0.0.1"                 ' hMailServer Server (DCOM)
   Const FROM_EMAIL = "system@yourdomain.com"    ' Replace this with the email address you want the report to come from
   Const REPORT_TO_EMAIL = "admin@yourdomain.com"' Replace this with the email address you want the report to be sent to
'   #### CONFIG END ####

'   Objects
   Dim oApp, oDomains, oDomain, oAccounts, oAccount, oMessages, oMessage

'   Numeric
   Dim AccountSize, NumMsgs, NumDeleted, iMessages, x, y, z, MessageID, DeleteCount, LoopCount

'   Strings / arrays
   Dim SearchFolders, FindFolders, FoundFolder, FolderList, aFolder, SpamFolder, SpamFolderAction, TotalMsgs
   Dim Message, CreateGUIDval, OutputMsg, w, FolderArray, SplitCount, xFolder

'   Flags
   Dim Skipped, ReturnValue : ReturnValue = 0

'   Date / time
   Dim MessageDate

'    On Error Resume Next

   SearchFolders = Split(MESSAGES_FOLDER, "|")

   Set oApp = CreateObject("hMailServer.Application", HMSSERVER)
   Call oApp.Authenticate(HMSADMINUSER, HMSADMINPWD)

   For x = 0 To oApp.Domains.Count - 1
     Set oDomain = oApp.Domains.Item(x)
      If oDomain.Active Then
         For y = 0 To oDomain.Accounts.Count - 1
            Set oAccount = oDomain.Accounts.Item(y)
            OutputMsg = OutputMsg & "<tr><td colspan=" & Chr(34) & "5" & Chr(34) & ">&nbsp;</td></tr>" & vbCrLf
            If oAccount.Active Then
              Skipped = ""
               For Each SpamFolder in SearchFolders
                  TotalMsgs = 0
                  AccountSize = 0
                  NumMsgs = 0
                  NumDeleted = 0
                  Set FindFolders = oAccount.IMAPFolders
                  FoundFolder = False
                  FolderList = ListFolders(FindFolders, 0, "")
                  aFolder = Split(Left(FolderList, Len(FolderList) - 1), "|")
                  For Each z in aFolder
                     If UCase(z)=UCase(Trim(SpamFolder)) Or _
                     (Right(trim(SpamFolder),1)="*" And UCase(left(z,len(trim(Spamfolder))-1)) = UCase(left(trim(SpamFolder),len(trim(SpamFolder))-1))) Then
                        FoundFolder = True
                        SpamFolderAction = z
                        Set oMessages  = GetInsideFolders(oAccount.IMAPFolders, SpamFolderAction)
                        NumMsgs = oMessages.Count
                        TotalMsgs = TotalMsgs + NumMsgs
                        iMessages = 0
                        DeleteCount = 0
                        LoopCount = 0
                        Do While oMessages.Count > (LoopCount - DeleteCount)
                           Set oMessage = oMessages.Item(iMessages)
                           AccountSize = AccountSize + oMessage.Size
                           MessageDate = oMessage.InternalDate
                           If (MessageDate < CDate(Now - DAYS_TO_KEEP_MESSAGES)) Then
                              If CLng(oMessage.ID) > 0 Then
                                 NumDeleted = NumDeleted + 1
                                 DeleteCount = DeleteCount + 1
                                 oMessages.DeleteByDBID(oMessage.ID)
                              Else
                                 Skipped = " *"
                                 ShowError("--->  Delete failed  <---")
                              End If
                           Else
                              iMessages = iMessages + 1
                           End If
                           If oMessages.Count = (NumMsgs - DeleteCount) Then
                              LoopCount = LoopCount + 1
                           Else
                              NumMsgs = oMessages.Count
                              iMessages = 0
                              DeleteCount = 0
                              LoopCount = 0
                           End If
                        Loop
                     End if
                  Next
                  If FoundFolder Then
                      WScript.Echo "Removed " & NumDeleted & " message(s) from " & SpamFolder & " folder in account " & oAccount.Address & vbCrLf
                      OutputMsg = OutputMsg & "<tr><td>" & oAccount.Address & "</td>"
                      OutputMsg = OutputMsg & "<td>" & SpamFolder & "</td>"
                      OutputMsg = OutputMsg & "<td align=" & Chr(34) & "right" & Chr(34) & ">" & FormatNumber(TotalMsgs, 0, True, False, True) & "</td>"
                      OutputMsg = OutputMsg & "<td align=" & Chr(34) & "right" & Chr(34) & ">" & FormatNumber(AccountSize, 0, True, False, True) & "K</td>"
                      OutputMsg = OutputMsg & "<td align=" & Chr(34) & "right" & Chr(34) & ">" & FormatNumber(NumDeleted, 0, True, False, True) & Skipped & "</td></tr>" & vbCrLf
                  Else
                      WScript.Echo SpamFolder & " folder Not Found in account " & oAccount.Address
                  End If
               Next
            End If
         Next
      End If
   Next

   OutputMsg = "<font face=" & Chr(34) & "Calibri" & Chr(34) & "><table border=" & Chr(34) & "1" & Chr(34) & "><tr><td><b>Email Account</b></td><td><b>Folder</b></td><td><b>Mail Count</b></td><td><b>Mail Size</b></td><td><b>Deleted</b></td></tr>" & vbCrLf & OutputMsg
   OutputMsg = OutputMsg & "</table>" & vbCrLf

   Set Message = CreateObject("hMailServer.Message", HMSSERVER)
   Message.HeaderValue("Message-ID") = "<" & CreateGUID & ">"
   Message.FromAddress = FROM_EMAIL
   Message.From = "Email Clearup Daemon <"& FROM_EMAIL & ">"
   Message.AddRecipient "System Administrator", REPORT_TO_EMAIL
   Message.Subject = "Email Clearup deletion report"
   Message.HTMLBody = OutputMsg
   Message.Save

   Wscript.Quit ReturnValue

   Function ShowError(strMessage)
      WScript.Echo strMessage
      WScript.Echo Err.Number & " Srce: " & Err.Source & " Desc: " &  Err.Description
      ReturnValue = Err.Number
      Err.Clear
   End Function

   Function ListFolders(obFolders, iRecursion, rootFolder)
      iRecursion = iRecursion + 1
      Dim sMessage
      Dim i
      For i = 0 To obFolders.Count -1
         Dim obFolder
         Set obFolder = obFolders.Item(i)
         If iRecursion > 1 Then
            FolderArray = Split(rootFolder, IMAP_DELIMITER)
            SplitCount=0
            for each xFolder in FolderArray
               SplitCount=SplitCount+1
               if SplitCount < iRecursion then
                  if SplitCount = 1 then
                     rootFolder = xFolder
                  else
                     rootFolder = rootFolder & IMAP_DELIMITER & xFolder
                  end if
               else
                  if SplitCount = iRecursion then
                     Exit for
                  End if
               end if
            next
            sMessage = sMessage & rootFolder & IMAP_DELIMITER & obFolder.Name & "|"
         Else
            sMessage = sMessage & obFolder.Name & "|"
         End If
         If iRecursion = 1 Then
            rootfolder =obFolder.Name
         Else
            rootFolder = rootFolder & IMAP_DELIMITER & obFolder.Name
         End If
         sMessage = sMessage & ListFolders(obFolder.SubFolders, iRecursion, rootFolder )
      Next
      iRecursion = iRecursion -1
      ListFolders = sMessage
   End Function

   Function GetInsideFolders(obFolders2, subFolders)
      Dim iRecursion2
      iRecursion2 = 1
      Dim SeprateFodlers
      SeprateFodlers = Split(subFolders, IMAP_DELIMITER)
      For Each w in SeprateFodlers
         If iRecursion2 = 1 Then
            Set obFolders2 = obFolders2.ItemByName(w)
            iRecursion2 = iRecursion2 + 1
         Else
            Set obFolders2 = obFolders2.SubFolders.ItemByName(w)
         End If
      Next
      Set GetInsideFolders = obFolders2.Messages
   End Function

   Function CreateGUID()
     ' Generate a random string.
      With CreateObject("hMailServer.Utilities", HMSSERVER)
         CreateGUID = Mid(.GenerateGUID, 2, 36) & "@randommail"
      End With
   End Function

--- Using an External MySQL server ---

The script is designed for installations where the database is on the same server as Hmailserver. However, if you have a MySQL database on a separate external server then you will need to make a slight adjustment to the scripts (courtesy of Mattv8, here):

Get the "mysqldump.exe" command line utility

Visit https://dev.mysql.com/downloads/mysql/ and download the latest version of MySQL community server as a zip archive. Once the zip file has downloaded, extract it and browse to the bin folder. Copy the file "mysqldump.exe" to somewhere convenient on your server. Note the directory that you copy it to.

Changes to the script BackupMail.bat

Next, you will need to make the following adjustments to the BAT script:
  1. Specify the MYSQLBINdir to the directory you just copied "mysqldump.exe" to.
  2. Add a new variable called MYSQLDBhost. Point this new variable to the ip address or hostname of your MySQL server. I just added this to the MySQL database section, which should now look something like the following:

    Code: Select all

    rem ** IF DBtype = MySQL set the following variables.
    rem ** Make sure mysqldump.exe is in the script folder, or update line 37 of this script.
    set MYSQLBINdir=C:\Program Files (x86)\hMailServer\Backup
    set MYSQLDBuser=root
    set MYSQLDBpass=password
    set MYSQLDBhost=192.168.0.123
    set MYSQLDBport=3306
  3. Edit section 4th to look like the following:

    Code: Select all

    :4th
    if "%DBtype%" == "mysql" (
       erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
       "%MYSQLBINdir%"\mysqldump.exe -h"%MYSQLDBhost%" -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
    ) ELSE (
       robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
       goto ROBOerrorcheck
    )
--- End of changes ---


LOG FILES

Some people might choose to save disk space by also clearing down their Hmailserver LOGS directory; in doing so you can perhaps save over to 90% in disk space initially taken up by the logs). The following code identifies the current location of the LOGS, zips the log files to a .7z compressed file, and then deletes the logs.

If wish this functionality then add the following code in to the Batch file somewhere similar to section "8th" or "9th" (or in to a separate batch file of your choice):

Code: Select all

FOR /F "eol=; eol=[ tokens=1,2* delims==" %%i in ('findstr /b /l /i "Logfolder"= "%programfiles(x86)%\hMailServer\Bin\hmailserver.ini"') DO set Logpath=%%~j
"%programfiles(x86)%\hMailServer\Bin\7za" a -t7z "%Logpath%\Logs.7z" -uq0 "%Logpath%\hmailserver_????-*.log" -xr!*.7z
del "%Logpath%"\hmailserver_????-*.log
(Note: I repeat, this code section is not included in the main Backup script and is additional).
Attachments
BackupScriptFiles.zip
Save contents in a 'script' folder for calling. Call the BAT with scheduler. (Updated 21 March 2018)
(5.54 KiB) Downloaded 162 times
BackupRestoreGuide.zip
Helpful Restore Guide DOCX to accompany the backup script.
(37.42 KiB) Downloaded 149 times
Last edited by jimimaseye on 2018-03-02 20:21, edited 3 times in total.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

evb
New user
New user
Posts: 13
Joined: 2008-11-05 09:33
Location: Belgium

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by evb » 2015-09-13 11:21

EDIT:

NOTE: THE FOLLOWING CORRECTIONS HAVE NOW ALREADY BEEN ADDED IN TO THE ABOVE SCRIPT. YOU DO NOT NEED TO MAKE THE CHANGES. This post is left here for tracking and narrative.

Jimimaseye


*******************************************

Thanks very much for your backup solution!
I was using till now the internal backup, but recently it was failing because of the size limit :(

I had to correct some problems in the script before it was running smootly on my system. So maybe my corrections will help other people.
1. first problem was the name of the backup logging file. With my regional settings for the date, it was a non valid filename as result
So I replaced the code

Code: Select all

set inDate=%date:~6,4%-%date:~3,2%-%date:~0,2%-%str%
with thanks to Google to (should be independent from regional settings now)

Code: Select all

for /F "usebackq tokens=1,2 delims==" %%i in (`wmic os get LocalDateTime /VALUE 2^>NUL`) do if '.%%i.'=='.LocalDateTime.' set ldt=%%j
set inDate=%ldt:~0,4%-%ldt:~4,2%-%ldt:~6,2%-%str%
2. Following problem were forgotten quotes around the folder path MYSQLBINdir giving the well know space problem in paths
old code

Code: Select all

if "%DBtype%" == "mysql" (
   erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
   %MYSQLBINdir%\mysqldump -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
) ELSE (
   robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
   goto ROBOerrorcheck
)
new code

Code: Select all

if "%DBtype%" == "mysql" (
   erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
   "%MYSQLBINdir%"\mysqldump -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
) ELSE (
   robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
   goto ROBOerrorcheck
)
3. The SpamAssassin update was always giving a failed message, even when there where no update, so I replaces the old code

Code: Select all

if errorlevel 1 set Failed=true& set FailedSAUpdate=Failed&set bold=b&set fontcolor=%colorfill%
by

Code: Select all

if errorlevel 4 set Failed=true& set FailedSAUpdate=Failed&set bold=b&set fontcolor=%colorfill%
(see SpamAssassin for the exitcodes: 4 and higher are real errors)

Regards,
hmailserver 5.3.3-B1879
MySQL server Ver 5.5, for Win32

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2015-09-13 13:58

Thank you evb!

I will amend the code to reflect your changes parts (1) and (2) - the missing quotes (2) are a mistake and the solution for universal time/date formatting (1) will help make the script usable wherever (hopefully. I trust your research. :wink: ) I have gone one further and shanged your final line to not rely on the %str% variable (which I will removed).

Code: Select all

set inDate=%ldt:~0,4%-%ldt:~4,2%-%ldt:~6,2%-%ldt:~8,6%
I chose not to make amends for exit code 4 on spamassassin because I feel its better to be made aware that there was no SA update performed as it is likely to be less the norm (my experience is that at least 6 our of 10 times there will be an update). The idea being that if you get a week or more notifications of 'no updates available' then it should raise a concern and if it continues then MAYBE there is a problem that needs researching (whereas simply replying on it and never being told you would never find the problems). Anyway, your post will alert others to the choice they can make in the script.

EDIT:
The updates script has now been amended and also the download ZIP will be changed to reflect the new code.

Thanks again.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2016-03-11 12:28

Original post/script updated to add missing important logging (regarding of service stops/restarts).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2016-04-08 08:35

Script modified to deal with situation where HMS service may have been restarted early by other process (ensuring emailing of the log continues).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2016-06-04 01:41

4 June 2016

Bug fix for completeness in handling multilevel subfolders.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-01-09 23:44

9 January 2017
  • Tidy up HTML formatting to cater for some fussy email clients (not displaying the email body correctly).
    Removed redundant lines of code
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

steveHmail
New user
New user
Posts: 13
Joined: 2017-01-11 12:14

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by steveHmail » 2017-01-11 12:17

Hi all,
I am wondering if anybody is using these scripts because i see only 2 downloads, mine.

I just want to make sure that everything is fine before i use them.

Thanks

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-01-11 12:21

MANY people are using them. As you can see from the thread (the initial post the the previous post), I only updated it 9th January - hence you are the first to download the new ones. (Before I updated, there was 49 downloads of the attachment - and that doesnt account for people that may have just copy/pasted the code themselves).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

steveHmail
New user
New user
Posts: 13
Joined: 2017-01-11 12:14

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by steveHmail » 2017-01-11 12:26

Thanks jimimaseye.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-01-11 14:32

steveHmail wrote:Thanks jimimaseye.
No worries. I would appreciate your feedback.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

steveHmail
New user
New user
Posts: 13
Joined: 2017-01-11 12:14

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by steveHmail » 2017-01-11 15:07

It run perfectly.Not a single problem. I already added it on the daily task scheduler. Thanks.
I am also checking the clear LOG FILES bat but my hMailserver in on c:\hMailserver so i don't think it will work for me without some changes.



Thanks again for the scripts.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-01-11 15:50

steveHmail wrote:It run perfectly.Not a single problem. I already added it on the daily task scheduler. Thanks.
I am also checking the clear LOG FILES bat but my hMailserver in on c:\hMailserver so i don't think it will work for me without some changes.
Yes. Just add the HMAILSERVERprogdir variable in place of the program directory and it will be ok.

So this is the modified version (add to the main backupdata.bat script):

Code: Select all

FOR /F "eol=; eol=[ tokens=1,2* delims==" %%i in ('findstr /b /l /i "Logfolder"= "%HMAILSERVERprogdir%"\Bin\hmailserver.ini') DO set Logpath=%%~j
"%HMAILSERVERprogdir%"\Bin\7za a -t7z "%Logpath%\Logs.7z" -uq0 "%Logpath%\hmailserver_????-*.log" -xr!*.7z
del "%Logpath%"\hmailserver_????-*.log
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

steveHmail
New user
New user
Posts: 13
Joined: 2017-01-11 12:14

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by steveHmail » 2017-01-11 17:11

Thanks jimimaseye.
You made my day. :)
I am ready now to upgrade from 5.4.2 to 5.6.6.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-01-17 13:16

17 January 2017
  • Mode to BAT code for tidiness (re-introduce <!--> rem tags ) - no update to existing installs necessary if using the version since 9th January 2017.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-02-19 15:30

19 February 2017

Added ability to use trailing asterix wildcard on the folder selection such as:

DEALT* which will match folders:
  • DEALT
    DEALT2014
INBOX.SUB1* which will match
  • INBOX.SUB1
    INBOX.SUB123
    INBOX.SUB1.SUBA
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

robinlee
New user
New user
Posts: 2
Joined: 2017-09-26 15:34

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by robinlee » 2017-09-26 15:42

Hi,

Thanks for the script. I have one question.
Since I am running MySql as the database on a different computer.
how should I config Database section on the script?

MYSQLBINdir=

because it's on a different computer and looks like I can not put the IP address of PC that runs Mysql

Could anyone help? thanks in advance.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-09-26 23:10

This part of the feature relies on the MySQL programs to be installed (as the 'mysqldump' is a mysql program). You could install MySQL locally - you dont need to be hosting the database. Add -h host_name (where host_name will be your remote mysql server machine) to the 'mysqldump' command to connect to the database.

Code: Select all

"%MYSQLBINdir%"\mysqldump  -h host_name -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l  .....etc
https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html

(Ive never done it or tried it - maybe you need to set your database to be accessible from a remote machine too if not already).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

robinlee
New user
New user
Posts: 2
Joined: 2017-09-26 15:34

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by robinlee » 2017-09-27 04:22

jimimaseye wrote:This part of the feature relies on the MySQL programs to be installed (as the 'mysqldump' is a mysql program). You could install MySQL locally - you dont need to be hosting the database. Add -h host_name (where host_name will be your remote mysql server machine) to the 'mysqldump' command to connect to the database.

Code: Select all

"%MYSQLBINdir%"\mysqldump  -h host_name -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l  .....etc
https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html

(Ive never done it or tried it - maybe you need to set your database to be accessible from a remote machine too if not already).
Thank you, I will give it try.

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2017-11-01 11:14

Awesome script!

I am having an issue, not sure if intentional or not.

The SpamAssassin update is running and the logfile reads:

Code: Select all

10:57:01.83 Stopping Spamassassin service...
.
The spamassassin service was stopped successfully.

10:57:04.39 Performing Spamassassin Update check...
Update finished, no fresh updates were available
However, it then marks the update as failed, and the report email states that the update failed.

Code: Select all

E:\Backups\Script>echo 11:09:49.26 Performing Spamassassin Update check... 1>>"E
:\Backups\Backup_2017-11-01-110946.log"

E:\Backups\Script>set FailedSAUpdate=Ok

E:\Backups\Script>c:

C:\>cd "C:\Program Files\JAM Software\SpamAssassin for Windows"

C:\Program Files\JAM Software\SpamAssassin for Windows>sa-update.exe -v --nogpg
--channelfile UpdateChannels.txt  1>>"E:\Backups\Backup_2017-11-01-110946.log"

C:\Program Files\JAM Software\SpamAssassin for Windows>set bold=!--  & set fontc
olor=!--

C:\Program Files\JAM Software\SpamAssassin for Windows>if errorlevel 1 set Faile
d=true  & set FailedSAUpdate=Failed  & set bold=b  & set fontcolor=font color="r
ed"

C:\Program Files\JAM Software\SpamAssassin for Windows>echo <tr><td><font color=
"red"><b>Spam Assassin Def Update:</td><td><font color="red"><b>Failed</td></tr>
  1>>"E:\Backups\Script\body.html"

Code: Select all

Wed 11/01/2017 11:00:39.22 Backup procedure FAILED!!! Zip: Ok, Script: Ok, SA Update: Failed, SA Service: Ok. 
Is this normal procedure because no files were updated, or is there something wrong with my config somewhere?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-11-01 11:30

That is perfectly normal and will happen most nights. (SA rule updates are comparatively rare)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2017-11-01 11:57

jimimaseye wrote:That is perfectly normal and will happen most nights. (SA rule updates are comparatively rare)
No worries, thanks!

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-11-01 14:25

sorry, I should hgave elaborated:

its because of:

"Update finished, no fresh updates were available"

If there was an update (and successfully installed), then it will not be seen as a 'failure'. I leave the status as a 'failiure' in there because of the possibility that the 'failure' was a DNS issue or network problem (or something else) and not simply because of it being "no fresh updates" (IOW its common but you should always be mindful to have a quick glance to ensure it is because of 'no fresh updates')
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2017-11-01 23:10

jimimaseye wrote:sorry, I should hgave elaborated:

its because of:

"Update finished, no fresh updates were available"

If there was an update (and successfully installed), then it will not be seen as a 'failure'. I leave the status as a 'failiure' in there because of the possibility that the 'failure' was a DNS issue or network problem (or something else) and not simply because of it being "no fresh updates" (IOW its common but you should always be mindful to have a quick glance to ensure it is because of 'no fresh updates')
Understood. Thanks again, I really appreciate all the help I am getting from these forums.

mibyge
New user
New user
Posts: 16
Joined: 2016-09-07 20:28

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by mibyge » 2017-11-27 00:51

Hi :)

First of all, thanks a lot for the script. Looks great.

I'm currently building a new server and intend to use this method for backup on the new server, so I wanted to test it a bit before using it in production. Based on my testing, I'd like to suggest the 2 following changes to the batch script.

1) You should be able to set the SCRIPTdir variable to the value %~dp0 and thereby automatically detect the path from where the script is being executed from. That gives the user one less variable to worry about. See https://stackoverflow.com/questions/382 ... in-windows for reference.

2) If your SCRIPTdir and HMAILSERVERprogdir is NOT on the same drive, the zip part will fail as the default "cd" command will not change the drive but only the folder. In the current script, the 7za command is called without the path which means the script won't be able to execute the command so that part of the script fails. You can solve it in 2 ways: Either add /D to the "cd" command so it also changes the drive or add "%HMAILSERVERprogdir%"\Bin when calling 7za.

I've made the above 2 changes to my script (added /D to the "cd" command), and it seems to be working fine.

Question: Is there a restore part (guide) for this script as well? Not that I hope I'll ever need it, but it would be nice to know exactly what the expected/best practice procedure would be if you need to restore either partially (i.e. just one domain or account) or fully (recover from catastrophic system failure).

Thanks :)

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-11-27 10:23

mibyge wrote:Hi :)
Question: Is there a restore part (guide) for this script as well? Not that I hope I'll ever need it, but it would be nice to know exactly what the expected/best practice procedure would be if you need to restore either partially (i.e. just one domain or account) or fully (recover from catastrophic system failure).
I have attached a guide to the original post for you.

Regarding your other comments and suggestions: thanks. I will review them and apply as necessary when I have a little more time.

Thanks again.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2017-11-27 13:26

Script amended to obtain current script path automatically and add /D paramter to CD command.

(Existing users/implementations do not need to update).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-03-04 18:40

* BUG FIX

Bat file amended to remove a bug when handling the errorlevel = 1 situations.

Existing users should redownload and implement.

Alternatively, follow these instructions to correct the error in your BAT file:

1, Using your editor, search for "if errorlevel 1" where the end of the line ends with "& echo %FailedHmailservice%! >> %BackLog%"
2, Separate the line at "& echo" so it becomes on a new line (and remove the & ampersand)
eg
this:

Code: Select all

if errorlevel 1 set Failed=true&set FailedHmailservice=Failed to stop. Backup of emails not performed!&set bold=b&set fontcolor=%colorfill% & echo %FailedHmailservice%! >> %BackLog%
becomes this:

Code: Select all

if errorlevel 1 set Failed=true&set FailedHmailservice=Failed to stop. Backup of emails not performed!&set bold=b&set fontcolor=%colorfill%
echo %FailedHmailservice%! >> %BackLog%
These lines exist in the :1st, :2nd, :5th and :6th sections

Without this fix implemented a failure still shows "Ok!" in the logfile (where it should have showed "Failed....!").
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

mattv8
New user
New user
Posts: 1
Joined: 2014-11-10 03:32

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by mattv8 » 2018-03-15 21:40

For those of us with external mysql databases, and don't have MySQL server installed on the hmail host machine, this might be helpful. I had to make a few minor changes to the script to get it to work.

Get the "mysqldump.exe" command line utility
Visit https://dev.mysql.com/downloads/mysql/ and download the latest version of MySQL community server as a zip archive. Once the zip file has downloaded, extract it and browse to the bin folder. Copy the file "mysqldump.exe" to somewhere convenient on your server. Note the directory that you copy it to.

Changes to the script BackupMail.bat
Next, you will need to make the following adjustments to jimimaseye's script:
  1. Specify the MYSQLBINdir to the directory you just copied "mysqldump.exe" to.
  2. Add a new variable called MYSQLDBhost. Point this new variable to the ip address or hostname of your MySQL server. I just added this to the MySQL database section, which should now look something like the following:

    Code: Select all

    rem ** IF DBtype = MySQL set the following variables.
    rem ** Make sure mysqldump.exe is in the script folder, or update line 37 of this script.
    set MYSQLBINdir=C:\Program Files (x86)\hMailServer\Backup
    set MYSQLDBuser=root
    set MYSQLDBpass=password
    set MYSQLDBhost=192.168.0.123
    set MYSQLDBport=3306
  3. Edit section 4th to look like the following:

    Code: Select all

    :4th
    if "%DBtype%" == "mysql" (
       erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
       "%MYSQLBINdir%"\mysqldump.exe -h"%MYSQLDBhost%" -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
    ) ELSE (
       robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
       goto ROBOerrorcheck
    )

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-03-15 22:27

Thanks mattv8. I will review your addition and see about updating the initial instructions.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-03-15 23:52

mattv8 wrote:
2018-03-15 21:40
For those of us with external mysql databases, and don't have MySQL server installed on the hmail host machine......
I have now incorporated your changes and advice in to the original instructions. Thanks again.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-03-22 00:39

* Modification to BAT code

Windows showed strange behaviour on starting the hmailserver service reporting errorlevel 2 although eventually it does start (I think errorlevel 2 might be 'delayed'). Consequently the script would then think it is 'out of order' and not send the final email out results. This change uses alternative method to check the hmailserver service is running for emailing the results instead of relying on the previous start service errorlevel.

EXISTING USERS:

I recommend the changes. Search and change:

Code: Select all

:GoEmail
if "%FailedHmailservice%" == "Failed to restart" echo Hmailservice not running - email not possible >> %emailer_log%&goto eof
with

Code: Select all

:GoEmail
net start | find "hMailServer" > nul
if not %errorlevel%==0 echo Hmailservice not running - email not possible >> %emailer_log%&goto eof
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-04-25 08:25

Hi

After running this script for about 6 months I have hit a snag and I can't seem to see what the issue is.

I have it setup to be triggered by task scheduler at midnight. This morning I came in and there was no backup file and no-one was getting emails. I checked the backup log and it gave me this
Backup Start: Wed 04/25/2018 0:00:15.36
0:00:15.36 Stopping Hmailserver service...
Ok!
0:00:24.07 Stopping Spamassassin service...
The spamassassin service is stopping.
The spamassassin service was stopped successfully.

0:00:26.65 Performing Spamassassin Update check...
Update available for channel updates.spamassassin.org: 1829763 -> 1829845
http: (lwp) hotpatching IO::Socket::INET by module IO::Socket::IP
http: (lwp) GET http://spamassassin.apache.org/updates/MIRRORED.BY, 200 OK
http: (lwp) GET http://sa-update.razx.cloud/1829845.tar.gz, 200 OK
http: (lwp) GET http://sa-update.razx.cloud/1829845.tar.gz.sha1, 200 OK
Update was available, and was downloaded and installed successfully
0:00:53.38 Starting Spamassassin service...
The spamassassin service is starting.
The spamassassin service was started successfully.
I then checked the hmailserver daily log and it gave me this
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Received stop-request from Windows."
"APPLICATION" 2652 "2018-04-25 00:00:18.275" "Stopping servers..."
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Application::StopServers() - Removing server work queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Stopping working queue Server queue."
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Interupt and join threads in working queue Server queue"
"DEBUG" 2072 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 2760 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 692 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "IOService::Stop()"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5750"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 2"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5722"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5548"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5525"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5718"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "IOService::DoWork() - removing Queue IOCP Queue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Stopping working queue IOCPQueue."
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Interupt and join threads in working queue IOCPQueue"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3900 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2036 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 1316 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3880 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3468 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2696 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3128 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 216 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3168 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2996 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 572 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3660 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2464 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 1036 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "All threads are joined in queue IOCPQueue."
"DEBUG" 3764 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Stopped IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Erased IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "IOService::Stop() - Complete"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Destructing work queue IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "All threads are joined in queue Server queue."
"DEBUG" 2652 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Stopped Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Erased Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Destructing work queue Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Application::StopServers() - Clearing caches"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Application::StopServers() - Destructing IOCP"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing DeliveryManager"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue SMTP delivery queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue SMTP delivery queue"
"DEBUG" 3124 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 3480 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 3948 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 1668 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 2296 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 3312 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 888 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 88 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 1604 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 300 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "All threads are joined in queue SMTP delivery queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Stopped SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Erased SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Destructing work queue SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing FetchManager"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "ExternalFetchManager::~ExternalFetchManager"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue External fetch queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue External fetch queue"
"DEBUG" 2784 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3036 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1552 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3936 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3888 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1780 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 544 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1172 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3732 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 4084 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 2304 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1548 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3924 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 4080 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3044 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "All threads are joined in queue External fetch queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Stopped External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Erased External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Destructing work queue External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "ExternalFetchManager::~ExternalFetchManager - Removed queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing Scheduler"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing Rest"
"APPLICATION" 2652 "2018-04-25 00:00:18.291" "Servers stopped."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue Maintenance queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue Maintenance queue"
"DEBUG" 1908 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 2736 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 2156 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 3916 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 3972 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "All threads are joined in queue Maintenance queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Stopped Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Erased Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Destructing work queue Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Asynchronous task queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue Asynchronous task queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue Asynchronous task queue"
"DEBUG" 4092 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3304 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2484 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3368 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3712 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2908 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3012 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 1944 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3276 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3884 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2032 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2412 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 792 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2824 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2652 "2018-04-25 00:00:18.322" "Still 2 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "IOService::~IOService - Destructing"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"ERROR" 2320 "2018-04-25 00:00:18.416" "Severity: 3 (Medium), Code: HM4308, Source: MailboxChangeNotifier::Unsubscribe, Description: An unknown error has occurred."
"DEBUG" 2320 "2018-04-25 00:00:18.447" "Ending session 5696"
"DEBUG" 2652 "2018-04-25 00:00:18.603" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:18.884" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:19.150" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:19.432" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:19.759" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.150" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.431" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.713" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.994" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:21.275" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:21.556" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:21.837" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:22.119" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:22.400" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:22.681" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
So, it looks like it stopped the server, completed the SA update and then hit an error somewhere in between 00:00:18 and 00:00:53 where it should be doing

Code: Select all

:3rd
set section=3rd
if "%FailedHmailservice:~0,14%" == "Failed to stop" goto 5th
robocopy "%MAILDATAdir%" "%BACKUPTEMPdir%"\hMailData /mir /ndl /r:43200 /np /w:1 >> %BackLog%
robocopy %HMAILSERVERprogdir%\Bin %BACKUPTEMPdir%\hMailserver\Bin\ hMailServer.INI /mir /ndl /r:43200 /np /w:1 >> %BackLog%
robocopy %HMAILSERVERprogdir%\Events %BACKUPTEMPdir%\hMailserver\Events /mir /ndl /r:43200 /np /w:1 >> %BackLog%
goto ROBOerrorcheck

:4th
if "%DBtype%" == "mysql" (
   erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
   "%MYSQLBINdir%"\mysqldump -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
) ELSE (
   robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
   goto ROBOerrorcheck
)
Then, it restarted SA but didn't seem to carry on with anything else, and didn't restart the mail service until I logged in this morning.

Any ideas?

Cheers
Dave

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-04-25 08:48

See my posts above with changes: viewtopic.php?p=202828#p202828 and viewtopic.php?p=203249#p203249. There are changes made to the Bat. It would be worth implementing them as they tackle oddities like this.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-04-25 09:27

jimimaseye wrote:
2018-04-25 08:48
See my posts above with changes: viewtopic.php?p=202828#p202828 and viewtopic.php?p=203249#p203249. There are changes made to the Bat. It would be worth implementing them as they tackle oddities like this.
Thanks, I have implemented and will monitor. I am having another issue with a few lines that I added myself to copy the backups to an external drive. Basically, it throws up an error every day and says failed but the copy has been completed correctly.

Code: Select all

REM :: Copy to NAS
echo %date% %time% Copying Zip to NAS >> %BackLog%
set FailedCopy=Ok
NET USE \\192.168.1.200\$IPC /U:192.168.1.200\username password >> %BackLog%
robocopy "%BACKUPdir%" "%NAS%" /maxage:1 >> %BackLog%
NET USE \\192.168.1.200\$IPC /D >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedCopy=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Copy:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedCopy%^</td^>^</tr^> >>%outf%
The log shows me the following:
Tue 04/24/2018 0:07:22.47 7Zip Creating archive E:\Backups\Completed\Serverdata_2018-04-24-000301.zip
Tue 04/24/2018 0:37:27.98 Copying Zip to NAS

-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------

Started : Tuesday, April 24, 2018 12:37:30 AM
Source : E:\Backups\Completed\
Dest : \\192.168.1.200\backup\drakakis Backups\Emails\

Files : *.*

Options : *.* /DCOPY:DA /COPY:DAT /MAXAGE:1 /R:1000000 /W:30

------------------------------------------------------------------------------

Total Copied Skipped Mismatch FAILED Extras
Dirs : 1 0 0 0 0 0
Files : 9 2 7 0 0 195
Bytes : 20.963 g 5.257 g 15.705 g 0 0 486.955 g
Times : 0:02:48 0:02:48 0:00:00 0:00:00


Speed : 33601920 Bytes/sec.
Speed : 1922.717 MegaBytes/min.
Ended : Tuesday, April 24, 2018 12:40:18 AM
No files have failed (the 195 are Extras), and there is nothing in the log, so I really don't know what the issue is.

Cheers

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-04-25 15:20

I think you need to move your 'errorlevel 1' check as you are doing the test of it after the NET USE command where you should be doing it after the ROBOCOPY command.

So:

Code: Select all

REM :: Copy to NAS
echo %date% %time% Copying Zip to NAS >> %BackLog%
set FailedCopy=Ok
NET USE \\192.168.1.200\$IPC /U:192.168.1.200\username password >> %BackLog%
robocopy "%BACKUPdir%" "%NAS%" /maxage:1 >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedCopy=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Copy:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedCopy%^</td^>^</tr^> >>%outf%
NET USE \\192.168.1.200\$IPC /D >> %BackLog%
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-04-26 07:36

jimimaseye wrote:
2018-04-25 15:20
I think you need to move your 'errorlevel 1' check as you are doing the test of it after the NET USE command where you should be doing it after the ROBOCOPY command.

So:

Code: Select all

REM :: Copy to NAS
echo %date% %time% Copying Zip to NAS >> %BackLog%
set FailedCopy=Ok
NET USE \\192.168.1.200\$IPC /U:192.168.1.200\username password >> %BackLog%
robocopy "%BACKUPdir%" "%NAS%" /maxage:1 >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedCopy=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Copy:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedCopy%^</td^>^</tr^> >>%outf%
NET USE \\192.168.1.200\$IPC /D >> %BackLog%
Thanks Jimimaseye, I have made the changes and will see what happens tonight!

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-04 10:12

jimimaseye wrote:
2018-04-25 08:48
See my posts above with changes: viewtopic.php?p=202828#p202828 and viewtopic.php?p=203249#p203249. There are changes made to the Bat. It would be worth implementing them as they tackle oddities like this.
Hi jimimaseye

Everything was fine for a few days, and now I came in again this morning, no backup and no-one was getting emails. As soon as I logged on to the server, then emails were all delivered, and everything is back to normal.

Something is happening with the backup and hmailserver is not restarting. We are only getting the emails from the few hours that the server was down as I am using our webhost as a mail2.domain.com as a backup and hmailserver periodically checks these mailboxes.

Is there anything I can check, I am at a bit of a loss here.

Help me jimimaseye, you're my only hope! (May 4th reference :lol:)

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-04 13:13

drakakistours wrote:
2018-05-04 10:12
Everything was fine for a few days, and now I came in again this morning, no backup and no-one was getting emails. As soon as I logged on to the server, then emails were all delivered, and everything is back to normal.
Ah, young Padawan.

This doesnt make in sense in context to what you originally say the problem is. When you say "logged on to the server" - do you mean as a Windows users or actually in to HmailAdmin? and once logged on, what do you then do?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-07 07:35

jimimaseye wrote:
2018-05-04 13:13
drakakistours wrote:
2018-05-04 10:12
Everything was fine for a few days, and now I came in again this morning, no backup and no-one was getting emails. As soon as I logged on to the server, then emails were all delivered, and everything is back to normal.
Ah, young Padawan.

This doesnt make in sense in context to what you originally say the problem is. When you say "logged on to the server" - do you mean as a Windows users or actually in to HmailAdmin? and once logged on, what do you then do?
Sorry Master

I log on as a Windows user, and prior to starting hMailAdmin, the mail server appears to start running again.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-07 12:42

Thats very peculiar. As far as I know, the running of a service (in this case the Hmailserver service), has no reliance on whether anyone logins in to windows or not. A NET START [service] should happen irrespective.

However, I know that TASK SCHEDULER jobs may depend on whether someone is logged in or not. And given that you are (probably) running this backup script via a Task Scheduler job, then perhaps the cause is somewhere around this. You should check that you have it set to run even if not logged on (and saved using windows Administrator level credentials):

Untitled.png
drakakistours wrote:
2018-05-07 07:35
I log on as a Windows user, and prior to starting hMailAdmin, the mail server appears to start running again.
"Appears to".

Look at EVENT VIEWER and maybe also the service itself (after you have logged on) to see the startup time of the service - does it tie in with the exact moment that you login in? Maybe if your task job is waiting for you to log in then it is actually waiting to START the job (with a NET STOP) and then progrsses to NET START a few seconds later?

Beyond the above, I cant offer any explanation or further ideas.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 19979
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by mattg » 2018-05-07 15:51

What dependencies does the hMailserver SERVICE have?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-14 09:24

mattg wrote:
2018-05-07 15:51
What dependencies does the hMailserver SERVICE have?
Hi Matt

The dependencies are MYSQL, DCOM Service Launcher and RPC Endpoint Mapper

Image

Jimimaseye - the Task is set to run regardless of user logged on, however the task is running without a problem, it's just quitting after the SpamAssassin update.

Thanks for all your help so far.
Attachments
Capture.PNG

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-14 09:29

Post a log of one that fails to start (until you log in).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-14 10:02

jimimaseye wrote:
2018-05-14 09:29
Post a log of one that fails to start (until you log in).
I did, a few posts ago
Backup Start: Wed 04/25/2018 0:00:15.36
0:00:15.36 Stopping Hmailserver service...
Ok!
0:00:24.07 Stopping Spamassassin service...
The spamassassin service is stopping.
The spamassassin service was stopped successfully.

0:00:26.65 Performing Spamassassin Update check...
Update available for channel updates.spamassassin.org: 1829763 -> 1829845
http: (lwp) hotpatching IO::Socket::INET by module IO::Socket::IP
http: (lwp) GET http://spamassassin.apache.org/updates/MIRRORED.BY, 200 OK
http: (lwp) GET http://sa-update.razx.cloud/1829845.tar.gz, 200 OK
http: (lwp) GET http://sa-update.razx.cloud/1829845.tar.gz.sha1, 200 OK
Update was available, and was downloaded and installed successfully
0:00:53.38 Starting Spamassassin service...
The spamassassin service is starting.
The spamassassin service was started successfully.
The hmailserver daily log and it gave me this
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Received stop-request from Windows."
"APPLICATION" 2652 "2018-04-25 00:00:18.275" "Stopping servers..."
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Application::StopServers() - Removing server work queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Stopping working queue Server queue."
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Interupt and join threads in working queue Server queue"
"DEBUG" 2072 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 2760 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 692 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "IOService::Stop()"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5750"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 2"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5722"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5548"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5525"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "TCP - AcceptEx failed. Error code: 995, Message: The I/O operation has been aborted because of either a thread exit or an application request"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Ending session 5718"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "IOService::DoWork() - removing Queue IOCP Queue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Stopping working queue IOCPQueue."
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Interupt and join threads in working queue IOCPQueue"
"DEBUG" 3832 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3900 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2036 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 1316 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3880 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3468 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2696 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3128 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 216 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3168 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2996 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 572 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3660 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 2464 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 1036 "2018-04-25 00:00:18.275" "Worker exited in work queue IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "All threads are joined in queue IOCPQueue."
"DEBUG" 3764 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Stopped IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Erased IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "IOService::Stop() - Complete"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Destructing work queue IOCPQueue"
"DEBUG" 3764 "2018-04-25 00:00:18.275" "Worker exited in work queue Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "All threads are joined in queue Server queue."
"DEBUG" 2652 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Stopped Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "WorkQueueManager::RemoveQueue - Erased Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Destructing work queue Server queue"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Application::StopServers() - Clearing caches"
"DEBUG" 2652 "2018-04-25 00:00:18.275" "Application::StopServers() - Destructing IOCP"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing DeliveryManager"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue SMTP delivery queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue SMTP delivery queue"
"DEBUG" 3124 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 3480 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 3948 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 1668 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 2296 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 3312 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 888 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 88 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 1604 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 300 "2018-04-25 00:00:18.291" "Worker exited in work queue SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "All threads are joined in queue SMTP delivery queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Stopped SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Erased SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Destructing work queue SMTP delivery queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing FetchManager"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "ExternalFetchManager::~ExternalFetchManager"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue External fetch queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue External fetch queue"
"DEBUG" 2784 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3036 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1552 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3936 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3888 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1780 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 544 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1172 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3732 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 4084 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 2304 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 1548 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3924 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 4080 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 3044 "2018-04-25 00:00:18.291" "Worker exited in work queue External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "All threads are joined in queue External fetch queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Stopped External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Erased External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Destructing work queue External fetch queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "ExternalFetchManager::~ExternalFetchManager - Removed queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing Scheduler"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Application::StopServers() - Destructing Rest"
"APPLICATION" 2652 "2018-04-25 00:00:18.291" "Servers stopped."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue Maintenance queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue Maintenance queue"
"DEBUG" 1908 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 2736 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 2156 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 3916 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 3972 "2018-04-25 00:00:18.291" "Worker exited in work queue Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "All threads are joined in queue Maintenance queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Stopped Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Erased Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Destructing work queue Maintenance queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "WorkQueueManager::RemoveQueue - Asynchronous task queue"
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Stopping working queue Asynchronous task queue."
"DEBUG" 2652 "2018-04-25 00:00:18.291" "Interupt and join threads in working queue Asynchronous task queue"
"DEBUG" 4092 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3304 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2484 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3368 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3712 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2908 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3012 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 1944 "2018-04-25 00:00:18.291" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3276 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 3884 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2032 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2412 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 792 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2824 "2018-04-25 00:00:18.307" "Worker exited in work queue Asynchronous task queue"
"DEBUG" 2652 "2018-04-25 00:00:18.322" "Still 2 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "IOService::~IOService - Destructing"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"DEBUG" 2320 "2018-04-25 00:00:18.354" "TCPServer::~TCPServer"
"ERROR" 2320 "2018-04-25 00:00:18.416" "Severity: 3 (Medium), Code: HM4308, Source: MailboxChangeNotifier::Unsubscribe, Description: An unknown error has occurred."
"DEBUG" 2320 "2018-04-25 00:00:18.447" "Ending session 5696"
"DEBUG" 2652 "2018-04-25 00:00:18.603" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:18.884" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:19.150" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:19.432" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:19.759" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.150" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.431" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.713" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:20.994" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:21.275" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:21.556" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:21.837" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:22.119" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:22.400" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
"DEBUG" 2652 "2018-04-25 00:00:22.681" "Still 1 remaining threads in queue Asynchronous task queue. First task: AsynchronousTask"
So, it looks like it stopped the server, completed the SA update and then hit an error somewhere in between 00:00:18 and 00:00:53 where it should be doing

Code: Select all


:3rd
set section=3rd
if "%FailedHmailservice:~0,14%" == "Failed to stop" goto 5th
robocopy "%MAILDATAdir%" "%BACKUPTEMPdir%"\hMailData /mir /ndl /r:43200 /np /w:1 >> %BackLog%
robocopy %HMAILSERVERprogdir%\Bin %BACKUPTEMPdir%\hMailserver\Bin\ hMailServer.INI /mir /ndl /r:43200 /np /w:1 >> %BackLog%
robocopy %HMAILSERVERprogdir%\Events %BACKUPTEMPdir%\hMailserver\Events /mir /ndl /r:43200 /np /w:1 >> %BackLog%
goto ROBOerrorcheck

:4th
if "%DBtype%" == "mysql" (
   erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
   "%MYSQLBINdir%"\mysqldump -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
) ELSE (
   robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
   goto ROBOerrorcheck
)

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-14 10:40

That doesnt look right to be begin with:
"ERROR" 2320 "2018-04-25 00:00:18.416" "Severity: 3 (Medium), Code: HM4308, Source: MailboxChangeNotifier::Unsubscribe, Description: An unknown error has occurred."
Also, in your logs you posted, you cut it off at the restart of Spamassassin
The spamassassin service is starting.
The spamassassin service was started successfully.
- its the logs immediately after that that is of interest.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-14 10:59

jimimaseye wrote:
2018-05-14 10:40
That doesnt look right to be begin with:
"ERROR" 2320 "2018-04-25 00:00:18.416" "Severity: 3 (Medium), Code: HM4308, Source: MailboxChangeNotifier::Unsubscribe, Description: An unknown error has occurred."
Also, in your logs you posted, you cut it off at the restart of Spamassassin
The spamassassin service is starting.
The spamassassin service was started successfully.
- its the logs immediately after that that is of interest.
The log ends there, there is nothing else.

Can you explain the error any further?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-14 11:16

Post the fill batch file - inedited - please. Lets take a look.

(No idea about the other error, sorry).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-14 11:33

jimimaseye wrote:
2018-05-14 11:16
Post the fill batch file - inedited - please. Lets take a look.

(No idea about the other error, sorry).

Code: Select all

rem   #### CONFIG START ####
rem  *******  FILL OUT VARIABLES BELOW  ***************************
set emailRecipient=it@drakakistours.com
set emailFrom="Backup Daemon <it@drakakistours.com>"

rem ---  SET LOCATIONS (without trailing '\')  ----
Set BACKUPdir=E:\Backups\Completed
Set NAS=\\192.168.1.200\backup\drakakis Backups\Emails
Set BACKUPTEMPdir=E:\Backups\Temp
set MAILDATAdir=E:\hMailServer\Data
set HMAILSERVERprogdir=E:\hMailServer
set BLATdir=C:\Blat\full\blat.exe
rem ---  WHERE are your housekeeping scripts located?  ----
Set SCRIPTdir=E:\Backups\Script

rem  Spamassassin installed and in use: enter "yes" or "no" (lowercase!). Also set program directory if 'yes'.
set SA_In_Use=yes
set SPAMASSASSINdir=C:\Program Files\JAM Software\SpamAssassin for Windows

rem  enter "internal" or "mysql" (lowercase!) as the database type in use
set DBtype=mysql

rem ** IF DBtype = 'internal' set the following variables
set InternalDATABASEdir=C:\Program Files (x86)\hMailServer\Database

rem ** IF DBtype = MySQL set the following variables
set MYSQLBINdir=C:\Program Files\MySQL\MySQL Server 5.7\bin
set MYSQLDBuser=root
set MYSQLDBpass=Dr4kM41l
set MYSQLDBport=3306

rem ---  HOW MANY recent backup Zips do you want to retain?  ----
set DAYSZIPTOKEEP=5

rem  ****  FILL OUT VARIABLES ABOVE  *******************************
rem   #### CONFIG END ####

for /F "usebackq tokens=1,2 delims==" %%i in (`wmic os get LocalDateTime /VALUE 2^>NUL`) do if '.%%i.'=='.LocalDateTime.' set ldt=%%j
set inDate=%ldt:~0,4%-%ldt:~4,2%-%ldt:~6,2%-%ldt:~8,6%
set BackLog="%BACKUPdir%\Backup_%inDate%.log"
set emailer_log="%SCRIPTdir%\Files\emailer.log"
set Failed=false
set FailedScript=Not performed
set FailedZip=Not performed
set FailedCopy=Not performed
set FailedSAUpdate=Not performed
set FailedSAService=Not performed
set FailedDNSService=Not performed
set FailedHmailservice=Not performed
set outf="%SCRIPTdir%\Files\body.html"

echo ^<table border='0'^> > %outf%

set colorfill=font color="red"
set fontBlack=!--
set noBold=!--

echo Backup Start: %date% %time% > %BackLog%

REM :: Perform backup to temporary directory

set section=1st

:1st
:maildata
echo %time% Stopping Hmailserver service...>> %BackLog%&net stop hmailserver >> %BackLog%
set FailedHmailservice=Ok
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true&set FailedHmailservice=Failed to stop. Backup of emails not performed!&set bold=b&set fontcolor=%colorfill%
echo %FailedHmailservice%! >> %BackLog%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Hmailserver service shutdown:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedHmailservice%^</td^>^</tr^> >>%outf%

:2nd
if not "%SA_In_Use%" == "yes" goto 3rd
echo %time% Stopping Spamassassin service...>> %BackLog%&net stop spamassassin >> %BackLog%
set FailedSAService=Ok
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedSAService=Failed to stop&set bold=b&set fontcolor=%colorfill%& echo %FailedSAService%! >> %BackLog%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Spamassassin service shutdown:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedSAService%^</td^>^</tr^> >>%outf%

@rem Update Spamassassin before restarting service
if "%FailedSAService%" == "Failed to stop" goto 3rd
echo %time% Performing Spamassassin Update check...>> %BackLog%
set FailedSAUpdate=Ok
c:
cd "%SPAMASSASSINdir%"
sa-update.exe -v --nogpg --channelfile UpdateChannels.txt >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedSAUpdate=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Spam Assassin Def Update:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedSAUpdate%^</td^>^</tr^> >>%outf%

:3rd
set section=3rd
if "%FailedHmailservice:~0,14%" == "Failed to stop" goto 5th
robocopy "%MAILDATAdir%" "%BACKUPTEMPdir%"\hMailData /mir /ndl /r:43200 /np /w:1 >> %BackLog%
robocopy %HMAILSERVERprogdir%\Bin %BACKUPTEMPdir%\hMailserver\Bin\ hMailServer.INI /mir /ndl /r:43200 /np /w:1 >> %BackLog%
robocopy %HMAILSERVERprogdir%\Events %BACKUPTEMPdir%\hMailserver\Events /mir /ndl /r:43200 /np /w:1 >> %BackLog%
goto ROBOerrorcheck

:4th
if "%DBtype%" == "mysql" (
   erase /Q "%BACKUPTEMPdir%"\hMailData\*.mysql >> %BackLog%
   "%MYSQLBINdir%"\mysqldump -u"%MYSQLDBuser%" -p"%MYSQLDBpass%" -q -A -l --add-drop-table -PMYSQLDBport >"%BACKUPTEMPdir%\hMailData\MYSQLDump_%inDate%.mysql"
) ELSE (
   robocopy "%InternalDATABASEdir%" "%BACKUPTEMPdir%"\Database /mir /ndl /r:43200 /np /w:1 >> %BackLog%
   goto ROBOerrorcheck
)

:5th
@rem Restart Spamassassin and Hmailserver services
if not "%FailedSAService%" == "Ok" goto 6th
echo %time% Starting Spamassassin service...>> %BackLog%&net start spamassassin >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedSAService=Failed to restart&set bold=b&set fontcolor=%colorfill%& echo %FailedSAService%! >> %BackLog%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Spam Assassin Service startup: ^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedSAService%^</td^>^</tr^> >>%outf%

:6th
if not "%FailedHmailservice%" == "Ok" set section=7th & goto 7th
echo %time% Starting Hmailserver service... >> %BackLog%&sc start hmailserver >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if %errorlevel% EQU 1056 set errorlevel=0
if %errorlevel% GEQ 1 set Failed=true& set FailedHmailservice=Failed to restart&set bold=b&set fontcolor=%colorfill%& echo %FailedHmailservice%! Errorlevel=%errorlevel% >> %BackLog%
set errorlevel=
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Hmailserver service startup:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedHmailservice%^</td^>^</tr^> >>%outf%


:10th
REM::  Clear down existing backup Zips over DAYSZIPTOKEEP
echo. & echo Clearing old archives
forfiles.exe /s /p "%BACKUPdir%" /m *.* /d -%DAYSZIPTOKEEP% /c "cmd /c echo Deleting over %DAYSZIPTOKEEP% days....@path & del /q @path" >> %BackLog%

REM :: Zip temporary directory and email log file
cd /D "%HMAILSERVERprogdir%"\Bin >> %BackLog%
echo %date% %time% 7Zip Creating archive %BACKUPdir%\Serverdata_%inDate%.zip >> %BackLog%
set FailedZip=Ok
7za a -tzip "%BACKUPdir%\Serverdata_%inDate%" "%BACKUPTEMPdir%\*"
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedZip=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Zip:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedZip%^</td^>^</tr^> >>%outf%

REM :: Copy to NAS
echo %date% %time% Copying Zip to NAS >> %BackLog%
set FailedCopy=Ok
NET USE \\192.168.1.200\$IPC /U:192.168.1.200\drakakistours panos1978@! >> %BackLog%
robocopy "%BACKUPdir%" "%NAS%" /maxage:1 >> %BackLog%
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedCopy=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Copy:^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedCopy%^</td^>^</tr^> >>%outf%
NET USE \\192.168.1.200\$IPC /D >> %BackLog%

REM :: Empty the Trash folder
echo. >> %BackLog%
set FailedScript=Ok
:runcleardown
echo Email clear up started at %time% ************************************ >> %BackLog%
cscript "%SCRIPTdir%"\EmailClearup.vbs //nologo //T:600   >> %BackLog%
echo Email clear up finished at %time% (errorlevel: %errorlevel%) **********************>> %BackLog%
if "%FailedScript%" == "RERUN-Ok" goto skiprun
if errorlevel 1 set FailedScript=RERUN-Ok& echo RERUN being called......***************>> %BackLog% & goto runcleardown
:skiprun
set bold=%noBold%&set fontcolor=%fontBlack%
if errorlevel 1 set Failed=true& set FailedScript=Failed&set bold=b&set fontcolor=%colorfill%
echo ^<tr^>^<td^>^<%fontcolor%^>^<%bold%^>Email Cleardown Script: ^</td^>^<td^>^<%fontcolor%^>^<%bold%^>%FailedScript%^</td^>^</tr^> >>%outf%

:test
set result=%date% %time% Backup procedure FAILED!!! Zip: %FailedZip%, Copy: %FailedCopy%, Script: %FailedScript%, SA Update: %FailedSAUpdate%, SA Service: %FailedSAService%.
set resulttext="!!! Backup completed WITH ERRORS.  CHECK LOG FILE FOR DETAILS!!!"
if not %Failed%==false goto Testend
:success
set result=%date% %time% Backup 7Zip Archive created - no errors encountered.
set resulttext="Backup completed. See attachment log file."
goto Testend

:ROBOerrorcheck
if errorlevel 16 echo ***FATAL ERROR*** >> %BackLog% & goto end
if errorlevel 15 echo OKCOPY + FAIL + MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 14 echo FAIL + MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 13 echo OKCOPY + FAIL + MISMATCHES >> %BackLog% & goto end
if errorlevel 12 echo FAIL + MISMATCHES>> %BackLog% & goto end
if errorlevel 11 echo OKCOPY + FAIL + XTRA >> %BackLog% & goto end
if errorlevel 10 echo FAIL + XTRA >> %BackLog% & goto end
if errorlevel 9 echo OKCOPY + FAIL >> %BackLog% & goto end
if errorlevel 8 echo FAIL >> %BackLog% & goto end
if errorlevel 7 echo OKCOPY + MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 6 echo MISMATCHES + XTRA >> %BackLog% & goto end
if errorlevel 5 echo OKCOPY + MISMATCHES >> %BackLog% & goto end
if errorlevel 4 echo MISMATCHES >> %BackLog% & goto end
if errorlevel 3 echo OKCOPY + XTRA >> %BackLog% & goto end
if errorlevel 2 echo XTRA >> %BackLog% & goto end
if errorlevel 1 echo OKCOPY  >> %BackLog% & goto end
if errorlevel 0 echo No Change >> %BackLog% & goto end
:end
if errorlevel 8 set Failed=true
if %section%==9th set section=10th
if %section%==8th set section=9th
if %section%==7th set section=8th
if %section%==6th set section=7th
if %section%==5th set section=6th
if %section%==4th set section=5th
if %section%==3rd set section=4th
if %section%==2nd set section=3rd
if %section%==1st set section=2nd
goto %section%

:Testend
if not %FailedZip%==Failed forfiles /p "%BACKUPdir%" /m Serverdata_%inDate%.zip /d +0 /c "cmd /c echo Created: @path @fsize" >> %BackLog%
echo ^<tr^>^<td^>^<br^>^</td^>^</tr^>^</table^>See attached log file below for details. >>%outf%
echo %result% >> %BackLog%

echo %time% %inDate% backup log.  Emailing.... >> %emailer_log%
:GoEmail
net start | find "hMailServer" > nul
if not %errorlevel%==0 echo Hmailservice not running - email not possible >> %emailer_log%&goto eof
"%BLATdir%" %outf% -mailfrom %emailFrom% -to %emailRecipient% -subject %resulttext% -server localhost -u it@drakakistours.com -pw Dr4k4k!sT@ur5 -attachi %BackLog% -log %emailer_log%
:eof
echo ********************************** >> %emailer_log%
del %outf%

:Finish

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-14 11:38

:3rd
set section=3rd
if "%FailedHmailservice:~0,14%" == "Failed to stop" goto 5th
.
.
.
:5th
@rem Restart Spamassassin and Hmailserver services
if not "%FailedSAService%" == "Ok" goto 6th
echo %time% Starting Spamassassin service...>> %BackLog%&net start spamassassin >> %BackLog%
.
.
.
:6th
if not "%FailedHmailservice%" == "Ok" set section=7th & goto 7th
.
.
.
:10th
REM:: Clear down existing backup Zips over DAYSZIPTOKEEP
echo. & echo Clearing old archives
.
.
"Goto 7th" - where is "7th" ??

Hence a Bat crash. Not sure why it would work once you log in though but it certaoinly explains why you dont get and data copied. (It needs changing to say "set section=10th & goto 10th")

Can you post the most recent LOG file?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-14 11:55

jimimaseye wrote:
2018-05-14 11:38
:3rd
set section=3rd
if "%FailedHmailservice:~0,14%" == "Failed to stop" goto 5th
.
.
.
:5th
@rem Restart Spamassassin and Hmailserver services
if not "%FailedSAService%" == "Ok" goto 6th
echo %time% Starting Spamassassin service...>> %BackLog%&net start spamassassin >> %BackLog%
.
.
.
:6th
if not "%FailedHmailservice%" == "Ok" set section=7th & goto 7th
.
.
.
:10th
REM:: Clear down existing backup Zips over DAYSZIPTOKEEP
echo. & echo Clearing old archives
.
.
"Goto 7th" - where is "7th" ??

Hence a Bat crash. Not sure why it would work once you log in though but it certaoinly explains why you dont get and data copied. (It needs changing to say "set section=7th & goto 7th")

Can you post the most recent LOG file?
Should it say 10th?

So, some days if fails to restart the service and therefore crashes? So, if the goto is correct, then I will get a backup but the service still wont be restarted until I login?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-14 12:09

Sorry, you beat me to it (I edited to say 10th, yes. See edited post).

The error (it seems) is that it fails to STOP the service:
echo %time% Stopping Hmailserver service...>> %BackLog%&net stop hmailserver >> %BackLog%
set FailedHmailservice=Ok
.
if errorlevel 1 set Failed=true&set FailedHmailservice=Failed to stop. Backup of emails not performed!
(you log file should show this)

then this checks that status and if it is not "ok" (which it wont be because it will be "Failed to stop. Backup of emails not performed!") it skips the restart (because it thinks it hasnt stopped):
:6th
if not "%FailedHmailservice%" == "Ok" set section=7th & goto 7th
echo %time% Starting Hmailserver service... >> %BackLog%&sc start hmailserver >> %BackLog%

The problem lies in determining if/why the service fails to stop in a controlled manner (ie, not errorlevel 0). And I suspect the other error might have something to do with it.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-15 08:16

jimimaseye wrote:
2018-05-14 12:09
Sorry, you beat me to it (I edited to say 10th, yes. See edited post).

The error (it seems) is that it fails to STOP the service:
echo %time% Stopping Hmailserver service...>> %BackLog%&net stop hmailserver >> %BackLog%
set FailedHmailservice=Ok
.
if errorlevel 1 set Failed=true&set FailedHmailservice=Failed to stop. Backup of emails not performed!
(you log file should show this)

then this checks that status and if it is not "ok" (which it wont be because it will be "Failed to stop. Backup of emails not performed!") it skips the restart (because it thinks it hasnt stopped):
:6th
if not "%FailedHmailservice%" == "Ok" set section=7th & goto 7th
echo %time% Starting Hmailserver service... >> %BackLog%&sc start hmailserver >> %BackLog%

The problem lies in determining if/why the service fails to stop in a controlled manner (ie, not errorlevel 0). And I suspect the other error might have something to do with it.
Thanks jimimaseye, that explains a lot. I wrongly assumed the OK! in the log, was actually it saying it had shot down the server correctly, but it appears not as it was missing the line "The hMailServer service was stopped successfully."

I will investigate further. In the 6 months I have been running the script, it has failed 4 times, and the first was the end of March, so this is a new error. I will investigate further.

Thanks for all the help, and for the awesome script!!

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-21 09:12

drakakistours wrote:
2018-05-15 08:16
Thanks jimimaseye, that explains a lot. I wrongly assumed the OK! in the log, was actually it saying it had shot down the server correctly, but it appears not as it was missing the line "The hMailServer service was stopped successfully."
.
.
.

I occasionally see windows misbehave and get itself in a tizzy with the stopping/starting of the services (similar to your experience) and its usually just after a windows update and during the 'reboot required' phase.

FYI this is how my log file looks:

Code: Select all

HMS Server Start Time: 2018-05-17 20:00:43
HMS Daily Spam Reject count: 0
HMS Daily Viruses Removed count: 0

20:00:00.29 Stopping Hmailserver service...

The hMailServer service was stopped successfully.

Ok! 
20:00:01.17 Stopping Spamassassin service...

The spamassassin service was stopped successfully.

Ok! 

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------

  Started : Fri May 18 20:00:01 2018

   Source : D:\Datastore\hMailData\
     Dest : D:\BackupTemp\hMailData\

    Files : *.*
	    
  Options : *.* /NDL /S /E /COPY:DAT /PURGE /MIR /NP /R:43200 /W:1 

------------------------------------------------------------------------------

	  *EXTRA File 		   7.1 m	D:\BackupTemp\hMailData\MYSQLDump_2018-05-17-200000.mysql
	    Newer     		       0	D:\Datastore\hMailData\ScheduledOutOfOffice.txt
	    New File  		   61381	D:\Datastore\hMailData\domain.co.uk\alan\1A\{1A8DF043-1768-4029-9EEC-14881CB8FBE8}.eml
	    .
	    .
	    
(I have a few extra lines to show the first 'Daily Spam and AV counts)'
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-05-28 22:47

jimimaseye wrote:
2018-05-21 09:12
drakakistours wrote:
2018-05-15 08:16
Thanks jimimaseye, that explains a lot. I wrongly assumed the OK! in the log, was actually it saying it had shot down the server correctly, but it appears not as it was missing the line "The hMailServer service was stopped successfully."
.
.
.

I occasionally see windows misbehave and get itself in a tizzy with the stopping/starting of the services (similar to your experience)
FYI: i had an example of it tonight:
20:00:39.50 Starting Hmailserver service...
Failed to restart! Errorlevel=2
No error feedback message or NET HELPMSG xxxx error.
And a search for errorlevel 2 shows nothing useful except that all values are likely to be value 2: https://ss64.com/nt/net-service.html

And yet by the time the point of emailing arrived in the script it had restarted (and successfully emailed the log). Note that this was only possible due to my recent updates to the script which ignores the above error and tests for the service being started anyway (at time of emailing).

So yes, weirdness happens.

[Entered by mobile. Excuse my spelling.]
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

drakakistours
Normal user
Normal user
Posts: 49
Joined: 2017-10-22 08:23

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by drakakistours » 2018-05-29 08:08

Thanks, it hasn't happened since so all going well.

I found the source of the other error I mentioned this morning. Basically the line I added

Code: Select all

NET USE \\192.168.1.200\$IPC /U:192.168.1.200\User Pass
is throwing an error saying I am already connected, but if I don't pass this line, I will get an error along the lines of that I am not connected. I will see what Google has to say about it.

palinka
Senior user
Senior user
Posts: 945
Joined: 2017-09-12 17:57

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by palinka » 2018-07-23 04:43

This script works great. Just one question. I received the emails after it finished. The "backup daemon" one has the SA update error (for no update). However, at the bottom of the message it says, "See attached log file below for details." but the are no attachments. Is this an oversight in the code or am I actually missing the attachment?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8066
Joined: 2011-09-08 17:48

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by jimimaseye » 2018-07-23 08:56

palinka wrote:
2018-07-23 04:43
However, at the bottom of the message it says, "See attached log file below for details." but the are no attachments. Is this an oversight in the code or am I actually missing the attachment?
You are missing the attachment as added here:
"%BLATdir%" %outf% -mailfrom %emailFrom% -to %emailRecipient% -subject %resulttext% -server localhost -attachi %BackLog%

Did you include/implement the "BLAT" program (as advised in "(2) implementation")?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

palinka
Senior user
Senior user
Posts: 945
Joined: 2017-09-12 17:57

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by palinka » 2018-07-23 11:47

Yes, I'm using blat. Here is last night's scheduled run in emailer.log:

Code: Select all

2018.07.23 03:03:38 (Mon)------------Start of Session-----------------
Blat v3.2.19 (build : Nov 18 2017 03:14:35)
32-bit Windows, Full, Unicode
Sending C:\scripts\HMSBackupCleardown\\body.html to admin@domain.com
Subject: !!! Backup completed WITH ERRORS.  CHECK LOG FILE FOR DETAILS!!!
Login name is Backup Daemon <system@domain.com>
Attached text file: X:\HMS-BACKUP\Backup_2018-07-23-030000.log
2018.07.23 03:03:39 (Mon)-------------End of Session------------------
********************************** 
It looks like it was supposed to attach but it didn't.

palinka
Senior user
Senior user
Posts: 945
Joined: 2017-09-12 17:57

Re: HOW TO: Ready-To-Go Backup and Cleardown script

Post by palinka » 2018-07-23 12:24

Whoa... I just noticed this:

Image

It looks like the archive files are way too small. The 7z files are from the HMS-included backup script. The zips are from this script.

Any idea what that's about? My scheduled task is set to run from SYSTEM and the manual runs were from a cmd window ran as administrator. So I don't think it's a permissions thing.

For whatever it's worth, the backup-temp folder is about 40 MB. My data dir is about 1.5 GB.

BackupMail.bat was copied from the post (not downloaded). The first run was "successful", although now I see it was not, then I added the bit about the logs (the separate part near the bottom of the first post) but I screwed something up, so I copied the script again and the only changes are the user variables. The log appears to show all messages copied.

Code: Select all

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------

  Started : Monday, July 23, 2018 3:00:07 AM
   Source : X:\HMS-DATA\
     Dest : X:\HMS-BACKUP\Backup-Temp\hMailData\

    Files : *.*
	    
  Options : *.* /NDL /S /E /DCOPY:DA /COPY:DAT /PURGE /MIR /NP /R:43200 /W:1 

------------------------------------------------------------------------------

	  *EXTRA File 		   9.0 m	X:\HMS-BACKUP\Backup-Temp\hMailData\MYSQLDump_2018-07-22-220518.mysql
	    New File  		    3830	X:\HMS-DATA\Data\{1CAEB17D-B70D-4FE6-B8B2-255A1A160880}.eml
	    New File  		    3784	X:\HMS-DATA\Data\{6B5B1373-777D-4E79-8EAF-9F0731966BBE}.eml
	    New File  		    3409	X:\HMS-DATA\Data\{9C37C091-7D48-4A77-B5A0-A5EE780F33EC}.eml
	    New File  		    3000	X:\HMS-DATA\Data\domain.com\a\03\{03E84872-31CD-4A54-BF4D-82D0718A9C80}.eml
	    
{~snip~}

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :      4024         1      4023         0         0         0
   Files :     10210      9728       482         0         0         5
   Bytes :   1.474 g   1.449 g   25.50 m         0         0   11.76 m
   Times :   0:01:44   0:01:40                       0:00:00   0:00:03


   Speed :            15477757 Bytes/sec.
   Speed :             885.644 MegaBytes/min.
   Ended : Monday, July 23, 2018 3:01:51 AM

OKCOPY + XTRA  
 3:01:56.68 Starting Spamassassin service...
The spamassassin service is starting.
The spamassassin service was started successfully.

Ok! 
 3:01:58.76 Starting Hmailserver service... 
The hMailServer service is starting.
The hMailServer service was started successfully.

Ok! Errorlevel=0 

What do you think?

Post Reply