Save all EML no users

Use this forum if you have problems with a hMailServer script, such as hMailServer WebAdmin or code in an event handler.
Post Reply
adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Save all EML no users

Post by adrianp » 2019-03-30 15:03

I need to set up hMail as an SMTP gateway only, no users defined, only domains. It needs to receives all emails for the domains configured, still go through AV/Spam checks. I'd like to have all emails that make it through saved to a single folder as an EML file. For reference I'm currently using Microsoft SMTP server which is what I'm trying to mimic, but want the additional security that hMail has.

For reference, I was looking at this, but wasn't sure since it's dealing with accounts. I'm not sure if what I want to do would require a catch all account or not.
https://www.hmailserver.com/forum/viewt ... ml#p173970

Thank you,
Adrian

User avatar
jimimaseye
Moderator
Moderator
Posts: 8080
Joined: 2011-09-08 17:48

Re: Save all EML no users

Post by jimimaseye » 2019-03-30 17:18

Set the domains and a single catch all account. Then set up the catch all account in the domain settings. Ensure you have the correct authentication for external to local deliveries in your IP ranges.

[Entered by mobile. Excuse my spelling.]
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 20003
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Save all EML no users

Post by mattg » 2019-03-31 01:16

adrianp wrote:
2019-03-30 15:03
I'd like to have all emails that make it through saved to a single folder as an EML file.
That can't happen with hMailserver, unless you specifically script it
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-01 14:21

mattg wrote:
2019-03-31 01:16
adrianp wrote:
2019-03-30 15:03
I'd like to have all emails that make it through saved to a single folder as an EML file.
That can't happen with hMailserver, unless you specifically script it
Yes, I figured as much, hence me posting in the Scripting forum.

User avatar
mattg
Moderator
Moderator
Posts: 20003
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Save all EML no users

Post by mattg » 2019-04-02 02:41

Perhaps something like this (not tested)

Code: Select all

Function SaveMessage(oMessage)
	dim oApp, OBJoutfile, FSO

	Set oApp = CreateObject("hMailServer.Application")
	' Give this script permission to access all
	' hMailServer settings.
	Call oApp.Authenticate("Administrator", "TopSecretPassword")
	Set FSO = CreateObject("Scripting.FileSystemObject")
	set OBJoutfile = FSO.CopyFile(oMessage.filename,"C:\SomeDirectory\" & oMessage.id & ".eml"
	set OBJOutFile = Nothing
End Function
With the function called from a Global rule, or as part of 'OnDeliverMessage'

Really depends on what other scripts / rules etc that you run
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
Dravion
Senior user
Senior user
Posts: 1414
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: Save all EML no users

Post by Dravion » 2019-04-02 08:33

Interresting Solution.

IMHO
A free MTA like Postfix or Exim or even Sendmail doing pickup Directories per default.

User avatar
SorenR
Senior user
Senior user
Posts: 3169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Save all EML no users

Post by SorenR » 2019-04-02 11:38

There is one major issue that will break this project. It is a bug I have reported a long time ago, Martin looked at it but I believe nothing was done.

CatchAll address translation!
  • hMailServer translates recipient to Alias or CatchAll when receiving email. Thus the need for:
    • oMessage.Recipients(i).Address --> mail is delivered TO
    • oMessage.Recipients(i).OriginalAddress --> mail was intended FOR
  • Assuming originating server do not split the email like commercial servers mostly do - free GMail don't.
  • When hMailServer receive same email twice to same recipient - same recipient in To: and CC: there will be two identical items in oMessage.Recipients(i).Address and hMailServer will subsequently skip one. This is all fine BUT!
  • When hMailServer receive email to multiple recipients into a CatchAll account. ALL oMessage.Recipients(i).Address are translated into the CatchAll account - AND duplicates are skipped leaving only ONE recipient.
    • One email with three recipients ...

      Code: Select all

      CatchAll = daffy.duck@acme.inc
      
      Originating Server  -->  oMsg.Recipients.Address  -->  oMsg.Recipients().OriginalAddress
      
       wile.e.coyote@acme.inc   daffy.duck@acme.inc           wile.e.coyote@acme.inc
       road.runner@acme.inc     daffy.duck@acme.inc           road.runner@acme.inc
       elmer.fudd@acme.inc      daffy.duck@acme.inc           elmer.fudd@acme.inc
       
            ...-...             daffy.duck@acme.inc           wile.e.coyote@acme.inc
      
    • Consolidating recipients is usually fine, it will save space and time - However in this particular case we loose information about "the other two intended recipients". Codewise hMailServer consolidate recipients between OnSMTPData(oClient, oMessage) and OnAcceptMessage(oClient, oMessage) so it is not possible to catch this using scripts and rules are only performed later still.
    I have modified the script to include X-Envelope-To (based on .OriginalAddress) and X-Envelope-From as this information is NOT in the .eml file.

    The code is untested - DO NOT try this on a production server without first testing the code on a TEST server! This script WILL DELETE the original message in the process.

    Code: Select all

    Sub OnDeliveryStart(oMessage)
    
       '
       ' Secure envelope sender and recipient(s)
       '
       Dim i, strTo
       For i = 0 To oMessage.Recipients.Count-1
          If (i = 0) Then strTo = oMessage.Recipients(i).OriginalAddress
          If (i > 0) Then strTo = strTo & ", " & oMessage.Recipients(i).OriginalAddress
       Next
       oMessage.HeaderValue("X-Envelope-To") = strTo
       oMessage.HeaderValue("X-Envelope-From") = oMessage.FromAddress
       oMessage.Save
    
       '
       ' Give this script permission to access all hMailServer settings.
       '
       Dim oApp : Set oApp = CreateObject("hMailServer.Application")
       Call oApp.Authenticate("Administrator", "TopSecretPassword")
    
       '
       ' Copy message file to external directory.
       '
       With CreateObject("Scripting.FileSystemObject")
          .CopyFile(oMessage.filename,"C:\SomeDirectory\" & oMessage.filename)
       End With
    
       '
       ' Housekeeping
       '
       oApp.Messages.DeleteByDBID(oMessage.ID)
       Set oApp = Nothing
       
    End Sub
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-02 16:01

For this service, only emails with a single recipient are supported. I took both examples and combined it to the following:

Code: Select all

Sub OnDeliverMessage(oMessage)
dim oApp, oEMLFile, FSO

	Set oApp = CreateObject("hMailServer.Application")
	' Give this script permission to access all
	' hMailServer settings.
	Call oApp.Authenticate("Administrator", "TopSecretPassword")
	
	Set FSO = CreateObject("Scripting.FileSystemObject")
	FSO.CopyFile oMessage.filename, "D:\Data\Emails\drop\" & oMessage.Id & ".eml"
	Set FSO = Nothing
	
	 oApp.Messages.DeleteByDBID(oMessage.Id)
	 Set oApp = Nothing
End Sub
There was compile error calling CopyFile.

Code: Select all

Script Error: Source: Microsoft VBScript compilation error - Error: 800A0414 - Description: Cannot use parentheses when calling a Sub - Line: 22 Column: 77 - Code: 		FSO.CopyFile(oMessage.filename, "D:\Data\Emails\drop\" & oMessage.id & ".eml")"
I removed the parenthesis, but now I'm getting another error.
"Script Error: Source: Microsoft VBScript runtime error - Error: 800A01CA - Description: Variable uses an Automation type not supported in VBScript - Line: 22 Column: 1 - Code: (null)"

User avatar
mattg
Moderator
Moderator
Posts: 20003
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Save all EML no users

Post by mattg » 2019-04-02 16:41

can you please include all of your eventhandlers.vbs, so that we can see which is line 22

Remember to remove your password
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
SorenR
Senior user
Senior user
Posts: 3169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Save all EML no users

Post by SorenR » 2019-04-02 16:58

adrianp wrote:
2019-04-02 16:01
For this service, only emails with a single recipient are supported. I took both examples and combined it to the following:

Code: Select all

Sub OnDeliverMessage(oMessage)
dim oApp, oEMLFile, FSO

	Set oApp = CreateObject("hMailServer.Application")
	' Give this script permission to access all
	' hMailServer settings.
	Call oApp.Authenticate("Administrator", "TopSecretPassword")
	
	Set FSO = CreateObject("Scripting.FileSystemObject")
	FSO.CopyFile oMessage.filename, "D:\Data\Emails\drop\" & oMessage.Id & ".eml"
	Set FSO = Nothing
	
	 oApp.Messages.DeleteByDBID(oMessage.Id)
	 Set oApp = Nothing
End Sub
There was compile error calling CopyFile.

Code: Select all

Script Error: Source: Microsoft VBScript compilation error - Error: 800A0414 - Description: Cannot use parentheses when calling a Sub - Line: 22 Column: 77 - Code: 		FSO.CopyFile(oMessage.filename, "D:\Data\Emails\drop\" & oMessage.id & ".eml")"
I removed the parenthesis, but now I'm getting another error.
"Script Error: Source: Microsoft VBScript runtime error - Error: 800A01CA - Description: Variable uses an Automation type not supported in VBScript - Line: 22 Column: 1 - Code: (null)"
Since you took out the only code that can identify the recipient, how will you 112% know who the mail is for? The "To:" header can be faked just as easy as my name is "Elmer Fudd" <elmer.fudd@looney.tunes.inc>
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-02 18:09

mattg wrote:
2019-04-02 16:41
can you please include all of your eventhandlers.vbs, so that we can see which is line 22

Remember to remove your password
The code is in the default EventHandler. Line 22 is FSO.CopyFile(oMessage.filename, "D:\Data\Emails\drop\" & oMessage.id & ".eml")"

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-02 18:13

SorenR wrote:
2019-04-02 16:58
Since you took out the only code that can identify the recipient, how will you 112% know who the mail is for? The "To:" header can be faked just as easy as my name is "Elmer Fudd" <elmer.fudd@looney.tunes.inc>
I'm not using hMAIL as it would typically be used, meaning it's not an email sender to email recipient system. We are only using email to receive documents. We have specific measures in place to limit connections to this gateway as well as other mechanism.

User avatar
SorenR
Senior user
Senior user
Posts: 3169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Save all EML no users

Post by SorenR » 2019-04-02 21:47

adrianp wrote:
2019-04-02 18:13
SorenR wrote:
2019-04-02 16:58
Since you took out the only code that can identify the recipient, how will you 112% know who the mail is for? The "To:" header can be faked just as easy as my name is "Elmer Fudd" <elmer.fudd@looney.tunes.inc>
I'm not using hMAIL as it would typically be used, meaning it's not an email sender to email recipient system. We are only using email to receive documents. We have specific measures in place to limit connections to this gateway as well as other mechanism.
Documents as in attachments?
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-02 21:49

Here's the contents of the complete file.

Code: Select all

'   Sub OnClientConnect(oClient)
'   End Sub

'   Sub OnSMTPData(oClient, oMessage)
'   End Sub

'   Sub OnAcceptMessage(oClient, oMessage)
'   End Sub

'	Sub OnDeliveryStart(oMessage)
'	End Sub

Sub OnDeliverMessage(oMessage)
dim oApp, oEMLFile, FSO

	Set oApp = CreateObject("hMailServer.Application")
	' Give this script permission to access all
	' hMailServer settings.
	Call oApp.Authenticate("Administrator", "TopSecretPassword")
	
	Set FSO = CreateObject("Scripting.FileSystemObject")
	FSO.CopyFile oMessage.Filename, "D:\Data\Emails\drop\" & oMessage.Id & ".eml"
	Set FSO = Nothing
	
	 oApp.Messages.DeleteByDBID(oMessage.Id)
	 Set oApp = Nothing
End Sub

'   Sub OnBackupFailed(sReason)
'   End Sub

'   Sub OnBackupCompleted()
'   End Sub

'   Sub OnError(iSeverity, iCode, sSource, sDescription)
'   End Sub

'   Sub OnDeliveryFailed(oMessage, sRecipient, sErrorMessage)
'   End Sub

'   Sub OnExternalAccountDownload(oFetchAccount, oMessage, sRemoteUID)
'   End Sub
Where can I see the definition of oMessage ? I didn't see it in the online documentation.

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-02 21:52

SorenR wrote:
2019-04-02 21:47
adrianp wrote:
2019-04-02 18:13
SorenR wrote:
2019-04-02 16:58
Since you took out the only code that can identify the recipient, how will you 112% know who the mail is for? The "To:" header can be faked just as easy as my name is "Elmer Fudd" <elmer.fudd@looney.tunes.inc>
I'm not using hMAIL as it would typically be used, meaning it's not an email sender to email recipient system. We are only using email to receive documents. We have specific measures in place to limit connections to this gateway as well as other mechanism.
Documents as in attachments?
Does that have any impact in getting the EML file saved in a specified folder?

User avatar
SorenR
Senior user
Senior user
Posts: 3169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Save all EML no users

Post by SorenR » 2019-04-02 23:03

OK, this works as intended. "Housekeeping" was a long shot anyways and appears to violate functionality.
No worries, you can create a "Rule" to delete messages as they come in.

File is saved as "{ <GUID> }.eml" so NO duplicates ... EVER :mrgreen:

Code: Select all

Sub OnDeliveryStart(oMessage)

   '
   ' Copy message file to external directory.
   '
   With CreateObject("Scripting.FileSystemObject")
      .CopyFile oMessage.filename, "C:\SomeDirectory\" & Mid(oMessage.filename, InStr(oMessage.filename, "{")), True
   End With

End Sub
The reason I asked about attachments is that you could wrap some code around this...

Code: Select all

'
' Beginning of some code here
'
' For i = 0 To oMessage.Attachments.Count -1
'    strFile = "C:\SomeDirectory\" & oMessage.Attachments.item(i).Filename
'    oMessage.Attachments(i).saveas(strFile)
' Next
'
' Some more code here
'
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
SorenR
Senior user
Senior user
Posts: 3169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Save all EML no users

Post by SorenR » 2019-04-02 23:32

adrianp wrote:
2019-04-02 21:49
Where can I see the definition of oMessage ? I didn't see it in the online documentation.
https://www.hmailserver.com/documentati ... ct_message
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
mattg
Moderator
Moderator
Posts: 20003
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Save all EML no users

Post by mattg » 2019-04-03 05:54

adrianp wrote:
2019-04-02 21:52
SorenR wrote:
2019-04-02 21:47
Documents as in attachments?
Does that have any impact in getting the EML file saved in a specified folder?
I guess the question is do you want the email, or do you want the attachments saved to a set directory.

I save attachments sent from fax servers to a specific directory, for 'pickup' in another program
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-03 14:37

mattg wrote:
2019-04-03 05:54
adrianp wrote:
2019-04-02 21:52
SorenR wrote:
2019-04-02 21:47
Documents as in attachments?
Does that have any impact in getting the EML file saved in a specified folder?
I guess the question is do you want the email, or do you want the attachments saved to a set directory.

I save attachments sent from fax servers to a specific directory, for 'pickup' in another program
No, I need them saved in the EML file. I have a service that processes the email and extracts the documents. I did a test last night with an attachment and that's coming through as expected.

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-03 14:56

SorenR wrote:
2019-04-02 23:03
File is saved as "{ <GUID> }.eml" so NO duplicates ... EVER
Ok, it's working now with the code that doesn't try to access the ID property which seems isn't set at this stage.

Regarding housekeeping, I see the original emails inside the Data\[DOMAIN NAME]\[ACCOUNT] folder. I need to delete any message that was successfully saved in my event handler. I started looking at doing this in the rules as you mentioned.

What happens if an error occurs in my OnDeliverMessage? Highly unlikely, but I'm curious if there is a way to handle it so that the email isn't deleted.
What about emails marked as SPAM or VIRUS, where do those end up? I'm asking because I want to be able to monitor for when this happens and have the ability if needed to examine the text EML file.

User avatar
SorenR
Senior user
Senior user
Posts: 3169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Save all EML no users

Post by SorenR » 2019-04-03 15:46

adrianp wrote:
2019-04-03 14:56
SorenR wrote:
2019-04-02 23:03
File is saved as "{ <GUID> }.eml" so NO duplicates ... EVER
Ok, it's working now with the code that doesn't try to access the ID property which seems isn't set at this stage.

Regarding housekeeping, I see the original emails inside the Data\[DOMAIN NAME]\[ACCOUNT] folder. I need to delete any message that was successfully saved in my event handler. I started looking at doing this in the rules as you mentioned.

What happens if an error occurs in my OnDeliverMessage? Highly unlikely, but I'm curious if there is a way to handle it so that the email isn't deleted.
What about emails marked as SPAM or VIRUS, where do those end up? I'm asking because I want to be able to monitor for when this happens and have the ability if needed to examine the text EML file.
Well, the ID property IS set but that's not the problem, it's the other half of that statement that is not initialized properly - somehow I should have known that :roll:

Regarding the rest of your questions... Read the documentation, search the forum or hire a programmer. It is no longer a 5 minute task.

Housekeeping can be done with Rules and by using custom mailheaders you can pass parameters from script to Rules.

Errorhandling is a complex task and you need to know hMailServer and Microsoft COM/DCOM well. With VBScript you can control errorhandling inside the code and hMailServer have builtin errorhandling that can trigger additional scripting.

Regarding SPAM... Spammers change there MO all the time, I'm well into my 10'th year seriously fighting SPAM with hMailServer/SpamAssassin and have probably posted a couple thousand lines of script code on the forum for people to pick from or to become inspired from.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-16 00:26

I modified the script to add a header to indicate that it's been copied. I then added a rule to delete emails with this custom header. I'm seeing the email still in the data folder with the custom header added so I know that part is working. But for some reason the rule didn't delete the email.

I won't need to hire a developer, I am one :) I just want to see how much I can do with the scripting and with mechanisms that exist before I start digging through the COM interface.

SPAM isn't a huge concern as we block a lot of traffic at the edge as well as it's not actual email recipients that receive these emails. So anything that fails our proprietary verification ends up in a black hole anyways. I just thought it'd be something extra we can enable. Anyways, this is minor next to the main thing I'm trying to do. Right now if I can get the email deleted correctly I think I'm good.

User avatar
mattg
Moderator
Moderator
Posts: 20003
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Save all EML no users

Post by mattg » 2019-04-16 03:18

Try two separate rules, rather than adding to the existing rule.

Depends on what your rule does, it may actually make a copy of the message, and then delete one or the other not both.

Enable debug logging, it will show which rules fire, and what actions are taken
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

adrianp
New user
New user
Posts: 13
Joined: 2019-03-30 14:55

Re: Save all EML no users

Post by adrianp » 2019-04-16 03:58

mattg wrote:
2019-04-16 03:18
Try two separate rules, rather than adding to the existing rule.

Depends on what your rule does, it may actually make a copy of the message, and then delete one or the other not both.

Enable debug logging, it will show which rules fire, and what actions are taken
I didn't add to an existing rule. I added (only have 1 rule) which looks for my custom header (X-Cleanup) with value True and is set to delete the email.


The copying of the email is done in the event handler, OnDeliverMessage. This is where I copy the email to the specific folder that I need it to be and I add the custom header, X-Cleanup = True.

Code: Select all

	oMessage.HeaderValue("X-Cleanup") = "True"
	oMessage.Save

Anyways, you got me headed in the right direction. I hadn't noticed the rules at the account level and had created it as a global rule. The global rules seem to execute before the event handler. After I created it as an account rule it started working correctly.

"DEBUG" 1720 "2019-04-15 20:54:38.505" "Applying rules"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Executing event OnMessageDeliver"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Event completed"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Performing local delivery"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Applying rules"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Applying rule Cleanup"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Performing rule action"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Local delivery completed"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Deleting message"
"DEBUG" 1720 "2019-04-15 20:54:38.505" "Deleting message file."

Thank you,
Adrian

Post Reply