Decoding base64 or quoted-printable Subject

Use this forum if you have problems with a hMailServer script, such as hMailServer WebAdmin or code in an event handler.
Post Reply
User avatar
katip
Senior user
Senior user
Posts: 1151
Joined: 2006-12-22 07:58
Location: Istanbul

Decoding base64 or quoted-printable Subject

Post by katip » 2024-02-11 21:16

I need to decode base64 or quoted-printable Subject header to something readable.
It looks like this is not an easy job but i see that HMS writes any Subject into metadata_subject very well human readable.
Is there an easy way to pass it also to a variable at some event stage?
Parsing raw Subject and decoding is indeed possible but may be sometimes quite tricky (plain text + base64 + quoted-printable combination in same Subject!!)

There is a similar topic here: viewtopic.php?t=36564#p229029
but doesn't help much, I need full readable text.

Or do I overlook something very basic?
Katip
--
HMS 5.7, MariaDB 10.4.10, SA 4.0.0, ClamAV 0.103.8

User avatar
SorenR
Senior user
Senior user
Posts: 6268
Joined: 2006-08-21 15:38
Location: Denmark

Re: Decoding base64 or quoted-printable Subject

Post by SorenR » 2024-02-11 22:33

This ??

Code: Select all

Function Base64Encode(ByVal sText)
   Dim fAsUtf16LE
   fAsUtf16LE=0

   ' Use an aux. XML document with a Base64-encoded element.
   ' Assigning the byte stream (array) returned by StrToBytes() to .NodeTypedValue
   ' automatically performs Base64-encoding, whose result can then be accessed
   ' as the element's text.
   With CreateObject("Msxml2.DOMDocument").CreateElement("aux")
      .DataType = "bin.base64"
      If fAsUtf16LE Then
         .NodeTypedValue = StrToBytes(sText, "utf-16le", 2)
      Else
         .NodeTypedValue = StrToBytes(sText, "utf-8", 3)
      End If
      Base64Encode = .Text
   End With

End Function


' Decodes the specified Base64-encoded string.
' If the decoded string's original encoding was:
' * UTF-8, pass False for fIsUtf16LE.
' * UTF-16 LE, pass True for fIsUtf16LE.
Function Base64Decode(ByVal sBase64EncodedText)
   Dim fAsUtf16LE
   fAsUtf16LE=0
   Dim sTextEncoding
   If fIsUtf16LE Then sTextEncoding = "utf-16le" Else sTextEncoding = "utf-8"

   ' Use an aux. XML document with a Base64-encoded element.
   ' Assigning the encoded text to .Text makes the decoded byte array
   ' available via .nodeTypedValue, which we can pass to BytesToStr()
   With CreateObject("Msxml2.DOMDocument").CreateElement("aux")
      .DataType = "bin.base64"
      .Text = sBase64EncodedText
      Base64Decode = BytesToStr(.NodeTypedValue, sTextEncoding)
   End With

End Function


' Returns a binary representation (byte array) of the specified string in
' the specified text encoding, such as "utf-8" or "utf-16le".
' Pass the number of bytes that the encoding's BOM uses as iBomByteCount;
' pass 0 to include the BOM in the output.
Function StrToBytes(ByVal sText, ByVal sTextEncoding, ByVal iBomByteCount)

   ' Create a text string with the specified encoding and then
   ' get its binary (byte array) representation.
   With CreateObject("ADODB.Stream")
      ' Create a stream with the specified text encoding...
      .Type = 2  ' adTypeText
      .Charset = sTextEncoding
      .Open
      .WriteText sText
      ' ... and convert it to a binary stream to get a byte-array
      ' representation.
      .Position = 0
      .Type = 1  ' adTypeBinary
      .Position = iBomByteCount ' skip the BOM
      StrToBytes = .Read
      .Close
   End With

End Function

' Returns a string that corresponds to the specified byte array, interpreted
' with the specified text encoding, such as "utf-8" or "utf-16le".
Function BytesToStr(ByVal byteArray, ByVal sTextEncoding)

   If LCase(sTextEncoding) = "utf-16le" Then
      ' UTF-16 LE happens to be VBScript's internal encoding, so we can
      ' take a shortcut and use CStr() to directly convert the byte array
      ' to a string.
      BytesToStr = CStr(byteArray)
   Else ' Convert the specified text encoding to a VBScript string.
      ' Create a binary stream and copy the input byte array to it.
      With CreateObject("ADODB.Stream")
         .Type = 1 ' adTypeBinary
         .Open
         .Write byteArray
         ' Now change the type to text, set the encoding, and output the
         ' result as text.
         .Position = 0
         .Type = 2 ' adTypeText
         .CharSet = sTextEncoding
         BytesToStr = .ReadText
         .Close
      End With
   End If

End Function
SørenR.

Old data analysts don’t die – they just get broken down by age and sex.

User avatar
katip
Senior user
Senior user
Posts: 1151
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Decoding base64 or quoted-printable Subject

Post by katip » 2024-02-12 08:35

SorenR wrote:
2024-02-11 22:33
This ??
Yes, and this for quoted-printable:

Code: Select all

Function QuotedPrintableDecode(SourceData, CharSet)
  'Create CDO.Message object For the encoding.
  Dim Message: Set Message = CreateObject("CDO.Message")

  'Set the encoding
  Message.BodyPart.ContentTransferEncoding = "quoted-printable"
  
  'Get the data stream To write source string data
  Dim Stream 'As ADODB.Stream
  Set Stream = Message.BodyPart.GetEncodedContentStream
  If VarType(SourceData) = vbString Then
    'Set charset To base windows charset
    Stream.charset = "windows-1254"

    'Write the VBScript string To the stream.
    Stream.WriteText SourceData
  Else
    'Set the type of the stream To adTypeBinary.
    Stream.Type = 1
    
    'Write the source binary data To the stream.
    Stream.Write SourceData
  End If
  
  'Store the data To the message BodyPart
  Stream.Flush
  
  'Get an encoded stream
  Set Stream = Message.BodyPart.GetDecodedContentStream
  
  'Set the type of the stream To adTypeBinary.
  Stream.CharSet = CharSet
  
  'You can use Read method To get a binary data.
  QuotedPrintableDecode = Stream.ReadText
End Function
As I said, decoding is the easy part - as long as you get the start and end of what to decode. But parsing and extracting (and re-assembling, see below) may not be so easy.

go figure:
Subject: [WT] FW: Request for Information:=?utf-8?Q?=20Company=20T=C3=9CRKIYE=20?= (Xyz Holding A.=?utf-8?B?wqo=?=.) Europe Road Freight Tender

ok, not an atom science... but i was looking for a simpler solution with internal calls & just a few lines VBS.

another solution could be to insert Message-ID and Subject into a seperate table and retrieve Subject from this table by Message-ID SQL lookup. Much simpler..
Katip
--
HMS 5.7, MariaDB 10.4.10, SA 4.0.0, ClamAV 0.103.8

User avatar
SorenR
Senior user
Senior user
Posts: 6268
Joined: 2006-08-21 15:38
Location: Denmark

Re: Decoding base64 or quoted-printable Subject

Post by SorenR » 2024-02-12 14:20

I don't get it... Are you looking at "Subject:" in the raw email or oMessage.Subject?

hMailServer will (usually) decode any text in headers to readable text. You can switch off encoding/decoding like this:

Code: Select all

Dim ECFlag : ECFlag = oMessage.EncodeFields
oMessage.EncodeFields = False

'Do stuff with raw headers...

oMessage.EncodeFields = ECFlag
SørenR.

Old data analysts don’t die – they just get broken down by age and sex.

User avatar
katip
Senior user
Senior user
Posts: 1151
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Decoding base64 or quoted-printable Subject

Post by katip » 2024-02-12 16:33

SorenR wrote:
2024-02-12 14:20
I don't get it... Are you looking at "Subject:" in the raw email or oMessage.Subject?

hMailServer will (usually) decode any text in headers to readable text. You can switch off encoding/decoding like this:

Code: Select all

Dim ECFlag : ECFlag = oMessage.EncodeFields
oMessage.EncodeFields = False

'Do stuff with raw headers...

oMessage.EncodeFields = ECFlag
i doubt this .EncodeFields works properly with oMessage.Subject. Already tested:

Code: Select all

	ECFlag = oMessage.EncodeFields
	oMessage.EncodeFields = False ' or True, makes no difference
	decSubject = oMessage.Subject
	oMessage.HeaderValue("X-HMS-decSubject") = decSubject
	oMessage.Save
	oMessage.EncodeFields = ECFlag
	
as a result, X-HMS-decSubject is the same garbage as in Subject in raw text mail source view. Should I expect human readable text like in a mail client?
Katip
--
HMS 5.7, MariaDB 10.4.10, SA 4.0.0, ClamAV 0.103.8

User avatar
SorenR
Senior user
Senior user
Posts: 6268
Joined: 2006-08-21 15:38
Location: Denmark

Re: Decoding base64 or quoted-printable Subject

Post by SorenR » 2024-02-12 17:39

katip wrote:
2024-02-12 16:33
SorenR wrote:
2024-02-12 14:20
I don't get it... Are you looking at "Subject:" in the raw email or oMessage.Subject?

hMailServer will (usually) decode any text in headers to readable text. You can switch off encoding/decoding like this:

Code: Select all

Dim ECFlag : ECFlag = oMessage.EncodeFields
oMessage.EncodeFields = False

'Do stuff with raw headers...

oMessage.EncodeFields = ECFlag
i doubt this .EncodeFields works properly with oMessage.Subject. Already tested:

Code: Select all

	ECFlag = oMessage.EncodeFields
	oMessage.EncodeFields = False ' or True, makes no difference
	decSubject = oMessage.Subject
	oMessage.HeaderValue("X-HMS-decSubject") = decSubject
	oMessage.Save
	oMessage.EncodeFields = ECFlag
	
as a result, X-HMS-decSubject is the same garbage as in Subject in raw text mail source view. Should I expect human readable text like in a mail client?
If you set oMessage.EncodeFields = "false" you should see exactly the same as in the raw email. If oMessage.Subject still looks like shite with oMessage.EncodeFields = "true" then hMailServer is not doing its job properly.
SørenR.

Old data analysts don’t die – they just get broken down by age and sex.

User avatar
katip
Senior user
Senior user
Posts: 1151
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Decoding base64 or quoted-printable Subject

Post by katip » 2024-02-12 19:29

SorenR wrote:
2024-02-12 17:39
If oMessage.Subject still looks like shite with oMessage.EncodeFields = "true" then hMailServer is not doing its job properly.
Yes, I think the same. I put this as very last action in OnDeliverMessage, tried with both oMessage.EncodeFields = True / False, in either case encoded garbage comes out.
5.7.0 b2643.6 by RvdH (you or he may want to have a look into this).

Anyway, I dump Message-ID and oMessage.Subject to a table and get readable Subject content from there whenever needed. Quick workaround..
Case closed. Thanks.
Katip
--
HMS 5.7, MariaDB 10.4.10, SA 4.0.0, ClamAV 0.103.8

Post Reply