Error HM5032 "Incorrect String Value"

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
chameenz
New user
New user
Posts: 29
Joined: 2004-06-23 16:48
Location: Baltimore, MD

Error HM5032 "Incorrect String Value"

Post by chameenz » 2012-09-26 17:43

Recently upgraded to 5.3.3. System is running mostly fine, but email clients periodically stop showing new messages. The client must be closed and reopened to show new messages. The problem is occurring in Outlook and Thunderbird. Going through the logs I see the following line repeated about 12 times during the day, but can not find a way to fix it.


"ERROR" 1384 "2012-09-26 11:16:14.107" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:17:34.166" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:18:54.086" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:20:15.740" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:21:37.269" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:22:59.345" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:24:19.000" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:25:38.841" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:26:58.777" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:28:18.525" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:29:38.851" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:30:58.974" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:32:21.206" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:33:41.167" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:35:01.852" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:36:22.272" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty "ERROR" 1384 "2012-09-26 11:37:41.848" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x92\x99 <...' for column 'metadata_cc' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (359, 2652, 8498400, '2012-04-19 15:27:00', 'someother user <someuser@notmydomain.com>', 'Re: Sales ticket #222212', '"user@mydomain.com" <user@mydomain.com>', 'Patty

Windows 2003, Xeon 1.8GHz, 4GB RAM, 100GB HD avail

^DooM^
Site Admin
Posts: 13862
Joined: 2005-07-29 16:18
Location: UK

Re: Error HM5032 "Incorrect String Value"

Post by ^DooM^ » 2012-09-26 17:46

Disable message indexing if it's causing issues with mail being delivered. try to run a repair on the hm_message_metadata table. If that fails to fix it, empty the hm_message_metadata table completely.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

tringstad
New user
New user
Posts: 7
Joined: 2011-12-12 19:49

Re: Error HM5032 "Incorrect String Value"

Post by tringstad » 2013-03-24 16:06

I now suddenly keep getting this same error. I have tried to empty cache and database table with no luck. When i activate Message Indexing it starts all over again... force new index or indexing the "normal" way does not fix anything...

1. Why does it do this?
2. How can i fix it?

Bill48105
Developer
Developer
Posts: 6189
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: Error HM5032 "Incorrect String Value"

Post by Bill48105 » 2013-03-24 18:42

tringstad wrote:I now suddenly keep getting this same error. I have tried to empty cache and database table with no luck. When i activate Message Indexing it starts all over again... force new index or indexing the "normal" way does not fix anything...

1. Why does it do this?
2. How can i fix it?
I see these now & then and have seen other reports. Based on the error I'd figure it's annoying & doesn't hurt anything. Odds are the email has some odd charset or string that SQL doesn't like. Since indexing will work fine if a message getting indexed fails (index just isn't just for that message) so it's not anything to worry about & not been a big priority to research too much. Odds are it's a spammer with some crap format.
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

Bill48105
Developer
Developer
Posts: 6189
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: Error HM5032 "Incorrect String Value"

Post by Bill48105 » 2013-04-04 22:23

Did some more research on this and while it appears to be the encoding of the subject not sure of best way to fix it. It's possible it's invalid encoding coming in & hmail and/or mysql are choking on it. Maybe dzekas can shed some light. :)

hmail error log shows:

Code: Select all

"ERROR"	3016	"2013-04-04 00:52:02.997"	"Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x9A\x99  ...' for column 'metadata_subject' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (3, 675, 433793, '2013-04-04 04:49:14', '"- FORD ONLINE BLOWOUT -" <NewFordCars@canadadepartmentstore.com>', '[Spam? - Score:19.8 Req:9.0] "
Raw headers after SA:

Code: Select all

To: xxxx@xxxx.com
Subject: [Spam? - Score:19.8 Req:9.0] =?utf-8?q?=F0=9F=9A=99?=   ONLINE FORD
 CLEARANCE! - EVENT 2013 ?utf-8?q?=F0=9F=9A=99?=
Date: Wed, 3 Apr 2013 21:49:14 -0700
Raw headers before SA:

Code: Select all

To: xxxx@xxxx.com
Subject:  =?utf-8?q?=F0=9F=9A=99?=   ONLINE FORD CLEARANCE! - EVENT 2013 ?utf-8?q?=F0=9F=9A=99?=
Message-ID: <4fa1d8c330fd77ea5ccfb66a8c0301ec@canadagasstation.com>
So in this case SA broke subject into 2 lines after adding the Spam tag but doubt that's the cause of the issue. I ran the UTF-8 strings thru tester & supposedly they are invalid. (Having to trust that as I don't encounter/see decoded UTF-8 myself)

From source this is being called for to/from/subject headers before adding to indexing table:

Code: Select all

      // De-code the value to plain text.
      AnsiString sRetVal;
      FieldCodeBase* pCoder = MimeEnvironment::CreateFieldCoder(pszFieldName);
      pCoder->SetInput(sRawFieldValue, sRawFieldValue.GetLength(), false);
      pCoder->GetOutput(sRetVal);

      AnsiString sCharset = pCoder->GetCharset();

      delete pCoder;

      String sWideStr = Charset::ToWideChar(sRetVal, sCharset);

      return sWideStr;

The problem is the crap utf-8 in subject but question is how to solve the problem & stop these errors from happening. The error makes it appear like hmail is decoding or encoding since = becomes \x so perhaps the answer is to change hmail doing that. I'll need to find the code & see and perhaps answer will be there.
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: Error HM5032 "Incorrect String Value"

Post by dzekas » 2013-04-05 17:59

UTF-8 symbol is valid.

Code: Select all

Every utf-8. character consists of n bytes. First byte is octal 300-375 (\xC0 to \xFD hex), other bytes - always octals 200-277 (\x80 to \xBF hex).

Character length in bytes depends on first byte value
 
 *                oct     dec    hex
 * Two byte   - 300-337 192-223 C0-DF
 * Three byte - 340-357 224-239 E0-EF
 * Four byte  - 360-367 240-247 F0-F7
 * Five byte  - 370-373 248-251 F8-FB
 * Six byte   - 374-375 252-253 FC-FD
You are looking at totally valid, but unassigned codepoint (U+1F699), which is outside of UCS2 (2 or 3 byte utf-8) range.

Although you could also be looking at Chinese GB2312 text (馃殭) incorrectly marked as utf-8.

From MIME encoding point of view only second '?utf-8?q?=F0=9F=9A=99?=' block is broken. starting tag is missing.

Bill48105
Developer
Developer
Posts: 6189
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: Error HM5032 "Incorrect String Value"

Post by Bill48105 » 2013-04-05 22:42

dzekas wrote:UTF-8 symbol is valid.

Code: Select all

Every utf-8. character consists of n bytes. First byte is octal 300-375 (\xC0 to \xFD hex), other bytes - always octals 200-277 (\x80 to \xBF hex).

Character length in bytes depends on first byte value
 
 *                oct     dec    hex
 * Two byte   - 300-337 192-223 C0-DF
 * Three byte - 340-357 224-239 E0-EF
 * Four byte  - 360-367 240-247 F0-F7
 * Five byte  - 370-373 248-251 F8-FB
 * Six byte   - 374-375 252-253 FC-FD
You are looking at totally valid, but unassigned codepoint (U+1F699), which is outside of UCS2 (2 or 3 byte utf-8) range.

Although you could also be looking at Chinese GB2312 text (馃殭) incorrectly marked as utf-8.

From MIME encoding point of view only second '?utf-8?q?=F0=9F=9A=99?=' block is broken. starting tag is missing.
OK thanks for the insight dzekas. I was talking to martin about this & suspect it's an hmail issue although to be honest charsets are not my strong point as I know squat about them except PITFA. lol

You mention starting tag missing on 2nd block. Do you think mysql would choke on "=?utf-8?q?=F0=9F=9A=99?= ONLINE FORD CLEARANCE! - EVENT 2013 ?utf-8?q?=F0=9F=9A=99?=" being saved in UTF-8 field? Although according to the logs hmail is encoding such as "\xF0\x9F\x9A\x99".. (Don't have the entire string as it's truncated but safe to say MySQL does not like it lol) So the question boils down to if hmail is at fault for encoding the string wrong or not encoding it correctly or if there is something that can be done to prevent the problem. Should we look for missing starting tags & fix? Should we encode differently not encode =?utf-8?q?=F0=9F=9A=99?= in 1st place if db is set to utf-8? Again I know very little about non-Latin characters & encoding.

Looking at the source this is called on the subject before the sql query:

Code: Select all

   Charset::ToWideChar(const AnsiString &sMultiByte, const AnsiString &sCharacterSet)
   {
      if (sCharacterSet.IsEmpty())
         return sMultiByte;

      int iToCodePage = CodePages::Instance()->GetCodePage(sCharacterSet);

      int iNeedSize = MultiByteToWideChar(iToCodePage, 0, sMultiByte.c_str(), -1, NULL, NULL);

      wchar_t *pOutput = new wchar_t[iNeedSize + 1];
      int i = MultiByteToWideChar(iToCodePage, 0, sMultiByte.c_str(), -1, pOutput, iNeedSize + 1);
      String sWideStr = pOutput;
      delete [] pOutput;

      return sWideStr;
   }
That function is what is converting from =?utf-8?q?=F0=9F=9A=99?= to \xF0\x9F\x9A\x99 but not sure why mysql doesn't like it or if that conversion should even be taking place.
Thx
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: Error HM5032 "Incorrect String Value"

Post by dzekas » 2013-04-06 08:03

Bill48105 wrote:That function is what is converting from =?utf-8?q?=F0=9F=9A=99?= to \xF0\x9F\x9A\x99 but not sure why mysql doesn't like it or if that conversion should even be taking place.
totally valid, but unassigned codepoint. That could be one of the problems. Symbol is valid only due to correct utf-8 formating.

It is 4 byte utf-8. That could be other problem.

Missing = means that parser does not have to decode and treats it as plain ASCII text. From user point of view they would see crap in subject. From program point of view they don't have to do anything about it.

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: Error HM5032 "Incorrect String Value"

Post by dzekas » 2013-04-06 09:08

Could you replace it with =?utf-8?q?=F0=90=8C=80?= and see if you get same error.

U+10300 OLD ITALIC LETTER A
UTF-8: 0xF0 0x90 0x8C 0x80

One of a few high Unicode letters Westerners can recognize.

ap_venkatesan
New user
New user
Posts: 1
Joined: 2013-04-17 09:51

Re: Error HM5032 "Incorrect String Value"

Post by ap_venkatesan » 2013-04-17 09:55

Is this problem addressed? Kindly let me know. I am facing this problem suddenly and do not have a solution.
Thanks

Bill48105
Developer
Developer
Posts: 6189
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: Error HM5032 "Incorrect String Value"

Post by Bill48105 » 2013-04-17 16:17

ap_venkatesan wrote:Is this problem addressed? Kindly let me know. I am facing this problem suddenly and do not have a solution.
Thanks
Not yet. Martin & I have discussed it & thanks to dzekas for helping out. The error is annoying but doesn't hurt anything so not been a huge priority.
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

molahs
New user
New user
Posts: 20
Joined: 2009-02-04 04:13

Re: Error HM5032 "Incorrect String Value"

Post by molahs » 2013-06-13 14:46

hmailserver will randomly 'freeze' by not allowing any smtp connection. and partly refuse imap connections, though it still accepts imap and pop though for most users
.
Having looked at the error log when this happens, I see many entries like this:

Code: Select all

"ERROR"	4644	"2013-06-13 07:59:15.359"	"Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: MySQL: Incorrect string value: '\xF0\x9F\x93\x8A N...' for column 'metadata_subject' at row 1 (Additional info: INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (72, 307, 3931182, '2013-05-14 23:38:26', '"ClassDojo" <hello@reports.classdojo.com>', '
Inspecting the MySQL log I see the following entry:

Code: Select all

		    1 Query	INSERT INTO hm_message_metadata (metadata_accountid, metadata_folderid, metadata_messageid, metadata_dateutc, metadata_from, metadata_subject, metadata_to, metadata_cc) VALUES (72, 307, 3931182, '2013-05-14 23:38:26', '"ClassDojo" <hello@reports.classdojo.com>', '📊 New feature: ClassDojo TrendSpotter', '"Mrs. Xxxxx Xxxxxxx" <info@xxxxxxxxxxxx.com>', '')
This message above has been in out logs for a long time now. Note that email metadata_dateutc is '2013-05-14 23:38:26' while the error is logged on "2013-06-13 07:59:15.359".

Since 14-05-2013 hmailserver, mysql and the server itself have been rebooted multiple times.


There are also entries like the following:

Code: Select all

"ERROR"	456	"2013-06-11 10:31:01.367"	"Severity: 1 (Critical), Code: HM5062, Source: IMAPFetch::_ReportCriticalError, Description: ERROR HM10001 - IMAP FETCH: Error when loading MIME message. - D:\hMailServer\data\somedomainhere.com\mariea\F4\{F4622EE2-55D4-4C0D-A518-37EA396665CD}.eml"
"ERROR"	456	"2013-06-11 10:31:01.727"	"Severity: 1 (Critical), Code: HM5062, Source: IMAPFetch::_ReportCriticalError, Description: ERROR HM10001 - IMAP FETCH: Error when loading MIME message. - D:\hMailServer\data\somedomainhere.com\mariea\4D\{4DED8591-731E-44A9-AF73-12045EF48B30}.eml"
"ERROR"	456	"2013-06-11 10:31:01.867"	"Severity: 1 (Critical), Code: HM5062, Source: IMAPFetch::_ReportCriticalError, Description: ERROR HM10001 - IMAP FETCH: Error when loading MIME message. - D:\hMailServer\data\somedomainhere\mariea\0D\{0D1373C4-123A-4556-8D01-EF33523FF2A4}.eml"
When I check the folder for the presense of the file {0D1373C4-123A-4556-8D01-EF33523FF2A4}.eml it is there.

Other errors happening at the same time are:

Code: Select all

"ERROR"	456	"2013-06-11 10:31:06.774"	"Severity: 3 (Medium), Code: HM5136, Source: TCPConnection::HandleRead, Description: An error occured while parsing data. Data length: 175, Data: A0008 FETCH 1:40 (UID RFC822.SIZE FLAGS INTERNALDATE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY)]). Remote IP: 172.16.16.27"
IP 172.16.16.27 is the IP address of hmailserver (internal address behind firewall)

Another few repeating erorrs are :

Code: Select all

"ERROR"	3220	"2013-06-11 10:31:08.571"	"Severity: 2 (High), Code: HM4208, Source: IOCPQueueWorkerTask::DoWork, Description: An unknown error occured while handling asynchronous requests."

Code: Select all

"ERROR"	736	"2013-06-11 10:31:09.696"	"Severity: 2 (High), Code: HM5325, Source: TCPConnection::Start, Description: An unknown error occurred while starting connection."

Code: Select all

"ERROR"	2084	"2013-06-11 10:31:20.680"	"Severity: 2 (High), Code: HM4223, Source: DeliveryTask::DoWork, Description: SMTPDeliverer::DeliverMessage() failed"

Code: Select all

"ERROR"	4040	"2013-06-11 10:31:29.352"	"Severity: 3 (Medium), Code: HM5141, Source: TCPConnection::HandleRead, Description: An error occurred while handling read operation. Remote IP: 76.109.0.57"
The remote ip of 76.109.0.57 has nothing to do with our systems. It's probably one of our users trying to send an email.

Here's a batch of errors following each other (see time stamp)

Code: Select all

"ERROR"	3548	"2013-06-11 10:32:04.711"	"Severity: 2 (High), Code: HM5000, Source: Temp::Temp, Description: Write operation failed."

"ERROR"	3548	"2013-06-11 10:32:04.711"	"Severity: 2 (High), Code: HM5329, Source: TCPConnection::Write, Description: An unknown error occurred while writing buffer."

"ERROR"	3548	"2013-06-11 10:32:04.711"	"Severity: 2 (High), Code: HM5339, Source: TCPConnection::HandleWrite, Description: An unknown error occurred while handling buffer write."

"ERROR"	3548	"2013-06-11 10:32:04.711"	"Severity: 2 (High), Code: HM4208, Source: IOCPQueueWorkerTask::DoWork, Description: An unknown error occured while handling asynchronous requests."
This complete smtp outage lasted for about 15 minutes. I don't know if it's a coincidence or not, but after giving up looking through logs to fix this (without restarting the mail server), I did a manual SMTP connection from the cmd prompt as follows:

Code: Select all

220 xxxxxxx Mail
helo 123
250 Hello.
mail from:<xxxxxxx@gmail.com>
250 OK
rcpt to:<xxxx@xxxxxxxx.com>
250 OK
data
354 OK, send.
Subject: test
From: mark
this is a test
.
250 Queued (11.969 seconds)

502 Unimplemented command.
quit
221 goodbye
It took almost 12 seconds because i'm a slow typist :)

I am hoping that all the additional information will help in identifying the exact bug (if it's in hmailserver) or if some kind soul will be able to point out what's going on with our mail server.

Thanks.

PS - hmailserver 5.3.2 running inside windows 2003 with 4gb ram on vmware ESXi.

Bill48105
Developer
Developer
Posts: 6189
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: Error HM5032 "Incorrect String Value"

Post by Bill48105 » 2013-06-13 16:50

molahs wrote:hmailserver will randomly 'freeze' by not allowing any smtp connection. and partly refuse imap connections, though it still accepts imap and pop though for most users

<SNIP!!>

I am hoping that all the additional information will help in identifying the exact bug (if it's in hmailserver) or if some kind soul will be able to point out what's going on with our mail server.

Thanks.

PS - hmailserver 5.3.2 running inside windows 2003 with 4gb ram on vmware ESXi.
Thanks for the thorough info. You have many different errors there and at this point we can't say if they are related or not. Fact that the time stamps are different means they likely are not we can't really rule out that they are either. The other issue is that you're on 5.3.2 and 5.3.x is KNOWN TO HAVE ISSUES so you really should consider upgrading to 5.4 to at least see if the errors continue or change. IOCP errors/crashes were very well known issue with 5.3.x and much rarer with 5.4 for example. The meta data ones are a known issue in all hmail versions & has to do with crap encoding of Subject by spammers that so far has been just an annoyance & will be looked into soon. (Either way 5.3.x will not likely get any more updates so you'd need to be on 5.4 to get the encoding fix anyway) All that said, you really should have started your own thread instead of tacking onto this one.
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
Spyd
Normal user
Normal user
Posts: 43
Joined: 2008-02-19 10:15
Location: Barcelona, Spain
Contact:

Re: Error HM5032 "Incorrect String Value"

Post by Spyd » 2015-06-02 10:04

I can confirm this is something related to rarely used UTF-8 codes. They can be icons or symbols or anything. You know they're rare because maybe they're displayed correctly in one computer/font or maybe the same code gives you a question mark inside a block on other computer/font.
I had this problem for ages. While the server keeps running ok, I hate finding error logs that are 500KB or greater from time to time.
My solution? get the message ID from the error log, look for the message and if it's not important (never is) I just delete it. Only marketing mails or spam put those funny characters anyway.

The most annoying part here is that it obscures the error log. If there's a error log file, I want it to contain only errors, not 99,9999% of it an error that's not important.

If it's not possible to fix (or is not worth the effort) is it possible to "demote" this error to a warning or something? Or having the option to ignore it entirely?

User avatar
jimimaseye
Moderator
Moderator
Posts: 7954
Joined: 2011-09-08 17:48

Re: Error HM5032 "Incorrect String Value"

Post by jimimaseye » 2015-06-02 11:06

This was discussed and identified in this thread explaining exactly the problem and potential fixes (depending on version of HMS and MySQL) here: viewtopic.php?f=7&t=26860

However, poster SorenR uses this script to remove such 'painful' characters to stop this problem. His script is here: viewtopic.php?f=9&t=27884
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

Post Reply