PurePOP3 library : POP3 functions

gnozal · Post by **gnozal** » Thu Feb 01, 2007 11:21 am

Update (both libs)

Changes :
- new functions PurePOP3_HeaderFindFirst() / PurePOP3_HeaderFindNext() to examine the message header

PurePOP3 still relies on boundaries as defined in the POP3 specifications (I hope this NOD32 bug will be fixed).

Frontier · Post by **Frontier** » Thu Feb 01, 2007 2:57 pm

gnozal wrote:1.
Frontier wrote:... but fails to save the attachments.
That seems like another issue with your antivirus or POP3 server.
Each line of the attachment base64 coded data should have the same length, except the last one. And this is not the case ...
Code: Select all
Dmcyc/3RdCbGvUwcMpgu269UyCJuAFlLInxn4WCZvldeNp8cjfPE8u/lNlEsftaZWXkmmqtQHvcT 
HHcY5X7yOD7A/MBuDnv2jH7peHht0utJvFQUNUmaYRqJddje1d9W2mxTUNDvS92vhhtC 
HaPUPLIlYXDmmi3yrLVXM4jZKCIz99YHNFOIwngJFr0V5i1o/s/waCy0in9MLJLuiodP00Q/HnQz 
5mgpW1ZlyqPeYb5h57i/VutrKcE2Zpa0ZknWLQKrmz820tG+nhZg0rF85lV4deMTTx2lQS87z9Ox
Are the attachments correctly saved with antivirus disabled ?
2. Any news from ESET ?
3. Isn't there any option to enable the scan but disable the message modification ?

Hello,

The strange thing is that neither Microsoft Outlook or Mozilla Thunderbird (both retrieve the messages with the same method like PurePOP3 does) have no issues whatsoever with attachments. Even pine works fine with NOD32, so this leads me to the conclusion that something with PurePOP3 has to do.

Eset (NOD32 makers) were kind enough to supply me with information regarding the inner workings of NOD32 IMON (Internet Monitor) and how e-mail get's scanned. They provided me with tests that I could perform to ensure that NOD32 was working fine (one of them had to do with pine); everything worked fine, PureSMTP sends the messages with the attachments OK, but PurePOP3 fails to extract the attachments completely (only the first 7kB get saved). In the meantime, pine, Thunderbird and Outlook get the messages with the attachments fine and with the correct sizes.

In conclusion: the latest beta code does indeed work regarding the detection of attachments (with NOD32 IMON fully enabled), but fails to save the attachments correctly. If the attachment is less than 7kB, then it's OK, but if it's bigger (which is in most of the times), the saved attachment is corrupt because only the first 7kBs are saved. I came to the conclusion that it is more likely that PurePOP3 is at fault than NOD32, simply by the fact that other POP3 solutions work fine with the same configuration.

Use the following link to download the trace files for this test:

Code: Select all

http://www.freefileupload.net/file.php?file=files/010207/1170337982/PurePOP3_dumps.rar

I am willing to help as much as I can to fix this, as this is a really wonderful and much needed library for our PureBasic systems.

Edit: I'll try the latest beta you've posted before my message. Of course I do not wish to take your time. Just to note that even with no message tagging (altering it), PurePOP3 still does not decode messages correctly.

I'll try the latest beta and give you more feedback if you wish.

gnozal · Post by **gnozal** » Thu Feb 01, 2007 3:48 pm

Frontier wrote:Eset (NOD32 makers) were kind enough to supply me with information regarding the inner workings of NOD32 IMON (Internet Monitor) and how e-mail get's scanned. They provided me with tests that I could perform to ensure that NOD32 was working fine

The dumps (raw data retrieved from the POP3 server) indicate that this is not the case : the message is modified in a way that the boundary is lost.

RFC 1521 :
... The Content-Type field for multipart entities requires one parameter, "boundary", which is used to specify the encapsulation boundary. The encapsulation boundary is defined as a line consisting entirely of two hyphen characters ("-", decimal code 45) followed by the boundary parameter value from the Content-Type header field...
... Note that the encapsulation boundary must occur at the beginning of a line, i.e., following a CRLF, and that the initial CRLF is considered to be attached to the encapsulation boundary rather than part of the preceding part. The boundary must be followed immediately either by another CRLF and the header fields for the next part, or by two CRLFs, in which case there are no header fields for the next part (and it is therefore assumed to be of Content-Type text/plain)...
... Encapsulation boundaries must not appear within the encapsulations, and must be no longer than 70 characters, not counting the two leading hyphens. The encapsulation boundary following the last body part is a distinguished delimiter that indicates that no further body parts will follow. Such a delimiter is identical to the previous delimiters, with the addition of two more hyphens at the end of the line ...

Changing the message part detection method from boundary to content type (if Outlook and co work that way ?) would imply changing a lot of code. And there are messages with several embedded boundaries ... It would be easier to rewrite most parts the library, and I don't have the time for this.

Why Outlook and co don't mind if the message is trashed by NOD32 I don't know, but the fact remains : the message IS changed and the boundary lost. What does Eset say about the lost data ?

Frontier · Post by **Frontier** » Thu Feb 01, 2007 4:54 pm

Thanks gnozal for the quick reply and explanation.

I will forward the rfc passage to Eset, so we'll know more tomorrow (perhaps).

netmaestro · Post by **netmaestro** » Fri Feb 02, 2007 7:44 pm

For a program that works flawlessly under XP, PureSMTP_SendMail is giving an Invalid Memory Access.

gnozal · Post by **gnozal** » Sat Feb 03, 2007 8:34 am

netmaestro wrote:For a program that works flawlessly under XP, PureSMTP_SendMail is giving an Invalid Memory Access.

Any crashing code example ?

netmaestro · Post by **netmaestro** » Sat Feb 03, 2007 12:31 pm

Yes, the sms program Heathen wrote. It works under XP, but when it's run in Vista the IMA comes at the PureSMTP_SendMail line. Could be a privilege-to-send-mail thing, not sure.

gnozal · Post by **gnozal** » Mon Feb 05, 2007 8:32 am

netmaestro wrote:Yes, the sms program Heathen wrote. It works under XP, but when it's run in Vista the IMA comes at the PureSMTP_SendMail line. Could be a privilege-to-send-mail thing, not sure.

Oh, I didn't get that the problem was with Vista ...

DoubleDutch · Post by **DoubleDutch** » Mon Feb 05, 2007 12:11 pm

For a program that works flawlessly under XP, PureSMTP_SendMail is giving an Invalid Memory Access

I thought he was being sarcastic and just plain cheeky...

Didn't know he was referring to vista. Maybe you should refund him your fee?

netmaestro · Post by **netmaestro** » Mon Feb 05, 2007 1:56 pm

Sorry, I don't know how I missed to say Vista in my post

DoubleDutch · Post by **DoubleDutch** » Mon Feb 12, 2007 12:15 am

Confimed.

The Invalid memory access problem is with the command "PureSMTP_SendMail" and Windows Vista.

gnozal · Post by **gnozal** » Mon Feb 12, 2007 9:56 am

DoubleDutch wrote:Confimed.

The Invalid memory access problem is with the command "PureSMTP_SendMail" and Windows Vista.

Could you please test this app http://freenet-homepage.de/gnozal/PureS ... A_TEST.zip and post the error message ?

DoubleDutch · Post by **DoubleDutch** » Mon Feb 12, 2007 11:28 am

Message sent!

The program that I'm testing that crashes uses the threaded mode.

NetMaestro: Is yours in threaded mode too?

gnozal · Post by **gnozal** » Mon Feb 12, 2007 11:42 am

DoubleDutch wrote:The program that I'm testing that crashes uses the threaded mode.

If threadsafe mode is the problem, then maybe it's a PB bug because the same PB code is used for the standard and the threadsafe lib, only the compiler switch is different.
Could you please test this app http://freenet-homepage.de/gnozal/PureS ... EST_TS.zip (same but compiled with threadsafe flag) ?

DoubleDutch · Post by **DoubleDutch** » Mon Feb 12, 2007 11:56 am

Sorry - nothing to do with threadsafe, just a coincidence.

It's to do with authentication.

I tested your example programs (without threadsafe):

PureSMTP_Test.pb -> works fine

PureSMTP_TEST_AUTH.pb -> crashes at PureSMTP_SendMail