Comment 1 Andre Klapper 2008-12-22 19:22:00 UTC

Now that bug 3700 has been fixed, this is a valid issue
(before that, using "-- " instead of "--" would have stripped completely the
entire message content in some mail applications when quoting).

This violates RFC 3676 Section 4.3 about usenet message signatures.

Comment 2 Andre Klapper 2009-05-26 12:15:40 UTC

Brave and willing to test this again with Modest from trunk in Scratchbox?
https://git.maemo.org/projects/modest/gitweb?p=modest

Now that extra space is not trimmed from the signature separator

Fixed in
Modest master df8ae49c96a166a2287cf7488335acb847303a55

Comment 4 Andre Klapper 2009-06-12 19:07:51 UTC

Verified that it works correctly now. And closing.

Comment 5 Andre Klapper 2009-07-14 13:38:23 UTC

Fix should be included in Fremantle SDK beta 2 hence updating Target Milestone.
If you are the reporter of this bug: Feel free to verify the fix if possible.

Hm, not quite there yet.  The sig separator modest inserts ("-- ") may look OK
to humans, but the last character is actually a UTF-8 no-break space (0xC2A0,
should be 0x20) so other MUAs still won't recognise it.

(In reply to comment #6)
> Hm, not quite there yet.  The sig separator modest inserts ("-- ") may look OK
> to humans, but the last character is actually a UTF-8 no-break space (0xC2A0,
> should be 0x20) so other MUAs still won't recognise it.

Well the nbr is inserted for HTML parts so every HTML renderer could display it
with no problem.

Am I missing anything?

Comment 8 Andre Klapper 2009-07-28 14:30:44 UTC

I'd say it's not about rendering but about parsing and clients that should
correctly strip "-- " and the following lines when answering such an email.

(In reply to comment #7)
> Am I missing anything?

text/plain messages :-)  When composing such, do the signature & separator
really have to be converted to HTML and back?  (Signature 'body' spaces are
also converted to no-break ones).

BTW if the n-b space is changed to a space manually while in the compose window
the resulting message that arrives at the other end is fine.

(In reply to comment #9)
> (In reply to comment #7)
> > Am I missing anything?
> 
> text/plain messages :-)  When composing such, do the signature & separator
> really have to be converted to HTML and back?  (Signature 'body' spaces are
> also converted to no-break ones).
> 
> BTW if the n-b space is changed to a space manually while in the compose window
> the resulting message that arrives at the other end is fine.

For text plain messages we insert a blank space. They're not converted to HTML
and back. If they're composed as plain text messages, then they won't be
converted to anything else.

(In reply to comment #10)
> If they're composed as plain text messages, then they won't be
> converted to anything else.

I promise you they are (at least the signature part)!

Steps to reproduce:
1. Build latest checkouts of modest & tinymail (using 3.0.17-rc27 & r3958 as of
a few minutes ago).
2. Start modest.
3. Tap "mcen_ti_new_message".
4. Tap menubar -> "mcen_me_message_settings" and make sure it's set to "plain
text".
5. Address the message to yourself, enter a subject and some body text
(preferably with some spaces for comparison), leaving the signature & separator
untouched.
6. Send.
7. Receive the message on another MUA and inspect it.

After doing the above, the received message's MIME headers are:

> Content-Type: text/plain; charset=utf-8
> Mime-Version: 1.0
> Content-Transfer-Encoding: 8bit

while a hex dump of the body ends in:

000002f0  72 65 20 74 65 73 74 0a  2d 2d c2 a0 0a 53 65 6e  |re test.--...Sen|
00000300  74 c2 a0 66 72 6f 6d c2  a0 4e 6f 6b 69 61 c2 a0  |t..from..Nokia..|
00000310  49 6e 74 65 72 6e 65 74  c2 a0 54 61 62 6c 65 74  |Internet..Tablet|
00000320  0a                                                |.|

Note the difference between offsets 0x2f2 and 0x2fa (and every other "space"
after that).

(In reply to comment #11)

I should add:

> 2. Start modest.

2a. Make sure a signature is set for the account used in the following steps.

> 7. Receive the message on another MUA and inspect it.

Or look at ~/.modest/local_folders/sent/cur/<latest message>, just to make sure
the message wasn't mangled in transit.

Comment 13 Andre Klapper 2009-08-04 12:22:29 UTC

Confirming with a hexeditor in ~/.modest/local_folders/sent/cur/<latest
message> : It's a c2 a0 while it should be just 20.

Already fixed in commit 63951a450b16e144c215ecbac88c37c80dd823a0

Comment 15 Andrew Flegg (reporter) 2009-09-30 19:50:50 UTC

Does 1.2009.39-9 include this fix, as no space (hard or soft) seems to be
present in that image.

Comment 16 Andre Klapper 2009-09-30 22:17:07 UTC

As this was committed on Sep 03 2009 it should be included, yes.

Fix can be seen here:
https://git.maemo.org/projects/modest/gitweb?p=modest;a=blobdiff;f=src/modest-text-utils.c;h=8161ae525541d813301f9a4f68198387bb4cbf85;hp=5b7c0bdda4987c18889e1e02416c75c9c4bcc023;hb=63951a450b16e144c215ecbac88c37c80dd823a0;hpb=115933dbb53fd67d529d4537bd9e3b85237d847c

Comment 17 Andre Klapper 2009-09-30 22:30:45 UTC

Taking a look at a plain text message in my 1.2009.40-4 Modest outbox I see:

-  -     S  e  n  t     f  r  o  m
2D 2D 0A 53 65 6E 74 20 66 72 6F 6D

The 0A still should be a 20 so it is NOT fixed.

(In reply to comment #17)
> Taking a look at a plain text message in my 1.2009.40-4 Modest outbox I see:

Confirmed also with current git, which definitely includes
63951a450b16e144c215ecbac88c37c80dd823a0.

> -  -     S  e  n  t     f  r  o  m
> 2D 2D 0A 53 65 6E 74 20 66 72 6F 6D
> 
> The 0A still should be a 20 so it is NOT fixed.

The 0A (LF) is fine, but there should be a 20 before it.  Looks like we're back
to the state in comment 0.

Created an attachment (id=1391) [details]
Convert space characters to &space; HTML entities

After looking at the wpeditor source (since it seems the root problem is that
wpeditor eats the trailing space) it seemed that &space; should do the trick,
and indeed it does.

(From update of attachment 1391 [details])
This has the unfortunate side-effect of displaying all spaces in incoming
messages as "&space;"

Created an attachment (id=1392) [details]
Convert spaces to " " instead

Although "&space;" is recognised by wpeditor, it isn't an official character
entity (<http://www.w3.org/TR/xhtml1/dtds.html#a_dtd_Latin-1_characters>) and
apparently gtkhtml doesn't know what to do with it.  "&#32;" works better :-)

Marking patches of interest to Diablo (Maemo4) community updates, please excuse
the noise.

Comment 23 Andrew Flegg (reporter) 2009-11-20 13:55:15 UTC

Still an issue in the sales release.

Patch looks good. Applied in 

modest-3-2 f334560f1bcf72748987e6a35f17ad14b31e80e6
master 62bc0ae5e6f609572d67921734ae10c7beebf6fd

Thank you very much Lucas for your effort and patch !!!

*** Bug 6749 has been marked as a duplicate of this bug. ***

Comment 26 Andre Klapper 2009-12-14 14:16:54 UTC

Taking a look at a plain text message in my 2009.50-7 Modest outbox I see:

-  -        S  e  n  t     f 
2D 2D 20 0A 53 65 6E 74 20 66

both in the plain part and in the HTML part.

Comment 27 Andrew Flegg (reporter) 2009-12-29 09:49:23 UTC

This is NOT fixed in 2.2009.51-1.

STEPS TO REPRODUCE:
1) Create a mail account containing a signature of 'Hello world'. Exclude the
sig sep from the signature.
2) Go into offline mode
3) Create a message and send it, putting it in the outbox.
4) Open X Terminal
5) Run the following command:
    fgrep -- '-- ' .modest/outboxes/*/cur/*

EXPECTED OUTCOME
One or more results

ACTUAL OUTCOME
No matches

REPRODUCIBILITY
always

Comment 28 Andre Klapper 2009-12-29 11:40:09 UTC

This has been fixed in package
modest 3.2.3-1+0m5
which is part of the internal build version
2009.50-7
(Note: 2009 is the year, and the number after is the week.)
This is not included in PR1.1 as it is another branch.

A future public update released with the year/week later than this internal
build version will include the fix. (This is not always already the next public
update.)
Please verify that this new version fixes the bug by marking this bug report as
VERIFIED after the public update has been released and if you have some time.


To answer popular followup questions:
 * Nokia does not announce release dates of public updates in advance.
 * There is currently no access to these internal, non-public build versions.
   A Brainstorm proposal to change this exists at
http://maemo.org/community/brainstorm/view/undelayed_bugfix_releases_for_nokia_open_source_packages-002/

Unfortunately this is broken (i.e. the signature separator is "--" instead of
the expected "-- " (note the trailing space) when composing an email in modest)
in 2.2009.51-1.

Tested with both "Formatted text" and "Plain text".

Comment 30 Andre Klapper 2010-01-14 14:26:40 UTC

Yes, it's not yet fixed in 2.2009.51-1, that's why the Version field still
mentions that version. :)

Created an attachment (id=2288) [details]
Patch for Diablo

There is one case where this still occurs:  when messages are sent in HTML
format (really multipart/alternative with both text/plain and text/html parts,
which is exactly right), both MIME parts have the trailing space (or nbsp in
the current Fremantle version) stripped.

For the HTML part it's a lost cause anyway, but the text/plain part should have
a correct sig separator so replies etc can do the right thing.

Comment 33 Andre Klapper 2010-02-16 20:23:12 UTC

Lucas: The fix was released in 10.2009.50-7 which is not the same branch as
2.2009.51-1/3.2010.02-8, hence I assume that you compiled from source?

(In reply to comment #33)
> Lucas: The fix was released in 10.2009.50-7 which is not the same branch as
> 2.2009.51-1/3.2010.02-8, hence I assume that you compiled from source?

Yes, for Diablo using the attached patch (context: community SSU).  I haven't
built Fremantle packages yet, but the vanilla 2.2009.51-1 modest does strip the
no-break space when composing in HTML format so I assume it would strip ASCII
spaces too.

Comment 35 Andre Klapper 2010-06-03 14:51:24 UTC

Fixed in git:
http://gitorious.org/modest/modest/commit/5302aa739add086e735f1eefb532ad3686d97f51

(Not closing as FIXED yet before it's integrated into internal builds.)

Comment 36 Andre Klapper 2010-06-10 18:32:58 UTC

This has been fixed in package
modest 3.4.0+0m5
which is part of the internal build version
2010.23-4
(Note: 2009/2010 is the year, and the number after is the week.)

A future public update released with the year/week later than this internal
build version will include the fix. (This is not always already the next public
update.)
Please verify that this new version fixes the bug by marking this bug report as
VERIFIED after the public update has been released and if you have some time.


To answer popular followup questions:
 * Nokia does not announce release dates of public updates in advance.
 * There is currently no access to these internal, non-public build versions.
   A Brainstorm proposal to change this exists at
http://maemo.org/community/brainstorm/view/undelayed_bugfix_releases_for_nokia_open_source_packages-002/

Comment 37 Andre Klapper 2010-10-25 17:12:23 UTC

The problem reported here should be fixed in the update that was released today
for public: The Maemo5 update version 20.2010.36-2 (also called "PR1.3"
sometimes). Please leave a comment if the problem is not fixed for you in this
update version.