Document Foundation Mail Archive › Moderators

Filtering "UTF-8"

Classic

List

Threaded

5 messages Options

James Wilde

Filtering "UTF-8"

Hi:

Don't know if others think it's risky, or whether you will laugh yourselves silly that I haven't done it a long time ago, but I have just tested a mail filter for moderation messages where either the From field or the Subject field (in the message body, of course) begin with "=utf-8?. I started by moving them to a special folder and marking them as read. The filter works fine, so now I just delete them.

Personally I don't remember any such messages which weren't spam, mostly with a russian address, and I can't see that a serious message to a mailing list will have fancy formatting in these two fields. If you think I'm being dangerously harsh let me know.

//James

--
Unsubscribe instructions: E-mail to [hidden email]
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.documentfoundation.org/www/moderators/
All messages sent to this list will be publicly archived and cannot be deleted

Florian Effenberger

Re: Filtering "UTF-8"

Hi,

James Wilde wrote on 2012-03-12 10:09:
> Personally I don't remember any such messages which weren't spam, mostly with a russian address, and I can't see that a serious message to a mailing list will have fancy formatting in these two fields. If you think I'm being dangerously harsh let me know.

I'd be cautious with that. I know that e.g. Google Mail can be set with
one click to use UTF-8, and there are other mailers natively using
UTF-8, so valid messages might indeed bear that charset. :)

Florian

--
Florian Effenberger <[hidden email]>
Chairman of the Board at The Document Foundation
Tel: +49 8341 99660880 | Mobile: +49 151 14424108
Skype: floeff | Twitter/Identi.ca: @floeff

--
Unsubscribe instructions: E-mail to [hidden email]
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.documentfoundation.org/www/moderators/
All messages sent to this list will be publicly archived and cannot be deleted

James Wilde

Re: Filtering "UTF-8"

On Mar 12, 2012, at 10:45 , Florian Effenberger wrote:

> Hi,
>
> James Wilde wrote on 2012-03-12 10:09:
>> Personally I don't remember any such messages which weren't spam, mostly with a russian address, and I can't see that a serious message to a mailing list will have fancy formatting in these two fields. If you think I'm being dangerously harsh let me know.
>
> I'd be cautious with that. I know that e.g. Google Mail can be set with one click to use UTF-8, and there are other mailers natively using UTF-8, so valid messages might indeed bear that charset. :)
>
I agree. My mailer does, but the code for UTF-8 in the From and Subject fields appears to be some kind of html code. Otherwise the UTF-8 coding appears somewhere in the header, I believe. And I've never seen the string I quoted anywhere other than in these two fields.

Regards

//J

--
Unsubscribe instructions: E-mail to [hidden email]
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.documentfoundation.org/www/moderators/
All messages sent to this list will be publicly archived and cannot be deleted

Christian Lohmaier-3

Re: Filtering "UTF-8"

Hi James, *,

On Mon, Mar 12, 2012 at 11:50 AM, James Wilde
<[hidden email]> wrote:

> On Mar 12, 2012, at 10:45 , Florian Effenberger wrote:
> [...]
> I agree. My mailer does, but the code for UTF-8 in the From and Subject fields appears to be some kind of html code.

Whenever there are non-ascii characters in the subject/any other
header field, it is mime-encoded.

http://tools.ietf.org/html/rfc2047#section-8 (examples)
#################
MIME (Multipurpose Internet Mail Extensions) Part Three:
Message Header Extensions for Non-ASCII Text
[...]
The following are examples of message headers containing 'encoded-
word's:

From: =?US-ASCII?Q?Keith_Moore?= <[hidden email]>
To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[hidden email]>
CC: =?ISO-8859-1?Q?Andr=E9?= Pirard <[hidden email]>
Subject: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=
=?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=
#################
all valid, although the From one is unnecessary. both quoted printable
as well as base64 variants are shown, examples with different charsets
are shown (although it is not common to use different encodings, and
even less common to mix them in one single header field)

I never saw html in from and/or subject lines ever, not in spam, not
in other posts.

> Otherwise the UTF-8 coding appears somewhere in the header, I believe. And I've never seen the string I quoted anywhere other than in these two fields.

You did not quote any string. you did say it starts with "=utf-8?"
(and probably missed a ? here), but didn't quote the string. A charset
definition is not spam nor html.

And while it is true that most spam has this trait (as most spam that
passes the filters is from china or russia and hence need to encode
the text), the opposite conclusion (everything that has encoded
characters is spam) is wrong.

ciao
Christian

--
Unsubscribe instructions: E-mail to [hidden email]
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.documentfoundation.org/www/moderators/
All messages sent to this list will be publicly archived and cannot be deleted

James Wilde

Re: Filtering "UTF-8"

OK I'll move these to a separate folder instead of deleting them.

//James

On Mar 12, 2012, at 14:14 , Christian Lohmaier wrote:

> Hi James, *,
>
> On Mon, Mar 12, 2012 at 11:50 AM, James Wilde
> <[hidden email]> wrote:
>
>> On Mar 12, 2012, at 10:45 , Florian Effenberger wrote:
>> [...]
>> I agree. My mailer does, but the code for UTF-8 in the From and Subject fields appears to be some kind of html code.
>
> Whenever there are non-ascii characters in the subject/any other
> header field, it is mime-encoded.
>
> http://tools.ietf.org/html/rfc2047#section-8 (examples)
> #################
> MIME (Multipurpose Internet Mail Extensions) Part Three:
> Message Header Extensions for Non-ASCII Text
> [...]
> The following are examples of message headers containing 'encoded-
> word's:
>
> From: =?US-ASCII?Q?Keith_Moore?= <[hidden email]>
> To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[hidden email]>
> CC: =?ISO-8859-1?Q?Andr=E9?= Pirard <[hidden email]>
> Subject: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=
> =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=
> #################
> all valid, although the From one is unnecessary. both quoted printable
> as well as base64 variants are shown, examples with different charsets
> are shown (although it is not common to use different encodings, and
> even less common to mix them in one single header field)
>
> I never saw html in from and/or subject lines ever, not in spam, not
> in other posts.
>
>> Otherwise the UTF-8 coding appears somewhere in the header, I believe. And I've never seen the string I quoted anywhere other than in these two fields.
>
> You did not quote any string. you did say it starts with "=utf-8?"
> (and probably missed a ? here), but didn't quote the string. A charset
> definition is not spam nor html.
>
> And while it is true that most spam has this trait (as most spam that
> passes the filters is from china or russia and hence need to encode
> the text), the opposite conclusion (everything that has encoded
> characters is spam) is wrong.
>
> ciao
> Christian
>
> --
> Unsubscribe instructions: E-mail to [hidden email]
> Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
> Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
> List archive: http://listarchives.documentfoundation.org/www/moderators/
> All messages sent to this list will be publicly archived and cannot be deleted
>

--
Unsubscribe instructions: E-mail to [hidden email]
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.documentfoundation.org/www/moderators/
All messages sent to this list will be publicly archived and cannot be deleted