Opened 11 months ago

#5496 new defect

DMARC pushes many mailing list messages to spam because of DKIM fail

Reported by: David Earl Owned by: Tom Hughes
Priority: minor Milestone:
Component: admin Version:
Keywords: Cc:

Description

More and more OSM mailing list messages are going straight to Gmail (and presumably other) spam, because mailman isn't handling DKIM properly when it forwards the message. In bt.com originated messages, they (and I guess others) have DMARC policies to force such failures to spam. Use of more strict DMARC policies is getting more common, and all the big providers now act on them, so this is only going to get worse.

I see this very old report from 2012 https://trac.openstreetmap.org/ticket/4480 but it was closed as "won't fix". It has now come back to bite this time.

Header shown below:


Delivered-To: David Earl Received: by 2002:a17:906:3603:0:0:0:0 with SMTP id q3-v6csp324599ejb;

Fri, 3 Aug 2018 16:47:19 -0700 (PDT)

X-Google-Smtp-Source: AA+uWPwyoeVgdH5aSUL1XXj3SAoFqpMSCyQpnfDDusqOYfkMdlvyPIfwQZVhU9Wruy5+dNgcokUm X-Received: by 2002:a5d:470e:: with SMTP id y14-v6mr815616wrq.229.1533340039744;

Fri, 03 Aug 2018 16:47:19 -0700 (PDT)

ARC-Seal: i=1; a=rsa-sha256; t=1533340039; cv=none;

d=google.com; s=arc-20160816; b=cXF1PN6LpvaBupa7nlcg8GwGKhpOAgGoLjL8RpTCGMZVf+0YpdygOCMTSq5tlROreb

QlPe4l1YwdAqOQ8e/a9RGuYS+O6U6cQlqWIj07TwlgpvHKts1QDR5YSBCSbmFeJCq52e w8kOqR7OpP8oA1CCBXsaEfGfDYqx08jc2Yf5Vnu5lXurqqB1+slG9xThE5e6UzxZi0fu 0BM6/U1KvMSvdStTjPjdjeWgxesRp1yI/HCd2AocuEY1LR4Qu6inSVJ1P8O17yiYcT3o rzp22zF94lIc7NcmKdZGHJTXEiiJnihuHXDclpG6sjfNt3hKhfjZg6q2RAWZK/0+ta3S xIPQ==

ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;

h=errors-to:list-subscribe:list-help:list-post:list-archive

:list-unsubscribe:list-id:precedence:subject:content-language :mime-version:user-agent:date:message-id:to:from:dkim-signature :arc-authentication-results;

bh=EUR9+3v28XG+meLgSjN3hVlrqbUURFfx0wb7uYzp23g=; b=jCDmRg+QOaQmTv8wcHcxApB/HY7B1EfeBoY6Icx8mlQ9uVjFrJ3V6mzuRO4IS5CQjK

VU4KUFYxbdLIxIBfwFz4PhOZWm5JoQnyEFZj9ylXjicPSFtZGeesU5QNIE/ZxmRNAhEw UDIxmIJsoVT12TV7Sd+sXpEbLTd4XRtJEgC7v43u4ik1jYTqWUXKO1FqMI+Tq2wMwSTC PjyGjF2TZdp84qXmbGPxYn3Bnj0RS7/VZ8QVHh8kSXBYaktLk/RnssmxH1epGE0TFAFT 7oMsdOBAMScSD5x+GwZ/CBDVIZB+CEDExq0CxRlH3WCG/0SuhXf2shLrMDmysu1e0i23 H+sA==

ARC-Authentication-Results: i=1; mx.google.com;

dkim=neutral (body hash did not verify) header.i=@btinternet.com header.s=btcpcloud header.b=JwJ7r5+S; spf=pass (google.com: domain of talk-gb-bounces@… designates 2001:41c9:1:400::32 as permitted sender) smtp.mailfrom=talk-gb-bounces@…; dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=btinternet.com

Return-Path: <talk-gb-bounces@…> Received: from shenron.openstreetmap.org (shenron.openstreetmap.org. [2001:41c9:1:400::32])

by mx.google.com with ESMTPS id j16-v6si443848wmj.207.2018.08.03.16.47.19 for <David Earl> (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 03 Aug 2018 16:47:19 -0700 (PDT)

Received-SPF: pass (google.com: domain of talk-gb-bounces@… designates 2001:41c9:1:400::32 as permitted sender) client-ip=2001:41c9:1:400::32; Authentication-Results: mx.google.com;

dkim=neutral (body hash did not verify) header.i=@btinternet.com header.s=btcpcloud header.b=JwJ7r5+S; spf=pass (google.com: domain of talk-gb-bounces@… designates 2001:41c9:1:400::32 as permitted sender) smtp.mailfrom=talk-gb-bounces@…; dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=btinternet.com

Received: from localhost ([::1]:51267 helo=shenron.openstreetmap.org) by shenron.openstreetmap.org with esmtp (Exim 4.82) (envelope-from <talk-gb-bounces@…>) id 1fljmd-0002dW-Hy; Fri, 03 Aug 2018 23:47:00 +0000 Received: from rgout01.bt.lon5.cpcloud.co.uk ([65.20.0.178]:61359) by shenron.openstreetmap.org with esmtp (Exim 4.82) (envelope-from <davefoxfac63@…>) id 1fljmV-0002cu-9Z for talk-GB@…; Fri, 03 Aug 2018 23:46:53 +0000 X-OWM-Source-IP: 109.147.134.83 (GB) X-OWM-Env-Sender: davefoxfac63@… X-RazorGate?-Vade-Classification: clean X-RazorGate?-Vade-Verdict: clean 0 X-VadeSecure?-score: verdict=clean score=0/300, class=clean X-SNCR-VADESECURE: CLEAN X-RazorGate?-Vade-Verdict: clean 0 X-RazorGate?-Vade-Classification: clean X-RazorGate?-Vade: gggruggvucftvghtrhhoucdtuddrgedtiedrleeigdduledvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuueftkffvkffujffvgffngfevqffopdfqfgfvnecuuegrihhlohhuthemuceftddtnecunecujfgurhephffuvffkffgfgggtsegrtderredtfeejnecuhfhrohhmpeffrghvvgcuhfcuoegurghvvghfohigfhgrtgeifeessghtihhnthgvrhhnvghtrdgtohhmqeenucffohhmrghinhepohhpvghnshhtrhgvvghtmhgrphdrohhrghdpohhvvghrphgrshhsqdhtuhhrsghordgvuhenucfkphepuddtledrudegjedrudefgedrkeefnecurfgrrhgrmhephhgvlhhopegludelvddrudeikedruddrvdduiegnpdhinhgvthepuddtledrudegjedrudefgedrkeefpdhmrghilhhfrhhomhepoegurghvvghfohigfhgrtgeifeessghtihhnthgvrhhnvghtrdgtohhmqecuuefqffgjpeekuefkvffokffogfdprhgtphhtthhopeeothgrlhhkqdfiueesohhpvghnshhtrhgvvghtmhgrphdrohhrgheqnecuvehluhhsthgvrhfuihiivgeptd Received: from [192.168.1.216] (109.147.134.83) by rgout01.bt.lon5.cpcloud.co.uk (9.0.019.26-1) (authenticated as davefoxfac63@…) id 5B321EA003860481 for talk-GB@…; Sat, 4 Aug 2018 00:46:46 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=btinternet.com; s=btcpcloud; t=1533340011;

bh=98S2MIdNVzVPDed9suyuhkzmq5JpwgnSApR1XVxZ0TM=; h=From:Subject:To:Message-ID:Date:MIME-Version; b=JwJ7r5+SeluntEiL01ljB8f1k/D0MZbUa3DHo3J5YN7uaJPuSEVMD1VGK44/qH0WEHdLI0N3QvrEyDv1V5+dHdfCScoG6wcDponWErXrWUd5TIQlFNGJEXkbIgC0qz6DlHakNxHQdtsFhBPGAf1/fsmp6mipG7mMS23FSSfP6Ls=

From: Dave F <davefoxfac63@…> To: "Talk-GB@…" <talk-GB@…> Message-ID: <8b293789-9a2e-42d2-258d-8dc0f9bce40b@…> Date: Sat, 4 Aug 2018 00:47:20 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Language: en-US X-Antivirus: Avast (VPS 180803-8, 03/08/2018), Outbound message X-Antivirus-Status: Clean Subject: [Talk-GB] 'C' class roads references. X-BeenThere?: talk-gb@… X-Mailman-Version: 2.1.16 Precedence: list List-Id: General discussion for users in Great Britain <talk-gb.openstreetmap.org> List-Unsubscribe: <https://lists.openstreetmap.org/options/talk-gb>, <mailto:talk-gb-request@…> List-Archive: <http://lists.openstreetmap.org/pipermail/talk-gb/> List-Post: <mailto:talk-gb@…> List-Help: <mailto:talk-gb-request@…> List-Subscribe: <https://lists.openstreetmap.org/listinfo/talk-gb>, <mailto:talk-gb-request@…> Content-Type: multipart/mixed; boundary="===============9213134699068287844==" Errors-To: talk-gb-bounces@…

--===============9213134699068287844== Content-Type: multipart/alternative; boundary="------------DA413704A325BA95B6B9A7EF" Content-Language: en-US


Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit

Hi

After many discussions over the years about the referencing of 'C' class roads there appeared to be a general consensus to keep them in the database but provide a unique tag to allow them not to be rendered.

This is a list of the discussions (there maybe others): https://lists.openstreetmap.org/pipermail/talk-gb/2011-May/011632.html https://lists.openstreetmap.org/pipermail/talk-gb/2013-March/014555.html https://lists.openstreetmap.org/pipermail/talk-gb/2013-April/014788.html https://lists.openstreetmap.org/pipermail/talk-gb/2014-August/016392.html https://lists.openstreetmap.org/pipermail/talk-gb/2015-May/017390.html https://lists.openstreetmap.org/pipermail/talk-gb/2015-May/017414.html

However this task was never undertaken. I decided to grab the bull by the horns.

I used variations of this Overpass query within JOSM to find the numerous 'C' refs keys (listed below) tagged to the different road classification. I uploaded in batches split by geography &/or tag values to make it easier for me to verify I used detailed changeset descriptions to make it easier to rectify if needed. If you spot any errors please let me know.

[maxsize:2073741824]; area(id:3600058447,3600058437,3600058446); England, Wales, Scotland [bbox:{{bbox}}];

way[highway=*highway classification"][~ref~"C[0-9]{1,4}$"] (area);

out tags; out center; (._;>;); out meta;

*Various keys used for 'C' refs: (listed most popular down)

  • ref

official_ref admin_ref admin:ref wcc_ref highway_ref designation offical_ref int_ref unsigned_ref reference local_ref

*Highway classes with 'C' refs:* (listed most popular down) tertiary (+_link) unclassified trunk (+_link) residential (error?) service (error?) pedestrian (error? Roads converted to pedestrian, but still classified?) track (error/prow_ref?) secondary (+_link) primary

I've amended them to *'highway_authority_ref*'. It was discussed in the May '15 thread where it was felt official_ref or admin_ref wasn't specific enough. Feel free to discuss here if you have strong objections to it. If there's a consensus to change it's quite easy now they're all under a single tag.

Note I didn't include Northern Ireland as I'm unsure whether they're signed on the ground or not. Is anyone able to verify?

These are the trunk, primary & secondary roads which previously either had a ref or highway_authority_ref: http://overpass-turbo.eu/s/AM7

Similarly these are the pedestrian, service, residential & track http://overpass-turbo.eu/s/AM8

These still have 'ref' tags: proposed, abandoned, construction, path, footway & cycleway; possibly copy paste errors or should be prow_ref?: http://overpass-turbo.eu/s/AMc

Please amend if you have local knowledge & believe any of the above are an error.

Cheers DaveF


Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

<html>

<head>

<meta http-equiv="content-type" content="text/html; charset=utf-8">

</head> <body text="#000000" bgcolor="#FFFFFF">

Hi<br> <br> After many discussions over the years about the referencing of 'C' class roads there appeared to be a general consensus to keep them in the database but provide a unique tag to allow them not to be rendered.<br> <br> This is a list of the discussions (there maybe others):<br> <a class="moz-txt-link-freetext"

href="https://lists.openstreetmap.org/pipermail/talk-gb/2011-May/011632.html">https://lists.openstreetmap.org/pipermail/talk-gb/2011-May/011632.html</a><br>

<a class="moz-txt-link-freetext"

href="https://lists.openstreetmap.org/pipermail/talk-gb/2013-March/014555.html">https://lists.openstreetmap.org/pipermail/talk-gb/2013-March/014555.html</a><br>

<a class="moz-txt-link-freetext"

href="https://lists.openstreetmap.org/pipermail/talk-gb/2013-April/014788.html">https://lists.openstreetmap.org/pipermail/talk-gb/2013-April/014788.html</a>

<br> <a class="moz-txt-link-freetext"

href="https://lists.openstreetmap.org/pipermail/talk-gb/2014-August/016392.html">https://lists.openstreetmap.org/pipermail/talk-gb/2014-August/016392.html</a><br>

<a class="moz-txt-link-freetext"

href="https://lists.openstreetmap.org/pipermail/talk-gb/2015-May/017390.html">https://lists.openstreetmap.org/pipermail/talk-gb/2015-May/017390.html</a><br>

<a class="moz-txt-link-freetext"

href="https://lists.openstreetmap.org/pipermail/talk-gb/2015-May/017414.html">https://lists.openstreetmap.org/pipermail/talk-gb/2015-May/017414.html</a><br>

<br> However this task was never undertaken. I decided to grab the bull by the horns. <br> <br> I used variations of this Overpass query within JOSM to find the numerous 'C' refs keys (listed below) tagged to the different road classification. <br> I uploaded in batches split by geography &amp;/or tag values to make it easier for me to verify<br> I used detailed changeset descriptions to make it easier to rectify if needed. If you spot any errors please let me know. <br> <br> [maxsize:2073741824];<br> area(id:3600058447,3600058437,3600058446); England, Wales, Scotland<br> [bbox:{{bbox}}]; <br>

way[highway=*highway classification"][~ref~"C[0-9]{1,4}$"]

(area);<br> out tags;<br> out center;<br> (._;&gt;;); out meta;<br> <br> <b>Various keys used for 'C' refs:<br>

(listed most popular down)<br>

</b> ref<br> official_ref<br> admin_ref<br> admin:ref<br> wcc_ref<br> highway_ref<br> designation<br> offical_ref<br> int_ref<br> unsigned_ref<br> reference<br> local_ref<br> <br> <b>Highway classes with 'C' refs:</b><br> <b><b>(listed most popular down)</b></b><br> tertiary (+_link)<br> unclassified<br> trunk (+_link)<br> residential (error?)<br> service (error?)<br> pedestrian (error? Roads converted to pedestrian, but still classified?) <br> track (error/prow_ref?)<br> secondary (+_link)<br> primary <br> <br> I've amended them to <b>'highway_authority_ref</b>'. It was discussed in the May '15 thread where it was felt official_ref or admin_ref wasn't specific enough. Feel free to discuss here if you have strong objections to it. If there's a consensus to change it's quite easy now they're all under a single tag. <br> <br> Note I didn't include Northern Ireland as I'm unsure whether they're signed on the ground or not. Is anyone able to verify? <br> <br> These are the trunk, primary &amp; secondary roads which previously either had a ref or highway_authority_ref:<br> <a class="moz-txt-link-freetext"

href="http://overpass-turbo.eu/s/ALX">http://overpass-turbo.eu/s/AM7</a><br>

<br> Similarly these are the pedestrian, service, residential &amp; track<br> <a class="moz-txt-link-freetext"

href="http://overpass-turbo.eu/s/AM1">http://overpass-turbo.eu/s/AM8</a><br>

<br> These still have 'ref' tags: proposed, abandoned, construction, path, footway &amp; cycleway; possibly copy paste errors or should be prow_ref?:<br> <a class="moz-txt-link-freetext"

href="http://overpass-turbo.eu/s/AMc">http://overpass-turbo.eu/s/AMc</a><br>

<br> Please amend if you have local knowledge &amp; believe any of the above are an error.<br> <br> Cheers<br> DaveF

</body>

</html>


--===============9213134699068287844== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline

X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVGFsay1HQiBt YWlsaW5nIGxpc3QKVGFsay1HQkBvcGVuc3RyZWV0bWFwLm9yZwpodHRwczovL2xpc3RzLm9wZW5z dHJlZXRtYXAub3JnL2xpc3RpbmZvL3RhbGstZ2IK --===============9213134699068287844==--

Change History (0)

Note: See TracTickets for help on using tickets.