[FIXED] How do I setup SPF to Pass for Mailchimp, SES and EC2 so that DMARC doesn't partially failing due to misalignment

Issue

One of my websites sends emails from various sources including:

• Mailchimp
• Mandrill
• Mailgun
• Amazon SES
• From the server itself

We have correctly configured SPF & DKIM settings for all of the sources so we decided to add DMARC too and at the same time we changed ?all to -all.

All of our emails have been going through just fine but we've noticed our Mailchimp emails are "SPF Incapable" and as such don't fully pass.

Most email clients / accounts show that DMARC has passed, however DMARCian & a few email clients throw up DMARC Fail-alignment

For example:

DMARC.io does however say the following:

SPF: SPF is not possible as MailChimp uses their own domain in the bounce address. Their domain authentication verification tool requires including Mailchimp, though. To work-around this without having to include Mailchimp servers, try including only 'ip4:205.201.128.0/20 ip4:198.2.128.0/18 ip4:148.105.8.0/21' in your SPF record.

So, I added those to my SPF however it still shows:

All of the following IP's are all covered by those CIDR notations so I am unable to work out why the SPF is still misaligned for SPF, but it fixes DKIM...

• 198.2.185.161
• 198.2.185.245
• 198.2.141.4
• 198.2.175.233
• 198.2.185.245
• 198.2.142.122

I did find the following on stackoverflow: https://stackoverflow.com/a/50471866/2487602 however that seems to be slightly wrong in advice because the bounce server is nearly always different. They've got hundreds of servers.

My SPF looks like:

v=spf1 +a +mx +ip4:94.237.30.75 +ip4:94.237.30.85 +ip4:94.237.30.86 +ip4:94.237.30.87 +ip4:54.77.177.67 +ip4:34.246.233.211 +ip4:52.18.62.128 +ip4:34.241.119.225 +ip4:205.201.128.0/20 +ip4:198.2.128.0/18 +ip4:148.105.8.0/21 include:servers.mcsv.net include:spf.mandrillapp.com include:mailgun.org include:amazonses.com -all

What have I done wrong/can I improve in order to get the SPF aligned?

On a side note, I assume for the first 4 IP's I could just use: +ip4:94.237.30.75/28 which would include those 4 IP's as well as a 14 or so other IP's which isn't really that risky is it especially when they won't have DKIM and as a result DMARC passing? To make it a little more secure I could just reduce those 4 records down to +ip4:94.237.30.75 +ip4:94.237.30.85/30 which would only include 1 IP that we don't use.

I do also want to send emails via EC2, rather than SES for some emails (potentially to replace Mailgun/SES) however when we send emails from those servers they get sent to spam for not having a reverse DNS.

I assume, the fix for that would be to request a PTR record to be setup by Amazon for the 3x EC2 instances that we use?

I did request one of these before, but I wasn't sure if it is supposed to be pointing to my main website's IP or the IP of the server that it's sending the emails from...

As for SES, even after adding the SPF include and the DKIM record, our SPF record shows "Neutral" rather than "Pass" for example the following is from the "Test Email" feature for SES/EC2

• SPF: NEUTRAL with IP 54.240.7.46

So to recap...

1. How do we make Mailchimp fully pass on SPF so that DMARC is aligned
2. How do we correctly setup a PTR record for a reverse dns, should the subdomain be pointing to the server it sent the email from or the server of the main domain?
3. How do I get Amazon SES to throw a SPF Pass, rather than SPF Neutral?

Solution

1. You can't. The answer in the thread you mentioned is correct. Mailchimp will set the bounce address to its own domain. So adding the ranges to your own SPF record will not change anything (SPF is checked on the bounce address, not the header.from address). While you're not authenticating fully redundantly, you're still passing DMARC and it should not impact reputation for your domain (or nobody would be using MailChimp).
2. Only the owner of the IP address can set the PTR record in DNS. It should point to the name of the server sending the emails, preferably the name in the EHLO/HELO. This thread has the correct answer, I think, though I've never set it up personally.
3. What is the bounce address domain used in your AmazonSES emails? Did you set up a Custom MAIL FROM address as explained here? A Neutral result is usually returned for an SPF record ending in ?all and not passing on anything that is on the left of it. Another thing that might be happening (if the SPF Domain is your own domain) is that you're hitting the "characters-per-string limit" for DNS TXT records. If you haven't cut the record up in at least 2 strings, the check might not get executed beyond the Mailchimp include. And with the absence of an all modifier (not evaluated), it defaults to neutral. You can see what your SPF record looks like with any lookup tool, but querying your domain with https://dns.google.com for TXT records should show you (escaped) double quote (") marks for where the string has been cut. As an example you can check this SPF record to see what it looks like: […]ip4:185.211.120.0\"\"/22 ip4:185.250.236.0/22[…]

One last thing to note is that you're close to the DNS lookup limit. It looks like you're at 8 lookups already. Beyond 10 lookups will cause the SPF check to result in a PERMERROR

Answered By - Reinto

Answer Checked By - Willingham (PHPFixing Volunteer)

Read More

[FIXED] How can I fix the problem with Gmail? Gmail marked email with passed SPF, DKIM and DMARC as spam

Issue

Gmail moves messages to spam folder. PTR records for ipv4 and ipv6 are available. I have the headers: SPF: PASS с IP-адресом 2a05:480:0:992f:0:0:0:2. DKIM: 'PASS', domain drom.com.ua DMARC: 'PASS'

Delivered-To: vostoknefteproduct@gmail.com
Received: by 2002:a9d:522:0:0:0:0:0 with SMTP id 31csp351612otw;
        Fri, 5 Apr 2019 05:26:18 -0700 (PDT)
X-Google-Smtp-Source: APXvYqzWa1UxYEepV1tor3akYHi1DLXojOZnr61fkANdKlbOkDM880aGrAB9RzQdjipusdfEcAVt
X-Received: by 2002:a2e:87d2:: with SMTP id v18mr7262855ljj.4.1554467178382;
        Fri, 05 Apr 2019 05:26:18 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1554467178; cv=none;
        d=google.com; s=arc-20160816;
        b=TTDI9m3Rt73dRkcImzRRAM5n98TJgwNxCnrbvKNVMAIOdrWk/iG+NU2OrR3/vDr7Gp
         fQmPmqt5QyEdZpO1G+He8uYckwETdCQFhVACDVcC5FJ8GWnWsys5p9vbFYzlYh89OZR+
         gFfJkt+lb5z607n03Sr9zMW9HiJVBGKLr+SFCsG8u/AOmZCF9wZtGwaP69E+7b916njn
         WZ+Pz9Q/+DyDKaCOQWrdUlTS5GvrfK/c86/5lA0AjoZN90/Rj+uGlbIROqKUXoLbA/z3
         G9LM9df3a6/kfAZBkzHB3PqCaiUhEKbSrcpLj1gT1fKHZKWOnkqb9hlcYwMX2OcHbpA8
         zTtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=content-language:thread-index:mime-version:message-id:date:subject
         :to:from:dkim-signature;
        bh=qI0pd1OCLQLqqFvztD8zxwZ0XJBtraToDzc2t4i88s0=;
        b=uLJIPEtdroC8P6tqoyfOikNhNzEmtmNRGpgoOefyyN/0wdAqExsSoUxZ9UKLL31lST
         AT7J8/ZOEMuGa/wn1dSUHfwRqiZl7d/rxM+AtT2a1PaSSLulOgz9HQDGw1r81QXKnZMI
         XzsGpuPmz9e3apEtczYBM0E3zYjm5Z2ITmpnrk9MluAnEuqE370R/24ixS1W2+EEG4mY
         FXZ6AsImGfZGFr75zc9kLTPgBxa1hxghD0mu6RIACf45b5ljQZQuspj8kRyhw5NX30SA
         FRwg1TwRjm3N+kmaUiDv4M64Y7s1d/PU7Gr4rb33v8Ia/aEOpeFfKLs2NfKmiiuvqBh7
         94vg==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@drom.com.ua header.s=dkim header.b=G8nYIwzB;
       spf=pass (google.com: domain of shop@drom.com.ua designates 2a05:480:0:992f::2 as permitted sender) smtp.mailfrom=shop@drom.com.ua;
       dmarc=pass (p=NONE sp=REJECT dis=NONE) header.from=drom.com.ua
Return-Path: <shop@drom.com.ua>
Received: from drom.com.ua (drom.com.ua. [2a05:480:0:992f::2])
        by mx.google.com with ESMTPS id z24si17168655ljj.124.2019.04.05.05.26.18
        for <vostoknefteproduct@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Fri, 05 Apr 2019 05:26:18 -0700 (PDT)
Received-SPF: pass (google.com: domain of shop@drom.com.ua designates 2a05:480:0:992f::2 as permitted sender) client-ip=2a05:480:0:992f::2;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@drom.com.ua header.s=dkim header.b=G8nYIwzB;
       spf=pass (google.com: domain of shop@drom.com.ua designates 2a05:480:0:992f::2 as permitted sender) smtp.mailfrom=shop@drom.com.ua;
       dmarc=pass (p=NONE sp=REJECT dis=NONE) header.from=drom.com.ua
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=drom.com.ua ; s=dkim; h=Content-Type:MIME-Version:Message-ID:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Transfer-Encoding:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=qI0pd1OCLQLqqFvztD8zxwZ0XJBtraToDzc2t4i88s0=; b=G8nYIwzBB+jpn8qK83ZCo7vu+m AM/RFrxq/PP8dITX1PNXqINAjHPSu2rBVsdE5QTMPDoaHYPbOeC9wFHFMfYspdy/JyQAh4A8hmSNz HgtnwRBVR14Nht8DAqeql7CCrOom0KNS0SQ95dTnOy42/5bFommziwUghHGgzyzjUav4=;
Received: from [176.98.31.190] (helo=DESKTOPV2831BB) by drom.com.ua with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from <shop@drom.com.ua>) id 1hCNvF-0000Rn-J9 for vostoknefteproduct@gmail.com; Fri, 05 Apr 2019 15:26:17 +0300
From: "DROM - служба обработки заказов" <shop@drom.com.ua>
To: <vostoknefteproduct@gmail.com>
Subject: Test title
Date: Fri, 5 Apr 2019 15:26:17 +0300
Message-ID: <005901d4ebaa$c4b3ea80$4e1bbf80$@drom.com.ua>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_005A_01D4EBC3.EA0197B0"
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AdTrqr/vXGG/i/4HRdu0LENUGEwBuw==
Content-Language: ru
X-Scanned-By: ClamAV 0.101.1; Fri, 05 Apr 2019 15:26:17 +0300

Where can be the problem? Thanks

Solution

Unfortunately this seems to be business as usual with gmail. Their spam filter seems entirely arbitrary and uncontrollable by recipients - for example adding addresses to your address book, marking messages as "not spam", or repeatedly moving messages from spam to your inbox does not help, and nor does following their guidelines, which include implementing measures such as SPF, DKIM and DMARC, just as you've done. On my own gmail account, I often find messages sent to myself end up in spam!

Their postmaster tools are also broken - this is supposed to provide a feedback mechanism that allows server admins to see why IPs or domains are being blocked or spam filtered, however, it doesn't work properly, and it's very common to see domains & IPs marked as "bad" despite having zero spam reports. There is also no support available for postmaster tools, so you can't even report such problems. It might be worth trying anyway just see what they think of you.

In short, you are entirely at google's mercy - even if you're doing everything that they ask, your messages may still get spam filtered, and you have no recourse.

Answered By - Synchro

Answer Checked By - Dawn Plyler (PHPFixing Volunteer)

Read More

[FIXED] How to track emails which are not DKIM aligned

Issue

I've set up DKIM on my domain and configured Postmark to send me a DMARC Daily Digest to monitor that everything is set up correctly. Each week, I have around half of my emails sent through Google marked as not DKIM aligned:

Any idea to how to track down the problem. I'm pretty confident, that I've correctly configured the email correctly in G Suite, which the 100% indicators on the two other IPs also show.

Solution

Many times, seeing DKIM (and / or SPF) misalignment in DMARC aggregate reports comes down to forwarding. In this particular case, you are likely seeing recipients hosted on Google having Distribution Groups forwarding to internal addresses. Google adds these DMARC authentication results to the DMARC aggregate reports it sends out, even though they are internally routed messages.

I've written an answer on this matter before here.

Answered By - Reinto

Answer Checked By - Mildred Charles (PHPFixing Admin)

Read More

[FIXED] How can you verify that an email was sent from the proper domain and that the contents of the email haven't been modified?

Issue

I'm examining my email headers (sent from Gmail) and am wondering which ones are best to verify that an email was sent from the proper domain and that the contents of the email haven't been modified?

Solution

I'll use an example message I received from Stack Overflow to illustrate what you can do to verify a message manually.

SPF isn't a great deal of use after delivery as it's a relatively minor (but important) step of verifying the origin of messages, but it's good to know that it was good at the time, and the headers tell you that. The parts of message content related to delivery are the Return-path header added by the receiver, along with the last Received header that shows where it was received from:

Return-Path: <bounces+3553988-07ba-marcus=example.com@em.stackoverflow.email>
Received: from o1.em.stackoverflow.email (o1.em.stackoverflow.email [167.89.81.234])
    (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
    (No client certificate requested)

Looking up this em.stackoverflow.email domain as a TXT record in DNS gives us:

# dig txt +short em.stackoverflow.email
u3553988.wl239.sendgrid.net.
"v=spf1 ip4:167.89.81.234 ip4:167.89.85.72 ip4:168.245.32.199 -all"

and we can see that the 167.89.81.234 IP that appears in the Received header is explicitly listed in the record, so SPF checks out. Other domains might have more complex needs to verify the SPF, e.g. requiring additional DNS lookups for include mechanisms. Significantly, SPF does not even look at the address used in the From header; it can be entirely unrelated to the return path domain.

DKIM is where the real action is. You can use the information in a DKIM-signature header to verify that a message has not been tampered with. For example:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; 
    d=stackoverflow.email; 
    h=from:subject:to:reply-to:mime-version:content-type; s=s1; 
    bh=IgKJJZNcUhfjH6LDr4XaWr3pBwq6wwxwyrGmf+k3DVo=; b=rR4J7VyvF3i0N
    20IX9bx0LGTKpSKj7XoHJurhBjcZLLTn/hXuZ8OMehfgMNFeXaMljlOz4tfoFwit
    aJ8UtK1oMVCPiv9200hpQViCh/5VsyYbs6k3YN6R3cFxMbrb7nflodXX+4Rp4xBu
    T+CloNFEDICtWJT4bVSrs/NRAUlJWY=

Look at the From header and check that it matches the domain shown in the d field of the DKIM signature, which it does:

From: Stack Exchange <do-not-reply@stackoverflow.email>

The DKIM signature has 2 signature parts: b is a signature for the message body, and bh is a signature for the message headers. The interaction of these is quite tricky. To generate a signature, first of all you calculate the body signature, then you calculate the header signature which includes the body signature, along with the canonicalised message headers listed in the h element except for the actual value of the bh element, because that would present a chicken & egg problem. The bh signature only includes the headers listed because mail servers may add other headers that should not be included, or that are not known at the time of sending, such as Return-path and Received. Stack overflow's signature is unusual in that it does not include the Date header, so the date on a message can be altered without breaking the DKIM signature.

To verify a signature you need the public key it was signed with, which you can get from DNS using the s field, which provides the selector, in this case s1:

# dig txt +short s1._domainkey.stackoverflow.email
s1.domainkey.u3553988.wl239.sendgrid.net.
"k=rsa; t=s; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCyJRzkL/aRo1F1+ChY+Crt2TARqqo7tGATw3fMfzW8MXFWaoW1rSvZsq4k1EIf2iW7gO/QZjU1Td7h1aZpS63/CmpKymmqNbHnnbTxZGvZziKPcL/R2PVL0g88MFcpAuSjIsGysYTeow0mnXQ5W03z5mtWqm5nxNM40A/TIlOegwIDAQAB"

One issue with this signature is that it only uses a 1024-bit key; gmail ignores DKIM signatures with less than 2048 bits, so I would not be surprised if SO has deliverability issues with gmail.

Actually verifying a DKIM signature is a complicated process, especially if it uses the relaxed canonicalisation algorithms (which make the signatures more likely to survive the email journey). I wrote a PHP DKIM validator that you could use to verify your own messages.

The final layer is DMARC which ties SPF, DKIM, and the From address header together. Looking back at the Return-path, we can see that the return path domain is a subdomain of the domain used in the From header. This means that it qualifies for "relaxed" rather than "strict" alignment. We can see SO's DMARC record:

# dig txt +short _dmarc.stackoverflow.email
"v=DMARC1;p=reject;sp=reject;pct=100;rua=mailto:dmarc-aggregates@stackoverflow.com;ruf=mailto:dmarc-forensics@stackoverflow.com;fo=1"

This tells us that SO wants receivers to enforce SPF and DKIM checks strictly, and to reject any messages that do not check out, and they have set up addresses to receive summary and forensic reports of any messages that fail.

There's lots more detail but this isn't really the place for it. There's a good article on it all here.

Answered By - Synchro

Answer Checked By - David Goodson (PHPFixing Volunteer)

Read More

[FIXED] How can I modify the return-path in a google workspace account that has the capability to send as another workspace account?

Issue

I haven't been able to find an answer to this, so forgive me if it's been asked somewhere before.

I'm working with non-profit who has a google workspace for non profits account. I'm working on getting our SPF/DKIM/DMARC records set up, and they work perfectly except for one situation.

We have two different domains, whedoncon.com, and thehellmouth.org. Some of our users have an email on both domains (i.e. user@whedoncon.com and user@thehellmouth.org are going to the same person). I can send emails individually from each domain, and they pass SPF, DKIM, and DMARC fine. The problem comes in when I set up the domains to be able to send from each other.

I've added the capability for user@whedoncon.com to be able to send mail as user@thehellmouth.org. The issue seems to be when I log in as user@whedoncon.com, and send a message as user@thehellmouth.org. Looking at the email headers, it seems that because I logged in as user@whedoncon.com, it sets the return-path to the whedoncon.com address regardless of what account I select to send out the email.

The problem with this, is it causes DMARC to fail whenever I send an email out as user@thehellmouth.org, even though SPF and DKIM both pass. It seems to be because the return-path is showing as user@whedoncon.com, but the DKIM is looking at hellmouth.org.

So, TL:DR, google seems to always default to the signed-in account for the return-path, and not the secondary account that it's actually sending from. Is there a way I can change the return-path so it matches the account the email is coming from, and not the account that I'm signed in as?

Solution

Google Workspace has a primary domain and users are assigned a primary address under that domain. When you have a domain alias, users are assigned an alias address under that alias domain.

The envelope sender address (also known as the return-path address) and the From: address for a message can be different or the same.

If users send email from their alias address, the return-path address will be their primary address, while the From: address will be their alias address.

To pass DMARC, a message must pass at least one of these checks:

• SPF authentication and SPF alignment
• DKIM authentication and DKIM alignment

SPF typically uses the message envelope sender address for authentication. DKIM uses the message From: address for authentication.

When the domain alias is setup correctly, both the SPF and DKIM authentication will pass. However, only DKIM alignment will pass, SPF alignment will not pass. But that is okay because DMARC does not require SPF alignment to pass as long ask DKIM authentication and DKIM alignment pass.

Answered By - bitPimps

Answer Checked By - Marilyn (PHPFixing Volunteer)

Read More

[FIXED] WHAT is "IN" in DNS records?

Issue

What is the proper meaning of the "IN" word in DNS records? DNS records are an important part of DNS zones. So I want to know that there is an "IN" word included in every record. what is the meaning?

It would help clarify what such a piece of information in DNS is?

Thank you for making all the efforts.

Solution

A simple INternet search would have brought you to Wikipedia, where the following is stated on DNS Record Classes:

The CLASS of a record is set to IN (for Internet) for common DNS records involving Internet hostnames, servers, or IP addresses. In addition, the classes Chaos (CH) and Hesiod (HS) exist.[34] Each class is an independent name space with potentially different delegations of DNS zones.

Find the link here.

Answered By - Reinto

Answer Checked By - Marie Seifert (PHPFixing Admin)

Read More