Thursday, April 30, 2015

The PTR DNS record, Email, SSL and You

What is the relationship of PTR DNS record, email, and SSL ?
This post will answer about that. Meanwhile, the background for this post are :
  • I am unable to make my new SMTP server instance deliver mail to a certain company's mailboxes. The company uses Trend micro's reputation list to block unwanted sender.
  • I have problems connecting to SMTP server using SSL with Android, and no problem using my Laptop and iPhone. I connect to the server using a load balancer IP address.

The analysis

First, to determine why the emails are not being delivered, I used the 'Delivery Status' display that available in the Cisco Ironport's Reports menu. It is quite useful when showing the reason why the email not being delivered.


Using the url shown in the 550 rejection message, I found out that the IP address being used for the email server is listed as having 'Bad' reputation. 


What is the meaning of DUL ? Quoting from Trend Micro website :
Dynamic User List (DUL) includes IP addresses in dynamic ranges identified by ISPs. Most legitimate mail sources have static IP addresses.
What that suppose to mean? It seems that the IP given to the smtp server is identified as dynamic IP range given from an ISP. I submitted a request to remove the IP from DUL. The nice person (or is it a bot? I don't know) from Trend Micro suggested to create PTR record that matches the A record given for the IP. That is, for each A record that states mysmtp.server.com is 1.2.3.4, we must create PTR record that states 4.3.2.1.in-addr.arpa is mysmtp.server.com. Of course it is not that simple if you are in a large organization, finding out which dns server to put the PTR and who is in charge for it is quite a challenge. Hint : its in your ISP who owns the IP range, and not the DNS server that has the authority of the server.com domain. After creating the reverse PTR record, replying to Trend Micro and asking them to recheck the IP, waiting a few hours, then there is no problem delivering email to the said company's servers.

For the second case, the symptom of the connection problem to said SMTP server is quite strange. There are no problems using Laptop, either using local WiFi and using internet Wifi from a GSM operator. As for Android tablet, we could connect using SSL to the load-balanced virtual server if we are using our internal WiFi network, and also WiFi from a GSM provider. But switching to 3G / HSDPA caused connections from the Android tablet to fail ('Cannot safely connect to server'). At first I worry about misconfigurations in the load balancer side, which could be tricky. But the same configuration works using local Wifi, if the LB is misconfigured then local Wifi shouldn't able to connect to the SMTP server either.

Enlightment comes after comparing packet traces using 3G with packet traces using Wifi. I used tPacketCapture for Android. That and a calm mind is needed for me to be able to do the next analysis.
Unsuccessful SSL connection
The client (10.8.0.1) established a three way handshake which are completed in frame 48 with the 'Ack' from the client. Compare the packet capture above to the healthy packet capture (which is very useful technique IMHO ) below :
Successful SSL connection
After the three way handshake (which finished in packet 31), the client immediately initiates TLS layer 1 handshake. Which is different from the unsuccessful capture above, where the client idles for a long time, and the server closes connection in frame 54 (with F/finish flag). Why does the client idles for that long ? The TLS Hello doesn't happen until about 28 second later in frame 73. What did the client do in the mean time ?
I set the packet filter to '(FrameNumber > 48) && (FrameNumber < 73)' to check (I am using Microsoft Network Monitor to view the captured packets).


We found that there is a DNS Query in frame #51,#53, #58, which all unanswered.
So the client does a reverse DNS lookup, which is looking for PTR record, that corresponds to the server's IP address. When it timed out, the client closes the connection. So the issue is that PTR records doesnt exist yet for load balancer's public IP and the GSM/3G provider's dns server is awfully quiet about that, causing time outs and connections not being made. We will try again after registering a PTR record and an A record for the load-balanced SMTP server.

The conclusion

If you are going to enlist A record to map a domain name to IP address, please be sure that you create a matching PTR record too in the right DNS server.
Android's SSL client does a DNS reverse lookup before we can establish SSL server.

No comments: