Thursday, January 4, 2007

Getting Postfix to work on Ubuntu with Gmail

Here's what I want to do. I have an Ubuntu box (Edgy-Eft) at home, and I want to be able to send out email, and I want to use gmail as my relayhost. There are several sites online that explain bits of how to do this, and Mike Chirico's is particularly good. I used his tutorial as a starting point, but I noticed I had to do a few things differently to get it working on my own system, so I'm documenting the differences.

Differences

Disclaimer: Different about my setup is that I am using the Ubuntu packages, whereas Chirico's tutorial has you compile the packages yourself. There's nothing wrong with doing that, in fact, it's probably good for your soul, but I'd prefer to make use of the Ubuntu package manager as much as possible. Further, I'm not interested in using fetchmail, so I've done nothing with that.

Installing Postfix

The first thing I did was install postfix.

# apt-get install postfix
I told the configuration script that I was installing for an internet site. Happily, debian/ubuntu's postfix comes with TLS and SASL compiled in.

Generate Your Certificates

In order to connect to gmail, you need a certificate. Here's what happened when I generated my certificate.
# /usr/lib/ssl/misc/CA.pl -newca
CA certificate filename (or enter to create)

Making CA certificate ...
Generating a 1024 bit RSA private key
.....................++++++
.........................++++++
writing new private key to './demoCA/private/cakey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Illinois
Locality Name (eg, city) []:Chicago
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Prancing Tarantula
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:Mattox Beckman
Email Address []:mattoxbeckman@gmail.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
Using configuration from /usr/lib/ssl/openssl.cnf
Enter pass phrase for ./demoCA/private/cakey.pem:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number:
            d5:0c:4b:bb:48:17:c3:b0
        Validity
            Not Before: Jan  4 22:42:34 2007 GMT
            Not After : Jan  3 22:42:34 2010 GMT
        Subject:
            countryName               = US
            stateOrProvinceName       = Illinois
            organizationName          = Prancing Tarantula
            commonName                = Mattox Beckman
            emailAddress              = mattoxbeckman@gmail.com
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                33:0A:41:44:07:7D:0F:4C:10:B8:8C:4A:89:8C:CC:0E:18:EF:CA:92
            X509v3 Authority Key Identifier: 
                keyid:33:0A:41:44:07:7D:0F:4C:10:B8:8C:4A:89:8C:CC:0E:18:EF:CA:92
                DirName:/C=US/ST=Illinois/O=Prancing Tarantula/CN=Mattox
Beckman/emailAddress=mattoxbeckman@gmail.com
                serial:D5:0C:4B:BB:48:17:C3:B0

            X509v3 Basic Constraints: 
                CA:TRUE
Certificate is to be certified until Jan  3 22:42:34 2010 GMT (1095 days)

Write out database with 1 new entries
Data Base Updated
Now generate a private key...
# openssl req -new -nodes -subj '/CN=prancingtarantula.net/O=Prancing Tarantula/C=US/ST=Illinois/L=Chicago/emailAddress=mattoxbeckman@gmail.com' -keyout FOO-key.pem -out FOO-req.pem -days 3650
Generating a 1024 bit RSA private key
.........................................++++++
....++++++
writing new private key to 'FOO-key.pem'
-----
And sign it...
# openssl ca -out FOO-cert.pem -infiles FOO-req.pem
Using configuration from
/usr/lib/ssl/openssl.cnf
Enter pass phrase for ./demoCA/private/cakey.pem:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number:
            d5:0c:4b:bb:48:17:c3:b1
        Validity
            Not Before: Jan  4 22:48:47 2007 GMT
            Not After : Jan  4 22:48:47 2008 GMT
        Subject:
            countryName               = US
            stateOrProvinceName       = Illinois
            organizationName          = Prancing Tarantula
            commonName                = prancingtarantula.net
            emailAddress              = mattoxbeckman@gmail.com
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Comment: 
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier: 
                16:B2:33:D3:E7:E9:4D:2B:76:71:5D:D7:EC:AF:47:22:FA:38:AB:54
            X509v3 Authority Key Identifier: 
                keyid:33:0A:41:44:07:7D:0F:4C:10:B8:8C:4A:89:8C:CC:0E:18:EF:CA:92

Certificate is to be certified until Jan  4 22:48:47 2008 GMT (365 days)
Sign the certificate? [y/n]:y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
Now I copied them to the /etc/postfix directory.
# cp demoCA/cacert.pem FOO-key.pem FOO-cert.pem /etc/postfix
# chmod 644 /etc/postfix/FOO-cert.pem /etc/postfix/cacert.pem
# chmod 400 /etc/postfix/FOO-key.pem
One difference from the tutorial: when running postfix, you may get warnings like this one:
Jan  4 17:21:59 calvin postfix/smtp[28881]: setting up TLS connection to
smtp.gmail.com
Jan  4 17:21:59 calvin postfix/smtp[28881]: certificate verification failed
for smtp.gmail.com: num=20:unable to get local issuer certificate
Jan  4 17:21:59 calvin postfix/smtp[28881]: SSL_connect error to
smtp.gmail.com: -1
I've copied them in so people searching for them will find this. These warnings are because postfix doesn't know where to find the Thawte certificate that gmail used to sign its own certificate. Ubuntu includes it in its ssl package. You need to append it to the cacert.pem file you generated earlier.
cat /etc/ssl/certs/Thawte_Premium_Server_CA.pem >> cacert.pem 

Transport

To cause the mail to be routed, you need a transport file. Here's mine:
# Contents of /etc/postfix/transport
#
# This sends mail to Gmail
*               smtp:[smtp.gmail.com]:587
Different from the tutorial is the specification of port 587. If you leave that off, postfix will attempt to connect to port 25, which is blocked by many ISPs now. If you get a timeout error in your log file, that's what's happening. The Gmail help pages say you should be able to use port 465 also, but that times out for me as well. You'll have to add another line if you expect to receive mail at your machine.

SASL

You now need to set the SASL passwords. My file looks like this one:
# Contents of sasl_passwd
#
[smtp.gmail.com]:587             mattoxbeckman@gmail.com:password
Of course, replace password and the email address with something appropriate for your system. Again, note the 587... if you leave that off, you will get very confusing log messages like this one:
Jan  4 18:20:30 calvin postfix/smtp[31770]: 49D438A6F: 
to=, orig_to=, 
relay=smtp.gmail.com[64.233.163.109]:587, delay=7661, 
delays=7660/0.1/0.19/0.03, dsn=5.5.1, status=bounced 
(host smtp.gmail.com[64.233.163.109] said: 
530 5.5.1 Authentication Required 16sm56842404nzo 
(in reply to MAIL FROM command))
This will be very frustrating because you will see the passwords are there, but they just aren't being used. Be sure to hash the files:
# postmap sasl_passwd
# postmap transport

Wrapup

The lines in tls_per_site, main.cf and master.cf are like the tutorial. Just paste them into your own versions, and you should be good to go.

33 comments:

Antonios said...

Great! I followed the original tutorial in freshmeat but I also stumbled upon the issues you describe, and I've been struggling for some hours without result.
Thanks for taking the time to clarify those issues.

Ryan said...

One might also want to deliver localmail directly rather than funneling it through the outgoing smtp server, the transport manpage (http://www.postfix.org/transport.5.html) has what one needs to do that in the Example section. Quickly in transport do

localhost :
localhost.localdomain :
* smtp:[smtp.gmail.com]:587

This should let you email user@localhost without sending that out to gmail stmp server.

bar said...

This really helped me with the generation of certs, but I am in the awful situation of not being an ubuntu user and therefore lacking the gmail certificate that you mentioned as being included in that software, so I can't tack it onto the end of my cacert.pem. Is there some alternative?

Thanks,
Barbara Shirtcliff

Brandon said...

Awesome! I've been following the tutorial on Sourceforge for quite a while and your tips cleared up all of my problems.

Thank you so much for helping those of us in need!

Wouter and Andrea said...

I don't get it to work. The comment
openssl req -new -nodes -subj '/CN=prancingtarantula.net/O=Prancing Tarantula/C=US/
seems not to be complete.
It generates a privkey.pem file and then the following command
openssl ca -out FOO-cert.pem -infiles FOO-req.pem
does not work, even when I use -infiles privkey.pem

Any suggestion?

Wouter and Andrea said...

Sorry, my browser did not render the page correctly. I checked the source of the page and found the complete command line.
Everything works fine now, thanks a lot!

Daniel said...

Excellent tutorial.
Worked great on Ubuntu 6.06 except for the Thawte certificate; so here's how to get it.
Download the Thawte root certificates from http://www.thawte.com/roots/; uncompress the file and look for a file called ThawtePremiumServerCA_b64.txt.
You can then "cat ThawtePremiumServerCA_b64.txt >>cacert.pem " ant it should work.

cioa

snooze said...

Thanks, it's just what I needed.

scott said...

great tutorial!!

Question. Lets say if you have multiple users with virtual domains how does one setup postfix and gmail to handle such a
situation

Jared Evans said...

Really appreciated this tip. This is more up to date. You solved all my problems!

butlimous said...

Thanks for the nice tutorial! I did it successfully...keep it up

Free PS3

William said...

Thanks for the helpful info. One point about using an alternate port like 587 is that you also need to edit the relayhost in the main.cf file to include the port.

For anyone trying to use this technique to connect to GoDaddy, use the following relayhost:

[smtpout.secureserver.net]:3535

Everything else is the same.


Cheers!
William McKee
Knowmad Technologies
http://www.knowmad.com

val.hoffman said...

Hi & thanks for the good tutorial. Just wondering if someone had this bug: root@mydesktop:~# openssl ca -out FOO-cert.pem -infiles FOO-req.pem
Using configuration from /usr/lib/ssl/openssl.cnf
Error opening CA private key ./demoCA/private/cakey.pem
11920:error:02001002:system library:fopen:No such file or directory:bss_file.c:352:fopen('./demoCA/private/cakey.pem','r')
11920:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:354:
unable to load CA private key

val.hoffman said...

Ok, for anyone interested - I solved it. In Ubuntu, I opened synaptic, and made a search for openssl. Everything that it found, I reinstalled. Hope this helps.
Val.

Gustavo A. said...

Hi!

This was very useful, but wanted to ask: every time i send and email from my server account, it delivers like was from the gmail account, so if a user replies that, the response will arrives of course to my gmail account and not to my server account...

Can i avoid this? since i want to receive the response to my server account and not to Gmail...

Thanks!

Josh said...

I've noticed that a number of the tutorials on using gmail to relay mail for postfix require the setup of SSL certificates on the client side. These are not necessary for establishing a TLS communication for relaying mail through gmail.

The only bits in main.cf that need to be set for this to work are:

relayhost= [smtp.gmail.com]:587

smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous

Set the password in /etc/postfix/sasl_password, run postmap on the file, restart postfix, and that's it.

Rick said...

Right on, Josh. You saved me a boatload of headache trying to get those certificates to work.

I guess without them we can't verify that gmail is who they say they are, but I'm not really concerned about it for the kind of status emails I'll be sending out from ubuntu.

Nathan said...

For those like me that couldn't find the Thawte_Premium_Server_CA.pem file, you can get it by installing the `ca-certificates` package.

e12win said...

Newbie question :)
Not sure if this is intended or not, but now, all email that is sent out from my server says that the sender is the email from the one in the sasl_password. How do I set it so that the sender becomes the sender that I specify instead of the account in sasl_password?

Gunfus said...

Woo I got caught twice by the same mistake.. so please make sure you read follow carefully, aparently I don't :)

In reply to the comment by Daniel posted on "Tue Jul 24, 06:41:00 PM CDT" about ThawtePremiumServerCA_b64.txt

When you download the certifacete and unzip the file you will find several certificates, make sure you use the one in "Thawte Server Roots" Otherwise you might get weird problems like:

Untrusted TLS connection established to smtp.gmail.com[72.14.205.109]:587: TLSv1 with cipher RC4-MD5 (128/128 bits)
Server certificate not trusted

Scott Lindner said...

This may be obvious to many folks but it wasn't to me. The main.cf and master.cf files cannot have whitespace prior to the key/value pairs. I was getting lots of "fatal: garbage after numerical service in server description" errors until I figured this out. When I copied the files from the referenced articles a lot of whitespace was copied. Remove this to avoid the mentioned error.

Scott Lindner said...

Now that I have the whitespace issue resolved I have the same issue noted by e12win. All email is being sent by the authenticating account. How is this resolved?

Scott Lindner said...

I apologize for the continual posts. It appears that Google is rewriting the From based on the authentication account. The only way this could be resolved is if postfix could use different authentication information for each From email address. This could become very tricky.

Mattox Beckman said...

Yes, that's correct; the only way would be to have each user somehow map to their own gmail account. Postfix was created in a time when email was accepted because it came from a certain machine. But routing through gmail is a method for verifying an individual. A similar problem exists with the venerable mutt mailer now: "we are not a mail agent" was fine before, but doesn't work now since machines on a dynamic IP won't have their mail accepted. I don't know of any system mail agents set up for this, but there's no reason why it couldn't be done.

seanmwhitney said...

Seems like gmail updated their cert today. I wasn't really interested in which one of the Equifax certs they used

So I ran this command and email seems to be working again. I'll leave it to someone else to determine exactly which cert is needed.

cat /etc/ssl/certs/Equifax_Secure_* >> cacert.pem

Sean

treefellow said...

Indeed, the certificate is changed now. It was a pain in the ..., but (again) the solution is simple:
get the root1 certificate from geotrust-website (Equifax Secure Certificate Authority (Base-64 encoded X.509) ) and copy it somewhere. Then include this one in ur cacert.pem instead of the Thawte certificate. http://www.geotrust.com/resources/root_certificates/

Good luck!

Gunfus said...

I was wondering why my email had stopped working since end the Jul. Thanks for updating the certificate steps.

treefellow said...

I said before to use the Equifax certificate INSTEAD of the Thawte. For some reason that is wrong now! My email spontanious stopped using gmail. Once I added Thawte Premium certificate to my cacert-file it worked fine again :-S
I didn't check if the Equifax is still used...

Mihai Vasilache said...

Hi!

Great tutorial.
I am using this (postfix/gmail) to send emails when some issues are modified on bugzilla.
But i have a problem: sometimes my gmail account is locked:

postfix/error[6910]:...status=deferred (delivery temporarily suspended: SASL authentication failed; server smtp.gmail.com[209.85.135.111] said: 535-5.7.1 Username and Password not accepted. Learn more at ?535 5.7.1 http://mail.google.com/support/bin/answer.py?answer=14257 e10sm16942073muf.14)


And i have to unlock the account manually using the gmail's captcha.

I suppose that is because postfix is sending too many request in a short period of time.
Do you have any ideea how to fix this? (Maybe a delay in postfix between emails sent).

Thank you,
Mihai

Mihai Vasilache said...

i received an answer from postfix-users@postfix.org:


/etc/postfix/main.cf:
smtp_destination_rate_delay = 60
relay_destination_rate_delay = 60

References:

http://www.postfix.org/postconf.5.ht...ion_rate_delay
http://www.postfix.org/postconf.5.ht...ion_rate_delay

Wietse


and

And if you discover that you are still seeing the "locked account"
problem where you have to re-enter the captcha code then try changing
your gmail password to something that the gmail password page accepts as
at least a STRONG (not FAIR or GOOD) password.

We are seeing our customers who use google apps hosted email having to
re-enter the captcha code and they definitely don't send emails out too
fast (individual, non-tech users on low bandwidth connections). It
happens only to customers who were set up on the hosted address with a
weak/poor/fair password. When they set a better password it stops happening.

Gerald

mootaccount said...

@Mihai Vasilache

What's the minimum

smtp_destination_rate_delay
relay_destination_rate_delay

allowed by Gmail?

Kamil Dworakowski said...

Why does postfix need a certifacate. I had ssmtp to send mail through smtp.gmail.com without generating any stupid certifacte. I am so totally not going to bother with postfix.

stevenboettcher said...

Great! Helped me fix some probs!