Originally Web posted Monday, 15 April 2013.
Content last modified Saturday, 9 January 2021 .
External links last verified Sunday, 27 August 2017.

Secure Email Setup for Mail 6 & OS 10.8 Mountain Lion

Note: To save on website hosting costs, screenshot images from other OS X versions (from other articles in this series) are used in this article when the only differences between the OS versions are cosmetic and minor, or involve commands or options not relevant to the current discussion.

Mountain Lion Stays The Course

As tends to be the case with “modified” cat name OS versions which seem to be mostly bugfixes and minor improvements over their predecessor OS, Mountain Lion OS 10.8 looks and works like Lion 10.7, in terms of secure email. It too continues with the revised security interface originally introduced with OS 10.5 Leopard. It even shares the same bug still in Lion as of this writing. Be sure to follow these instructions carefully to work around the remaining bugs!

The following instructions have been formulated running OS 10.8.3, its Mail 6.3, Safari 6.0.3, Keychain Access 7.0, Certificate Assistant 5.0 and all current updates to these items as of 26 March 2013. They might possibly work with earlier versions of OS 10.8 and its components, but i never had those earlier versions to test with. These instructions will probably continue to apply to any later updates as well, though versions newer than those listed above have not been explicitly tested by me.

These instructions assume you already have at least one working email account set up and properly operating in Mail (in the usual insecure manner of standard Internet email). If you don’t, please go set it up and test it now, following Apple’s/other’s instructions.

Creating a Self-Signed S/MIME Certificate-key pair

  1. Quit Mail, if it is running.
  2. Open Keychain Access (from the Utilities folder). From its eponymous menu, select Certificate Assistant and Create a Certificate…:
  3. You’ll be presented with the Create Your Certificate window:
    This window allows you to name the certificate and lists the default parameters which will be used to create the certificate.
    Self Signed Root is exactly the type of certificate we wish to create. Given that S/MIME (Email) is most definitely the Certificate Type we want, one might think that it would be OK to merely verify the name and click Continue. But Apple chose inappropriate default settings, so default certificates can never work correctly! The biggest problem is that Certificate Assistant never asks for/confirms the email address, without which the certificate is useless for email! In many (but not all) cases, Certificate Assistant fails to insert any email address whatsoever into a default certificate, rendering it useless for email. Even in cases where Certificate Assistant finds one or more email addresses, when there is more than one, Certificate Assistant does not allow the user to select which one to use. Equally severe, Apple neglected to default select Key Encipherment for the Key Usage Extension, so even if the email address is present and correct in the default certificate, it cannot be used for encrypting outgoing messages… only for signing them. As of this writing, Apple has not fixed this problem, all the way through the most recent version tested, OS 10.9 Mavericks. Note that even though OS X versions 10.6 and later now contain an added layer of complexity to work around these mistakes, this added layer is not backward-compatible with older OS X versions! It is also unlikely to be compatible with non-Apple systems.

    Even if Apple hadn’t made the several errors which render a default certificate useless for secure email, the default lifetime for the certificate is set for one year. Most people will find renewing their certificate each year—and getting all their correspondents to renew it as well!—a huge hassle not worth undertaking. For all these reasons, be sure to check Let me override defaults.

  4. Click Continue, and you’ll see:
    Warning that a self-signed certificate needs to be explicitly computer-trusted by recipients.
    If, after reading Apple’s warning, any of you are not comfortable at this point and want to go to the trouble of paying a reliable Certificate Authority (CA) for a guaranteed certificate and following their instructions for installation, or wish to hunt around for any CAs who may be still offering free certificates for personal use (and jump through their hoops), by all means go ahead… see ya ’round! The rest of us will simply click Continue and move on.
  5. Because of the necessity of overriding the defaults, you’ll now be presented with a series of detailed settings windows. The first of the series of detail windows—Certificate Information—asks you to fill in basic information:
    Please specify: Serial Number and Validity Period (days)
    I have not found that the Serial Number matters, in my testing. If you think you may wind up having multiple certificates for the same user name and/or email address on one system, it is probably a good idea to ensure that the serial numbers differ (though i did not find any issues in brief testing where they were the same). I strongly recommend choosing a longer Validity Period, to minimize the hassle of re-creating a new certificate, distributing it to all your secure correspondents, and getting them to do the work to get the new one trusted on their system(s). The tradeoff with a longer time is that there is a much longer time span over which the valid certificate may be compromised by some malefactor getting ahold of your security credential set (certificate + public & private keys) and pretending to be you. I’m finding that 10 years is working as a decent compromise, so i usually fill in the box with 3652 (approximately 10 years, measured in days). Once these items are verified/set, go ahead and click Continue.
  6. The second Certificate Information window asks for “personal information […] to be used in the certificate”. As discussed in the Points common to all OS versions section, the only essential item is the email address. Entering a Common Name is highly recommended, as it will make it easier to find the certificate in keychains, on backups, etc. The other items are optional. They may be filled out frivolously, though if the certificate might ever be used for other purposes or on non-Apple systems, there might be cases of failure if certain fields do not match reality (or at least expectations). Note that all this information becomes part of the certificate you’ll be sharing with the outer world.

    click Continue when finished.
  7. You will now be asked to approve Key Pair Information. The defaults of 2048 bits key size and RSA algorithm are good (to the point that i did not bother testing any other options. The newer ECC option looks intriguing, but it would not be backwards-compatible with Leopard 10.5 nor Tiger 10.4 systems, and possibly not with non-Apple systems, therefore i cannot recommend it). These values are generally regarded as secure (as of the creation date of this article). 2048 bits is the largest (most secure) option for RSA on all versions of OS X from Tiger through Mavericks (and hopefully will remain compatible with newer OS X/Mail versions released in the future).
    Key Size: 2048 bits, Algorithm: RSA
    Click Continue.
  8. Next you’ll be offered the Key Usage Extension settings. Details for this extension are discussed on the separate page X.509 v3 Certificate Extensions. Here again, Apple made a mistake: the certificate cannot work for encrypting outgoing email messages unless Key Encipherment is selected!
    Use Key Usage Extension box is unchecked.
    I used to recommend disabling (unchecking) “Include Key Usage Extension”, however due to changes made by Apple as of 10.12 Sierra, the Key Usage Extension MUST be included for a certificate to be valid. If you wish for your certificate to work on Sierra or newer as well as all OS X/Mail versions back to 10.4 Tiger/Mail 2, you must enable Key Encipherment (and leave Signature enabled). Once you’ve made your selection(s), click Continue.
  9. Next you’ll be hit with the Extended Key Usage Extension settings:
    Include, Critical, and Email Protection are the three items checked.
    This one’s defaults are OK. It is OK to include it if you want, or disable it if you want—both ways work, in my testing. I tend to leave it engaged with settings as shown, in case future software starts to get picky about seeing specific usages indicated within certificates. As always, click Continue to move on.
  10. The Basic Constraints Extension gets us back into a controversial area where the experts are not in agreement. Some say that this extension should always be enabled and “Use this certificate as a certificate authority” (which will appear in this window when Basic Constraints is enabled) should be checked, for all self-signed certificates. Others disagree. In my testing, i left it at its default—disabled—and things worked fine.
    “Include Basic Constraints Extension” is UNchecked.
  11. The last of the array of extra-work-making extensions is the Subject Alternate Name Extension (which i’ll abbreviate: SANE):
    Default includes the extension, but the email address may be missing!
    As discussed in the SANE section of the X.509 v3 Certificate Extensions page, SANE is not required. I did a lot of testing with and without it, with no difference in results. If it is used, it is essential that the rfc822Name field include the same exact email address as listed above under Certificate Information. Normally, Certificate Assistant fills this in automatically and correctly, so all you’ll need to do is double-check it. If for some reason it shows up empty as in the screenshot above, carefully type it in, correctly: just the you@youresp.com part… nothing else. Once you’ve made your choices, click Continue.
  12. Finally, Certificate Assistant will ask you to Specify a Location For The Certificate:

    The default location of the login keychain is the correct answer. When you click Continue, Certificate Assistant will now (finally) create the usual three items in your login keychain: the certificate, your public key, and your private key. See New Security Credentials part of the Points common to all OS versions section of the main article for further information.

Trusting a Certificate

Your public and private keys need no further attention—they’re ready to run. All they need to do is sit there on your system and work.

In Keychain Access, looking at your login keychain, double-click your newly-made certificate. You can select the Certificates category in the lower left column to filter out items in the main window frame which are not certificates, to make it easier to find. Click the arrow next to the word Trust just below the certificate icon, and the trust options will be revealed. Here’s what they’ll look like at first:

“This root certificate is not trusted”, all pop-ups at defaults.
Only two of these need to be adjusted, for secure email: Set each of these to Always Trust:
“When using this certificate:” now shows “Use Custom Settings”.
Close the window. You’ll be asked to type in your current OS X user’s account credentials to allow Keychain Access to make the needed changes:
Once that’s done, you’ll see Keychain Access showing the certificate as trusted:
White + sign in blue circle icon with “This certificate is marked as trusted for this account”

Preserve Your New Security Credentials: Back Up!

Now is the time to back up your new certificate, public key, and private key, archive them, or both!

Please see Backing up your new security credentials on the common page for all the step-by-step details.

Other Machines?

As discussed in the Sharing One Set of Credentials with Other Machines part of the Points common to all OS versions section of the main article page, one set of email security credentials should be used on all devices you own.

The easiest and possibly best way to do this is to place a copy of the keychain you made with your 3 security credentials on each device and import the items from there. Here’s how it works moving these credentials to a different Mac:

  1. Place a copy of your secure email credentials keychain on the Mac. I tend to put it on the desktop.
  2. Double-click the credentials keychain file. Keychain Access should open, and the keychain should be visible in the list, shown locked.
  3. Select the credentials keychain. Select all 3 items (certificate, public key, private key) within the keychain and drag them to the login keychain.
  4. You’ll get a warning that Keychain Access is attempting to modify the system keychain. (I don’t know why this is showing up since we’re dragging into the login keychain and not the System keychain, but there you have it.) Authenticate as an administrator to continue.
  5. Keychain Access will now ask for the password/passphrase for the credentials keychain. Enter it.
  6. You’ll get another warning about modifying the system keychain. Type in the administrator credentials yet again.
  7. A sheet will drop down in Keychain Access asking whether you want to trust the imported certificate. Interestingly, the only option listed is X.509 Basic Policy. It doesn’t really matter whether you select Always Trust or cancel out of this offer by selecting Don’t Trust.
  8. If you chose to Always Trust in the last step, you’ll now need to OK a change of certificate trust settings with your current OS X user credentials.
  9. The certificate is not ready for email yet. Double-click it in the login keychain. Be sure both Secure Mail (S/MIME) and X.509 Basic Policy are set to Always Trust. Close the window.
  10. You will (maybe again) see the change to certificate trust settings window. OK the change of certificate trust settings with your current OS X user credentials.
  11. Delete the credentials keychain (now empty) from within Keychain Access via the Delete Keychain option in the File menu.

    Select Delete References & Files to get rid of both the references inside the Keychain Access application itself and the actual credentials file, now empty (you did use a copy of this file in the first step of this sequence, didn’t you?). The deleted keychain might vanish immediately or remain in the list of keychains until Keychain Access is quit and reopened (no need to do that now), depending on things like permissions.
  12. Quit Keychain Access. Done!

If instead of saving the 3 credential items in a separate keychain you exported each item individually, you will need to use File > Import Items… in Keychain Access on each of your backed up/archived .cer, .pem, and .p12 files for the certificate, public key, and private key, respectively.

Due to bugs in Keychain Assistant (and/or the security infrastructure) for OS X all the way from Tiger through Lion (at which point i ceased testing this individual item method), during my testing the certificate went in to the new system just fine, but there was an error message for both the public and private keys:

“An error has occurred. Unable to import an item. The contents of this item cannot be retrieved.”

The private key went in despite this error message; the public key did not. This problem occurs whether i double-click the file in Finder or use File > Import Items… within Keychain Access. Guess we’d better hope that the public key truly is redundant when importing!

Verify that Setup is Complete

This completes the setup for your email security credentials at your end. Quit Keychain Access. Launch Mail, and open a new message form. If all is well, you should see a new pair of icons in the lower right part of the header area, just above the white message body area:

Yours may or may not look exactly like the screenshot: your seal may have an x in it instead of a checkmark. If you have more than one email account and the currently-selected account in Mail is not the one for which you just created security credentials, both icons may be grayed out. As long as you see these two icons in some form (gray or black, selected or not selected), you have successfully completed setup. The final step is to exchange certificates with your correspondents.

Trusting Others’ Certificates

When you first receive an email message with a new, self-signed certificate, Mail will not be able to trust the certificate and will alert you to this fact, first with a banner:

“Unable to verify message signature”

Due to bugs currently in place in OS 10.7 Lion and 10.8 Mountain Lion, do Not do Not do Not go through the process of showing details and eventually trusting the certificate from within Mail—it’s broken and it doesn’t work! (If you want to see what it would look like if it did work, go to this section on the page in this series of articles for OS 10.6 Snow Leopard.)

Quit Mail and follow the same instructions above to use Keychain Access to trust the certificate, as you did previously for your own.

Bugs and Problems

Can’t Encrypt a reply email immediately after trusting an incoming certificate

Padlock remains grayed out, even though the certificate shows as trusted. This currently happens if you try to allow Mail to change the trust settings of a certificate.

Workaround: If you find yourself stuck in this situation:

  1. Quit Mail, if it is running.
  2. Open Keychain Access. Double-click the troublesome certificate (the one you received from someone else and mistakenly tried to trust within Mail).
  3. Change “When using this certificate” to “Use System Defaults”. Close the certificate window to save the changes (very important), authenticating as needed.
  4. Re-open (double-click) the certificate and its trust settings area. Change both Secure Mail (S/MIME) and X.509 Basic Policy to Always Trust. Close the certificate window (again) to save the changes.

Anyone at Apple wanting to look into this should check my Radar bug, #11742852. Hopefully this bug will be fixed soon, before Apple abandons Mountain Lion. Feel encouraged to send Apple feedback if you’d like to see this bug fixed.

Can’t Encrypt an outgoing email message: Can’t Lock Padlock

Many things can cause this failure, esp. in Lion and Mountain Lion. Here are the ones i know:

Security Information Fails to Display

First off, ensure that the Details are showing on the message:
Details hidden. Word “Details” in aqua blue. Details showing. “Hide Details” in aqua blue.
If that didn’t solve your problem, read on.

This is especially likely for pre-existing messages already on the server, in any server mailbox. You may also see “This message has no content” in the preview lines of the Message Viewer (the window with your Inbox, etc.):

Message with no preview lines. Needs to be rebuilt.

Assuming you’re using IMAP for your incoming email (as most of the world is today), go to the Mailbox menu and select the Rebuild command at the bottom of the menu. Your messages will briefly disappear, then come back in. This time you should see however many lines of preview text you’ve selected in your Mail preferences (default is 2), for each encrypted message which has actual text:

Same message with correct preview text.

Looking over at the message pane or re-opening the message window (if you’re like me and prefer not having the smaller message pane in Mail), you should now see security information as expected. This should be all you need for incoming emails. Note that each version of Mail 5-7 for OSes 10.7 Lion through 10.9 Mavericks has its own issues with displaying security information for outgoing messages. I have found cases where the Sent box needed to be rebuilt again to see the security information of a message i’d just sent.

Note that if your incoming email is via POP, you might lose messages with a mailbox rebuild! I haven’t used POP since the early 2000s, so i have no advice for you (other than seriously look into IMAP and convert over).

Comments? Found an error? Let me know, and i’ll see what i can do.