Originally Web posted Monday, 10 September 2018.
Content last modified Monday, 10 September 2018 .
External links last verified Monday, 10 September 2018.

Key Size, Algorithm, Security, and Compatibility Considerations for S/MIME in Apple Mail Mac

Time marches on, or at least keeps clocking moment by moment through future-present-past. Errors or new math breakthroughs render formerly secure technical protocols less secure or insecure. The progression of computing power makes formerly impractical-to-brute-force security credentials practical to crack and exploit.

I am a decades-long Mac user and former software QA tester at Apple, not any sort of cryptography expert. Up until the 2018 revision of this article (of which this is one page), i blithely ignored the problem, ASSuming that a 2048 bit RSA algorithm key set up as the default in Certificate Assistant was sufficient. It was the biggest, most secure key that OS 10.4 Tiger’s Certificate Assistant could generate and was the default, and was the only RSA option in Certificate Assistant for Yosemite and El Capitan, so why make things more difficult?

Recent reading and especially starting to encounter long-reach (Tiger to the current macOS) interoperability issues during testing for this S/MIME article have driven me to publish this page, in hopes of helping provide guidance from someone with deep Apple Mail S/MIME experience (but only very basic cryptography understanding) to at least get you to consider the questions (and likely do your own research) to make key size and algorithm decisions optimized to your needs.

The Basics: What We Know (even non-experts)

In my very brief reading on the subject, there does not appear to be any clear consensus on how big an S/MIME encryption key needs to be. As i compose the initial version of this page in early September 2018, the 2048 bit RSA key that this article (beyond this one page) has always assumed from the beginning to be secure enough for nearly anyone’s purpose is no longer so considered by many, but not all, experts. One source (RSA Laboratories, as of 2003) considered 2048 bit RSA to be real-world secure until about 2030, barring any unexpected major breakthroughs. Others (NSA, January 2016 (PDF) for one) recommend deprecating it right now and moving to higher RSA bit counts, or a different algorithm. I did not see anyone claiming that 2048 bit RSA is vulnerable right now in early September 2018, with the differences in recommendations resting mostly upon informed, educated speculation regarding how long encrypted emails will be retained, and what each author believes is most likely to happen in terms of improved cracking abilities between now and when the emails may be discarded/obliterated as unneeded.

Please note that i may not bother to update external site links on this page very often, or even ever at all. Try a search term like:

is 2048 bit RSA encryption still secure?

to find current information on whatever specific key algorithm and bit size you’re contemplating using. The Wikipedia article on Key size may also be a good starting point.

Certificate Assistant Key Generation Options Chart

This table presents the available algorithm and key size options for generating S/MIME encryption keys in various versions of Certificate Assistant and OS X/macOS. The left column lists the Certificate Assistant version (abbreviated CA, not to be confused with a Certificate Authority, build number in parentheses), followed by the OS version with which it originally shipped.

Important: This table lists the capabilities of each version of Certificate Assistant, not what i have actually tested. The very few key sizes and algorithms that i have actually tested have a check mark ✓ in front of the text. RSA 2048 has been tested almost exclusively by me, as it has been Apple’s default from 2005 through the most recent revision of this page.

“Generally considered strong/secure” is my subjective conclusion from my brief reading. It may not be correct, will become obsolete, and may not apply to your needs. Be sure to do your own research and draw your own conclusions regarding what key strength works for you!

Legend

Strong
Supported by this version and generally considered strong/secure as of the last revision of this page.
OK
Supported by this version, with disagreement concerning strength/security as of the last revision of this page.
Weak
Supported by this version and generally considered obsolete/weak, no longer viable for secure usage as of the last revision of this page.
No
Not supported by this version.
All versions of Certificate Assistant released through the last modification date of this page default to RSA 2048. When the DSA algorithm is selected (when available) the default is also 2048 bits. When ECC is selected (when available) on Certificate Assistants included through OS 10.12 Sierra, the default is 521 bits. Interestingly on OS 10.13 High Sierra the ECC default is 384 bits.
Algorithm DSA RSA ECC
Key Size 512 1024 1200 1400 1600 2048 512 1024 1200 1400 1600 2048 4096 8192 256 384 521
CA 1.0 (30)
10.4 Tiger
Weak Weak No No No OK Weak Weak No No No ✓ OK No No No No No
CA 2.0 (32175)
10.5 Leopard
Weak Weak Weak Weak Weak OK Weak Weak Weak Weak Weak ✓ OK No No No No No
CA 3.0 (36996)
10.6 Snow Leopard
Weak Weak Weak Weak Weak OK Weak Weak Weak Weak Weak ✓ OK No No OK Strong Strong
CA 4.4 (55106.3)
10.7 Lion
Weak Weak Weak Weak Weak OK Weak Weak Weak Weak Weak ✓ OK No No OK Strong Strong
CA 5.0 (55108.3)
10.8 Mountain Lion
Weak Weak Weak Weak Weak OK Weak Weak Weak Weak Weak ✓ OK No No OK Strong Strong
CA 5.0 (55125)
10.9 Mavericks
No Weak Weak Weak Weak OK No Weak Weak Weak Weak ✓ OK No No OK Strong Strong
CA 5.0 (55128)
10.10 Yosemite
No Weak Weak Weak Weak OK No Weak Weak Weak Weak ✓ OK No No OK Strong Strong
CA 5.0 (55132.40.1)
10.11 El Capitan
No No No No No No No No No No No ✓ OK No No OK Strong Strong
CA 5.0 (55165.50.2)
10.12 Sierra
No No No No No No No No No No No ✓ OK No No OK Strong Strong
CA 5.0 (55174.50.1)
10.13 High Sierra
No No No No No No No No No No No ✓ OK ✓ Strong ✓ Strong ✓ OK ✓ Strong ✓ Strong

Keychain Access Ability to Utilize Chart

This table presents the few combinations of algorithm and key size of S/MIME encryption keys generated by various versions of Certificate Assistant that i have personally tested (or perhaps site correspondents report to me work) in various versions of Keychain Access in OS X/macOS. The left column lists the Keychain Access version (abbreviated KA, build number in parentheses), followed by the OS version with which it originally shipped.

The point of this table is that sometimes versions of Keychain Access whose accompanying Certificate Assistant is incapable of generating a particular key variant can still process certificates and keys made by a different version of Certificate Assistant on a different system. As one specific example, Keychain Access 3.3 in OS 10.4 Tiger is able to work with RSA 4096 bit key credentials made on a High Sierra system: it can open them, store them in a keychain, and verify their integrity, amongst other functions.

Legend

Works
This algorithm and key size has been tested and found working
[empty cell]
This combination has not been tested
Fail
This combination has been tested and does not work
* indicates cases where things generally worked, but some instability was noted. For example: Tiger’s Keychain Access 3.3 initially crashed when i double-clicked a keychain made on a High Sierra system. Relaunching KA from the crash dialog then repeating the Finder double-click of the same keychain worked fine, thereafter the keychain and its contents worked as expected. Consider anything marked this way to be an edge case, worthy of your own more extensive testing before deploying on daily-use critical systems.
1 “The data does not appear to be a valid certificate.”
2 Appears to work correctly, but makes a broken certificate which works for signing, but not for encryption. See footnote 4 below for more information.
Algorithm DSA RSA ECC
Key Size 512 1024 1200 1400 1600 2048 512 1024 1200 1400 1600 2048 4096 8192 256 384 521
KA 3.3 (25367)
10.4 Tiger
Works Works* Fail1 Fail Fail
KA 4.0.2 (35210)
10.5 Leopard
Works
KA 4.1.1 (55004)
10.6 Snow Leopard
Works Works Fail1 Fail2 Fail2 Fail2
KA 5.4 (55120.6)
10.7 Lion
Works Works Fail1 Fail2 Fail2
KA 7.0 (55121.4)
10.8 Mountain Lion
Works Fail1 Fail2
KA 9.0 (55153)
10.9 Mavericks
Works Fail1 Fail2
KA 9.0 (55161)
10.10 Yosemite
Works Fail1 Fail2
KA 9.0 (55171.20.2)
10.11 El Capitan
Works Works Fail2
KA 9.0 (55208.60.1)
10.12 Sierra
Works Works Works Works
KA 10.0 (55237.50.3)
10.13 High Sierra
Works Works Works Works Works Works

Secure Email in Mail Chart

This table presents the few combinations of algorithm and key size of S/MIME encryption keys generated by various versions of Certificate Assistant that i have personally tested (or perhaps site correspondents report to me work) in various versions of Apple Mail in OS X/macOS. The left column lists the Mail version followed by the OS version with which it originally shipped.

The point of this table is that there are several cases where Keychain Access for a given OS version can handle a certain algorithm and key size just fine, but Mail itself cannot, or vice-versa.

Legend

Works
This algorithm and key size has been tested and found working
[empty cell]
This combination has not been tested
Fail
This combination has been tested and does not work (details in footnotes)
Algorithm DSA RSA ECC
Key Size 512 1024 1200 1400 1600 2048 512 1024 1200 1400 1600 2048 4096 8192 256 384 521
Mail 2.1.3
10.4 Tiger
Works Works Fail Fail Fail
Mail 3.6
10.5 Leopard
Totally Broken
Mail 4.5
10.6 Snow Leopard
Works Works Fail Fail3 Fail3 Fail3
Mail 5.3
10.7 Lion
Works Works Fail Fail4 Fail4
Mail 6.6
10.8 Mountain Lion
Works Fail Fail4
Mail 7.3
10.9 Mavericks
Works Fail Fail4
Mail 8.2
10.10 Yosemite
Works Fail Fail4
Mail 9.3
10.11 El Capitan
Works Works Fail4
Mail 10.3
10.12 Sierra
Works Works Works Works
Mail 11.5
10.13 High Sierra
Works Works Works Works Works Works

3 Mail 4.5 says it’s unable to verify its own-made certificate, even though that certificate is fully trusted and verified by Snow Leopard’s own Keychain Access, whose Certificate Assistant made the certificate. This was tested for all ECC key sizes both directly from two accounts on the same iMac running Snow Leopard 10.6 (which should be the case which always works) and —for the 256 bit ECC case only—from a High Sierra Mail 11.5 system to Snow Leopard. The comments in the next footnote regarding stealth broken certificates may apply here too—i have not tested this theory.

4 Mail 5.3-9.3 happily show an S/MIME file rather than an error message, which file, when double-clicked, opens Keychain Access. And nothing useful happens. More specifically:

While this looks like a Mail bug, further testing reveals that using an ECC certificate made by High Sierra works, at least with limited testing placing this certificate onto a 10.9 Mavericks test system. Note: during my brief testing of this configuration, while signing and encryption worked correctly, the status indicators disappeared from Mail and would not come back. This is getting too crazy for me to test all possible parameters and variations. If you have an actual use case, be sure to test your specific case thoroughly. I’m marking it as a Fail because staying within an OS for all components is an utter fail. It’s a much lighter color (for those seeing colors) because it appears it can be made to work, if a newer Certificate Assistant makes the ECC certificate (tested so far only with High Sierra). Note that i only light-marked the version(s) i actually tested, but this probably applies to all Fails with superscript 4, and possibly also 3.

In general, testing was between accounts in the same OS with the same key credentials, all created and utilized with the software versions bundled with that OS. In cases where a given OS’s Certificate Assistant is incapable of generating a certificate with certain parameters, i used a newer version of CA to generate the certificate, putting it directly into a keychain on the desktop of the creating system, then file sharing it to the other Mac. Most testing like this i did in September 2018 from High Sierra, so assume this was the source unless otherwise indicated.

Because my time is not infinite, i have done “longest reach” testing: testing between the newest and oldest OS (and related software) which works, omitting in-between versions, ASSuming they all work. This may not be a valid assumption, and if you plan to use one of these intermediate OS/Mail/Keychain Access/Certificate Assistant versions, you owe it to yourself to test the combination you intend to use for awhile before doing anything meaningful/serious with it. These cross-OS tests are not explicitly differentiated from same-OS tests in the table, the general sense being that if it works cross-OS with no notable issues it’ll work within the OS, and vice-versa.


Have You Tested Other Combinations?

If you’ve done reasonable testing and know that certain combinations currently reported as untested on this page work for routine secure emailing and would like to contribute to this page (credit to you as you wish or not at all as you prefer) please let me know.

Return to where you may have been in the main article (or use your browser’s Back button, especially from an individual OS setup article)


Secure Email in OS X, Apple Mail Sonic’s signatureThe Sonically Pure Pages

This Siber-Sonically Pure Page is Valid CSS! and Valid HTML 4.01! Transitional compliant.