Originally Web posted Monday, 10 September 2018.
Content last modified Saturday, 9 January 2021 .
External links last verified Thursday, 17 September 2020.

Key Size, Algorithm, Security, and Compatibility Considerations for S/MIME in Apple Mail Mac

Time marches on, or at least keeps clocking moment by moment through future-present-past. Errors or new math breakthroughs render formerly secure technical protocols less secure or insecure. The progression of computing power makes formerly impractical-to-brute-force security credentials practical to crack and exploit.

I am a decades-long Mac user and former software QA tester at Apple, not any sort of cryptography expert. Up until the 2018 revision of this article (of which this is one page), i blithely ignored the problem, ASSuming that a 2048 bit RSA algorithm key set up as the default in Certificate Assistant was sufficient. It was the biggest, most secure key that OS 10.4 Tiger’s Certificate Assistant could generate and was the default, and was the only RSA option in Certificate Assistant for Yosemite and El Capitan, so why make things more difficult?

Recent (as of summer 2018) reading and especially starting to encounter long-reach (Tiger to the current macOS) interoperability issues during testing for this S/MIME article have driven me to publish this page, in hopes of helping provide guidance from someone with deep Apple Mail S/MIME experience (but only very basic cryptography understanding) to at least get you to consider the questions (and likely do your own research) to make key size and algorithm decisions optimized to your needs.

The Basics: What We Know (even non-experts)

More bits in a given key make for a more secure key
Newer algorithms might be more secure than older ones
Interoperability between different macOS and Mail versions requires that both handle the same key size and algorithm
Future-proofing against greater computational power matters more for items being kept a long time, less for transitory items or items retained for shorter time frames
We’re using asymmetric (public key) cryptography in S/MIME (and PGP). Asymmetric keys require many more bits than symmetric keys for the same equivalent cracking resistance

In my very brief reading on the subject, there does not appear to be any clear consensus on how big an S/MIME encryption key needs to be. As i compose the initial version of this page in early September 2018, the 2048 bit RSA key that this article (beyond this one page) has always assumed from the beginning to be secure enough for nearly anyone’s purpose is no longer so considered by many, but not all, experts. One source (RSA Laboratories, as of 2003) considered 2048 bit RSA to be real-world secure until about 2030, barring any unexpected major breakthroughs. Others (NSA, January 2016 (PDF) for one) recommend deprecating it right now and moving to higher RSA bit counts, or a different algorithm. I did not see anyone claiming that 2048 bit RSA is vulnerable right now in early September 2018, with the differences in recommendations resting mostly upon informed, educated speculation regarding how long encrypted emails will be retained, and what each author believes is most likely to happen in terms of improved cracking abilities between now and when the emails may be discarded/obliterated as unneeded.

Two opinions to consider which i’ve found useful:

keylength.com (the Factoring Modulus column corresponds to RSA key length)
How big an RSA key is considered secure today? (current as of 1 August 2020, and a history of being updated often)

Please note that i may not bother to update external site links on this page very often, or even ever at all. Try a search term like:

is 2048 bit RSA encryption still secure?

to find current information on whatever specific key algorithm and bit size you’re contemplating using. The Wikipedia article on Key size may also be a good starting point.

Certificate Assistant Key Generation Options Chart

This table presents the available algorithm and key size options for generating S/MIME encryption keys in various versions of Certificate Assistant and OS X/macOS. The left column lists the Certificate Assistant version (abbreviated CA, not to be confused with a Certificate Authority, build number in parentheses), followed by the OS version with which it originally shipped.

Important: This table lists the capabilities of each version of Certificate Assistant, not what i have actually tested. The very few key sizes and algorithms that i have actually tested have a check mark ✓ in front of the text. RSA 2048 has been tested almost exclusively by me, as it has been Apple’s default from 2005 through the most recent revision of this page.

“Generally considered strong/secure” is my subjective conclusion from my brief reading. It may not be correct, will become obsolete, and may not apply to your needs. Be sure to do your own research and draw your own conclusions regarding what key strength works for you!

Legend

Strong: Supported by this version and generally considered strong/secure as of the last revision of this page.
OK: Supported by this version, with disagreement concerning strength/security as of the last revision of this page.
Weak: Supported by this version and generally considered obsolete/weak, no longer viable for secure usage as of the last revision of this page.
No: Not supported by this version.

Algorithm	DSA						RSA								ECC
Key Size	512	1024	1200	1400	1600	2048	512	1024	1200	1400	1600	2048	4096	8192	256	384	521
CA 1.0 (30) 10.4 Tiger	Weak	Weak	No	No	No	OK	Weak	Weak	No	No	No	✓ OK	No	No	No	No	No
CA 2.0 (32175) 10.5 Leopard	Weak	Weak	Weak	Weak	Weak	OK	Weak	Weak	Weak	Weak	Weak	✓ OK	No	No	No	No	No
CA 3.0 (36996) 10.6 Snow Leopard	Weak	Weak	Weak	Weak	Weak	OK	Weak	Weak	Weak	Weak	Weak	✓ OK	No	No	OK	Strong	Strong
CA 4.4 (55106.3) 10.7 Lion	Weak	Weak	Weak	Weak	Weak	OK	Weak	Weak	Weak	Weak	Weak	✓ OK	No	No	OK	Strong	Strong
CA 5.0 (55108.3) 10.8 Mountain Lion	Weak	Weak	Weak	Weak	Weak	OK	Weak	Weak	Weak	Weak	Weak	✓ OK	No	No	OK	Strong	Strong
CA 5.0 (55125) 10.9 Mavericks	No	Weak	Weak	Weak	Weak	OK	No	Weak	Weak	Weak	Weak	✓ OK	No	No	OK	Strong	Strong
CA 5.0 (55128) 10.10 Yosemite	No	Weak	Weak	Weak	Weak	OK	No	Weak	Weak	Weak	Weak	✓ OK	No	No	OK	Strong	Strong
CA 5.0 (55132.40.1) 10.11 El Capitan	No	No	No	No	No	No	No	No	No	No	No	✓ OK	No	No	OK	Strong	Strong
CA 5.0 (55165.50.2) 10.12 Sierra	No	No	No	No	No	No	No	No	No	No	No	✓ OK	No	No	OK	Strong	Strong
CA 5.0 (55174.50.1) 10.13 High Sierra	No	No	No	No	No	No	No	No	No	No	No	✓ OK	✓ Strong	✓ Strong	✓ OK	✓ Strong	✓ Strong
CA 5.0 (55174.260.1) 10.14 Mojave	No	No	No	No	No	No	No	No	No	No	No	✓ OK	✓ Strong	✓ Strong	✓ OK	✓ Strong	✓ Strong
CA 5.0 (55188) 10.15 Catalina	No	No	No	No	No	No	No	No	No	No	No	✓ OK	✓ Strong	✓ Strong	✓ OK	✓ Strong	✓ Strong

All versions of Certificate Assistant released through the last modification date of this page default to RSA 2048. When the DSA algorithm is selected (when available) the default is also 2048 bits. When ECC is selected (when available) on Certificate Assistants included through OS 10.12 Sierra, the default is 521 bits. Interestingly on OS 10.13 High Sierra through at least OS 10.15 Catalina the ECC default is 384 bits.

Keychain Access Ability to Utilize Chart

This table presents the few combinations of algorithm and key size of S/MIME encryption keys generated by various versions of Certificate Assistant that i have personally tested (or perhaps site correspondents report to me work) in various versions of Keychain Access in OS X/macOS. The left column lists the Keychain Access version (abbreviated KA, build number in parentheses), followed by the OS version with which it originally shipped.

The point of this table is that sometimes versions of Keychain Access whose accompanying Certificate Assistant is incapable of generating a particular key variant can still process certificates and keys made by a different version of Certificate Assistant on a different system. As one specific example, Keychain Access 3.3 in OS 10.4 Tiger is able to work with RSA 4096 bit key credentials made on a 10.13 High Sierra system: it can open them, store them in a keychain, and verify their integrity, amongst other functions.

Important: Just because Keychain Access can work with certificates of a certain format and size does not mean that the accompanying Apple Mail bundled in the same OS can. Mail compatibility differs, and is covered in the next table below.

Legend

Works: This algorithm and key size has been tested and found working
[empty cell]: This combination has not been tested
Fail: This combination has been tested and does not work

Algorithm	DSA						RSA								ECC
Key Size	512	1024	1200	1400	1600	2048	512	1024	1200	1400	1600	2048	4096	8192	256	384	521
KA 3.3 (25367) 10.4 Tiger												Works	Works*	Fail¹	Fail	Fail
KA 4.0.2 (35210) 10.5 Leopard												Works
KA 4.1.1 (55004) 10.6 Snow Leopard												Works	Works	Fail¹	Fail²	Fail²	Fail²
KA 5.4 (55120.6) 10.7 Lion												Works	Works	Fail¹	Fail²		Fail²
KA 7.0 (55121.4) 10.8 Mountain Lion												Works		Fail¹	Fail²
KA 9.0 (55153) 10.9 Mavericks												Works		Fail¹	Fail²
KA 9.0 (55161) 10.10 Yosemite												Works		Fail¹	Fail²
KA 9.0 (55171.20.2) 10.11 El Capitan												Works		Works	Fail²
KA 9.0 (55208.60.1) 10.12 Sierra												Works		Works	Works		Works
KA 10.0 (55237.50.3) 10.13 High Sierra												Works	Works	Works	Works	Works	Works
KA 10.5 (55237.250.3) 10.14 Mojave												Works	Works
KA 10.5 (55270.0.1) 10.15 Catalina												Works	Works	Works	Works	Works	Works

* indicates cases where things generally worked, but some instability was noted. For example: Tiger’s Keychain Access 3.3 initially crashed when i double-clicked a keychain made on a High Sierra system. Relaunching KA from the crash dialog then repeating the Finder double-click of the same keychain worked fine, thereafter the keychain and its contents worked as expected. Individual certificates—not keychains—of these sizes made by Certificate Access in OS 10.14 Mojave & 10.15 Catalina worked problem-free when USB flash drive transferred over and opened in Tiger Keychain Access. Consider anything marked this way to be an edge case, worthy of your own more extensive testing before deploying on daily-use critical systems.

¹ “The data does not appear to be a valid certificate.”

² Appears to work correctly, but makes a broken certificate which works for signing, but not for encryption. See footnote 6 below for more information.

Secure Email in Mail Chart

This table presents the few combinations of algorithm and key size of S/MIME encryption keys generated by various versions of Certificate Assistant that i have personally tested (or perhaps site correspondents report to me work) in various versions of Apple Mail in OS X/macOS. The left column lists the Mail version followed by the OS version with which it originally shipped.

The point of this table is that there are several cases where Keychain Access for a given OS version can handle a certain algorithm and key size just fine, but Mail itself cannot, or vice-versa.

Legend

Works: This algorithm and key size has been tested and found working
[empty cell]: This combination has not been tested
Fail: This combination has been tested and does not work (details in footnotes)

Algorithm	DSA						RSA								ECC
Key Size	512	1024	1200	1400	1600	2048	512	1024	1200	1400	1600	2048	4096	8192	256	384	521
Mail 2.1.3 10.4.11 Tiger PPC												Works³	Works³	Fail	Fail	Fail
Mail 3.6 10.5 Leopard	Totally Broken
Mail 4.5 10.6.8 Snow Leopard												Works	Works	Fail	Fail⁵	Fail⁵	Fail⁵
Mail 5.3 10.7.5 Lion												Works	Works	Fail	Fail⁶		Fail⁶
Mail 6.6 10.8 Mountain Lion												Works		Fail	Fail⁶
Mail 7.3 10.9 Mavericks												Works		Fail	Fail⁶
Mail 8.2 10.10 Yosemite												Works		Fail	Fail⁶
Mail 9.3 10.11 El Capitan												Works		Works	Fail⁶
Mail 10.3 10.12.6 Sierra												Works		Works	Works		Works
Mail 11.5 10.13 High Sierra												Works	Works	Works	Works	Works	Works
Mail 12.? 10.14 Mojave												Works⁴	Works⁴	Works	Works	Works	Works
Mail 13.4 10.15.6 Catalina												Works⁴	Works⁴	Works	Works	Works	Works

³ Fully forwards-compatible through 10.13 High Sierra systems with Mail 11.5, but no newer. Mail 2.1.3 crashes any time it tries to open an incoming email message containing a certificate sent by Mail 12.? in OS 10.14 Mojave or Mail 13.4 in OS 10.15 Catalina, such as a signed but not encrypted message. Note that i tested a known-safe working certificate generated on 10.12 Sierra and working perfectly via that OS and its Mail with the Tiger system, and when that certificate was sent by Catalina Mail, Tiger Mail still crashed. Carefully inspecting the raw email sources sent by Mail 10.3 in Sierra and 13.4 in Catalina revealed no differences. Given what i can read about the crash report, something is happening when Tiger Mail hands off the received certificate to the Tiger security infrastructure that somehow must differ between older OSes and the newer ones where the crashes happen. Interestingly, it is possible for Tiger Mail to receive an encrypted but not signed message from Mojave and Catalina Mail.

⁴ Fully backwards-compatible through 10.6 Snow Leopard systems. Signed messages sent by Mail 12.? with 10.14 Mojave or Mail 13.4 with 10.15 Catalina consistently crash Mail 2.1.3 on Tiger systems. Encrypted but not signed messages sent by Mail 12.? or 13.4 appear to work on Mail 2.1.3 in limited testing (did not test repeated back-and-forth replies). 10.5 Leopard systems were not tested, because their S/MIME email isn’t real-world usable with latter-day security and performance updates. See above footnote 3 for more information: this is the mirror image of the same issue.

⁵ Mail 4.5 says it’s unable to verify its own-made certificate, even though that certificate is fully trusted and verified by Snow Leopard’s own Keychain Access, whose Certificate Assistant made the certificate. This was tested for all ECC key sizes both directly from two accounts on the same iMac running Snow Leopard 10.6 (which should be the case which always works) and —for the 256 bit ECC case only—from a High Sierra Mail 11.5 system to Snow Leopard. The comments in the next footnote regarding stealth broken certificates may apply here too—i have not tested this theory.

⁶ Mail 5.3-9.3 happily show an S/MIME file rather than an error message, which file, when double-clicked, opens Keychain Access. And nothing useful happens. More specifically:

Signing works, when there is no encryption
Encryption may work when the older OS is sending to the newer OS, encrypting with the newer OS’s ECC certificate
Encryption from the newer OS to the older one fails, with the raw S/MIME file sitting as an unreadable attachment in the message

While this looks like a Mail bug, further testing reveals that using an ECC certificate made by High Sierra works, at least with limited testing placing this certificate onto a 10.9 Mavericks test system. Note: during my brief testing of this configuration, while signing and encryption worked correctly, the status indicators disappeared from Mail and would not come back. This is getting too crazy for me to test all possible parameters and variations. If you have an actual use case, be sure to test your specific case thoroughly. I’m marking it as a Fail because staying within an OS for all components is an utter fail. It’s a much lighter color (for those seeing colors) because it appears it can be made to work, if a newer Certificate Assistant makes the ECC certificate (tested so far only with High Sierra). Note that i only light-marked the version(s) i actually tested, but this probably applies to all Fails with superscript 6, and possibly also 5.

In general, testing was between accounts in the same OS with the same key credentials, all created and utilized with the software versions bundled with that OS. In cases where a given OS’s Certificate Assistant is incapable of generating a certificate with certain parameters, i used a newer version of CA to generate the certificate, putting it directly into a keychain on the desktop of the creating system, then file sharing it to the other Mac. Most testing like this i did in September 2018 from High Sierra, so assume this was the source unless otherwise indicated.

Testing Limitations: the Long Reach

Because my time is not infinite, i have done “longest reach” testing: testing between the newest and oldest OS (and related software) which works, omitting in-between versions, ASSuming they all work. This may not be a valid assumption, and if you plan to use one of these intermediate OS/Mail/Keychain Access/Certificate Assistant versions, you owe it to yourself to test the combination you intend to use for awhile before doing anything meaningful/serious with it. These cross-OS tests are not explicitly differentiated from same-OS tests in the table when they work as expected, the general sense being that if it works cross-OS with no notable issues it’ll work within the OS, and vice-versa. Only cross-OS outright failure cases are noted (footnote and lighter table background color).

Similarly, whether across OSes or within one, i did not always make time to test intermediate key sizes (lengths), ASSuming that if the largest and smallest of a given encryption format worked, the intermediate size(s) would work too. I have yet to find a case where this was false, but maybe it could happen.

Have You Tested Other Combinations?

If you’ve done reasonable testing and know that certain combinations currently reported as untested on this page work for routine secure emailing and would like to contribute to this page (credit to you as you wish or not at all as you prefer) please let me know.

Return to where you may have been in the main article (or use your browser’s Back button, especially from an individual OS setup article)