Since this is still open and the issue keeps coming up:
TLDR: There are lots of things in OpenSSL that implement standards including AES, but the key derivation part of enc
is partly nonstandard (at least by default)
First, OpenSSL has several commandline operations it calls commands (although they usually aren't separate programs, as typical commands are on Unix), and a whole range of library calls, that do encryption and decryption with numerous symmetric algorithms of which AES is only one as well as several asymmetric algorithms. Major ones are PKCS#7/CMS and semantically equivalent S/MIME; PKCS#8/rfc5208 and PKCS#12/rfc7292; and the SSL/TLS protocols for which it is named.
The file format with Salted__
is used by one specific OpenSSL command enc
. Although if the commandline executable is given a cipher name instead of a command it silently translates to enc
-- for example openssl aes-128-cbc ...
becomes openssl enc -aes-128-cbc ...
-- so people often perceive and describe this as 'OpenSSL AES-128-CBC encryption' as if it were the only AES-128-CBC encryption OpenSSL does.
enc
can encrypt and decrypt files (including anything the OS can provide as standard input and output, such as a pipe) with a wide range of options in (mostly) two dimensions:
the symmetric cipher algorithm and mode, and padding. OpenSSL library provides and enc
can use several algorithms, which may vary depending on build but currently default to Blowfish CAST DES DES-EDE(3) (usually called TripleDES TDES or TDEA) IDEA RC2 RC4 Camellia SEED and AES; I haven't bothered linking all the standards. Some algorithms, especially AES, have options for key size. All except RC4 are block ciphers and must be used with a mode such as CBC ECB or CTR; although algorithms and modes are actually defined by separate standards, enc
(and the EVP
API) combines them in a single name like aes-128-cbc
above. (added) A list of the algorithm-and-modes is available through version 1.0.2 from openssl list-cipher-commands
or just openssl enc -?
, or version 1.1.0 from openssl enc -ciphers
. Do not use openssl ciphers
which shows something quite different: the ciphersuites available for SSL/TLS protocol communication.
Some block modes (CBC and ECB) require data be a multiple of the cipher's block size; since in general real data doesn't do this, it is necessary to add padding when encrypting and remove it when decrypting. By default OpenSSL uses the padding scheme defined by PKCS#7 which extends a scheme defined in PKCS#5, and therefore is usually still called PKCS#5 or just PKCS5 padding. If you specify -nopad
this is not done, and (for these modes) encrypting (or decrypting) wrong-size data gives an error.
all symmetric ciphers require a key, and some modes require an Initialization Vector (IV). enc
can use a 'raw' key and if applicable IV specified on the commandline, with -K
(uppercase!) and -iv
and in hexadecimal. However its default and usual mode is to do Password Based Key Derivation (combined with encryption to give Password-Based Encryption, PBE), and this in turn has several options.
The password can be entered several ways, using legacy option -k
(lowercase) or -kfile
or newer option -pass
or prompted for and typed with no terminal echo (if possible).
The newer, preferred and default option is salted key derivation, which strengthens the derivation (but see below) by using 8-byte random salt. It is also possible to specify a specific salt with -S
in hex, or disable salting with -nosalt
. If salt is used, random or not, a very simple header is added to the ciphertext consisting of the ASCII characters Salted__
and the 8 bytes of salt. (Standardized PBEs like PKCS#12 also convey the salt, but use much more extensive and flexible formats.)
The key derivation process is based on PBKDF1 from PKCS#5/rfc2898 but modified, and is implemented by the API function EVP_BytesToKey
whose man page should be available in any Unix system with OpenSSL installed and on the website. A bit confusingly, and inconsistent with more recent best practice, this 'key derivation' actually derives both the key and IV if applicable. To lay out the relationship exactly:
PBKDF1 uses a choice of hash from MD2 MD5 SHA1, and an iteration count, and (required) salt. It initially forms password || salt
and hashes it, then hashes the first hash, then hashes the second hash, iterating to a total of count times. PKCS5 PBES1 uses PBKDF1 to produce both key and IV -- but only for RC2 and (single) DES, neither of which is acceptable today.
EVP_BytesToKey
uses any hash supported by EVP
, which may depend on build options but currently defaults to MD4 MD5 MDC2 RIPEMD RIPEMD160 SHA1 and the original SHA2 family (but not the later '512 slash' additions), plus iteration count and optional salt. It does one block like PBKDF1 as H^count (password||salt)
with the salt omitted if not used, but if the required 'key material' is more than one block, it then does additional blocks as H^count (prevblock||password||salt)
ditto. Whether the EVP_BytesToKey
result is used for IV depends on the caller.
enc
uses EVP_BytesToKey
with salt by default (but see options above); (edit) hash MD5 by default through 1.0.2 or SHA256 by default in 1.1.0 released 2016-08 but you can specify otherwise with the option -md
which was only documented from 1.0.0 but actually available before; and iteration count ONE. Thus the key and IV if applicable used by enc
before 1.1.0 defaults to
key[||IV] = b0=md5(t=pw[||salt]) || b1=md5(b0||t) || b2=md5(b1||t) ...
This is a poor PBKDF. Because all the available hashes (including default MD5 or SHA256) are fast and this PBKDF 'iterates' only once, an attacker can quickly try large numbers of possible passwords to find yours, unless your password contains enough entropy without any significant 'strengthening' -- and most people don't choose, and even if given can't remember, such strong passwords.
But we are stuck with it for backward compatibility.
Thus for examples:
openssl rc4-128 [-e|-d] -k sekrit -nosalt
# uses RC4 with 128-bit key (RC4 is a stream cipher and uses no IV)
# derived using no salt, MD5 (unless 1.1.0), and count 1 from 'sekrit'
openssl aes-256-cbc [-e|-d] -k sekrit -md sha1
# uses AES in CBC mode with 256-bit key and 128-bit IV
# (both) derived using random salt, SHA1 and count 1 from 'sekrit'
# and adds the Salted__ header with the salt to the ciphertext
And finally, if you specify -base64
or the abbreviation -a
, the ciphertext (including header if any) is encoded to base64 on encryption, and decoded from base64 on decryption; this is fairly commonly needed because some applications, systems, or protocols cannot handle the arbitrary (quasi-random) binary bytes needed for ciphertext. (edit) Similarly if you specify -z
plaintext is compressed before encryption or decompressed after decryption, but only if OpenSSL was built with compression (zlib), and after the CRIME and BREACH attacks some builders or packagers disable compression entirely. These options are independent of the cipher and key/PBKDF, and in fact you can use them alone to only base64 encode/decode or zlib compress/decompress without doing encryption or decryption at all.
Base64 is used with slight variations in many standards but OpenSSL mostly follows the original PEM in 4.3.2.4 (no internal hyperlink because 1993 was before WWW became popular). Two caveats: base64 decoding silently ignores part and sometimes all of a line with a non-base64 character; this is useful when OpenSSL uses it internally to read PEM-format files, but may not be so in other applications. And before 1.1.0, it silently ignores lines longer than the 76-character limit specified in MIME unless you add -A
(uppercase). zlib compression is related to, but not quite the same as, gzip
.
Inter-version compatibility: (added) using password-based encryption and decryption depends critically on using the same parameters and especially the same hash in the PBKDF for both operations. As noted above, the default hash used by the enc
command changed in 1.1.0 to SHA256 versus MD5 in lower versions. Thus data encrypted using the default hash in lower versions won't decrypt with the default in 1.1.0 or vice versa.
if data was encrypted with the default (MD5) on an older version, decrypt in 1.1.0 (and presumably higher when released) with -md md5
if data was encrypted with the default (SHA256) on 1.1.0 (or presumably higher), decrypt in an older version (at least back to 0.9.8, possibly before) with -md sha256
to proactively prevent a problem, specify -md
consistently for both encryption and decryption even when it's redundant
UPDATE: OpenSSL 1.1.1 (2018-09) adds an option for -pbkdf2
-- with default iteration count 10000 -- and an option -iter
to change that count -- and gives a warning when you don't use these i.e. when you default to the old scheme, described above. This is the standard PBKDF2 from PKCS5v2/RFC2898, although 10k is rather low (but much better than 1!); even my aging laptop with a Celeron can do 100-200k without noticeable delay. Compare
Can I dynamically calculate an appropriate number of iterations for PBKDF2 based on the system time, rather than using a fixed value?
https://security.stackexchange.com/questions/110084/parameters-for-pbkdf2-for-password-hashing/
https://security.stackexchange.com/questions/3959/recommended-of-iterations-when-using-pkbdf2-sha256 .
But the iteration count isn't recorded in the file, so you must either remember it (which makes varying it inconvenient) or store it elsewhere such as the filename or other metadata like an NTFS alternate stream.
Also, it still uses the traditional enc
behavior of deriving both the key and IV (if needed), not choosing the IV by a separate random process, as the encryption scheme PBES2 in PKCS5v2 does.