Cryptography Stack Exchange is a question and answer site for software developers, mathematicians and others interested in cryptography. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Many AES-encrypted things (files, strings, database entries, etc.) start with "Salted__" ("U2FsdGVkX1" in base64). I hear it's some sort of OpenSSL interoperability thing: a b c.

Is there some standard reference somewhere (perhaps an RFC?) that explains how such OpenSSL-inter-operable AES-encrypted things are produced and later decrypted?

Ideally, an answer would link to a standard reference for the entire process, or perhaps a brief summary list of steps with a link to a standard reference for each step, something like:

To decrypt such a thing beginning with "U2FsdGVkX1" and with a known password,

  • first do base64 decoding -- see Wikipedia: base64. The result will start with "Salted__". Be careful not to use C strings, because the result may include several 0x00 bytes.
  • ...
  • (I guess something about salting and the IV goes here?)
  • (I guess something about CBC or CTR goes here?) -- see Wikipedia: block cipher modes of operation.
  • (Perhaps something about message authentication goes here?)
  • ...
  • AES-decrypt each block with decrypt_one_AES_block( key, block_of_128_bits ) -- see Wikipedia: AES article and A Stick Figure Guide to the Advanced Encryption Standard (AES).
  • save the result of each decrypt_one_AES_block(), concatenate them all together, and that's your plaintext. If the original was human-readable text or an HTML file, it may be OK to store the result in a C string; but other kinds of things may include several 0x00 bytes incompatible with C strings.

It may sound like I'm planning to write an implementation myself. I reassure you that I plan to use one of several available libraries -- it's just that when reviewing the libraries, I'd like to know what they should be doing. What should OpenSSL libraries be doing?

Is there a standard for OpenSSL-interoperable AES encryption?

share|improve this question
1  
related: "Salting when encrypting?" – David Cary Oct 2 '12 at 16:10
1  
1  
A very good question, and I have been programming down this rabbit hole recently as well. @Dave correctly notes that OpenSSL does some weird things that are proprietary, but he is incorrect that "nothing else" is compatible. Today I stumbled upon a nearly complete implementation of 'openssl enc' in Java via: juliusdavies.ca/commons-ssl/pbe.html You can, with the java class org.apache.commons.ssl.OpenSSL, do the equivalent in Java of nearly any command line such as 'openssl enc -aes-128-cbc -iv foo -key bar -base64', or 'openssl enc -aes-128-cbc -pass /foo/bar'. The ciphertext is interoperable – Douglas Held Aug 7 '15 at 21:42
up vote 4 down vote accepted

There's no standard for it, it's a proprietary format that OpenSSL invented. So it's interoperable with every other version of OpenSSL out there, but nothing else.

share|improve this answer
1  
Slightly more information would be welcome, e.g. a reference and a very short description of what the actual format is. A pointer to CMS containers would also be great. – Maarten Bodewes Aug 9 '15 at 10:05
    
OpenSSL doesn't use CMS containers, it's a proprietary format. The thing with proprietary formats is that they're, well, proprietary. You'd have to reverse-engineer the OpenSSL source code to figure out what the actual format is. – Dave Apr 28 at 16:06
    
Ah, so openssl pkcs7 doesn't exist? Note the final question: "Is there a standard for OpenSSL-interoperable AES encryption?" – Maarten Bodewes Apr 28 at 16:39
    
You can use PKCS #7/CMS, but the original question was about OpenSSL's base64 format. If it was CMS it'd be either raw binary or denoted with "----- BEGIN PKCS7 MESSAGE -----" or something similar. There is a standard for CMS-based AES encryption (and PGP-based AES encryption), but not for whatever format OpenSSL uses with its base64 format. – Dave Apr 28 at 16:49

Since this is still open and the issue keeps coming up:

TLDR: There are lots of things in OpenSSL that implement standards including AES, but the key derivation part of enc is partly nonstandard

First, OpenSSL has several commandline operations it calls commands (although they usually aren't separate programs, as typical commands are on Unix), and a whole range of library calls, that do encryption and decryption with numerous symmetric algorithms of which AES is only one as well as several asymmetric algorithms. Major ones are PKCS#7/CMS and semantically equivalent S/MIME; PKCS#8/rfc5208 and PKCS#12/rfc7292; and the SSL/TLS protocols for which it is named.

The file format with Salted__ is used by one specific OpenSSL command enc. Although if the commandline program is given a cipher name instead of a command it silently translates it to enc -- for example openssl aes-128-cbc ... is the same as openssl enc -aes-128-cbc ... -- so people often perceive and describe this as 'OpenSSL AES-128-CBC encryption' as if it were the only AES-128-CBC encryption OpenSSL does.

enc can encrypt and decrypt files (including anything the OS can provide as standard input and output, such as a pipe) with a wide range of options in (mostly) two dimensions:

  • the symmetric cipher algorithm and mode, and padding. OpenSSL library provides and enc can use several algorithms, which may vary depending on build but currently default to Blowfish CAST DES DES-EDE(3) (usually called TripleDES TDES or TDEA) IDEA RC2 RC4 Camellia SEED and AES; I haven't bothered linking all the standards. Some algorithms, especially AES, have options for key size. All except RC4 are block ciphers and must be used with a mode such as CBC ECB or CTR; although algorithms and modes are actually defined by separate standards, enc (and the EVP API) combines them in a single name.

    In addition some modes (CBC and ECB) require data be a multiple of the cipher's block size; since in general real data doesn't do this, it is necessary to add padding when encrypting and remove it when decrypting. By default OpenSSL uses the padding scheme defined by PKCS#7 which extends a scheme defined in PKCS#5, and therefore is usually still called PKCS#5 or just PKCS5 padding. If you specify -nopad this is not done, and for a block mode encrypting (or decrypting) wrong-size data fails.

  • all symmetric ciphers require a key, and some modes require an Initialization Vector (IV). enc can use a 'raw' key and if applicable IV specified on the commandline, with -K (uppercase!) and -iv and in hexadecimal. However its default and usual mode is to do Password Based Key Derivation (combined with encryption to give Password-Based Encryption, PBE), and this in turn has several options.

    The password can be entered several ways, using legacy option -k (lowercase) or -kfile or newer option -pass or prompted for and typed with no terminal echo (if possible).

    The newer, preferred and default option is salted key derivation, which strengthens the derivation (but see below) by using 8-byte random salt. It is also possible to specify a specific salt with -S in hex, or disable salting with -nosalt. If salt is used, random or not, a very simple header is added to the ciphertext consisting of the ASCII characters Salted__ and the 8 bytes of salt. (Standardized PBEs like PKCS#12 also convey the salt, but use much more extensive and flexible formats.)

    The key derivation process is based on PBKDF1 from PKCS#5/rfc2898 but modified, and is implemented by the API function EVP_BytesToKey whose man page should be available in any Unix system with OpenSSL installed and on the website. A bit confusingly, and inconsistent with more recent best practice, this 'key derivation' actually derives both the key and IV if applicable. To lay out the relationship exactly:

    • PBKDF1 uses a choice of hash from MD2 MD5 SHA1, and an iteration count, and salt. It initially forms password || salt and hashes it, then hashes the first hash, then hashes the second hash, iterating to a total of count times. PBES1 does use PBKDF1 to produce both key and IV -- but only for RC2 and (single) DES, neither of which is acceptable today.

    • EVP_BytesToKey uses any hash supported by EVP, which may depend on build options but currently defaults to MD4 MD5 MDC2 RIPEMD RIPEMD160 SHA1 and the original SHA2 family (but not the later '512 slash' additions), plus iteration count and optional salt. It does one block like PBKDF1 as H^count (password||salt) with the salt omitted if not used, but if the required 'key material' is more than one block, it then does additional blocks as H^count (prevblock||password||salt) ditto.

    • enc uses EVP_BytesToKey with salt by default (but see options above), hash MD5 by default (but you can specify otherwise with the previously undocumented option -md) and iteration count ONE. Thus the key and IV if applicable used by enc normally is

      key[||IV] = b0=md5(t=pw[||salt]) || b1=md5(b0||t) || b2=md5(b1||t) ...
      

    This is not a good PBKDF. Because MD5 is fast and 'iterated' only once, an attacker can quickly try large numbers of possible passwords to find yours, unless your password contains enough entropy without any significant 'strengthening' -- and most people don't choose, and even if given can't remember, such strong passwords. But we are stuck with it for backward compatibility.

Thus for examples:

openssl rc4-128 [-e|-d] -k sekrit -nosalt 
# uses RC4 with 128-bit key derived using no salt, MD5, and count 1 from 'sekrit'
# RC4 is a stream cipher and uses no IV

openssl aes-256-cbc [-e|-d] -k sekrit -md sha1
# uses AES in CBC mode with 256-bit key and 128-bit IV 
# (both) derived using random salt, SHA1 and count 1 from 'sekrit'
# and adds the Salted__ header with the salt to the ciphertext

And finally, if you specify -base64 or the abbreviation -a, the ciphertext (including header if any) is encoded to base64 on encryption, and decoded from base64 on decryption. This is independent of the other options, and in fact you can use openssl enc -base64 [-e|-d] to only base64 encode or decode without doing any encryption or decryption at all. Base64 is used, with slight variations, in many standards, but OpenSSL follows the original PEM in 4.3.2.4 (no internal hyperlink because 1993 was before WWW became popular).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.