# XORing a header with SHA512-hash - secure?

I am writing a application that encrypts text several times and stores the result. The result looks like complete random bytes, therefore I add some random bytes at the end. To let the program know where the real encrypted text ends I put an header at the beginning of the file. The header consists of 8 bytes: If $h$ is the header, then $h_0$ and $h_1$ are the magic numbers 0 and 204 (I have chosen this numbers randomly to check the file), $h_2$, $h_3$, $h_4$ and $h_5$ show the length of the text in as 32-bit integer and $h_6$ and $h_7$ are just completely random just to fill the header up to 8 bytes. To make the header look like random bytes (a program could find such files easily if I don't do that) I XOR the header-bytes with the SHA512-hash of the password. If the hash is $s$, then

$h_i = h_i \oplus s_i \oplus s_{i+8} \oplus s_{i+16} \oplus s_{i+24} \oplus s_{i+32} \oplus s_{i+40} \oplus s_{i+48} \oplus s_{i+56}$

That makes the file look like random bytes, but after all i think that this reveals information about the sha512-hash of the password, since at least 2 bytes of the header are known and 4 others can be predicted by looking at the size of the file. So this could help an attacker to find the password with a rainbow table, because even the magic numbers help the attacker to "eliminate" 65535 out of 65536 password-hashes (since the magic numbers are known, only 1 out of 65536 hashes will XOR the magic numbers to the bytes given in a file).

My question is: Is this way of securing the header really as insecure as I think it is and if so, how can I prevent this (I don't know why, but I would prefer a method just with XORing and hashing)

-
To summarize: you want the header to be as secure as the text — which is encrypted? –  CL. Apr 20 '14 at 15:11
For starters you could use a proper padding scheme. What algorithm are you using to encrypt the rest of the text? –  rath Apr 20 '14 at 15:58
@CL. The text is encrypted several times, and the header is just XORed with the password hash. Due to the random bytes at the end you cannot encrypt the header with the text and due to my programs structure I preferred no encryption-algorithm and just wanted to XOR the header bytes with something. So the text is encrypted safely but the header not –  Sirac Apr 20 '14 at 16:17
@rath Thanks for your advise, but a proper padding scheme does not help me because I append random bytes at the end, so I have to know the length of the text somehow. About the used algorithms: I use AES, Twofish and TripleDES (and I am aware of the fact that multiple encryption does not increase the security very much) –  Sirac Apr 20 '14 at 16:22
Yes. $\:$ You could skip that header and use bit padding on the plaintext. $\:$ Also, you should $\hspace{.98 in}$ use a MAC or some directly authenticated encryption. $\;\;\;\;$ –  Ricky Demer Apr 20 '14 at 16:49

Leaving text encryption and padding questions aside and focusing on the header stuff, here's how I'd approach the problem:

FileHeader = {
BYTE Salt[16]  # Random bytes, K = KDF(Salt, Password)
BYTE EncHdr[]  # EncHdr = AES-GCM(K, h0...h5)
}


Salt is a sequence of random bytes (it's there to prevent Rainbow table attacks) EncHdr is the encrypted version of your current header containing magic bytes and length (you can store other information in there, too) KDF is any key-derivation function, such as PBKDF2, crypt, scrypt, etc.

Now, if you don't want to use proper encryption and KDF, you can try this:

FileHeader = {
BYTE Salt[16]  # K = HMAC-SHA512(Password, Salt), split K into (K1, K2)
BYTE EncHdr[]  # EncHdr = Encrypt(K1, Header)
BYTE MacHdr[]  # MacHdr = HMAC(K2, EncHdr)
}


So, instead of computing K = SHA-512(Password) you compute K = HMAC-SHA512(Password, Salt) and then split resulting 64-byte K into two 32-byte subkeys K1 and K2 (e.g. K1 is first 32 bytes of K and K2 is the last 32 bytes of K).

You then use K1 to encrypt header payload (magic and length) similarly to as you did before. After that you use K2 to authenticate encrypted data by computing HMAC.

Compared to your scheme, this one is (a) resistant to Rainbow tables (because salt is used to derive key in addition to password) and (b) protects from unauthorised modifications of encrypted header data by authenticating it.

-