I assume AESCrypt uses AES.
AEScrypt uses AES-256 in CBC mode. However, the program also uses SHA-256.
When a file is encrypted with AEScrypt, there is no limit to the length of the password.
The Linux implementation of AEScrypt limits password to 1024 characters. This means: it is practically unlimited, and you can use even very long passphrases. (The other versions of the program may vary in password length.)
What is the relation between the AESCrypt password and the key size of the underlying AES encryption?
Nothing. The AES key will be always 256 bits. However, if the password has insufficient entropy, the AES key although 256-bits long will not be secure against password brute-forcing attempts.
The program uses salted (128 bit IV) SHA-256 to derive key to use from password. The hash is invoked 8192 times, which is quite small. This means that possible attackers are able to try brute-forcing quite a w few hundred thousand or a million passwords per second. Thus, passwords containing e.g. little entropy or just few dictionary words are likely broken with ease.
It is very likely that password is the weakest link. To have as much entropy as full AES-256 key, passphrase needs to contain around 25 words. Password would need to be somewhere around 50-100 characters. Generally, nobody often uses that long passwords (AEScrypt would allow it, see above).
The result of key derivation is a key that is used for AES (encryption/decryption) and HMAC-SHA256 (protect file integrity/prevent modifications). The same key is used for both purposes.
Issues (detected when analyzing code)
- Ad-hoc salted hashing would be better replaced with scrypt. I'd like PBKDF2 more than hash, because it is well studied construct although it gives little other benefits.
- It is not recommended to use one key for different purposes. (It does not seem to vulnerable, though.)
- The AES implementation used is not side-channel free.
- The /dev/urandom is used much more than is minimum needed. (Works badly together with any software that uses /dev/random.)
None of these appears very bad (i.e. easy to exploit in usual usage).