Why don't we use hill cipher of 100 × 100? or even bigger. That would be closely unbreakable. The number of keys possible 2 × 2 hill cipher is 157248. for 100 × 100 number is beyond limits.
The Hill cipher is vulnerable to known-plaintext attack. Once the attacker gets $n$ plaintext/ciphertext pair it can break the cipher by solving a system of linear equations. Consider AES, it is not proved but considered secure against known-plaintext attack, see this question for details.
And, also, key size itself doesn't represent the security. High key sizes are necessary but not sufficient. As an example from history, Enigma was beyond its time as having around $87$ to $88$-bit key size which is greater than the key size of DES. We have already know that Enigma is broken.
Answer to comment: We work in Kerckhoffs's principles and are not accepting security by obscurity. So the attacker knows everything but the key.