Quoting the comment on the limit value:
// In NIST SP 800-38D, GCM input size is limited to be no longer
// than (2^36 - 32) bytes. Otherwise, the counter will wrap
// around and lead to a leak of plaintext.
// However, given the current GCM spec requirement that recovered
// text can only be returned after successful tag verification,
// we are bound by limiting the data size to the size limit of
// java byte array, e.g. Integer.MAX_VALUE, since all data
// can only be returned by the doFinal(...) call.
private static final int MAX_BUF_SIZE = Integer.MAX_VALUE;
In other words, decrypting has to buffer the whole ciphertext (even if you pass to
update in pieces) which it does in a Java byte array, and by the long established JVM spec a Java array can't exceed 2^31-1 elements (or available heap space, if less, but you can't generally determine that in advance). This is actually a problem only for decrypting, but they apparently added the check on encrypting so you can't encrypt data and then be unable to decrypt it and thus permanently lose all the data you considered valuable, which some users would consider undesirable. (Although you could work around it using BouncyCastle, which doesn't enforce the 800-36D restriction -- or OpenSSL, ditto, but less easy to use in Java.)
Encryption of big files in Java with AES/GCM