AES Decryption performance
Hi, We're using Nuxeo 10.10-HF30 with encryption activated (nuxeo.core.binarymanager=org.nuxeo.ecm.core.blob.binary.AESBinaryManager) running with openjdk version 1.8.0_282. Everything seems just fine, except that the time taken to read files is too long for big files, and increasing O(n^2) with file size.
I made some unit-tests using TestAESBinaryManager class from nuxeo-core-api module:
Encrypt 200KB in 38ms
Encrypt 400KB in 47ms
Encrypt 800KB in 53ms
Encrypt 1500KB in 47ms
Encrypt 3MB in 69ms
Encrypt 6MB in 132ms
Encrypt 12MB in 211ms
Encrypt 24MB in 397ms
Decrypt 200KB in 47ms
Decrypt 400KB in 68ms
Decrypt 800KB in 171ms
Decrypt 1500KB in 494ms
Decrypt 3MB in 1583ms
Decrypt 6MB in 5625ms
Decrypt 12MB in 22423ms
Decrypt 24MB in 89946ms
Doing some profiling, it seems that most of the time is spent deep inside the JDK crypto classes:
update:683, CipherCore (com.sun.crypto.provider)
engineUpdate:380, AESCipher (com.sun.crypto.provider)
update:1835, Cipher (javax.crypto)
getMoreData:140, CipherInputStream (javax.crypto)
read:248, CipherInputStream (javax.crypto)
read:223, CipherInputStream (javax.crypto)
copyLarge:2314, IOUtils (org.apache.commons.io)
copy:2270, IOUtils (org.apache.commons.io)
copyLarge:2291, IOUtils (org.apache.commons.io)
copy:2246, IOUtils (org.apache.commons.io)
decrypt:524, AESBinaryManager (org.nuxeo.ecm.core.blob.binary)
the getMoreData method is called for every 512 bytes chunk of the file, and CipherCore.update allocates a byte array big enough to store all the previous chunks, and ends with a Arrays.fill call. Both the allocation cost and the Arrays.fill cost are proportionals to the size of the array, and the number of calls to getMoreData is also proportional to the file size, hence a O(n^2) cost.
What puzzle me is that I found no other people experiencing this behavior (Nuxeo users or other java users). I tried with other JDKs with no improvements (openjdk-8u292-b10 and openjdk-11.0.11+9)
Has anybody an idea to solve this problem? Should we use another encryption algorithm or library?
Thanks
 |
Didier DUQUENNOY
Member (200) 1 0 2 |
08/30/2021 ( History) |
I finally managed to override AESBinaryManager class to use BouncyCastle instead of the JDK API to decrypt files.To do so, create a sub-class and:
add this static bloc:
static { if (Security.getProvider("BC") == null) { Security.addProvider(new BouncyCastleProvider()); } }override the method
protected void decrypt(InputStream in, OutputStream out)with the same code, except that thenew CipherInputStreamshould instanciate org.bouncycastle.jcajce.io.CipherInputStream instead of javax.crypto.CipherInputStreamoverride the method
protected Cipher getCipher()with the same code, except thatCipher.getInstance(AES_GCM_NOPADDING)becomesCipher.getInstance(AES_GCM_NOPADDING, "BC")to specify the provider.
Now my 24MB files is decrypted in less than a second :-) Note that Nuxeo 10.10-HF30 already includes dependencies to BouncyCastle 1.64
|
Didier DUQUENNOY
Member (200) 1 0 2 |
09/03/2021 ( History) |
Additional infos: https://stackoverflow.com/questions/60575897/cipherinputstream-hangs-while-reading-data tells about known problems with CipherInputStream class for decryption of large files. The link https://crypto.stackexchange.com/questions/20333/encryption-of-big-files-in-java-with-aes-gcm confirm that “CipherInputStream in general is horrible”.
|
Didier DUQUENNOY
Member (200) 1 0 2 |
08/31/2021 ( History) |