Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A lot of ANRs in Google Play Console #26

Open
aliyailina opened this issue Feb 28, 2024 · 14 comments
Open

A lot of ANRs in Google Play Console #26

aliyailina opened this issue Feb 28, 2024 · 14 comments
Labels

Comments

@aliyailina
Copy link

Describe the bug:

We started using EncryptedSharedPreferences and now receive a lot of ANR from Google Play Console. There is not much information, but some of ANRs points to com.google.crypto.tink library.

What was the expected behavior?

No ANRs

How can we reproduce the bug?

Unfortunately, we have no reproduce, only data from Google Play Console.

Do you have any debugging information?

Console points to that lines of code (they are all different, it is not single stacktrace):

com.google.crypto.tink.shaded.protobuf.MessageSchema.newSchemaForRawMessageInfo+3804

com.google.crypto.tink.daead.AesSivProtoSerialization.parseKey

com.google.crypto.tink.internal.MutablePrimitiveRegistry.registerPrimitiveWrapper

com.google.crypto.tink.proto.AesSivKeyFormat.dynamicMethod

javax.crypto.Cipher.init+66

com.google.crypto.tink.shaded.protobuf.ProtobufArrayList.mutableCopyWithCapacity

com.google.crypto.tink.KeyManagerRegistry$2.getUntypedKeyManager

com.google.crypto.tink.shaded.protobuf.MessageSchema.mergeFromHelper+8504

What version of Tink are you using?

1.8.0 inside AndroidX.Security.Crypto-1.1.0-alpha06

Can you tell us more about your development environment?

Xamarin.Android
Android 24+
There is no statistically significant differences in ANR per devices or Android version.

@juergw
Copy link
Contributor

juergw commented Feb 28, 2024

This is similar to tink-crypto/tink#638. But that problem has been fixed.

But note that under the hood we are using Android Keystore, which will fail if too many threads try to access it. That might be the problem you are facing here. So what you could try is to minimize the creation of new EncryptedSharedPreference objects. For example, create the ones you need at startup, and keep reusing them. And do the same for the MasterKey object: only create one of them at startup.

I hope that helps.

@juergw juergw removed their assignment Feb 28, 2024
@juergw juergw added the p3 label Feb 28, 2024
@aliyailina
Copy link
Author

@juergw Thank you for your quick response!
Yes, recreation can really be the problem. We will try to change the way we use it and see if it works.

@juergw juergw self-assigned this Mar 6, 2024
@juergw
Copy link
Contributor

juergw commented Mar 7, 2024

Did this work?

@aliyailina
Copy link
Author

@juergw We have no reproduce, so we need to release first and then see the results in console. I will let you know.

@aliyailina
Copy link
Author

aliyailina commented Mar 13, 2024

@juergw Unfortunately, don't think it has helped, we still see related ANRs in Console.

#00  pc 0x000000000004f55c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
#01  pc 0x0000000000232858  /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks+140)
#02  pc 0x000000000045df98  /apex/com.android.art/lib64/libart.so (artJniMethodEnd+336)
#03  pc 0x00000000005c0dfc  /apex/com.android.art/lib64/libart.so (art_jni_method_end+12)
at java.lang.Thread.nativeCreate (Native method)
at java.lang.Thread.start (Thread.java:976)
at android.app.SharedPreferencesImpl.startLoadFromDisk (SharedPreferencesImpl.java:142)
at android.app.SharedPreferencesImpl.<init> (SharedPreferencesImpl.java:130)
at android.app.ContextImpl.getSharedPreferences (ContextImpl.java:577)
at android.app.ContextImpl.getSharedPreferences (ContextImpl.java:558)
at android.content.ContextWrapper.getSharedPreferences (ContextWrapper.java:217)
at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.readKeysetFromPrefs (AndroidKeysetManager.java:251)
at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.build (AndroidKeysetManager.java:288)
at androidx.security.crypto.EncryptedSharedPreferences.create (EncryptedSharedPreferences.java:169)
at androidx.security.crypto.EncryptedSharedPreferences.create (EncryptedSharedPreferences.java:130)
at <id>.MainApplication.n_onCreate (Native method)
at <id>.MainApplication.onCreate (MainApplication.java:26)

Now looking at stack trace I'm not even sure if it is your problem.

@juergw juergw removed their assignment Mar 19, 2024
@juergw
Copy link
Contributor

juergw commented Mar 19, 2024

I unassigned myself because there is nothing I can do at the moment. But I'll leave this issue open for now.

@alvindizon
Copy link

In our case, our app which uses Encrypted DataStore, which in turn uses tink, have experienced ANRs from two devices, all of which are running Android 12. If you are interested, here is the stack trace:

main (timed waiting):tid=1 systid=10495 
       at java.lang.Thread.sleep(Native method)
       at java.lang.Thread.sleep(Thread.java:450)
       at java.lang.Thread.sleep(Thread.java:355)
       at android.security.KeyStoreSecurityLevel.interruptedPreservingSleep(KeyStoreSecurityLevel.java:206)
       at android.security.KeyStoreSecurityLevel.createOperation(KeyStoreSecurityLevel.java:115)
       at android.security.keystore2.AndroidKeyStoreCipherSpiBase.ensureKeystoreOperationInitialized(AndroidKeyStoreCipherSpiBase.java:334)
       at android.security.keystore2.AndroidKeyStoreCipherSpiBase.engineInit(AndroidKeyStoreCipherSpiBase.java:171)
       at javax.crypto.Cipher.tryTransformWithProvider(Cipher.java:2985)
       at javax.crypto.Cipher.tryCombinations(Cipher.java:2892)
       at javax.crypto.Cipher$SpiAndProviderUpdater.updateAndGetSpiAndProvider(Cipher.java:2797)
       at javax.crypto.Cipher.chooseProvider(Cipher.java:774)
       at javax.crypto.Cipher.init(Cipher.java:1144)
       at javax.crypto.Cipher.init(Cipher.java:1085)
       at com.google.crypto.tink.integration.android.AndroidKeystoreAesGcm.encryptInternal(AndroidKeystoreAesGcm.java:77)
       at com.google.crypto.tink.integration.android.AndroidKeystoreAesGcm.encrypt(AndroidKeystoreAesGcm.java:61)
       at com.google.crypto.tink.integration.android.AndroidKeystoreKmsClient.validateAead(AndroidKeystoreKmsClient.java:289)
       at com.google.crypto.tink.integration.android.AndroidKeystoreKmsClient.getAead(AndroidKeystoreKmsClient.java:157)
       at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.readMasterkeyDecryptAndParseKeyset(AndroidKeysetManager.java:367)
       at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.build(AndroidKeysetManager.java:303)
       at androidx.security.crypto.EncryptedFile$Builder.build(EncryptedFile.java:172)
       at io.github.osipxd.security.crypto.EncryptedPreferenceDataStoreSingletonDelegate$getValue$1$1.invoke(EncryptedPreferencesDataStoreDelegate.kt:121)
       at io.github.osipxd.security.crypto.EncryptedPreferenceDataStoreSingletonDelegate$getValue$1$1.invoke(EncryptedPreferencesDataStoreDelegate.kt:110)
       at io.github.osipxd.security.crypto.EncryptedPreferenceDataStoreFactoryKt.createEncrypted(EncryptedPreferenceDataStoreFactory.kt:76)
       at io.github.osipxd.security.crypto.EncryptedPreferenceDataStoreSingletonDelegate.getValue(EncryptedPreferencesDataStoreDelegate.kt:110)
       at io.github.osipxd.security.crypto.EncryptedPreferenceDataStoreSingletonDelegate.getValue(EncryptedPreferencesDataStoreDelegate.kt:83)
       at com.someapp.common.preferences.di.DataStoreModule.getEncryptedDataStore(DataStoreModule.kt:23)
       at com.someapp.common.preferences.di.DataStoreModule.provideDataStore(DataStoreModule.kt:28)
       at com.someapp.common.preferences.di.DataStoreModule_ProvideDataStoreFactory.provideDataStore(DataStoreModule_ProvideDataStoreFactory.java:50)
       at com.someapp.DaggerSomeApp_HiltComponents_SingletonC$SingletonCImpl$SwitchingProvider.get(DaggerSomeApp_HiltComponents_SingletonC.java:1320)
       at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
       at com.someapp.SomeApp_HiltComponents_SingletonC$SingletonCImpl$SwitchingProvider.get(DaggerSomeApp_HiltComponents_SingletonC.java:1317)
       at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
       at com.someapp.DaggerSomeApp_HiltComponents_SingletonC$SingletonCImpl.injectSomeApp2(DaggerSomeApp_HiltComponents_SingletonC.java:1264)
       at com.someapp.DaggerSomeApp_HiltComponents_SingletonC$SingletonCImpl.injectSomeApp(DaggerSomeApp_HiltComponents_SingletonC.java:1240)
       at com.someapp.Hilt_SomeApp.hiltInternalInject(Hilt_SomeApp.java:51)
       at com.someapp.Hilt_SomeApp.onCreate(Hilt_SomeApp.java:42)
       at com.someapp.SomeApp.onCreate(SomeApp.java:76)
       at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1211)
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6765)
       at android.app.ActivityThread.access$1600(ActivityThread.java:253)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2090)
       at android.os.Handler.dispatchMessage(Handler.java:106)
       at android.os.Looper.loopOnce(Looper.java:201)
       at android.os.Looper.loop(Looper.java:288)
       at android.app.ActivityThread.main(ActivityThread.java:7881)
       at java.lang.reflect.Method.invoke(Native method)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:568)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1045)

AFAIK we are only invoking this once on startup, i.e., during the creation of objects via Dagger/Hilt.

@BlueTurtle3
Copy link

We have noticed the same behaviour on Android 12 devices on native Android apps using our library. In our case we are creating an EncryptedSharedPreference object with dagger once on app initialisation.

Android 12
library: androidx.security:security-crypto:1.1.0-alpha06
Native apps

@Toubap
Copy link

Toubap commented Aug 14, 2024

Same here.
Updating tink as transitive dependency might help?

@juergw
Copy link
Contributor

juergw commented Aug 15, 2024

Sorry I haven't replied earlier. Thanks for the stack-trace, that is helpful.

Tink's AndroidKeystoreAesGcm.encrypt calls Android Keystore to encrypt. But here, Android Keystore is busy, and the thread keeps retrying until it causes an ANR. This is the code where it fails:
https://android.googlesource.com/platform/frameworks/base/+/refs/heads/main/keystore/java/android/security/KeyStoreSecurityLevel.java#207

So this is not a problem in Tink. The problem is that there are too many requests to Android Keystore at the same time.

I think the problem is that dagger tries to create several EncryptFile / EncryptedSharedPreferances objects in parallel in different threads, and all threads try to decrypt the encrypted keyset at the same time. And Android keystore can't handle so many requests.

I don't know dagger. Can you make sure that these objects are created in the same thread?

@Toubap
Copy link

Toubap commented Oct 17, 2024

I do not use Dagger, and the object is created only once in a kotlin lazy property. And still experiencing the issue

@juergw
Copy link
Contributor

juergw commented Oct 21, 2024

The main problem here is that both jetpack and Tink tried to give a too simple API to the user. That API works fine in most cases, but when something goes wrong, it is much more difficult for the user of the API to figure out what is happening.

We have decided to provide a more low-level API. It requires the user to write more code themselves, but I think it makes it easier to understand what is going on, which will make it easier to debug and maintain.

Here is the new API (this will be in the next release):
https://github.com/tink-crypto/tink-java/blob/main/src/main/java/com/google/crypto/tink/integration/android/AndroidKeystore.java

and here is how a user might use this API in their app:
https://github.com/tink-crypto/tink-java/blob/main/examples/android/helloworld/app/src/main/java/com/helloworld/TinkApplication.java

@viethoangtien
Copy link

viethoangtien commented Oct 31, 2024

Hi @juergw, now a new version 1.1.0-alpha06 of androidx security which includes Tink Java version 1.8.0.
Does this solve this issue?

@juergw
Copy link
Contributor

juergw commented Oct 31, 2024

No, it doesn't solve the issue. Note that androidx.security:security-crypto:1.1.0-alpha06 was released on April 19, 2023, so it's not new. And it has been deprecated in the mean time:
https://developer.android.com/privacy-and-security/cryptography#jetpack_security_crypto_library

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants