Implement SplitInstallService #2500

DaVinci9196 · 2024-08-19T11:25:03Z

Fix maps language modification and Fix Smart Lens translation language download

CoelacanthusHex · 2024-08-19T11:45:07Z

Will it fix #1629 and partially resolve #2225?

vending-app/src/main/AndroidManifest.xml

mar-v-in · 2024-08-19T17:32:16Z

This PR seems to be mixing completely unrelated features (Phenotype / Experiment Flags and SplitInstallService). I don't think SplitInstallService strictly needs Phenotype related functionality. Please clean this up so the PR only contains the SplitInstallService

vending-app/build.gradle

vending-app/src/main/kotlin/com/android/vending/ExperimentAndConfigs.kt

vending-app/src/main/kotlin/com/android/vending/PhenotypeDatabase.kt

DaVinci9196 · 2024-08-21T14:31:19Z

This PR seems to be mixing completely unrelated features (Phenotype / Experiment Flags and SplitInstallService). I don't think SplitInstallService strictly needs Phenotype related functionality. Please clean this up so the PR only contains the SplitInstallService

I submitted a new revision

...nearby/core/src/test/java/com/google/android/finsky/splitinstallservice/ExampleUnitTest.java

vending-app/src/main/AndroidManifest.xml

vending-app/src/main/java/org/microg/vending/billing/Constants.kt

mar-v-in · 2024-08-21T14:41:32Z

vending-app/src/main/kotlin/com/google/android/phonesky/header/PhoneskyHeaderValue.kt

+            context,
+            SettingsContract.CheckIn.getContentUri(context),
+            arrayOf(SettingsContract.CheckIn.ANDROID_ID)
+        ) { cursor: Cursor -> cursor.getLong(0) }
+
+        millis = System.currentTimeMillis()
+        timestamp
+            .container1Wrapper(
+                TimestampContainer1Wrapper.Builder()
+                    .androidId(androidId.toString())
+                    .container(
+                        TimestampContainer1.Builder()
+                            .timestamp(millis.toString() + "000")
+                            .wrapper(makeTimestamp(millis))
+                            .build()
+                    )
+                    .build()
+            )
+        val encodedTimestamps = String(
+            Base64.encode(
+                Util.encodeGzip(timestamp.build().encode()),
+                Base64.URL_SAFE or Base64.NO_WRAP or Base64.NO_PADDING
+            )
+        )
+        val locality = Locality.Builder()
+            .unknown1(1)
+            .unknown2(2)
+            .countryCode("")
+            .region(
+                TimestampStringWrapper.Builder()
+                    .string("")
+                    .timestamp(makeTimestamp(System.currentTimeMillis())).build()
+            )
+            .country(
+                TimestampStringWrapper.Builder()
+                    .string("")
+                    .timestamp(makeTimestamp(System.currentTimeMillis())).build()
+            )
+            .unknown3(0)
+            .build()
+        val encodedLocality = String(
+            Base64.encode(locality.encode(), Base64.URL_SAFE or Base64.NO_WRAP or Base64.NO_PADDING)
+        )
+        val header = LicenseRequestHeader.Builder()
+            .encodedTimestamps(StringWrapper.Builder().string(encodedTimestamps).build())
+            .triple(
+                EncodedTripleWrapper.Builder().triple(
+                    EncodedTriple.Builder()
+                        .encoded1("")
+                        .encoded2("")
+                        .empty("")
+                        .build()
+                ).build()
+            )
+            .locality(LocalityWrapper.Builder().encodedLocalityProto(encodedLocality).build())
+            .unknown(IntWrapper.Builder().integer(5).build())
+            .empty("")
+            .languages(externalxpsrh)
+            .deviceMeta(
+                DeviceMeta.Builder()
+                    .android(
+                        AndroidVersionMeta.Builder()
+                            .androidSdk(org.microg.gms.profile.Build.VERSION.SDK_INT)
+                            .buildNumber(org.microg.gms.profile.Build.ID)
+                            .androidVersion(org.microg.gms.profile.Build.VERSION.RELEASE)
+                            .unknown(0)
+                            .build()
+                    )
+                    .unknown1(UnknownByte12.Builder().bytes(ByteString.EMPTY).build())
+                    .unknown2(1)
+                    .build()
+            )
+            .userAgent(
+                UserAgent.Builder()
+                    .deviceName(org.microg.gms.profile.Build.DEVICE)
+                    .deviceHardware(org.microg.gms.profile.Build.HARDWARE)
+                    .deviceModelName(org.microg.gms.profile.Build.MODEL)
+                    .finskyVersion(FINSKY_VERSION)
+                    .deviceProductName(org.microg.gms.profile.Build.MODEL)
+                    .androidId(androidId) // must not be 0
+                    .buildFingerprint(org.microg.gms.profile.Build.FINGERPRINT)
+                    .build()
+            )
+            .uuid(
+                Uuid.Builder()
+                    .uuid(UUID.randomUUID().toString())
+                    .unknown(2)
+                    .build()
+            )
+            .build().encode()
+        return String(Base64.encode(Util.encodeGzip(header), Base64.URL_SAFE or Base64.NO_WRAP or Base64.NO_PADDING))
+    }
+
+    private fun getHeaders(): Map<String, String> {
+        headerMap["X-PS-RH"] = getXHeaders()
+        headerMap["Authorization"] = "Bearer " + AccountManager.get(context).getAuthToken(
+            user, tokenType, null, false, null, null
+        ).result.getString(AccountManager.KEY_AUTHTOKEN)
+        return this.headerMap
+    }


This seems to mostly overlap with getLicenseRequestHeaders in LicenseRequestHeaders.kt. Maybe you can refactor this into a common codebase.

… downloaded

DaVinci9196 · 2024-08-28T13:25:09Z

We modified it according to your opinion

fynngodau

I've had a look at the code and provide some suggestions for improvement. Note that I haven't tested the changes and thus can't tell whether they are working as intended.

fynngodau · 2024-09-10T11:30:32Z

vending-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/extensions.kt

+    get(url = requestUrl, headers = getLicenseRequestHeaders(auth, 1).toMutableMap().apply {
+        val xPsRh = String(
+            Base64.encode(
+                getDefaultLicenseRequestHeaderBuilder(1).languages(RequestLanguagePackage.Builder().language(requestLanguagePackage).build()).build().encode().encodeGzip(),
+                Base64.URL_SAFE or Base64.NO_WRAP or Base64.NO_PADDING
+            )
+        )
+        put("X-PS-RH", xPsRh)


I see no reason why adding the language field should not be integrated with the code LicenseRequestHeaders (perhaps as an optional parameter). Then you wouldn't have to calculate the X-PS-RH value again here. Also, probably the file and methods in LicenseRequestHeaders should be renamed and pulled out of the .license package, considering they are now not only used for license checking (as well as renaming the corresponding proto files).

fynngodau · 2024-09-10T12:02:34Z

vending-app/src/main/res/values/strings.xml

@@ -28,4 +28,5 @@
    <string name="tips_forget_passwd">Forget password?</string>
    <string name="tips_more_details">Learn more</string>
    <string name="text_verify_button">Verify</string>
+    <string name="split_install">%s Downloading subpackages</string>


This is not very descriptive (and also not well-formatted). I would suggest

Suggested change

<string name="split_install">%s Downloading subpackages</string>

<string name="split_install">Downloading required components for %s</string>

fynngodau · 2024-09-10T12:09:59Z

vending-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/extensions.kt

+    sendCompleteBroad(context, intent)
+}
+
+private suspend fun getDownloadUrls(context: Context, httpClient: HttpClient, packageName: String, langName: Array<String>, splitName: Array<String>): ArrayList<Array<String>> {


I would change the return type to either List<Pair<String,String>> or even List<DownloadUrl> with DownloadUrl as either a typealias to Pair<String,String> or as a separate type.

You can then make the code that adds to the list easier to read using a to b instead of arrayOf(a, b), and the compiler guarantees that each item has exactly two values.

fynngodau · 2024-09-10T12:20:56Z

vending-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/extensions.kt

+    if (lastSplitPackageName != null && lastSplitPackageName != pkg) {
+        if (mutex.isLocked) mutex.unlock()


When does this case occur?

fynngodau · 2024-09-10T12:27:15Z

vending-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/extensions.kt

+
+private suspend fun installSplitPackage(context: Context, httpClient: HttpClient, downloadUrl: ArrayList<Array<String>>, packageName: String, language: String?): Intent {
+    Log.d(TAG, "installSplitPackage downloadUrl: ${downloadUrl.firstOrNull()}")
+    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) {


This check is only executed after calling getDownloadUrls. I would suggest performing the check earlier in the flow, and annotating relevant methods with @RequiresApi(Build.VERSION_CODES.LOLLIPOP).

fynngodau · 2024-09-10T12:33:36Z

...ing-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallService.kt

+    override fun startInstall(pkg: String, splits: List<Bundle>, bundle0: Bundle, callback: ISplitInstallServiceCallback) {
+        Log.d(TAG, "Method <startInstall> Called by package: $pkg")
+        lifecycleScope.launch {
+            trySplitInstall(context, httpClient, pkg, splits)
+            Log.d(TAG, "onStartInstall SUCCESS")
+            callback.onStartInstall(CommonStatusCodes.SUCCESS, Bundle())


It seems to me like the identity of the caller is not verified. I would expect packages are only allowed to request split installs for themselves, right? We should ensure that pkg is indeed the caller themselves by calling PackageUtils.getAndCheckCallingPackage.

DaVinci9196 · 2024-09-12T09:32:53Z

@fynngodau Accepted, thanks for the suggestion.
In practice, the packageName provided by the calling method cannot be verified by PackageUtils.getAndCheckCallingPackage to be the same calling

fynngodau · 2024-09-14T15:25:16Z

Thanks for providing us with a new iteration!

I now noticed that HeaderProvider and com.android.vending.extensions are partially overlapping and duplicate, and the latter is not well-integrated with the classes (AuthData and DeviceEnvInfo) used by the former. Would it be possible to harmonize and merge the two files? Though I see the difficulty considering that some calls seem to want an x-ps-rh header and others do not.

Is there a reason you are using 1 instead of the real (microG) Android ID in your queries?

In practice, the packageName provided by the calling method cannot be verified by PackageUtils.getAndCheckCallingPackage to be the same calling

I can't see why it wouldn't work the way you tried, but maybe @mar-v-in has an idea.

jonathanklee · 2024-09-16T11:29:28Z

...ing-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallManager.kt

+            val intent = Intent(context, InstallResultReceiver::class.java).apply {
+                putExtra(KEY_BYTES_DOWNLOADED, totalDownloaded)
+            }
+            val pendingIntent = PendingIntent.getBroadcast(context, sessionId, intent, 0)


Suggested change

val pendingIntent = PendingIntent.getBroadcast(context, sessionId, intent, 0)

val pendingIntent = PendingIntent.getBroadcast(context, sessionId, intent, PendingIntent.FLAG_UPDATE_CURRENT | PendingIntent.FLAG_IMMUTABLE)

@jonathanklee With FLAG_IMMUTABLE, I am not receiving any response values in the InstallResultReceiver, so the receiver can't detect if installation was successful or not. So FLAG_MUTABLE seems better.

jonathanklee · 2024-09-16T11:32:04Z

...ing-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallManager.kt

+    private suspend fun requestDownloadUrls(callingPackage: String, authToken: String, packs: MutableSet<String>): ArraySet<Triple<String, String, Int>> {
+        val versionCode = PackageInfoCompat.getLongVersionCode(context.packageManager.getPackageInfo(callingPackage, 0))
+        val requestUrl =
+            StringBuilder("https://play-fe.googleapis.com/fdfe/delivery?doc=$callingPackage&ot=1&vc=$versionCode&bvc=$versionCode&pf=1&pf=2&pf=3&pf=4&pf=5&pf=7&pf=8&pf=9&pf=10&da=4&bda=4&bf=4&fdcf=1&fdcf=2&ch=")


I tried to execute the split installations from TikTok, but I wasn't able to download the split APKs (the url triples were always null) until I changed the URL to:

https://play-fe.googleapis.com/fdfe/delivery?doc=$callingPackage&ot=1&vc=$versionCode

so removing the last part:

&bvc=$versionCode&pf=1&pf=2&pf=3&pf=4&pf=5&pf=7&pf=8&pf=9&pf=10&da=4&bda=4&bf=4&fdcf=1&fdcf=2&ch=

What app did you use to test the split install feature ?

When testing with TikTok, I face the following error;

SplitInstallManager: InstallResultReceiver onReceive: install fail -> INSTALL_FAILED_MISSING_SPLIT: Missing split for com.zhiliaoapp.musically

@DaVinci9196 I was able to install a language split apk on PayPal 👍

I tried to execute the split installations from TikTok, but I wasn't able to download the split APKs (the url triples were always null) until I changed the URL to:

https://play-fe.googleapis.com/fdfe/delivery?doc=$callingPackage&ot=1&vc=$versionCode

so removing the last part:

&bvc=$versionCode&pf=1&pf=2&pf=3&pf=4&pf=5&pf=7&pf=8&pf=9&pf=10&da=4&bda=4&bf=4&fdcf=1&fdcf=2&ch=

What app did you use to test the split install feature ?

Tested by switching languages in Google Maps and Bolt

fynngodau · 2024-09-21T15:58:44Z

vending-app/src/main/java/org/microg/vending/billing/core/HttpClient.kt

+                        throw IOException("Failed to create directories: ${parentDir.absolutePath}")
+                    }
+                    val fos = FileOutputStream(downloadFile)
+                    fos.write(response.data)


Volley loads the entire response into memory. This is a problem for large downloads, as it will consume a lot of RAM (temporarily – or even fail the download if not enough memory is available). The data from the network stream should be written to disk immediately instead.

Per homepage:

Volley is not suitable for large download or streaming operations, since Volley holds all responses in memory during parsing.

fynngodau · 2024-09-21T16:05:39Z

...ing-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallManager.kt

+    private fun updateSplitInstallRecord(callingPackage: String, triple: Triple<String, String, Int>) {
+        splitInstallRecord[callingPackage]?.let { triples ->
+            val find = triples.find { it.first == triple.first }
+            find?.let { triples.remove(it) }
+            triples.add(triple)
+        } ?: run {
+            val triples = ArraySet<Triple<String, String, Int>>()
+            triples.add(triple)
+            splitInstallRecord[callingPackage] = triples
+        }
+    }


This method would be entirely unnecessary if splitInstallRecord had the data type MutableMap<String, Map<PackageComponent, DownloadStatus>> instead of HashMap<String, ArraySet<Triple<String, String, Int>>>, with these new classes:

data class PackageComponent( val componentName: String, val url: String ) enum class DownloadStatus { PENDING, FAILED, COMPLETE }

Or even simpler, a MutableMap<PackageComponent, DownloadStatus> if PackageComponent additionally contained the package's name.

You can find my suggestions here: https://github.com/e-foundation/android_packages_apps_GmsCore/blob/c3398b989ab1a01aaf0bed7a6900cc0f46225ad6/vending-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallManager.kt

fynngodau · 2024-09-21T16:30:51Z

...ing-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallManager.kt

+            )
+        }
+        NotificationCompat.Builder(context, NOTIFY_CHANNEL_ID).setSmallIcon(android.R.drawable.stat_sys_download)
+            .setContentTitle(context.getString(R.string.split_install, context.getString(R.string.app_name))).setPriority(NotificationCompat.PRIORITY_DEFAULT)


The placeholder in R.string.split_install should be filled in with the downloading app's name, not with microG's name.

fynngodau · 2024-09-21T20:10:16Z

...ing-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallManager.kt

+            }
+    }
+
+    private fun Context.splitSaveFile() = File(filesDir, FILE_SAVE_PATH)


Installed packages are deleted after installation, but if installation fails and never retried (for instance: one large package is downloaded successfully, then a small package is downloaded unsuccessfully due to a network error, then the user uninstalls the app because it doesn't work), this can still lead to storage leaks from downloaded-but-never-installed apps. We should use cacheDir instead: it will allow the system and users to clear the cache themselves.

fynngodau · 2024-09-22T09:41:23Z

...ing-app/src/main/kotlin/com/google/android/finsky/splitinstallservice/SplitInstallManager.kt

+            putInt(KEY_ERROR_CODE, 0)
+            putInt(KEY_SESSION_ID, 0)
+            putLong(KEY_TOTAL_BYTES_TO_DOWNLOAD, intent.getLongExtra(KEY_BYTES_DOWNLOADED, 0))
+            putString(KEY_LANGUAGES, intent.getStringExtra(KEY_LANGUAGE))


It doesn't seem like this intent would have this field, given that it is not set upon creation of the intent in installSplitPackage.

fynngodau · 2024-09-22T11:30:48Z

I opened PR #2553, which contains the code from this PR; I performed some major refactoring to SplitInstallService there.

SplitInstallService Additions

34ccbe2

DaVinci9196 added 2 commits August 19, 2024 21:33

add permission

5e6f01e

ISplitInstallServiceCallback added

1962f33

ale5000-git reviewed Aug 19, 2024

View reviewed changes

vending-app/src/main/AndroidManifest.xml Outdated Show resolved Hide resolved

grote reviewed Aug 20, 2024

View reviewed changes

vending-app/build.gradle Outdated Show resolved Hide resolved

vending-app/src/main/kotlin/com/android/vending/ExperimentAndConfigs.kt Outdated Show resolved Hide resolved

vending-app/src/main/kotlin/com/android/vending/PhenotypeDatabase.kt Outdated Show resolved Hide resolved

remove Phenotype / Experiment

c9bfbc0

mar-v-in requested changes Aug 21, 2024

View reviewed changes

DaVinci9196 and others added 4 commits August 22, 2024 09:24

Merge branch 'microg:master' into split_install_service

796c3ab

Optimizing the code

e8a0058

SplitInstallService add call super.onBind

b8d2b20

Fixed the issue that multiple versions of language packs could not be…

6cdd488

… downloaded

mar-v-in added this to the 0.3.4 milestone Aug 27, 2024

DaVinci9196 closed this Aug 27, 2024

DaVinci9196 deleted the split_install_service branch August 27, 2024 11:48

DaVinci9196 restored the split_install_service branch August 27, 2024 11:48

DaVinci9196 reopened this Aug 27, 2024

DaVinci9196 added 2 commits August 27, 2024 19:56

Formatting Code

36651db

Formatting Code

ff837d9

DaVinci9196 changed the title ~~SplitInstallService Additions~~ Implement AssetModuleService Aug 27, 2024

DaVinci9196 changed the title ~~Implement AssetModuleService~~ Implement SplitInstallService Aug 27, 2024

fynngodau suggested changes Sep 10, 2024

View reviewed changes

Execute processing suggestions

8a68044

cleanup

5a9f1b3

jonathanklee reviewed Sep 16, 2024

View reviewed changes

fynngodau reviewed Sep 21, 2024

View reviewed changes

fynngodau reviewed Sep 22, 2024

View reviewed changes

fynngodau mentioned this pull request Sep 22, 2024

Work app store #2553

Open

11 tasks

mar-v-in removed this from the 0.3.4 milestone Sep 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement SplitInstallService #2500

Implement SplitInstallService #2500

DaVinci9196 commented Aug 19, 2024

CoelacanthusHex commented Aug 19, 2024

mar-v-in commented Aug 19, 2024

DaVinci9196 commented Aug 21, 2024

mar-v-in Aug 21, 2024

DaVinci9196 commented Aug 28, 2024

fynngodau left a comment

fynngodau Sep 10, 2024

fynngodau Sep 10, 2024

fynngodau Sep 10, 2024

fynngodau Sep 10, 2024

fynngodau Sep 10, 2024

fynngodau Sep 10, 2024

DaVinci9196 commented Sep 12, 2024

fynngodau commented Sep 14, 2024 •

edited

Loading

jonathanklee Sep 16, 2024

fynngodau Sep 21, 2024

jonathanklee Sep 16, 2024 •

edited

Loading

jonathanklee Sep 16, 2024

jonathanklee Sep 16, 2024

DaVinci9196 Sep 18, 2024

fynngodau Sep 21, 2024 •

edited

Loading

fynngodau Sep 21, 2024 •

edited

Loading

fynngodau Sep 21, 2024

fynngodau Sep 21, 2024

fynngodau Sep 21, 2024

fynngodau Sep 22, 2024

fynngodau commented Sep 22, 2024

	<string name="split_install">%s Downloading subpackages</string>
	<string name="split_install">Downloading required components for %s</string>

		if (lastSplitPackageName != null && lastSplitPackageName != pkg) {
		if (mutex.isLocked) mutex.unlock()

	val pendingIntent = PendingIntent.getBroadcast(context, sessionId, intent, 0)
	val pendingIntent = PendingIntent.getBroadcast(context, sessionId, intent, PendingIntent.FLAG_UPDATE_CURRENT \| PendingIntent.FLAG_IMMUTABLE)

Implement SplitInstallService #2500

Are you sure you want to change the base?

Implement SplitInstallService #2500

Conversation

DaVinci9196 commented Aug 19, 2024

CoelacanthusHex commented Aug 19, 2024

mar-v-in commented Aug 19, 2024

DaVinci9196 commented Aug 21, 2024

Choose a reason for hiding this comment

DaVinci9196 commented Aug 28, 2024

fynngodau left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DaVinci9196 commented Sep 12, 2024

fynngodau commented Sep 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathanklee Sep 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fynngodau Sep 21, 2024 • edited Loading

Choose a reason for hiding this comment

fynngodau Sep 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fynngodau commented Sep 22, 2024

fynngodau commented Sep 14, 2024 •

edited

Loading

jonathanklee Sep 16, 2024 •

edited

Loading

fynngodau Sep 21, 2024 •

edited

Loading

fynngodau Sep 21, 2024 •

edited

Loading