[prototype] Rewrite managed JNI names after R8 obfuscation

[simonrozsival]

Context

Fixes/advances #12535.

.NET for Android currently supplies -dontobfuscate to R8. Java shrinking and optimization are enabled, but Java names are not shortened because managed bindings contain the original JNI names as data. Those names are used at runtime to find Java classes, methods, and fields and to register managed native callbacks. If R8 changes the Java names while the managed strings retain their original values, JNI lookup fails.

This draft prototypes a build-time solution: let R8 choose the final Java names, consume its mapping.txt, and rewrite the corresponding JNI names in the managed assemblies before they are packaged.

The goal is to keep runtime JNI dispatch direct and allocation-free. The final application does not carry a mapping table and does not perform original-to-obfuscated name translation at runtime.

Initial assumptions and ideas

The investigation started with several possible designs:

1. Prescribe shortened names to R8. If the build generated stable short names itself, managed bindings and R8 could theoretically use the same names. R8's -applymapping helps reuse an existing mapping, but it does not preserve dynamically accessed members or make the build's independently generated names authoritative. R8 optimization and inlining also mean the output mapping is not always a simple one-to-one rename table. The prototype therefore treats R8's emitted mapping as the source of truth.

2. Ship the mapping and translate names at runtime. This would work without rewriting existing binding assemblies, and resembles the broad shape of managed Android remapping support. It would, however, add application data, runtime lookup cost, startup work, and complexity to every JNI operation. Since Release build time is not the optimization target here, the preferred design moved toward paying the cost once during the build.

3. Run R8 first, then rewrite assemblies before ILLink/ILC and typemap generation. This looked attractive because every downstream system would naturally observe the obfuscated names. In the current build graph, however, R8 depends on Java sources and configuration derived from the linked managed application, while native application configuration and compressed-assembly descriptors are generated before R8. The prototype consequently runs after R8 and rewrites all managed artifacts that are ultimately packaged, including generated trimmable typemap assemblies. Production integration still needs to make this ordering explicit and incremental.

4. Patch string heaps in place. The first mental model was that most changes would be substitutions of string literals. In-place patching is insufficient: obfuscated values can be longer, one metadata/user-string heap value can be shared by use sites needing different owner-specific rewrites, custom-attribute blobs encode lengths, IL tokens may need new user-string handles, and UTF-8 data can live in FieldRVA-backed structures. The implementation evolved into a two-pass planner plus complete PE reconstruction using System.Reflection.Metadata.

Mono.Cecil is intentionally not used.

Feature design

The prototype consists of an R8 mapping model, a use-site-aware rewrite planner, and a managed PE rebuilder.

1. Parse R8 output

R8Mapping parses class, field, method, and constructor mappings and exposes JNI-oriented lookups.

Real MAUI mappings changed the initial parser design in two important ways:

• R8 emits qualified inline call-frame records for retracing. These are not runtime member mappings for the current class and must be ignored.
• One source method can be inlined into multiple surviving destination methods. Such a mapping has no unique runtime name, so the parser records it as ambiguous rather than selecting a potentially incorrect name.

Class names are normalized to JNI slash-separated form while method parameter types retain the Java-source representation used by mapping.txt.

2. Plan exact use-site rewrites

JniRewritePlanner scans the complete assembly and records replacements by metadata handle or exact IL operand location rather than globally replacing string values.

It currently handles:

• RegisterAttribute JNI type and member names
• JNI type, method, and constructor signature attributes
• JniPeerMembers encoded method and field identifiers
• RegisterNatives / fast native-registration strings
• JNI descriptors, including every embedded parameter and return type
• generated native-method names and signatures stored as UTF-8 FieldRVA data
• direct JNIEnv lookup patterns where a referenced JNI class is followed by a member name and descriptor

The owner-sensitive plan matters because a shared string such as run.()V can map to different Java names depending on the declaring JNI type. If a shared FieldRVA datum would require conflicting values, the rewriter fails instead of silently producing an invalid assembly.

The direct JNIEnv case was discovered only after the first fully packaged app reached managed startup. Android.App.Application.Context finds net/dot/android/ApplicationRegistration, then asks for the static Context field using separate string operands. Rewriting only attributes and encoded JniPeerMembers strings left that field name unchanged and produced a real NoSuchFieldError. The planner now recognizes this class/member/descriptor pattern as well.

3. Rebuild the PE

AssemblyRebuilder reconstructs the managed PE with System.Reflection.Metadata.Ecma335 so replacements can have arbitrary lengths and distinct use sites can receive distinct values.

The reconstruction preserves:

• metadata table row ordering and tokens
• method bodies and exception regions
• signatures, constants, properties, events, generics, P/Invokes, and nested types
• managed resources
• native PE resources
• debug-directory data and portable PDB identity
• FieldRVA data and required alignment
• strong-name signature reservation

A rewritten strong-named assembly cannot retain its old signature. The prototype clears the signed flag while preserving the signature-directory reservation and reports that the assembly is left delay-signed. Authenticode data is omitted because any PE rewrite invalidates it; the enclosing APK is subsequently signed.

Assemblies with zero planned replacements are returned byte-for-byte unchanged. This optimization became a correctness requirement during device testing: rebuilding every framework assembly changed System.Private.CoreLib.dll after its expected decompression size had already been recorded. It also avoids stripping signatures and spending reconstruction time on the majority of assemblies.

4. Keep compressed-assembly metadata consistent

Release applications record each assembly's uncompressed size and descriptor index in generated native data. Rewriting can change an assembly's size, so the native descriptors must be regenerated before libxamarin-app.so is finalized.

Simply regenerating descriptors from the rewritten item list was incorrect because item ordering differed from the original list used by the assembly store. That associated sizes with the wrong descriptor indices; for example, CoreLib received a 2 KiB typemap assembly's size.

GenerateCompressedAssembliesNativeSourceFiles therefore accepts optional size-source assemblies. It reuses the already registered descriptor layout and indices, substitutes only the rewritten file sizes by package key and ABI, and emits updated native data. The controlled experiment then recompiles and relinks the application native library.

This is an important production integration constraint: the rewrite, compression-size regeneration, native recompilation, assembly compression, and packaging stages must agree on the same assembly identities and descriptor indices.

Current build flow demonstrated by the prototype

1. ILLink produces the trimmed managed application.
2. Trimmable typemap assemblies and Java sources are generated.
3. R8 shrinks, optimizes, and obfuscates Java bytecode and writes mapping.txt.
4. RewriteJniNamesForR8 scans and rewrites the packaged managed assemblies and generated typemap assemblies.
5. Compressed-assembly native metadata is regenerated using the original descriptor ordering and rewritten file sizes.
6. The application native library is recompiled/relinked.
7. Rewritten assemblies are compressed and packaged, and the APK is signed.

The task and supporting implementation are included in this PR, but the experiment-only MSBuild hook is deliberately not. This draft does not remove the shipped -dontobfuscate rule and does not yet enable the feature for product builds.

MAUI device validation

The prototype was tested with a Release dotnet new maui --sample-content application using:

• .NET 11 CoreCLR
• android-arm64
• API 35 arm64 emulator
• trimmable typemap
• R8 shrinking, optimization, and full name obfuscation
• ReadyToRun disabled

Artifact	Baseline	Obfuscated	Difference
Signed APK	16,062,517 bytes	15,677,493 bytes	-385,024 bytes (-2.40%)
classes.dex	5,049,332 bytes	4,228,304 bytes	-821,028 bytes
Compressed classes.dex	2,250,194 bytes	1,868,788 bytes	-381,406 bytes

Both APKs were rebuilt against the same matched managed/native runtime artifacts.

The final obfuscated APK:

• installed with a non-incremental streamed install
• cold-launched successfully
• remained alive after 12 seconds
• remained the top resumed activity
• rendered the MAUI sample-content UI
• produced no JNI, Java, managed, assembly-store, or native fatal errors
• verified with APK Signature Scheme v2 and v3

Tests

The focused suite contains 69 passing tests covering:

• real and synthetic managed PE reconstruction
• attributes and owner-specific user strings
• JNI descriptors and encoded peer-member identifiers
• native registration blocks
• UTF-8 FieldRVA data
• resource, exception-region, generic, P/Invoke, and token preservation
• strong-name and Authenticode behavior
• portable PDB compatibility
• runtime loading of rewritten assemblies
• no-op byte identity and in-place timestamp preservation
• R8 parser edge cases, including inline frames and ambiguous inlining
• native resource relocation and bounds checking

Xamarin.Android.Build.Tasks also builds successfully.

Open design and productization work

• Integrate the ordering into shipped targets without rerunning large native stages unnecessarily.
• Decide whether rewriting should occur in-place or in a dedicated project-local staging directory.
• Define the production strong-name re-signing strategy.
• Determine whether direct JNI lookup analysis needs broader IL dataflow support beyond the currently observed patterns.
• Add incremental build inputs/outputs and multi-ABI coverage.
• Validate Mono and NativeAOT paths where applicable.
• Remove -dontobfuscate only when the complete rewrite path is enabled.
• Consider whether existing runtime remapping remains useful as a compatibility fallback for assemblies that cannot be rewritten.