[runtime] Move JNI remapping lookup to managed code - #12796
Closed
simonrozsival wants to merge 20 commits into
Closed
simonrozsival wants to merge 20 commits into
simonrozsival wants to merge 20 commits into
Conversation
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The stale NativeAOT source entry and missing managed lookup regression coverage must be addressed.
Get a fresh assessment by requesting another Copilot review.
Review tier: Lite
Findings: 1
Open (2)
What changed in this PR
This pull request moves JNI remapping lookups from native runtime code into managed code across Mono, CoreCLR, and NativeAOT.
Changes:
- Adds shared remapping metadata and managed UTF-8 binary-search lookups.
- Passes remapping data through runtime initialization.
- Removes obsolete native lookup implementations and P/Invoke registrations.
- Updates generators, runtime stubs, build files, and tests.
| File | Summary |
|---|---|
src/Xamarin.Android.Build.Tasks/Utilities/JniRemappingAssemblyGenerator.cs |
Emits aggregate remapping data. |
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/GenerateJniRemappingNativeCodeTests.cs |
Updates generated-symbol and ordering assertions. |
src/native/nativeaot/include/runtime-base/internal-pinvokes.hh |
Updates NativeAOT internal P/Invoke declarations. |
src/native/nativeaot/host/jni-remapping-tables-stub.cc |
Supplies NativeAOT remapping table stubs. |
src/native/nativeaot/host/host.cc |
Wires remapping data into the NativeAOT host. |
src/native/nativeaot/host/CMakeLists.txt |
Updates NativeAOT build sources. |
src/native/native.targets |
Updates runtime inputs; retains a stale deleted-header entry. |
src/native/mono/xamarin-app-stub/xamarin-app.hh |
Updates the Mono application interface. |
src/native/mono/xamarin-app-stub/application_dso_stub.cc |
Provides Mono remapping data. |
src/native/mono/runtime-base/internal-pinvokes.hh |
Updates Mono internal P/Invoke declarations. |
src/native/mono/pinvoke-override/pinvoke-tables.include |
Updates generated P/Invoke table entries. |
src/native/mono/pinvoke-override/generate-pinvoke-tables.cc |
Updates P/Invoke table generation. |
src/native/mono/monodroid/monodroid-glue.cc |
Wires remapping data during initialization. |
src/native/mono/monodroid/jni-remapping.hh |
Removes obsolete native remapping declarations. |
src/native/mono/monodroid/jni-remapping.cc |
Removes the native lookup implementation. |
src/native/mono/monodroid/internal-pinvokes.cc |
Removes obsolete native P/Invoke implementations. |
src/native/mono/monodroid/CMakeLists.txt |
Updates Mono build sources. |
src/native/common/include/managed-interface.hh |
Defines the shared remapping-data ABI. |
src/native/clr/xamarin-app-stub/application_dso_stub.cc |
Provides CoreCLR remapping data. |
src/native/clr/runtime-base/jni-remapping.cc |
Removes the CoreCLR native lookup implementation. |
src/native/clr/runtime-base/CMakeLists.txt |
Updates CoreCLR build sources. |
src/native/clr/pinvoke-override/precompiled.cc |
Updates P/Invoke override compilation. |
src/native/clr/include/xamarin-app.hh |
Updates the CoreCLR application interface. |
src/native/clr/include/runtime-base/jni-remapping.hh |
Removes the obsolete remapping header. |
src/native/clr/include/runtime-base/internal-pinvokes.hh |
Updates CoreCLR internal P/Invoke declarations. |
src/native/clr/host/internal-pinvokes-shared.cc |
Updates shared host P/Invoke wiring. |
src/native/clr/host/host.cc |
Wires shared remapping data into the host. |
src/Mono.Android/Microsoft.Android.Runtime/JniRemappingLookup.cs |
Implements managed remapping lookups; runtime coverage is still needed. |
src/Mono.Android/Android.Runtime/RuntimeNativeMethods.cs |
Removes obsolete P/Invoke declarations. |
src/Mono.Android/Android.Runtime/JNIEnvInit.cs |
Passes remapping data during initialization. |
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 15, 2026 19:49
05ff47f to
5c655e9
Compare
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 16, 2026 08:39
5c655e9 to
bc65226
Compare
simonrozsival
removed this pull request from stack #12797
September 16, 2026 08:40
simonrozsival
added this pull request to stack #12805
September 16, 2026 08:40
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
3 times, most recently
from
September 16, 2026 17:58
477ca05 to
3203cc6
Compare
32 tasks
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 17, 2026 14:30
accf258 to
2e82c37
Compare
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 18, 2026 09:20
2e82c37 to
8a702d3
Compare
Carry stable UTF-8 replacement type and method-name pointers through remapping results and use them directly for JNI lookup while preserving string fallbacks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Document the unmanaged memory ownership and lifetime contract for remapping pointers, and exercise pointer-backed instance/static method lookups with signature fallback in the JVM tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Carry replacement type, method name, and signature pointers through JniPeerMembers and JNI lookup without eagerly decoding native memory. Preserve string-based compatibility paths and defer decoding to explicit diagnostics and Debug metadata access. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Require replacement pointers to remain stable for the associated runtime because cached JNI metadata may retain them, and use UTF-8-first precedence when formatting failed lookup diagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The UTF-8-signature compatibility case relies on the test fixture's custom JniTypeManager, which Android does not support. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the mixed representation coverage in the host type manager while registering the equivalent generated remap for Android test runs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 18, 2026 09:22
8a702d3 to
4b1489e
Compare
Return string and UTF-8 replacement representations from one virtual lookup so pointer-aware managers can report handled misses without triggering a duplicate string-table search. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Route string replacement queries through the combined lookup and make non-zero UTF-8 pointers authoritative when an override supplies both representations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Emit the existing Intune remapping tables as a self-describing data block, pass it through JNIEnv initialization, and share UTF-8 binary lookup across MonoVM and CoreCLR without per-call P/Invokes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use System.Text.Ascii for the common UTF-8/UTF-16 comparison path and a streaming Rune fallback for non-ASCII names. Keep replacement types and exact method signatures in generated native memory, avoid source metadata strings, and cache generator UTF-8 sort keys. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exercise allocation-free UTF-8/UTF-16 comparison for a non-ASCII replacement type and method name through the CoreCLR device remapping path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Compare UTF-8 and UTF-16 ASCII names in vector-sized chunks so binary-search ordering rescans at most one chunk after Ascii.Equals reports a mismatch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The native assembler populates the JniRemappingData table pointers during code generation, so suppress CS0649 for those two fields just like the existing generated methods pointer. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Update the Xamarin.Forms CoreCLR R8 Release APK description from the matching failed CI test attachment. macOS and Windows produced the same package size; their assembly stores differed by only 16 bytes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use the generator UTF-8 sort-key cache when computing serialized string lengths so each distinct remapping string is encoded only once. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use Encoding.UTF8.GetByteCount for serialized lengths so signatures and other non-sort values are not retained in the sort-key cache. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Override the combined replacement-type lookup in both Android type managers so generated UTF-8 table misses do not fall through into a second compatibility string search. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 18, 2026 09:53
4b1489e to
2ab1f9c
Compare
Base automatically changed from
simonrozsival-optimize-jni-remapping
to
main
September 18, 2026 19:13
simonrozsival
added a commit
that referenced
this pull request
Sep 18, 2026
## Motivation The existing Intune/MAM JNI method-remapping path stores target type and method names as stable NUL-terminated UTF-8 strings in generated native data. The old path materialized those names as managed UTF-16 strings and then encoded them back to UTF-8 for `FindClass` and `GetMethodID`. That round trip is unnecessary and becomes more important as remapping is reused by larger consumers such as R8. ## Approach - Add pointer-backed target type, method-name, and method-signature values to `ReplacementMethodInfo`. - Keep pointer and string representations independent so reading a compatibility string property does not implicitly decode native memory. - Add `JniType` lookup paths for pointer/pointer and mixed pointer/span member names and signatures. - Let `JniPeerMembers` retain a replacement type pointer and use it directly for `FindClass` and later member-remapping lookups. - Keep native pointers in `JniMethodInfo` Debug metadata and decode them only if `Name`, `Signature`, or `ToString()` is explicitly requested. - Retain the existing string/span paths for custom `JniTypeManager` implementations. - Document UTF-8 encoding, NUL termination, ownership, and lifetime requirements for every pointer API. The successful generated-remapping path therefore passes the pregenerated UTF-8 type, method name, and optional signature directly to JNI without copying them or converting them to a managed string. This PR is method-only and does not add R8, field remapping, reverse-type mapping, inherited-member fallback, or NativeAOT remapping support. ## Relationship to other PRs - This PR is based directly on `main` and provides the Java.Interop representation and JNI lookup primitives. - #12796 supplies the generated pointers, performs managed table search, and removes the native remapping P/Invokes. - #12692 can build on this stack to add the R8-specific field, reverse-type, inherited-member, and NativeAOT pieces. ## Validation - Java.Interop Debug build. - Java.Interop `JniPeerMembersTests`: 12 passed, 1 skipped. - The JVM fixture exercises stable unmanaged UTF-8 type/name/signature storage, signature fallback, instance/static lookup, and instance-to-static remapping.
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 18, 2026 19:13
2ab1f9c to
25652d2
Compare
Member
Author
|
/review |
Contributor
|
✅ Android PR Reviewer completed successfully!
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
⚠️ Needs Changes
Findings: 0 errors, 0 warnings, 1 suggestion.
The managed/native ABI stays aligned across MonoVM and CoreCLR, the generated UTF-8 ordering matches the managed binary-search comparator, and stable generated pointers preserve the required Java.Interop lifetimes. The added device scenario also validates non-ASCII type and method remapping. I left one inline suggestion for focused 16-byte comparator-boundary coverage.
CI is not yet complete: 41 checks have passed, 2 are running, and the aggregate dotnet-android check is queued. No failing checks were reported, but this cannot be considered green until the remaining matrix finishes.
Generated by Android PR Reviewer for #12796 · copilot · gpt56 · 272.9 AIC · ⌖ 9.79 AIC · ⊞ 17.6K
Comment /review to run again
Exercise 15-, 16-, and 17-byte method names and adjacent names that first differ in the second comparison chunk through the generated remapping table and JNI lookup path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
force-pushed
the
simonrozsival-managed-jni-remapping
branch
from
September 19, 2026 04:28
25652d2 to
d338ba5
Compare
Member
Author
|
/review |
Contributor
|
✅ Android PR Reviewer completed successfully!
|
Contributor
There was a problem hiding this comment.
⚠️ Needs Changes
Findings: 0 errors, 0 warnings, 1 suggestion.
The managed/native ABI remains aligned, the UTF-8 comparison and binary-search paths preserve the generated ordering, pointer lifetimes are documented and backed by process-lifetime generated data, and all 44 CI checks passed. I left one inline testing suggestion: the new aggregate log assertions do not prove that each boundary source name maps to its intended target, so a swapped second-chunk result can pass unnoticed.
Generated by Android PR Reviewer for #12796 · copilot · gpt56 · 230.7 AIC · ⌖ 8.26 AIC · ⊞ 18.1K
Comment /review to run again
Require the ASCII chunk-boundary target messages to appear in invocation order so swapped adjacent lookup results cannot satisfy the regression test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Member
Author
simonrozsival
added a commit
that referenced
this pull request
Oct 1, 2026
Depends on #12846. This is the consolidated runtime foundation for the R8 runtime-remapping stack. It supersedes the earlier split runtime implementation in #12796 and #12817 without abandoning that workstream. It adds the generic JNI remapping support shared by Java.Interop, Mono.Android, MonoVM, CoreCLR, and NativeAOT, including forward/reverse type remapping, descriptor-aware method and constructor remapping, field remapping, Java hiding/fallback behavior, and focused tests. The generated application object contains read-only remapping tables and a single `jni_remapping_data` descriptor. Native startup passes that descriptor to managed initialization; type, reverse-type, method, and field lookup algorithms remain in `JniRemappingLookup.cs`. No C++ remapping lookup implementation or lookup P/Invokes are introduced. This PR intentionally contains no R8 build orchestration and no managed assembly rewriting. Producer-side R8 mapping ingestion and generated-table wiring are in the next stack layer, #12848; public mode and build orchestration follow in #12692, with the modern task-assembly boundary in #12844. ## APK size and disabled-remapping footprint The measured Simple/CoreCLR baselines were refreshed from the complete test attachments in [CI build 1618183](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1618183): | Configuration | Previous APK bytes | Current APK bytes | Change | |---|---:|---:|---:| | Without R8 | 6,526,395 | 6,530,491 | +4,096 (0.063%) | | With R8 | 6,526,395 | 6,526,395 | 0 | The test failure was the **per-file** threshold on `libxamarin-app.so`, not a large APK regression: that library grew from 11,744 to 12,368 bytes (+624, 5.05%). Isolated re-links using the archived CI object files attribute this exactly: - **400 bytes** for the expanded empty remapping ABI: the 48-byte descriptor, reverse-type and field placeholders, and their ELF symbol/hash/string/relocation bookkeeping. - **224 bytes** for the two runtime configuration entries that explicitly disable remapping. Removing only these entries produces 12,144 bytes; replacing only the remapping object with the previous two-table layout produces 11,968 bytes. Removing both reproduces the previous 11,744-byte library. The `.text` section remains **36 bytes** in all four variants; this is fixed data/configuration overhead, not added remapping executable code. For this no-remapping application, all four table counts are zero and both remapping switches are configured `false`. Inspection of the actual linked **`Mono.Android.Runtime.dll`** confirms that `JniRemappingLookup` is absent. The earlier approximately 100 KiB cost from retaining the managed remapping implementation has not returned. The full baseline comparison also crosses the **.NET 11 RC2 to .NET 12 alpha update inherited from `main` in #12939**, so its other changes must not be attributed wholesale to this PR. The APK's CoreCLR, JIT, globalization, and System.Native binaries are byte-identical to the new runtime pack; the cached previous runtime pack matches the old reference sizes. For example, `libcoreclr.so` shrank 103,792 uncompressed bytes, while `libclrjit.so` grew 15,896 bytes. The assembly store grew 23,880 bytes and `libmonodroid.so` shrank 2,896 bytes. Summed across all entries, **uncompressed contents actually shrank 65,868 bytes**. APK size measures the **compressed and signed archive**, not the sum of those uncompressed sizes. The current APK uses DEFLATE for its native libraries, an 8 KiB signing block, and 4 KiB signing alignment. The nearest retained pre-upgrade comparison, [CI build 1617831](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1617831), shows the larger assembly store (+23,216 compressed bytes) and JIT (+6,970) almost offset by the smaller CoreCLR (-28,694) and other entries: compressed payload grows just **1,034 bytes**, while ZIP/signing/alignment overhead grows **3,062 bytes**, producing the observed **4 KiB APK step**. `libxamarin-app.so` itself adds only **107 compressed bytes** in that comparison. **Comparison limitation:** the exact reference APK was generated locally before the runtime update and was not retained. Build 1617831 has slightly different dex/store/monodroid entries, although its total APK size matches the reference. The compressed/padding breakdown is therefore an explicitly identified historical CI comparison, not an exact reconstruction of the local reference or a same-toolchain `main`-versus-PR A/B test. The isolated 400/224-byte native attribution and removal of the managed lookup are independently confirmed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The existing Intune/MAM JNI remapping implementation performs a managed-to-native transition for every type and method lookup and maintains separate linear-search implementations in MonoVM and CoreCLR. Native replacement strings are then decoded to managed UTF-16 and encoded back to UTF-8 before JNI receives them.
The generated tables are already sorted and live for the process lifetime. Shared managed lookup can read them directly, avoid the repeated P/Invoke boundary, and keep successful remaps in native UTF-8 memory through JNI.
Approach
jni_remapping_datastructure directly in generated LLVM IR. It contains the existing type-replacement and method-index table pointers and counts.JnienvInitializeArgsinstead of a boolean remapping flag.System.Text.Ascii.IsValid()and chunkedAscii.Equals(ReadOnlySpan<byte>, ReadOnlySpan<char>)handle the common ASCII path without transcoding or allocation.Rune.DecodeFromUtf8()/Rune.DecodeFromUtf16()fallback preserves correct ordering and equality for non-ASCII names without creating a full converted buffer.SourceJniMethodNameorSourceJniMethodSignature; the requesting spans remain available to the caller for fallback and diagnostics.The search is not hash-based. Tables are ordered lexicographically by raw UTF-8 bytes. UTF-8 byte order and Unicode scalar order are equivalent, so the mixed UTF-8/UTF-16 comparator preserves the generated sort order.
Scope and relationship to #12692
This PR is based on #12795 and covers the existing Intune/MAM type and method remapping contract. It intentionally does not contain R8-specific field remapping, reverse-type mapping, inherited-member fallback, or NativeAOT remapping support.
#12692 can be rebased on this stack and add those R8-specific extensions rather than this optimization depending on #12692.
Performance characterization
Direct search benchmark
The R8 sample-content table contains 1,014 methods across 186 types. A linear control used identical UTF-8 data and comparisons without hashing.
205.8 nslinear vs.204.6 nsbinary).30.5%faster.70.5%faster.83.0%faster.Matched R2R startup benchmark
To benchmark the two PRs without depending on #12692's conflicting implementation, the real R8 sample's 483 method mappings and two type mappings were converted to identity remaps. Baseline and treatment APKs therefore ran identical unobfuscated MAUI sample-content code with the same large lookup-table distribution.
CoreCLR APKs were restored and built with
PublishReadyToRun=true. Measurements used a Samsung Galaxy A16 (Android 16, arm64-v8a), fresh installs, two warmups per install, five measured launches, animations disabled, and counterbalanced ordering.The initial mixed comparator rescanned an entire common prefix after
Ascii.Equals()reported a mismatch and measured+0.81%(95% CI+0.04%..+1.57%). The final implementation compares vector-sized chunks so a mismatch rescans at most one chunk.Final 160-launch A/B (80 launches per variant, 16 paired blocks):
am start -WTotalTimeFinal paired difference:
+0.22%, 95% CI-0.87%..+1.32%. The final implementation has no statistically measurable whole-app startup regression.A separate 180-launch R2R control matrix compared runtime remapping with equivalent R8 private-member optimization:
+0.02%, 95% CI-1.14%..+1.19%.Stack
Depends on #12795. The bottom PR is based on
main.Validation
JniPeerMembersTests: 12 passed, 1 skipped.TypeAndMemberRemapping: Debug and Release passed; NativeAOT remains intentionally skipped because remapping is not supported there onmain.