Couple USDT note section with target code - #17
Merged
Merged
Conversation
Historically, .note.stapsdt was emitted as a plain non-allocatable
note section with no formal connection to the code section containing
the probe site it describes. This falls apart once the linker starts
removing code sections: with -ffunction-sections + -Wl,--gc-sections
uncalled functions are discarded, and COMDAT copies of C++ inline
functions are deduplicated across compilation units. BFD ld treats
relocations from note sections as GC roots, but fails the link
outright when a note references a discarded COMDAT copy. LLD doesn't
follow relocations from non-allocatable sections at all and silently
resolves them to 0 + addend, leaving behind dangling USDT notes with
bogus location/base/semaphore values that break probe parsing and
attachment by tracing tools (e.g., libbpf refuses to attach to a USDT
as soon as one dangling copy of it is present, even if other copies
are valid).
Fix this by tying each USDT note to the code section containing its
probe site with SHF_LINK_ORDER ("o" section flag, linked to a
uniquely-named per-probe .Lusdt_nop_N label, as assemblers don't
accept a numeric label backreference like 990b as a linked-to symbol)
and to the probe site's COMDAT group, if any ("?" section flag). The
linker then discards or deduplicates notes together with the code they
describe. Live notes also keep the sections they reference alive, so
implicit semaphores in .probes and .stapsdt.base survive GC as well.
The "o" flag requires GNU assembler >= 2.35 or Clang >= 6; older GNU
as fails hard on unknown section flags, so provide an escape hatch:
defining USDT_NO_SHF_LINK_ORDER before including usdt.h drops the "o"
flag and the linked-to operand, keeping COMDAT deduplication working
("?" is supported by much older assemblers), but losing --gc-sections
protection under LLD, which matches the historical behavior.
Bump library version to 0.2.0.
Add two regression tests for USDT note integrity when the linker
removes probe site code:
- gc_sections defines probes in an uncalled function and is built
with -ffunction-sections and -Wl,--gc-sections, validating that
garbage-collected probe code doesn't leave dangling notes behind
(and that live probes' base/semaphore references survive, which
is asserted by the BASE*/SEMA* stubs in USDT_SPECS);
- cxx_comdat triggers a USDT in a C++ inline function instantiated
in two translation units, validating that COMDAT deduplication
discards the corresponding note copy instead of failing the link
(BFD ld) or leaving a dangling note (LLD).
Detection is implemented in check-usdt-locs.awk, which run_test.sh
now runs for every test binary (and shared library): each USDT note's
location must fall inside an executable PT_LOAD segment of its ELF
file, mirroring the lookup tracing tools perform at attach time
(e.g., libbpf's find_elf_seg()); a location outside any executable
segment means the note's relocation was resolved against a section
discarded by the linker.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By default, each USDT note in .note.stapsdt section is tied to the code
section containing its probe site ("o" section flag, SHF_LINK_ORDER,
linked to a per-probe .Lusdt_nop_N label) and to the probe
site's COMDAT group, if any ("?" section flag). This way linker garbage
collection (-Wl,--gc-sections) and COMDAT deduplication (e.g., of C++
inline functions defined in headers) discard USDT notes together with the
probe site code they describe, instead of either failing the link (BFD
ld) or leaving behind dangling USDT notes with bogus location/base/
semaphore values that break probe parsing and attachment by tracing tools
(LLD).