Under -DANYPS5_ENABLE_ASAN=ON with ASAN_OPTIONS=detect_leaks=0, 14 of
the 849 tests outside the AGC driver, shader and VideoOut suites failed
on main for reasons that come from the ASan build itself, not from the
code they test:
- libc copies the main executable's file-backed writable pages before
remapping them anonymously. In an instrumented test executable those
pages hold ASan's redzones around globals, so the memcpy reported a
global-buffer-overflow at load (patched_libraries,
guest_memory_lifetime). The ASan build copies the pages word by word
without instrumentation.
- Five tests check that a local variable is on the thread's stack.
With detect_stack_use_after_return, which is on by default on Linux,
instrumented locals live on ASan's fake stack. These tests now run
with it off. guest_kernel_is_stack also bounded the worker stack by
the application area without the ASan build's arena offset.
- The relinker module tests and loadexec_arguments load instrumented
libraries into a Python process or a relinked guest, so the ASan
runtime is not first in the initial library list. They now run with
verify_asan_link_order=0. Preloading libasan instead put its
__cxa_throw interceptor ahead of libc.prx's and broke the unwind of
an exception that LoadExec throws and catches.
- guest_allocator_failures limits RLIMIT_DATA to 128 MiB and then
commits 300 GiB. ASan maps 37.5 GiB of shadow for that commit, which
the limit refuses, and aborts. The ASan build skips that section.
- fiber checks the address of a local against the fiber's context too,
so it runs without fake stacks as well. Its CheckActiveContext then
reads the fiber stack word by word above that local, through the
redzones of the frames on that stack, and is no longer instrumented.
- agc_multisample_guest_access passed aligned_alloc a size that is not
a multiple of the alignment, which C11 does not allow and ASan
rejects. It now rounds the size up.
add_asan_test_options appends to ASAN_OPTIONS for one test in an ASan
build and does nothing otherwise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux, glibc maps the stack of every guest thread near the top of the
host address space (around 0x7fff00000000). On the console, application
stacks lie inside the application area below 0xFC00000000, and guest code
relies on that: ChakraCore's garbage collector indexes a 4 MiB table with
address >> 18, so it only covers addresses below 1 TiB and reports
Js::OutOfMemoryException for a stack above it.
Map each guest thread's stack with DoMapAnon, which allocates from the
guest arena, behind a PROT_NONE guard page, and pass it to pthread_attr_setstack.
The stack is freed after a join; the stack of a detached thread is freed once its
host thread is gone from /proc/self/task.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Track the stack range of each running guest thread and answer
sceKernelIsStack from the calling thread's stack (the host limits for an
adopted thread) or from that registry. A mapped address outside every
stack succeeds with null bounds and an unmapped one returns EACCES; the
output pointers may be null.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
