Culprit: none in 892a4f1328..b85d972f42; race in the Windows vnode watch (50b06110b guarded only the loop). Run 39473930499, guest_kqueue: a watched file deleted before registration is snapshotted under $Extend\$Deleted, WatchParent got ERROR_ACCESS_DENIED and threw.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
kevent threw NotImplemented for EVFILT_WRITE with NOTE_LOWAT above 1 on a socket, although the socket readiness check already reports free send-buffer space (SO_SNDBUF less SIOCOUTQ on Linux). The write event now fires once that space reaches the mark, as FreeBSD 11's filt_sowrite compares sbspace with kn_sdata, and stays silent while the space is below it. The debt entry no longer says a write mark is limited to 1. Reference: https://github.com/freebsd/freebsd-src/blob/release/11.0.0/sys/kern/uipc_socket.c
kevent threw for EVFILT_WRITE on any descriptor that is not a socket. On the write end of a pipe it now follows FreeBSD 11's filt_pipewrite: data is the free buffer space, and once the read end is closed the event reports EV_EOF with data 0. Adding the event after the read end is closed fails with EPIPE, as in pipe_kqfilter. EVFILT_WRITE on the read end still throws, because FreeBSD's pipe ends are bidirectional and host pipes are not.
kevent threw for EV_CLEAR on a socket or the read end of a pipe; Boost.Asio's kqueue reactor, among others, registers every socket that way. On Linux such a knote now watches its host descriptor through an edge watch. It is armed when added, on every later EV_ADD or EV_ENABLE while enabled, and by each host wakeup; when armed it is checked with the level-triggered readiness test and reported with the current data and EV_EOF, or disarmed if no longer ready, as kqueue_scan re-runs the filter; reporting it disarms it until the next edge. Waits block on the watch's epoll descriptor, so a socket that stays readable does not wake them. Windows and macOS still throw. GuestSockets::NativeDescriptor exposes the host socket behind a guest one for the watch.
FreeBSD activates an EV_CLEAR knote on each state change the object reports through KNOTE (sorwakeup, sowwakeup, pipeselwakeup), not on its level, so polling the level cannot tell an EV_CLEAR knote when to fire again. EdgeWatch::Watch registers the host descriptor with a private epoll instance in edge-triggered mode: every wakeup of the descriptor with a matching mask (data arriving, buffer space freed, a hang-up) queues one edge, which TakeEdge consumes, and the epoll descriptor itself is readable while an edge is pending, so a wait can block on it. Off Linux opening a watch throws.
WaitForDescriptors always handed the knote's own descriptor to the host wait. A filter that tracks readiness through a descriptor of its own, such as an edge watch, needs the wait to block on that descriptor instead, or a descriptor that stays ready keeps waking the wait. KernelFilter gets a waitDescriptor, and the filter's owner is held for the length of the wait so the descriptor cannot be closed under it.
A kqueue descriptor is readable while one of its events is triggered and
is never writable or exceptional, as FreeBSD's kqueue_poll answers, on
both hosts. On Windows select threw for it, and on Linux its /dev/null
placeholder was reported readable and writable.
A descriptor that stops being a socket, a kqueue or a pipe during the
wait now fails the call with EBADF when it was closed and is selected
again with the rest of the timeout when its number was reused, as
FreeBSD's kern_select looks every descriptor up again in each pass,
instead of throwing. On Windows a host descriptor that is not a pipe
throws when select starts, before waiting.
Depends on #7139.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PPSA34015 (EA SPORTS FC 27) registers EVFILT_READ for descriptor 0 in a
new kqueue at start-up, ignores the result, and stopped with "kevent:
EVFILT_READ on a non-socket descriptor not implemented". A read event on
the host stream on descriptor 0 is now accepted and never triggers, as
select never reports descriptor 0 readable.
On Linux, FIONREAD on a datagram socket reports the next datagram only,
where FreeBSD's sbavail counts every queued byte, so a read mark above 1
there could never fire. kevent now throws for it as before; stream
sockets and marks of 1 are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kevent threw for NOTE_LOWAT. On a socket's EVFILT_READ it now sets the
low-water mark as FreeBSD 11's filt_soread does: the event fires once the
buffered bytes reach it, or at end of file or on a listening socket, and
a later change of the knote replaces it. EVFILT_WRITE takes a mark of 1,
where the host's writability stands in for a free byte; other write marks
and marks below 1 still throw. A pipe ignores the mark, as filt_piperead
does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kevent threw for any descriptor that is not a guest socket. The read end
of a pipe now gets the descriptor knote, following FreeBSD 11's
filt_piperead: data is the number of buffered bytes and EV_EOF is set
once the write end is closed. GuestSockets::Ready falls back to the new
PipeReadState for a descriptor that is not a socket, and WaitAny waits on
the pipe with ppoll on Linux or in 1 ms WSAPoll slices on Windows, only
when the queue holds a pipe. On Windows the equeue waker now starts
Winsock itself, since a queue can wait on a pipe before any socket exists.
On a new knote EV_FORCEONESHOT is dropped from the flags; on an
existing one it sets EV_ONESHOT and fires the knote, as
kqueue_register does at l.1339-1341. A modify no longer ORs
EV_ONESHOT, EV_CLEAR or EV_DISPATCH into the knote.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A knote registered or modified with EV_DISABLE stays registered but is
not polled, fired or reported until an EV_ENABLE change re-arms it, as
kern_event.c does with KN_DISABLED. EV_DISPATCH reports an event once
and then disables the knote, and EV_FORCEONESHOT registers as oneshot.
A negative nevents fails with EINVAL instead of throwing.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Windows error for a refused FindFirstChangeNotification was one string shared
by WatchParent and WatchContents, so the log could not tell which handle or path
was denied. Build it from the watcher, the path the API was given and
GetLastError instead.
Still throws: the EVFILT_VNODE attach path is vfs_kqfilter, which returns only
EINVAL, ENOMEM or 0, with EBADF from fget before it, and consults no directory
permission, so no FreeBSD errno stands for a host that refuses to notify.
Windows 10 1903 deletes a file with open handles by moving it into
$Extend\$Deleted until the last handle closes, so the next snapshot of a
watched file that a title (or the test) deleted saw a new path, tried to
watch that directory's changes, got ERROR_ACCESS_DENIED and threw from
the watch thread, which aborts the process; the path change was also
reported as NOTE_RENAME next to NOTE_DELETE. A snapshot with a pending
delete or no links now reports NOTE_DELETE alone and drops the parent
directory watch instead of replacing it, as the file has no directory
to be renamed or removed from any more.
libkernel numbered guest sockets from GuestSockets::FirstDescriptor
(0x10000000), so a title that puts a socket in a 1024-bit fd_set writes
fd / 8 bytes past it. The Epic Online Services SDK does this on its
RtcNetworkThread and three titles stop there (#1794):
Palworld PPSA23126 (queeeef, main a11c8f7f7):
FATAL: unhandled exception 0xc0000005 on thread 37164 'RtcNetworkThread'
rip 0x00007ffcdb1ea2be eossdk-ps5-shipping.prx.guest.prx+0xdea2be
rcx 0000000010000005 rsi 0000000000400000 r12 0000000010000005
The Mound PPSA26223 (xXPBSXx, main 0a8b1d32b):
FATAL: unhandled exception 0xc0000005 on thread 'RtcNetworkThread',
rip eossdk-ps5-shipping.prx.guest.prx+0xefc6f4, rcx = 0x10000005,
rsi = 0x400000
Dying Light 2 PPSA02262 (tv7):
FATAL: unhandled exception 0xc0000005 on thread 'RtcNetworkThread'
rcx 0000000010000005 ... rsi 0000000000400000 ... r12 0000000010000005
On FreeBSD 11.4 (8c639945) FD_SETSIZE is 1024 (sys/sys/select.h:58-59)
and sockets take the lowest free entry of the one process descriptor
table that files use (kern_descrip.c:1643-1660 fdalloc,
uipc_syscalls.c kern_socket through falloc).
Every socket number is now a real host descriptor: a zero-access NUL
handle registered with _open_osfhandle on Windows (as directory
descriptors already are) and an F_DUPFD_CLOEXEC duplicate of the host
socket on Linux. The socket table maps that number to the socket, so
"is this a socket" is a table lookup instead of a range check at the 31
former FirstDescriptor comparisons. close releases both the socket entry
and the number, dup and dup2 duplicate the number through the host table
and keep the socket shared, dup2 of a file onto a socket number drops the
socket entry, and sceKernelClose now closes sockets too. sceNetSocket and
sceNetAccept take their ids from the same table instead of a counter from
32, so libkernel and libSceNet numbers never collide.
select polls guest sockets alongside libSceNet sockets, the random device,
stdio and (on Linux) host descriptors; poll throws only for an open
descriptor that is not a guest socket and reports POLLNVAL otherwise.
File calls that used to fail with EBADF on the socket range return
FreeBSD's answer for a socket: lseek ESPIPE (kern_lseek), ftruncate and
getdirentries EINVAL (invfo_truncate, getvnode), flock EOPNOTSUPP;
fstat on a socket throws. pread on a closed descriptor returns EBADF.
title failing on host descriptor numbers; these three titles are that
case, and this change keeps host numbering for files and only moves
sockets into it.
Tests: guest_socket_descriptors (new: socket below 1024, select on a
socket and a file together, close frees the number for a file, dup of a
socket, sceKernelClose), guest_select (socket and pipe together, Linux),
guest_poll, guest_dup, guest_socket_pair, guest_fstatfs,
guest_posix_filesystem and guest_sce_net updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 20378375b31eb5db922db6e55f45e44a4dae6445)
inotify queues IN_MOVED_FROM and IN_MOVED_TO of a rename one after the
other. The watcher thread could read the first before the second was
queued and report a rename inside the directory as a move to another
directory (NOTE_WRITE | NOTE_EXTEND instead of NOTE_WRITE), which made
guest_kqueue abort on loaded runners.
The watcher now reads the queue until it is empty and, when a move half
has no partner, calls getdents64 on the directory before reading again:
that takes the directory lock rename holds until both halves are queued.
Return the event's data when EV_ERROR is set in its flags and 0 otherwise, as KytyPS5 does and as FreeBSD 11 kevent stores the errno of a failed change. A null event throws like the other accessors.
FreeBSD's vfs_kqfilter checks neither the vnode type nor fflags, so the
registration is valid; it is only delivered for a move between directories,
which the Windows directory handle cannot tell from a removal plus an addition.
Also moves the TechnicalDebt entries to docs/dev/debt after main split the file.
Co-Authored-By: Claude <noreply@anthropic.com>
Cloudberry Kingdom (PPSA23827, #3177) registers its first kevent change
as a file watch: ident 3, EVFILT_VNODE, flags 0x25
(EV_ADD|EV_ENABLE|EV_CLEAR), fflags 0x37
(NOTE_DELETE|WRITE|EXTEND|LINK|RENAME). Palworld's Discord module
(PPSA23126, #2648) imports sceKernelAddFileEvent and
sceKernelDeleteFileEvent, which no library exported.
EVFILT_VNODE watches the host file or directory behind the descriptor
and turns host changes into FreeBSD 11 vnode hints, added whole when
they share a bit with the requested fflags as filt_vfsvnode does
(sys/kern/vfs_subr.c, release/11.0.1 34bd3475): a growing write is
NOTE_WRITE|NOTE_EXTEND, a shrinking truncate or an attribute change
NOTE_ATTRIB, a link count change NOTE_LINK or NOTE_DELETE, a pending
delete NOTE_DELETE, a new path NOTE_RENAME, and in a directory a new,
removed or renamed entry NOTE_WRITE (with NOTE_LINK for a subdirectory,
as vop_mkdir_post/vop_rmdir_post). Windows uses FindFirstChangeNotification
on the parent directory (and the watched directory) and compares handle
snapshots every 50 ms, since a write through an open handle is not
reported until it is flushed or closed (measured); Linux uses inotify.
Closing the descriptor, deleting the event or the queue stops the
watcher.
sceKernelAddFileEvent(eq, fd, watch, udata) and
sceKernelDeleteFileEvent(eq, fd) take the OpenOrbis 0a1aaf9d
(libkernel.h:35,109) and SharpProspero 5dfdb4b0 (KernelEqueue.cs:164,168)
signatures; NIDs cG3t15OK4Ow and UmXngHKB6is confirmed with
tools/nid_names.py. The watch bits are the fflags and the event is added
with EV_CLEAR, which is not confirmed on a title.
Throws: NOTE_REVOKE, NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, NOTE_READ
and unknown bits, NOTE_EXTEND on a directory on Windows (a move from
another directory cannot be told from a creation) and descriptors that
are not files, directories, pipes or sockets (pipes, sockets and kqueues
fail with EINVAL as in FreeBSD). Recorded in TechnicalDebt.
guest_kqueue adds a vnode watch that sees a write to a temporary file
with Cloudberry's flags, fflags filtering, a re-registration, the sce
pair on an equeue, EINVAL for sockets, pipes and the kqueue itself, the
close of a watched descriptor and a directory watch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Cloudberry Kingdom (PPSA23827, #3177) calls kqueue right after start-up
and stopped on the throwing declarations merged with #3337; Palworld's
Discord module (PPSA23126, #2648) imports kevent.
kqueue reserves a host null-device descriptor in the file descriptor
space and registers it as an event queue that reports each ready event
once per call and moves level-triggered ones behind the others, as
kqueue_scan does. close and sceKernelClose close the queue and drop the
descriptor's events from every queue, dup2 drops them for the target,
and dup/dup2 of a kqueue throw, so a reused descriptor number is never
read as a stale kqueue.
kevent follows FreeBSD 11 kern_event.c (release/11.0.1 34bd3475):
kqueue_kevent for the changelist (filter 0 skipped, EV_SYSFLAGS
cleared, EV_ERROR receipts with the errno in data, EV_RECEIPT, the
early return when nevents is exhausted), kqueue_register for EV_ADD,
EV_DELETE, EV_ENABLE, EBADF, ENOENT and EINVAL, kqueue_scan for the
timeout (EINVAL for a bad timespec, zero polls, above INT32_MAX waits
forever) and filt_timerattach, filt_usertouch for the timer and user
filters. Read and write events reuse the guest socket poll of
sceKernelAddReadEvent/WriteEvent.
Throws: other filters, EV_DISABLE, EV_DISPATCH, EV_FORCEONESHOT,
NOTE_LOWAT, EV_CLEAR on a socket, read/write events on non-socket
descriptors or another kqueue, changes to an existing timer (11.0 and
11.4 differ), a NOTE_NSECONDS fraction above one second and a negative
nevents. Recorded in TechnicalDebt.
Test guest_kqueue covers creation and close, timers, a user event
triggered from another thread, read and write events on a socket pair,
EV_ERROR receipts, the timeout and the error cases.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A user event added with sceKernelAddUserEvent stays triggered until it is
deleted. The collection loop kept re-emitting it until the caller's event
buffer was full, so a wait with num > 1 returned the same event num times.
Like kevent, each event is now reported at most once per call; distinct
queued completions of one event are still delivered in order.
(cherry picked from commit db28ebbb7f)
Forwards _sceUltWaitingQueueResourcePoolCreate and _sceUltSemaphoreCreate to the existing calls, like the other underscore Ult exports. Declares sceNetShowIfconfig, sceNpWebApi2GetMemoryPoolStats, Json::Value::setNullAccessCallBack, kqueue, kevent and one pad call of unknown name without an implementation.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A wait on an equeue that holds socket events now blocks in poll/WSAPoll on
those sockets plus a per-thread loopback wake socket, instead of rechecking
every millisecond. Adding, triggering or deleting an event and closing the
equeue wake the blocked waiters. Closing a socket removes its read and write
events, as on FreeBSD. A read event also reports EV_EOF once the peer has
shut down its sending side.
sceKernelAddReadEvent/DeleteReadEvent and sceKernelAddWriteEvent/
DeleteWriteEvent register EVFILT_READ and EVFILT_WRITE events on guest
sockets. The equeue polls the host socket for readiness on each wait
and reports the readable byte count or the send buffer size in data,
with EV_EOF on hang-up or error.
AMPR events were registered and raised as EVFILT_USER. Titles that
dispatch their AMPR queue by filter ignore those events, so reads that
complete through WriteKernelEventQueue never finish. Add, delete and
trigger them with EVFILT_AMPR (-25).
sceKernelGetEventId, sceKernelGetEventFilter, sceKernelGetEventFflags,
sceKernelGetEventData and sceKernelGetEventUserData return the matching
field of the KernelEvent filled by sceKernelWaitEqueue and throw on a
null event. sceKernelGetEventError still throws NotImplemented.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bring in the resolved#25 branch, which already carries upstream main
including boykopovar/AnyPS5#10, and re-apply the KernelErrors.hpp
refactor onto the rewritten files:
- Equeue: drop the local EQUEUE_ERROR_* constants and use the shared
SCE_KERNEL_ERROR_* names.
- Pthread Attr/Thread: drop local error constants and include
KernelErrors.hpp.
- Pthread Cond/Mutex: take upstream; they no longer define local
SCE_KERNEL_ERROR_* constants.
- Semaphore: drop KERNEL_SEMA_ERROR_* in favour of KernelErrors.hpp.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018emyZ4Rk8yoSJvxTFBe4DL
Kernel error codes were redeclared per file under three prefixes
(SCE_KERNEL_ERROR_, EQUEUE_ERROR_, KERNEL_SEMA_ERROR_), which let wrong
values go unnoticed. Define them once as 0x80020000 | errno for errno
1..94 and use the SCE_KERNEL_ERROR_ names everywhere. Values are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SCE kernel errors are 0x80020000 plus the FreeBSD errno. The equeue and
direct memory constants used an unrelated 0x8001xxxx range with mismatched
errno values (e.g. ETIMEDOUT was 0x80010023), so a guest checking
sceKernelWaitEqueue for SCE_KERNEL_ERROR_ETIMEDOUT saw an unexpected error.
SCE_KERNEL_ERROR_EDEADLK in Mutex.cpp held the EAGAIN value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
libkernel: a TimedWait service for condition variables, semaphores, event flags, rwlocks and
equeues with the scheduler tick; mutex/cond/sem corner cases the title hits; job worker
affinity is opt-in (APS5_JOB_AFFINITY). libc: calloc/realloc heap slot swap, a default
terminate handler with a backtrace, CPU topology helper. libSceRtc export fixes with tests.
libSceVideoOut: APS5_NO_PAD_INPUT ignores host input for unattended measurements,
APS5_FLIP_QUEUE_WAIT keeps a stalled process alive for a debugger. ApplicationHeap and
GuestRtc tests wired into the test build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
- libc: reserve guest memory in one arena below the PS5 map limit with
MEM_WRITE_WATCH (GuestArena) and a caching guest heap; the allocation
registry gains a generation counter, an invalidator callback for the
driver's page cache and a pin waiter for leased ranges. This replaces
the fault-based GuestMemoryTracking/GuestMemoryBacking layer, whose
files are removed.
- libkernel: mmap/mprotect/munmap on the arena, flexible memory budget,
real sceKernelVirtualQuery; POSIX pthread layer, rwlocks, timed locks,
TSD, event flags; 0.5 ms scheduler tick and high-resolution sleeps;
TSC/clock/time functions; file APIs return SCE errno codes; the APR
file-streaming API.
- libc: vectored crash reporter with backtraces for fatal exceptions,
SIGABRT, UCRT invalid parameters and process exit; SSE4a EXTRQ/INSERTQ
emulation for Intel hosts; %ls/%lc/%S/%C with the guest's 16-bit
wchar_t; hand-written setjmp/longjmp for the guest jmp_buf; a path
alias table for short save-data mount points; more string, stdio and
math exports.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
Federico Marras