Raise host fd limit to back the guest fd table#148
Conversation
The guest is told RLIMIT_NOFILE is FD_TABLE_SIZE (1024), and every guest descriptor is backed by at least one host descriptor, but macOS starts processes with a soft limit of only 256. On a stock host the guest therefore hits EMFILE at around 250 open fds, far below the limit it was promised. This is why the test-stress fd-exhaustion subtest fails from a fresh shell but passes once ulimit -n is raised. Raise the soft limit to min(OPEN_MAX, rlim_max) in fdtable_init(), before any guest code runs, following the macOS setrlimit(2) man page. A limit that is already high enough is left alone, and a failed setrlimit is ignored. The extra headroom over FD_TABLE_SIZE also covers emulations that need more than one host fd per guest fd, such as eventfd self-pipes and per-watch inotify fds. Host fds may now exceed FD_SETSIZE, which is fine: sys_pselect6 already falls back to poll() when a host fd does not fit in an fd_set. Developer machines usually run with raised shell limits, so a regression here would go unnoticed. Add a check step that reruns test-stress with the soft limit forced back to 256 in a child shell.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 2 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="mk/tests.mk">
<violation number="1" location="mk/tests.mk:86">
P2: The `test-fd-limit-stock` test unconditionally sets the soft fd limit to 256, which will fail if the host hard limit is lower. This causes spurious `make check` failures in constrained CI or container environments. Consider checking the hard limit first and either capping the requested soft limit or skipping the test gracefully when the environment cannot support it.</violation>
</file>
<file name="src/syscall/fdtable.c">
<violation number="1" location="src/syscall/fdtable.c:169">
P2: If raising the host fd limit fails (e.g. in a constrained container or sandbox), the guest-visible `RLIMIT_NOFILE` is left at the default `FD_TABLE_SIZE` (1024) while the actual host limit stays low. Host syscalls can then fail with `EMFILE` well below the guest's advertised capacity, recreating the host/guest divergence this patch is meant to fix.
Consider syncing the guest limit down when `setrlimit` fails — the codebase already has `fd_set_rlimit_nofile()` for this purpose. Saving the original `rl.rlim_cur` before the `setrlimit` call and passing it to `fd_set_rlimit_nofile()` on failure would keep the two limits aligned.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| FD_LIMIT_STOCK_DEPS += $(BUILD_DIR)/test-stress | ||
| endif | ||
| test-fd-limit-stock: $(FD_LIMIT_STOCK_DEPS) | ||
| @bash -c 'ulimit -S -n 256 && exec "$(ELFUSE_BIN)" "$(TEST_DIR)/test-stress"' |
There was a problem hiding this comment.
P2: The test-fd-limit-stock test unconditionally sets the soft fd limit to 256, which will fail if the host hard limit is lower. This causes spurious make check failures in constrained CI or container environments. Consider checking the hard limit first and either capping the requested soft limit or skipping the test gracefully when the environment cannot support it.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At mk/tests.mk, line 86:
<comment>The `test-fd-limit-stock` test unconditionally sets the soft fd limit to 256, which will fail if the host hard limit is lower. This causes spurious `make check` failures in constrained CI or container environments. Consider checking the hard limit first and either capping the requested soft limit or skipping the test gracefully when the environment cannot support it.</comment>
<file context>
@@ -74,6 +76,15 @@ check: $(ELFUSE_BIN) $(TEST_DEPS) check-syscall-coverage \
+ FD_LIMIT_STOCK_DEPS += $(BUILD_DIR)/test-stress
+endif
+test-fd-limit-stock: $(FD_LIMIT_STOCK_DEPS)
+ @bash -c 'ulimit -S -n 256 && exec "$(ELFUSE_BIN)" "$(TEST_DIR)/test-stress"'
+
## Hot-syscall performance guardrail: ensure getpid, libc clock_gettime,
</file context>
| if (rl.rlim_cur >= want) | ||
| return; | ||
| rl.rlim_cur = want; | ||
| (void) setrlimit(RLIMIT_NOFILE, &rl); |
There was a problem hiding this comment.
P2: If raising the host fd limit fails (e.g. in a constrained container or sandbox), the guest-visible RLIMIT_NOFILE is left at the default FD_TABLE_SIZE (1024) while the actual host limit stays low. Host syscalls can then fail with EMFILE well below the guest's advertised capacity, recreating the host/guest divergence this patch is meant to fix.
Consider syncing the guest limit down when setrlimit fails — the codebase already has fd_set_rlimit_nofile() for this purpose. Saving the original rl.rlim_cur before the setrlimit call and passing it to fd_set_rlimit_nofile() on failure would keep the two limits aligned.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/syscall/fdtable.c, line 169:
<comment>If raising the host fd limit fails (e.g. in a constrained container or sandbox), the guest-visible `RLIMIT_NOFILE` is left at the default `FD_TABLE_SIZE` (1024) while the actual host limit stays low. Host syscalls can then fail with `EMFILE` well below the guest's advertised capacity, recreating the host/guest divergence this patch is meant to fix.
Consider syncing the guest limit down when `setrlimit` fails — the codebase already has `fd_set_rlimit_nofile()` for this purpose. Saving the original `rl.rlim_cur` before the `setrlimit` call and passing it to `fd_set_rlimit_nofile()` on failure would keep the two limits aligned.</comment>
<file context>
@@ -145,11 +147,35 @@ static int fd_bitmap_find_free(int minfd)
+ if (rl.rlim_cur >= want)
+ return;
+ rl.rlim_cur = want;
+ (void) setrlimit(RLIMIT_NOFILE, &rl);
+}
+
</file context>
|
existing issue related to this PR:
Please clamp the
The guest should be allowed to change its own Please virtualize guest |
| */ | ||
| void fdtable_init(void) | ||
| { | ||
| fd_raise_host_nofile(); |
There was a problem hiding this comment.
fdtable_init() is called from syscall_init(), and fork children call syscall_init() again while rebuilding state in fork_ipc_recv_fd_table(). That means a guest that lowers RLIMIT_NOFILE before fork() can get a child whose host soft limit is silently raised back to OPEN_MAX, which violates Linux rlimit inheritance semantics.
Please either run the host raise only during initial bootstrap, or serialize/restore the guest nofile state across the fork path before guest code resumes.
| ifndef GUEST_TEST_BINARIES | ||
| FD_LIMIT_STOCK_DEPS += $(BUILD_DIR)/test-stress | ||
| endif | ||
| test-fd-limit-stock: $(FD_LIMIT_STOCK_DEPS) |
There was a problem hiding this comment.
Please wrap this target with the same 60-second timeout used by the normal test driver, or move the lowered-rlimit case into a driver script that already has timeout handling.
2 participants
Fixes #68
The test-stress FD-exhaustion subtest fails whenever make check runs from a shell with the stock macOS soft fd limit (256). The guest is told RLIMIT_NOFILE is 1024 (FD_TABLE_SIZE) and every guest fd is backed by a host fd, so once the host process runs out at ~250 the guest sees EMFILE far below its advertised limit.
Fix it in fdtable_init(): raise the host soft limit to min(OPEN_MAX, rlim_max) before any guest code runs, following the macOS setrlimit(2) man page. A limit that is already high enough is left alone, and a failed setrlimit is ignored. Host fds can now exceed FD_SETSIZE, which is fine — sys_pselect6 already falls back to poll() when a host fd doesn't fit in an fd_set.
Since dev machines usually run with raised shell limits, a regression here would go unnoticed, so this also adds a check step (test-fd-limit-stock) that reruns test-stress with the soft limit forced to 256 in a child shell.
Tested on an M3 Max: the repro above fails on unpatched main and passes with the patch; full make check stays green.
Summary by cubic
Raises the host soft file descriptor limit at startup so the guest can use its full fd table without hitting EMFILE on macOS’s stock 256 limit. Adds a regression check that reruns the stress test under the stock limit.
fdtable_init(), raiseRLIMIT_NOFILEtomin(OPEN_MAX, rlim_max); leave higher limits as-is and ignoresetrlimitfailures.test-fd-limit-stockto runtest-stresswithulimit -S -n 256to catch regressions.Written for commit 6d66e60. Summary will update on new commits.