Skip to content

bf: validate --eval-filter up-front, fail with a clear error (#440)#441

Open
yarikoptic wants to merge 2 commits into
mainfrom
bf-issue-440-eval-filter-syntax
Open

bf: validate --eval-filter up-front, fail with a clear error (#440)#441
yarikoptic wants to merge 2 commits into
mainfrom
bf-issue-440-eval-filter-syntax

Conversation

@yarikoptic

Copy link
Copy Markdown
Member

Summary

con-duct ls -e EXPR used to call eval(EXPR, ...) inside the per-file loop
of load_duct_runs(), inside a broad except Exception block. The reporter
in #440 had a hidden U+00A0 non-breaking space in their filter expression
(easy to produce on macOS with Option+Space, or when copy-pasting from a
browser/document). The resulting SyntaxError was caught, misattributed
to each JSON file, and logged as:

Failed to load file <_io.TextIOWrapper name='.duct/logs/…_info.json'>: invalid non-printable character U+00A0 (<string>, line 1)

— which sent the user hunting through their JSON logs for corruption that
was never there.

This PR centralises filter compilation so a bad expression fails once,
up-front, with a clear message
, before any file is opened.

  • New compile_eval_filter() helper compiles the expression once via
    compile(..., '<eval-filter>', 'eval'). On SyntaxError it raises
    ValueError naming the offending expression and explicitly hinting at
    U+00A0 / macOS Option+Space as a common cause.
  • ls() calls it first thing, catches ValueError, logs it via
    lgr.error, and returns exit code 2 — no traceback, no per-file noise.
  • load_duct_runs() now accepts either a string (which it compiles) or a
    pre-compiled CodeType, and reuses it in the per-file eval() loop —
    also avoids re-parsing the expression once per file.

After the fix, the reporter's command produces:

[ERROR   ] con_duct.ls: Invalid --eval-filter expression '"fmriprep" in command and\xa0exit_code == 1': invalid non-printable character U+00A0 (<eval-filter>, line 1). Check for hidden non-printable characters (e.g. U+00A0 non-breaking space produced by macOS Option+Space or copy-paste from a browser/document).

and exits with code 2.

Test plan

  • tox -e py3 -- test/test_ls.py — 29 passed (23 pre-existing + 6 new)
  • Full suite: tox -- test/ — 397 passed
  • tox -e lint — clean
  • tox -e typing — clean
  • Manual repro of the running con-duct ls on Mac leads to many error messages #440 command (with a real U+00A0 injected via
    printf '%b') now prints the single ERROR above and exits 2.

New tests added to test/test_ls.py:

  • test_compile_eval_filter_none_returns_none
  • test_compile_eval_filter_valid
  • test_compile_eval_filter_rejects_nbsp (regression test for running con-duct ls on Mac leads to many error messages #440)
  • test_compile_eval_filter_rejects_syntax_error
  • test_load_duct_runs_fails_fast_on_bad_filter — verifies no file is
    even opened when the filter is invalid, and no per-file "Failed to load
    file" warning is emitted.
  • test_ls_exits_cleanly_on_bad_filter — verifies exit code 2 and a
    user-facing ERROR log line mentioning U+00A0.

`con-duct ls -e EXPR` used to call `eval(EXPR, ...)` per-file inside a
broad `except Exception` block. A stray U+00A0 non-breaking space in
EXPR (easy to produce with macOS Option+Space or when copy-pasting
from a browser or document) then surfaced as one misleading
"Failed to load file <_io.TextIOWrapper name=...>: invalid non-printable
character U+00A0 (<string>, line 1)" warning per file, blaming the
JSON files for a filter-expression typo.

Compile the filter once up-front via a new `compile_eval_filter()`
helper. On `SyntaxError`, raise `ValueError` with the offending
expression and an explicit hint about hidden non-printable characters
like U+00A0. `ls()` catches that, logs it via `lgr.error`, and returns
exit code 2 -- so the user gets a single, clear message with no
traceback and no per-file noise.

Reuse the compiled `CodeType` in the per-file `eval()` loop, which
also avoids re-parsing the expression once per file.

Co-Authored-By: Claude Code 2.1.209 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.01%. Comparing base (cc80272) to head (5f1d59b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #441      +/-   ##
==========================================
- Coverage   91.08%   91.01%   -0.07%     
==========================================
  Files          15       15              
  Lines        1245     1258      +13     
  Branches      170      171       +1     
==========================================
+ Hits         1134     1145      +11     
- Misses         77       78       +1     
- Partials       34       35       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes confusing per-file “Failed to load file …” noise when con-duct ls --eval-filter contains a syntax error (notably hidden U+00A0 non-breaking spaces), by compiling/validating the expression once up-front and failing with a single clear error message and exit code 2.

Changes:

  • Add compile_eval_filter() to compile --eval-filter once (with <eval-filter> filename) and convert SyntaxError into a user-facing ValueError.
  • Update ls() to validate the filter before globbing/opening any files and to exit cleanly (rc=2) on invalid expressions.
  • Update load_duct_runs() to accept a precompiled CodeType and reuse it in the per-file eval() loop; add tests covering the new behavior and the gh-440 regression.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
src/con_duct/ls.py Adds upfront filter compilation and updates ls() / load_duct_runs() to reuse a compiled filter and fail cleanly on invalid expressions.
test/test_ls.py Adds unit/regression tests for filter compilation, fast failure before file I/O, and ls() exit code/logging behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/con_duct/ls.py Outdated
Comment thread test/test_ls.py Outdated
Comment thread test/test_ls.py Outdated
Comment thread test/test_ls.py Outdated
Comment thread test/test_ls.py Outdated
- Replace three literal U+00A0 non-breaking spaces in the test source
  with ` ` escapes.  A literal invisible byte in the source is
  fragile (an editor, pre-commit hook, or accidental copy-paste could
  silently strip or replace it — the very failure mode this test is
  guarding against); the escape re-materialises as the same U+00A0 at
  runtime, so the test still asserts identical behavior.

- Fix nonstandard spelling "mis-classified" → "misclassified" in the
  `compile_eval_filter` docstring and a comment in
  `test_load_duct_runs_fails_fast_on_bad_filter`.

Spotted by Copilot review on PR #441.

Co-Authored-By: Claude Code 2.1.209 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@yarikoptic yarikoptic added the semver-patch Increment the patch version when merged label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-patch Increment the patch version when merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

running con-duct ls on Mac leads to many error messages

3 participants