Skip to content

fix(fetcher): restore Stack Overflow Q&A extraction#149

Merged
chaliy merged 1 commit into
mainfrom
codex/fix-stackoverflow-fetcher
Jul 5, 2026
Merged

fix(fetcher): restore Stack Overflow Q&A extraction#149
chaliy merged 1 commit into
mainfrom
codex/fix-stackoverflow-fetcher

Conversation

@chaliy

@chaliy chaliy commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

What

Fix Stack Overflow question fetching so Q&A pages return meaningful markdown again, including the question body, accepted/top answers, code snippets, links, and metadata.

Why

The Stack Exchange API rejects the old withbody_markdown filter with HTTP 400, so the specialized fetcher could no longer return the requested Q&A content.

How

  • Use the supported withbody API filter for questions and answers.
  • Convert Stack Exchange HTML bodies through FetchKit's existing HTML-to-markdown converter.
  • Add a minimal-content guard so empty/block-like Q&A responses are not reported as successful.
  • Add a mocked regression test for the reported Stack Overflow question shape.
  • Add an Unreleased changelog note.

Risk

  • Low
  • Stack Overflow/Stack Exchange fetch output formatting changes from API-provided markdown to FetchKit-converted markdown, but this restores working content and uses the same converter as normal HTML fetches.

Security

  • Reviewed SSRF/network handling: API host remains hardcoded and policy-pinned through existing transport_request flow.
  • Reviewed resource exhaustion: API body reads still use existing max body size and timeout handling.
  • Reviewed HTML conversion: uses existing converter that strips unsafe/noisy elements.

Checklist

  • Unit tests are passed
  • Smoke tests are passed
  • Documentation/changelog is updated
  • Specs are up to date and not in conflict
  • Security review performed

Validation:

  • cargo test --workspace
  • cargo build --workspace --exclude fetchkit-python
  • cargo run -p fetchkit-cli -- --help
  • cargo run -p fetchkit --example fetch_urls
  • MCP initialize smoke via cargo run -p fetchkit-cli -- mcp
  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps

@chaliy chaliy merged commit f9c56f1 into main Jul 5, 2026
11 checks passed
@chaliy chaliy deleted the codex/fix-stackoverflow-fetcher branch July 5, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant