Skip to content

FIX: Render full error content in GUI instead of generic 500#2145

Open
jsong468 wants to merge 1 commit into
microsoft:mainfrom
jsong468:gui_error
Open

FIX: Render full error content in GUI instead of generic 500#2145
jsong468 wants to merge 1 commit into
microsoft:mainfrom
jsong468:gui_error

Conversation

@jsong468

@jsong468 jsong468 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Description

When a message send fails in the GUI (e.g. sending to a model that returns a connection
error, or any target that raises), the chat showed a cryptic
unknown: Internal server error. Check server logs for details. bar. Yet navigating to the
same conversation via the conversations panel showed the full error turn — a
processing-error message with the traceback. The immediate send view and the reload view were inconsistent for the exact same event. See below:
image

This PR makes the send-time view consistent with the reload view by surfacing the already-stored error turn inline. See below for the updated behavior:
image

Root cause

PromptNormalizer.send_prompt_async persists a full error piece to memory
(response_error="processing", converted_value = exception + traceback) and then re-raises (pyrit/prompt_normalizer/prompt_normalizer.py). In the GUI flow that re-raise propagated up through add_message_async to the route's catch-all except Exception, which returned an HTTP 500 (in add_message route in attacks.py) with a generic detail, discarding the real error. The frontend rendered that 500 as an ephemeral unknown error bar via its catch path. On reload (GET), the stored error piece rendered normally (hence the mismatch).

Fix

In add_message_async, wrap the send in a try/except. Because the normalizer already persisted
the error turn before raising, when the exception is accompanied by a newly stored error piece
we swallow-and-log, then return the normal AddMessageResponse (HTTP 200) whose messages
include that error turn. This makes the POST (send) response render identically to the GET
(reload) view. If no new error piece was stored — i.e. the failure happened before the send
(target lookup, validation) — we re-raise so the route still reports a genuine 404/400/500.

  • pyrit/backend/services/attack_service.py: added a module logger; wrapped
    _send_and_store_message_async with a guard that keys off has_error() on a piece not present before the send.

Why the service layer (not the route or the normalizer)

  • The normalizer's re-raise is a deliberate signal to programmatic callers (attack strategies
    retry on it). This is left untouched.
  • The GUI service is the presentation boundary: for a human operator the error turn is already
    captured as displayable conversation data, so translating the raise into that stored turn
    belongs here — not in the generic route handler, and not by weakening the normalizer.
  • The guard is generic (has_error()), so it covers any error type that stores a piece then
    raises.

Behavior changes

  • Only the raised-exception (processing) send path changes: HTTP 500 → 200, and the frontend now renders it via the success path as a normal assistant error turn (bar + traceback body) instead of the ephemeral unknown bar.
  • Unchanged: everything persisted to memory; the reload view; the blocked (content-filter) and
    empty paths (they never raised); programmatic/attack callers; genuine pre-send failures (still
    error out); what is replayed to the target on subsequent sends.
  • Side effect of the 500→200 flip: the frontend no longer enters its catch, so the "repopulate
    failed input for re-send" convenience no longer fires for this case (the error is now a
    persisted turn, consistent with reload).

Testing

  • pytest tests/unit/backend/test_attack_service.py -k add_message → all add_message tests pass,
    including the 2 new tests and the existing target-not-found re-raise test.

"""When the normalizer stores an error piece then raises, the send returns it inline (no raise)."""
ar = make_attack_result(conversation_id="test-id")
mock_memory.get_attack_results.return_value = [ar]
mock_memory.get_conversation_messages.return_value = []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: test_add_message_surfaces_stored_error_piece_on_send_failure mocks get_conversation_messages to [], so it only checks we didn't raise - not that the error turn actually comes back. Might want it to return the error piece and assert response_error == "processing"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants