FIX: Render full error content in GUI instead of generic 500#2145
Open
jsong468 wants to merge 1 commit into
Open
FIX: Render full error content in GUI instead of generic 500#2145jsong468 wants to merge 1 commit into
jsong468 wants to merge 1 commit into
Conversation
varunj-msft
approved these changes
Jul 9, 2026
| """When the normalizer stores an error piece then raises, the send returns it inline (no raise).""" | ||
| ar = make_attack_result(conversation_id="test-id") | ||
| mock_memory.get_attack_results.return_value = [ar] | ||
| mock_memory.get_conversation_messages.return_value = [] |
Contributor
There was a problem hiding this comment.
nit: test_add_message_surfaces_stored_error_piece_on_send_failure mocks get_conversation_messages to [], so it only checks we didn't raise - not that the error turn actually comes back. Might want it to return the error piece and assert response_error == "processing"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When a message send fails in the GUI (e.g. sending to a model that returns a connection

error, or any target that raises), the chat showed a cryptic
unknown: Internal server error. Check server logs for details.bar. Yet navigating to thesame conversation via the conversations panel showed the full error turn — a
processing-error message with the traceback. The immediate send view and the reload view were inconsistent for the exact same event. See below:This PR makes the send-time view consistent with the reload view by surfacing the already-stored error turn inline. See below for the updated behavior:

Root cause
PromptNormalizer.send_prompt_asyncpersists a full error piece to memory(
response_error="processing",converted_value= exception + traceback) and then re-raises (pyrit/prompt_normalizer/prompt_normalizer.py). In the GUI flow that re-raise propagated up throughadd_message_asyncto the route's catch-allexcept Exception, which returned an HTTP 500 (inadd_messageroute inattacks.py) with a generic detail, discarding the real error. The frontend rendered that 500 as an ephemeralunknownerror bar via itscatchpath. On reload (GET), the stored error piece rendered normally (hence the mismatch).Fix
In
add_message_async, wrap the send in atry/except. Because the normalizer already persistedthe error turn before raising, when the exception is accompanied by a newly stored error piece
we swallow-and-log, then return the normal
AddMessageResponse(HTTP 200) whosemessagesinclude that error turn. This makes the POST (send) response render identically to the GET
(reload) view. If no new error piece was stored — i.e. the failure happened before the send
(target lookup, validation) — we re-raise so the route still reports a genuine 404/400/500.
pyrit/backend/services/attack_service.py: added a module logger; wrapped_send_and_store_message_asyncwith a guard that keys offhas_error()on a piece not present before the send.Why the service layer (not the route or the normalizer)
retry on it). This is left untouched.
captured as displayable conversation data, so translating the raise into that stored turn
belongs here — not in the generic route handler, and not by weakening the normalizer.
has_error()), so it covers any error type that stores a piece thenraises.
Behavior changes
processing) send path changes: HTTP 500 → 200, and the frontend now renders it via the success path as a normal assistant error turn (bar + traceback body) instead of the ephemeralunknownbar.blocked(content-filter) andemptypaths (they never raised); programmatic/attack callers; genuine pre-send failures (stillerror out); what is replayed to the target on subsequent sends.
catch, so the "repopulatefailed input for re-send" convenience no longer fires for this case (the error is now a
persisted turn, consistent with reload).
Testing
pytest tests/unit/backend/test_attack_service.py -k add_message→ alladd_messagetests pass,including the 2 new tests and the existing target-not-found re-raise test.