Qwen3.5 keeps thinking

[abdullah-cod9]

Environment

• Model: Qwen3.5-2B-BF16.gguf
• Chat Handler: Qwen35ChatHandler(enable_thinking=True, preserve_thinking=False)
• Library: llama-cpp-python (latest main branch)
• Context: n_ctx=5120, swa_full=True

What I Expect

I want to strictly limit the model's reasoning/thinking phase so it doesn't consume the entire max_tokens budget. I used reasoning_budget with reasoning_start_in_prompt=True because the chat handler injects think into the assistant prefix.

What I Tried

response = llm.model.create_chat_completion(
    messages=core_messages,
    stream=True,
    max_tokens=1024,
    reasoning_budget=64,  # Also tried 128, 256
    reasoning_start_in_prompt=True,
    reasoning_budget_message="\\n[reasoning budget exhausted]\\n",
)

What the Logs Show

The ReasoningBudgetSampler does activate and force-closes the first block:

ReasoningBudgetSampler: initialized (state=COUNTING, budget=64, ...)
ReasoningBudgetSampler: reasoning budget exhausted; entering FORCING state.
ReasoningBudgetSampler: forced end sequence completed; switching to DONE passthrough.

The Actual Problem

The model continues generating tokens as if it is still "thinking." It does not stop at the forced end tag. Instead, it keeps producing text until it hits the global max_tokens limit.

My Question

what is the recommended way to truly cap the total reasoning length so the model transitions to the actual answer promptly?

Any guidance or workaround would be appreciated.

[abdullah-cod9]

Update: Root Cause Found

Root cause: min_p=0.0

I was setting min_p=0.0 explicitly because it's listed in the model's recommended sampling parameters page. I didn't realize this was the actual culprit until several rounds of isolating each parameter individually.
Once min_p is omitted entirely (not passed as 0.0, but left out of the call altogether), the ReasoningBudgetSampler behaves as expected: the forced end sequence is honored, and the model transitions cleanly into the answer phase right after the budget is exhausted.

code
llm.model.create_chat_completion(
messages=core_messages,
stream=True,
max_tokens=1024,
# tools=tools_schema,
temperature=1.0,
top_p=0.95,
top_k=20,
# min_p=0.0,
present_penalty=1.5,
repeat_penalty=1.0,
reasoning_budget=64,
reasoning_start="<think>",
reasoning_end="</think>",
reasoning_start_in_prompt=True,
reasoning_budget_message="\n...Wait, I have been thinking long enough. Let me start answering the user's question.\n",
)

[JamePeng]

👌Good practical analysis