Does gpt-oss:20b’s thinking output cause more confusion than help in multi-step tasks?

Posted by Prestigious_Peak_773@reddit | LocalLLaMA | View on Reddit | 7 comments

I have been experimenting with gpt-oss:20b on Ollama for building and running local background agents.

What works

Creating simple agents work well. The model creates basic agent files correctly and the flow is clean. Attached is a quick happy path clip.

On my M5 MacBook Pro it also feels very snappy. It is noticeably faster than when I tried it on M2 Pro sometime back. The best case looks promising.

What breaks

As soon as I try anything that involves multiple agents and multiple steps, the model becomes unreliable. For example, creating a workflow for producing a NotebookLM type podcast from tweets using ElevenLabs and ffmpeg works reliably with GPT-5.1, but breaks down completely with gpt-oss:20b.

The failures I see include:

forgetting earlier steps
getting stuck in loops
mixing tool instructions with content
losing track of state across turns

Bottom line: it often produces long chains of thinking tokens and then loses the original task.

I am implementing system_reminders from this blog to see if it helps:
https://medium.com/@outsightai/peeking-under-the-hood-of-claude-code-70f5a94a9a62.
Would something like this help?

[-]

aldegr@reddit

Are you passing the reasoning back between tool calls? The looping issue and forgetting previous steps seems to indicate you are not. GPT-OSS does what other models call “interleaved thinking,” which requires keeping the reasoning between tool calls until the final assistant message. I created a notebook showing how tool calling performance degrades when you don’t.

teleprint-me@reddit

Simplify the workflow. Overwhelming the model with information will degrade performance.

Simplify the tool usage and offload the difficulty to those tools. Make those tools available to the model and keep the tool count as low as possible.

Only feed the information relevant to the workflow to the model, then let the model chain tool calls.

For example, if an error occurs, the tool should inform the model exactly what went wrong and it should have utilities in place for self correcting.

Sometimes lowering the logit entropy can help. Improving model performance is a bit of an art form. It's a lot of trial and error.

SlowFail2433@reddit

Small qwens get confused by their own reasoning too

Prestigious_Peak_773@reddit (OP)

yeah makes sense. I feel like when it chains many tool calls before responding, thats when it gets overwhelmed by its own reasoning and loses track of earlier context.

ravage382@reddit

The 120b version does really well with agent tasks if you can manage to run it.

Thanks for the pointer! I'm not able to run 120b on my machine - will try it on Ollama cloud.

I am testing these workflows inside a CLI I am building for running background agents locally.
If anyone wants to reproduce the setup, the repo is here:
https://github.com/rowboatlabs/rowboat