There's been some research available discussing how tone and prompt quality can drastically impact the output of the LLMs. Anything from a negative tone to a spelling mistake could potentially result in significant changes to the results - partially due to their tokenization scheme as well as training data.

This got me thinking - should we be running a sanitization pass on prompts before they hit the main model doing the work? Essentially feeding user input through a lightweight LLM whose only job is to clean it up. Change tone, fix spelling, normalize casing, tighten grammar; then passing that polished version to a second LLM to do the real work.

I have been working on internal tools at work to help empower my colleagues with AI driven tools. When I've done my internal testing and evaluation - I generally get satisfactory results, but I've been having difficulty in getting consistent outputs when having others try to leverage the tools. I think part of it is in the prompt quality (e.g. some users expect they can paste in internal company-specific documents or phrases and the LLM will automatically understand it).

So I'm curious:

• Is anyone running a pre-processing LLM in front of their main model to sanitize input?
• Are you using a smaller/cheaper model for the cleanup pass, or the same model with a system prompt?
• How does diversity of the input sanitization LLM impact the main model (e.g. using GPT to feed Claude models vs Claude to Claude)
• Are there open-source tools or frameworks already doing this? I have seen some tools using smaller models for things like web-search or file search operations, then pass the results to the larger model - but nothing for the input sanitization.

It's been hard to understand the true impact of understanding how our inputs are impacting our results. Internally it always feels like the answer is that the model isn't good enough yet - but maybe it's just the way we're asking it that is making the impact.