Validating a visual orchestration tool for local LLMs (concept feedback wanted)

Posted by HarjjotSinghh@reddit | LocalLLaMA | View on Reddit | 6 comments

Hey r/LocalLLaMA,

Before I build this, I want to know if it's actually useful.

The Problem (for me): Running multiple local models in parallel workflows is annoying: - Writing Python scripts for every workflow - Managing async execution - Debugging when things break - No visual representation of what's happening

What I'm considering building:

Visual orchestration canvas (think Node-RED but for LLMs):

Features (planned): - Drag-and-drop blocks for Ollama models - Parallel execution (run multiple models simultaneously) - Real-time debugging console - Export to Python (no lock-in) - Local-first (API keys never leave the machine)

Example workflow: Question → 3 local models in parallel: - Llama 3.2: Initial answer - Mistral: Fact-check - Mixtral: Expand + sources

All running locally. Target: <10 seconds.

Tech stack (if I build it): - Mext.js + React Flow (canvas) - Express.js/Hono backend - WebSockets + SSE (real-time updates) - LangChain (orchestration layer) - Custom Ollama, LMStudio, and vLLL integrations

Why I'm NOT building yet:

Don't want to spend 3 months on something nobody wants.

The validation experiment: - IF 500 people sign up → I'll build it - If not, I'll save myself 3 months

Current status: 24/500 signups

Questions for local LLM users:

Is visual orchestration useful or overkill?
What local-model workflows would you build?
Missing features for local deployment?
Would you PAY $15/month for this? Or should it be open-source?

What I need from r/LocalLLaMA:

Brutal technical feedback: - Is this solving a real problem? - What integrations matter most? - Performance concerns with Ollama? - Should I open-source the Ollama connector?

Mockups: Link in comments - concept only, no product yet.

The ask:

If this sounds useful, sign up (helps me validate) If this sounds dumb, roast it (saves me 3 months)

Thanks for the feedback!

[-]

Trick-Rush6771@reddit

This is exactly the kind of friction people keep running into, and it sounds like you already have a sensible feature set in mind. We often see the biggest wins come from making execution deterministic and visible so non-devs can reason about failures, and from exporting a canonical JSON/Python representation so teams do not get locked in.

Some options people evaluate for this space are LlmFlowDesigner, LangFlow, and n8n depending on whether you want a more LLM-focused canvas or a general automation tool; regardless of tooling, make sure the canvas shows live token usage and a replayable prompt path for each run so debugging and cost tradeoffs are obvious up front.

If you want feedback on a specific node model or the UX for parallel model runs I can share a few gotchas folks usually hit.

HarjjotSinghh@reddit (OP)

Thank you so much for the feedback! Would love to hear the ideas!

Marksta@reddit

Sounds like it won't be opened source so nobody will be interested in it.

We’re planning on making it open-source

nunodonato@reddit

so, n8n?

N8N for normies and non-devs with an exceptionally low learning curve.