What comes closest to ComfyUI for LLMs?

[-]

SomeOddCodeGuy@reddit

You actually have a few choices here. * [Omnichain is literally comfyui for LLMs](https://www.reddit.com/r/LocalLLaMA/comments/1dhyyob/comfyui_for_llms_second_tutorial_is_up_build_your/), if its the interface you are after. * [Langflow ](https://github.com/langflow-ai/langflow)is the most well known workflow project * [Flowise ](https://flowiseai.com/)is a big one as well * This guy made a fancy [ui for a no-code workflow for LLMs](https://www.reddit.com/r/LocalLLaMA/comments/1e1c3h9/vectorvein_the_opensource_nocode_ai_workflow/) * And I personally use [my own project, Wilmer](https://www.reddit.com/r/LocalLLaMA/comments/1dnsfh9/sorry_for_the_wait_folks_meet_wilmerai_my_open/). It doesn't have a UI yet and may cause your eyes to bleed trying to learn how to use it. But I can say that it works, and it works well, as my wife and I have been using it exclusively to interact with LLMs for the past 3 months. It's my passion project, and[ I can do neat things with it](https://www.reddit.com/r/LocalLLaMA/comments/1ctvtnp/almost_a_year_later_i_can_finally_do_this_a_small/) You definitely have options. Each has their own strength: * Omnichain's developer specifically states *"OmniChain is more of a generalist automation platform that doesn't over-burden the language model with all the tasks"* which makes it great for repeatable results, and probably the best option for commercial options. Even with a bad LLM, omnichain will likely produce good results * Langflow and Flowise are very well known so lots of tutorials * Wilmer is the antithesis of Omnichain- the whole purpose of Wilmer is definitely to over-burden the language model with all the tasks lol. Wilmer can either just do a workflow, or it can look at your incoming prompt, let the LLM decide what kind of workflow should be used for it, and do it. If you pick the wrong, Wilmer will produce terrible results. But if you want "Jarvis at home", Wilmer is probably the foundation you want (*that's why I'm building the thing, after all... lol*) Right now, I have 3 instances of Wilmer running to allow me 3 setups: a dev team made up of multiple LLMs, an "advisory board" of multiple LLMs who all work together to help me solve generalized problems, and a single AI assistant who has 6 LLMs working in tandem to generate a single response.

Reply

[-]

0x-dawg@reddit

Now that we can build full-stack AI apps literally in seconds thanks to Cursor, we don't use no-code solutions anymore. However they are still great to grasp modern AI concepts fast. I personally like flowise, which is quite close to langflow, but I would recommend to use them mainly for prototyping or testing purposes. For production-ready agents, you should consider moving to Dify. You also have VectorShift or Voiceflow, depending on your specific needs, but those two are not open sourced

Reply

[-]

Open_Channel_8626@reddit (OP)

Omnichain looks good, I like how you can set everything like context length and sampler settings all in one node. I get the impression that Flowise and Langflow are the two "big ones", and that Langflow is designed to work well with langchain, which might make it a good starting point. Wilmer does look really interesting. I could see the group chat functionality as being useful for coding as you can get multiple candidate snippets from different LLMs

Reply

[-]

SomeOddCodeGuy@reddit

>Wilmer does look really interesting. I could see the group chat functionality as being useful for coding as you can get multiple candidate snippets from different LLMs Yea the group chat was an unexpected perk. I really didn't set out to be able to do that, but once it worked I couldn't stop lol Wilmer's main purpose is to be able to use a bunch of LLMs at once for a single API call. So you send a coding question to the Wilmer api. It categorizes that prompt, realizes its coding, and you go to a workflow where 1. Maybe the first model is Codestral 22b set to deterministic presets that answers the question, 2. Then the second is Codegeex set to deterministic that code reviews 3. Then the third model is Codestral again that takes in both outputs of the previous nodes and fixes the issue 4. Finally Llama 3 8b with Default settings (0.7 temp, etc) responds using the final output from the 3rd codestral code as RAG to respond. So Wilmer's main goal is basically to upgrade my AI assistant beyond what local LLMs could normally do. I got tired of proprietary AI being that much more powerful than my local models, but realize a bunch of fine-tuned models working together could do the trick. Even if we never got another local model again starting today, I've only scratched the surface of how powerful I can make my assistant with Wilmer using what we already have, so I'll still be tinkering with this a year from now =D

Reply

[-]

tolitius@reddit

take a look at [towel](https://github.com/tolitius/towel) more powerful than chains, simpler than graphs: space_trip = plan([ step(pick_planet), pin('are_you_ready'), step(how_ready_are_you), route(lambda result: 'book' if result['how_ready_are_you']['score'] > 95 else 'train'), pin('train'), step(space_bootcamp), route(lambda x: 'are_you_ready'), pin('book'), step(reserve_spaceship) ]) where steps are just Python functions powered by LLMs: @towel(prompts={'available': """list three unexpected planets or moons that are currently available for human space tourism. include a brief, funny description for each."""}) def pick_planet(): planets = llm.think(prompts['available'], response_model=List[Planet]) choice = int(input("enter the number of your planet to go to: ")) - 1 return {'destination': planets[choice].name}

Reply

[-]

Open_Channel_8626@reddit (OP)

Thanks this is an interesting approach

Reply

[-]

prompt_seeker@reddit

make comfyui custom node. it is super easy. or there are some LLM node for comfyui, so you can use them.

Reply

[-]

IONaut@reddit

What we need to complete the whole thing is a comfy node to run other comfy workflows as a sub-process.

Reply

[-]

tommitytom_@reddit

Keep an eye on this PR, as it will eventually allow subgraph support: [https://github.com/comfyanonymous/ComfyUI/pull/2666](https://github.com/comfyanonymous/ComfyUI/pull/2666)

Reply

[-]

Open_Channel_8626@reddit (OP)

This looks great and apparently might be coming soon ```The new Comfy Org team (Comfy/me/robin/yoland/data/hcl/pythongossss/...) are considering this PR a priority to get figured out and merged. It's a massive change that needs a lot of validation, but it's a very important foundational feature to enable a lot of future expansion of what ComfyUI can be capable of natively running. ```

Reply

[-]

Open_Channel_8626@reddit (OP)

I might make my own LLM custom node yeah. I use some already for example the florence2 one

Reply

[-]

bankimu@reddit

I made a comfy ui node that takes a prompt and a model and outputs the result as a text. It's in GitHub but I can't link it because I don't want to dox myself.

Reply

[-]

Storhemulen@reddit

[https://rivet.ironcladapp.com/](https://rivet.ironcladapp.com/) is kinda cool, and does the whole bubble diagram thing.

Reply

[-]

Open_Channel_8626@reddit (OP)

Thanks, this one looks nice, slightly different style to langflow etc which is cool

Reply

[-]

cyan2k@reddit

ComfyUi is the closest to ComfyUi. I don’t know why but except NativeInstrument‘s Reactor I absolutely hate any kind of node based software. It’s all so clunky even if you optimize your graph nobody except you understands anything about it unless the user spends hours following virtual cables, then you install a node by accident that hacks your pc because security is an afterthought (and really hard to do in such an environment), after reformatting your pc you think „ah I just install the most popular flows“ and after downloading ten of those not a single one works out of the box and fixing those takes four times as long as implementing the shit on your own. (Try it out: blank comfy install->civitai sort most popular workflows. Like two out of the top 50 work. ) Then you throw it away and are cloning the diffusers repo because it’s basically ConfyUI but instead of drawing spaghetti you write three lines of code. And that’s what I don’t get the most. If you love your modularity and being able to make crazy low level shit why don’t you just code lol. What do nodes give you what code can’t? „I can’t code :(„ pls if you can use comfy you can use the diffuser lib. Instead drawing lines you write a line of text that define which output goes into which input (and save 30min of time getting your lines looking good). That’s basically all the difference. There’s nothing difficult about it especially in a time in which LLMs can literally guide you through whatever you want. So even if you think you can’t code check out hughingface‘s diffuser lib (or transformer for LLMs) because they are also amazing starting points to learn that shit because millions of examples, a good documentation and community, and you learn an actual useful skill.

Reply

[-]

Dead_Internet_Theory@reddit

I love Blender's nodes and also Davinci Resolve Fusion's, and if I was really into simulations I think I'd love Houdini too, Substance Designer as well. Nodes can be an excellent user interface, doubly so if the alternative is 🤢Python 🤮

Reply

[-]

Open_Channel_8626@reddit (OP)

Nodes seems to be getting more popular yes

Reply

[-]

randomanoni@reddit

typing has got your back, bro.

Reply

[-]

Aischylos@reddit

I code in diffusers for my diffusion projects, but comfyui does have some advantages over it. Diffusers doesn't always have the best documentation and there are times that comfy nodes exist before a diffusers implementation.

Reply

[-]

Open_Channel_8626@reddit (OP)

> and there are times that comfy nodes exist before a diffusers implementation. Yes but I would put it even more strongly- I have used hundreds of nodes that don't have a diffusers implementation yet. I hope diffusers can catch up in the long run, but this is something that will happen on the scale of multiple years.

Reply

[-]

Open_Channel_8626@reddit (OP)

It seems like node-based audio software like Native Instruments’ Reaktor are about as spaghetti as Comfyui from what I can see. There are issues with comfy package management yes (the famous security issue, not having enough node packs installed by default and not adding node packs fast enough to keep up with the github community.) If you broadly install large amounts of the contents of comfy manager on day 1, that helps an enormous amount. After that you only have to hunt for new custom nodes occasionally and its usually just a case of github search and then add the URL to comfy terminal. I'm actually going in the other direction, I learnt huggingface transformers and diffusers first and I moved to comfy ui after that. For images in particular I quite like having the nodes for fast iteration. I can tweak buttons, reroute a couple of spaghetti noodles and then press generate within a few seconds. I think this is why such an interface is so common for creative software (audio/video/image/GFX/3D rendering etc)

Reply

[-]

ArtyfacialIntelagent@reddit

This rant is basically just a more polite version of "learn to code, noob" and is not helpful. I code professionally and still think ComfyUI is an excellent tool for designing and sharing advanced txt2img and img2img workflows in the context of Stable Diffusion. I can only imagine how indispensable it is for people with less coding experience but who want to dig deeper into generative AI than the easy web UIs allow. If it's not for you, fine, but OP's question is perfectly legitimate so don't diss it.

Reply

[-]

randomanoni@reddit

You don't say it explicitly, but do you mean that coding and using a code graph go hand in hand? Graphs are a good tool to help with reading and understanding code for us that don't have infinite working memory like some legendary rock star coders seem to have? On the other hand, using graphs to "get things done because I don't have time to read and understand all this code" is risky. I understand the temptation really well. Where do we draw the line for auditing code ourselves? If you're actually able to audit code at all you're in a relative luxury position. But <100 coding IQ people or equivalent exist according to the normal distribution, so what should "we" do when people distribute their free-money.exe files on ai-codingstorehub.co? This is going nowhere, sorry. FWIW, I give this rant to random elderly people I meet on the street too. I'm planning on building a large building that people can see from afar where me and my followers can teach my ways using song and fear.

Reply

[-]

milksteak11@reddit

His main problem is trying to use other people's old workflows when nodes, comfy itself and the manager are updated usually daily. Recommending diffusers over comfy is hilarious though

Reply

[-]

adkallday@reddit

This is my vote : https://github.com/langflow-ai/langflow It’s just like what you’re describing

Reply

[-]

Open_Channel_8626@reddit (OP)

yeah langflow integrating well with langchain is a decent bonus

Reply

[-]

kristaller486@reddit

Dify I think. Maybe this is not a full analog, but the same concept. https://github.com/langgenius/dify

Reply

[-]

Open_Channel_8626@reddit (OP)

thanks, looks like a good langflow/flowise competitor

Reply

[-]

Brahvim@reddit

I was about to mention text-generation-webui, but...

Reply

[-]

MicBeckie@reddit

I never tried by myself, but maybe that’s the tool you’re looking for: https://github.com/FlowiseAI/Flowise

Reply

[-]

tvetus@reddit

Haven't tried it yet, but saw Langflow.

Reply

[-]

hapliniste@reddit

I think comfyui can do LLM too

Reply

What comes closest to ComfyUI for LLMs?

Reply to Post

32 Comments

SomeOddCodeGuy@reddit

0x-dawg@reddit

Open_Channel_8626@reddit (OP)

SomeOddCodeGuy@reddit

tolitius@reddit

Open_Channel_8626@reddit (OP)

prompt_seeker@reddit

IONaut@reddit

tommitytom_@reddit

Open_Channel_8626@reddit (OP)

Open_Channel_8626@reddit (OP)

bankimu@reddit

Storhemulen@reddit

Open_Channel_8626@reddit (OP)

cyan2k@reddit

Dead_Internet_Theory@reddit

Open_Channel_8626@reddit (OP)

randomanoni@reddit

Aischylos@reddit

Open_Channel_8626@reddit (OP)

Open_Channel_8626@reddit (OP)

ArtyfacialIntelagent@reddit

randomanoni@reddit

milksteak11@reddit

adkallday@reddit

Open_Channel_8626@reddit (OP)

kristaller486@reddit

Open_Channel_8626@reddit (OP)

Brahvim@reddit

MicBeckie@reddit

tvetus@reddit

hapliniste@reddit