NeKot - a terminal interface for interacting with local and cloud LLMs

Posted by Balanceballs@reddit | LocalLLaMA | View on Reddit | 25 comments

Been working on this for a while, since I could not find a decent solution that is not abandoned or has all the features I need.

Supports Gemini, OpenAI and OpenRouter APIs as well as almost any local solution (tested with llama-cpp + llamaswap, ollama, lmstudio).
Has support for images, presets (each preset can have it's own settings and system prompt), sessions.
Written in GO , so no interpreter or runtime required.
Has support for basic vim motions.

Repo: https://github.com/BalanceBalls/nekot

[-]

natufian@reddit

Looks beautiful! Truly.

But, as an end user it is my sacred duty to offer unsolicited criticisms and feature requests 😅.

In the config.json example on github the localhost example given lists the port as 1143, I think it is more generally 11434 (spells out L.L.A.M.A).

-perhaps drop the OPENAI_API_KEY requirement when working local? (not a big deal)

TAB progresses focus to the Next window. Hopefully Shift + Tab for previous tab is in the works?
Can't copy paste any of the output ... 😢
right arrow takes you from Settings to [Presets] but left arrow doesn't let you out again.
No arrows keys to select Setting / Preset / Session options. And no vim motion for the same. (unless I'm using it wrong).

So many criticism because this app is legit good enough to be a daily driver. Hope you stick with it man, this program is potentially a game changer!

[-]

Mangleus@reddit

I LOVE the design of NeKot!!

I only do localai and have no openai key. Doing this did not get me unstuck. The app reports error and then i get stuck.

export OPENAI_API_KEY=1

(I also tried randomised key sk- and 48 chars to no avail).

[-]

Thanks for the feedback! I'll see what I can do :)
The 1143 is a typo indeed. I tried to make shift+tab work, but I think there were some limitations from the library I use for interface, will check if that's still the case.
You can't copy text with a mouse (also limitations from the lib, will try to figure it out later), you can copy it from a visual line mode - focus on the output pane, hit space (or shift+v), navigate to lines you want to copy and the press space (or shift+v) again. After you selected the lines - hit 'y' to copy the selected lines

[-]

CV514@reddit

I've almost lost it when I remembered powershell is not able to do setx natively as instructed, despite directly stating SUCCESS in the log. Thanks, Microsoft.

[-]

Balanceballs@reddit (OP)

Yep, never tested koboldcpp, it needs some fixes. There is a ticket for that already https://github.com/BalanceBalls/nekot/issues/54

[-]

CV514@reddit

thb, koboldcpp does not require any interface since it's bundled, but that's not a way, messing around is cool. Thanks!

[-]

Hot-Employ-3399@reddit

Does it support non-chat mode (base models, in llama.cpp it's /completion endpoint)?

[-]

Balanceballs@reddit (OP)

no, for OpenAI compatible APIs it uses /v1/completions endpoint only. What would be the use case for the non-chat mode? I was only focusing on the conversation use case basically

[-]

Hot-Employ-3399@reddit

> What would be the use case for the non-chat mode?

Painless story writing. (chatgpt/gemini have canvas these days at least but local UIs are not very good at it and who knows if models are good)

[-]

noriilikesleaves@reddit

Maybe this is a noob question, but how does something like this compare to LMStudio?

[-]

Balanceballs@reddit (OP)

LMStuido is a whole different beast. It provides a full package - multiple inference backends, api and a GUI. My app is basically a terminal UI for the inference APIs

[-]

noriilikesleaves@reddit

Got it. I should have elaborated. I use LMS to download models, then maybe connect them to vscode. It's not clear to me if your UI can path to and load a saved model file. Can it do that?

[-]

jwpbe@reddit

No, you need to get llama.cpp and use llama-server to serve the model for this program

[-]

MumeiNoName@reddit

What would you say the best alternatives to your software are, and what sets yours apart from them?

This tool looks like what I need

[-]

Hey_You_Asked@reddit

sst/opencode

[-]

Balanceballs@reddit (OP)

I think that would be https://github.com/pythops/tenere . But is lacks support for images, certain APIs and has a simpler UI. Also it is written in rust which I don't know, I guess my other requirement kinda was that I could modify a project to my needs

[-]

Balanceballs@reddit (OP)

this is up to a classifier apparently

[-]

sourceholder@reddit

*identifies as

[-]

jwpbe@reddit

it looks really good, makes me wish i knew Go so that I could add websearch / stdin piping to it

[-]

Balanceballs@reddit (OP)

I actually plan to implement web search for the next release

[-]

chaosmetroid@reddit

I love this give me a retro hifi vibe.