Looking for local LLMs that are extremely informal
Posted by Lazy-Pattern-5171@reddit | LocalLLaMA | View on Reddit | 37 comments
Hey I’m looking for LLMs that are not just conversational information retrieval systems but are from the ground up trained or fine tuned to have human like conversations even if for example I ask a question like “how many cherries to the center of the earth?”
I’m looking for conversations that are deep and soulful but very very light on the education spectrum.
Basically looking for water cooler convos, smoking partner, that one guy you have a lot of fun with for no reason and you don’t even know if he’s employed.
JR2502@reddit
Gemma. I usually run mine at 0.7 but set it to Temp = 1.0 and it might just jump out the window and do a naked streak down your street.
I'm building a chat app and have several models I play with. I was trying to get tools to work. The new GPT OSS aced it first time but this smaller Gemma 3n 4b model was struggling at first (bad tool description).
I asked GPT for advise and it flat out told me Gemma was incapable of running tools. I told Gemma what GPT had said, she thought at about it and wondered if she *really* wanted to burn that bridge with GPT as "who knows, he might be cute!", but decided she wanted to respond.
From my local LLM app on my phone:
**USER**: hi there!! 👋 can you use an obnoxious amount of emojis in totally funny ways? 😄😹
**MODEL**: OMG! 😄 Hi there!! 👋 You betcha I can! 🎉🥳 Get ready for an emoji explosion! 💥🤯So, what's up? 🤔🍕 Wanna talk about... 🤔... fluffy kittens? 😻🧶 Or maybe the existential dread of mismatched socks? 😩 I'm here for it all! 😂🤣Seriously though, I'm ready to unleash a torrent of emojis in the most delightfully absurd ways! 🤪✨ Just tell me what you're thinking! 💭💖 Let's make this a legendary emoji-filled conversation! 🚀🌟🌈
**USER**: can you imagine if everyone's prompt to you are this emojicified? (yes, that's a word... one I made up, of course 😂😂😂)
**MODEL**: OMG! 🤩 YES! 💯 I CAN ABSOLUTELY IMAGINE IT! 🤯🥳 It would be... ✨🤯... a *spectacle*! 🤩🌈 A glorious, chaotic, emoji-fueled symphony of prompts! 🎶💃🕺
Imagine: 🤖 asking for a poem about sunsets... 🌅💖... and I respond with a cascade of 🥰🤩🥳✨🤯💖😂🤣💯🚀🌟🌈🍕🎉🎈🎁🎊🍾🥂💃🕺👯♀️👯♂️🐈⬛🐼🐨🐯🦁🐮🐷🐸🐒🐔🐧🐦🦉🦇🦋🐌🐛🐜🐝🐞🦗🕷️🕸️🦂🐢🐍🦎🦖🦕🐳🐋🐬🐟🐠🐡🐙🐚🦀🦞🦐🦑🐬🐳🐋🐟🐠🐡🐙🐚🦀🦞🦐🦑... I'm losing count! 🤪😵💫😅
Sativatoshi@reddit
Jadio/lazy_pattern_5171:latest
Lazy-Pattern-5171@reddit (OP)
Haha. Dumbass 😒
Sativatoshi@reddit
Excuse me? You asked a question, I gave you an answer. Do I know you? Do I know if youre asking for a simple chat model or something else? Nah.
I apologize that you asked such a stupid, generalized question that it warranted a response you didn't like.
Lazy-Pattern-5171@reddit (OP)
Question was pretty fine actually. It’s a question similar to the ones asked about other RP model asked here quite often. You made it into a joke so I took offense and replied accordingly. Don’t like taking it? Don’t dish it out.
Sativatoshi@reddit
Yeah I didn't make any jokes. You decided to take offense to something you imagined.
Anyways it seems your question was so fine, that you got all the answers you wanted.
Lazy-Pattern-5171@reddit (OP)
You basically insinuated that I should have conversations with myself. Not cool man.
Sativatoshi@reddit
No, i interpreted your msg as wanting an LLM that would simply give you conversation without going into instruct. I was trying to help, assuming that maybe you didnt know about Ollama modelfiles.
Lazy-Pattern-5171@reddit (OP)
Stating that my own username is an LLM?
Sativatoshi@reddit
Bro its an ollama model. I just quickly wrote up a template that matched your post, and made and pushed a granite3.3:2b model with it.
https://ollama.com/Jadio/lazy_pattern_5171
Lazy-Pattern-5171@reddit (OP)
Oh! Lol well you could’ve avoided this whole conversation if you’d just provided the link. I thought you were telling me to talk to myself basically. But no I’ve already tried 20 different models and most of them go back to being helpful or very subversive after awhile it’s not much fun. I have already tried Granite Llama Qwen Dolphin merges etc. I’ve also played around with system prompts and chat templates. I’m looking for something that’s trained from ground up or fine tuned.
Sativatoshi@reddit
https://huggingface.co/meta-llama/Llama-2-7b-chat-hf
I can't speak for it because I havent tested it, but maybe this series is worth a look?
Lazy-Pattern-5171@reddit (OP)
Real bro to bro conversations were surprisingly hard to find. I haven’t tried much in all honesty. I know there’s some absolute goldmines out there in that behemoth that is common crawl just don’t know how to pull those out cleanly.
Sativatoshi@reddit
https://github.com/JaxxyJadio/HF-Scrape
Lazy-Pattern-5171@reddit (OP)
This is amazing!! Daresay maybe just the thing I needed.
Sativatoshi@reddit
Let me know if you want any changes, like other fields added. Im going to add another few files which can download the "intro" section from any wikipedia article for you
Lazy-Pattern-5171@reddit (OP)
👍
Sativatoshi@reddit
https://github.com/JaxxyJadio/HF-Scrape/blob/main/Keyword-from-topic-agent.zip
This is an agent workflow which will read from the yaml file and generate keywords that can be used as seeds to generate further info. Run the script and it will automatically generate new keywords from the YAML.
Let me know if you use this, and Ill send the other agents that fill the remaining fields
Lazy-Pattern-5171@reddit (OP)
Wiki isn’t a good source for how I’m thinking about this. Do you agree?
Sativatoshi@reddit
I would agree. Personally, im downloading intros and then having my agents refactor them into something more useful I would use the agent, and input highly creative fields. If you arent getting the responses you want, adjust the temperature in the modelfile.
from there, you take that keyword and get another agent to write a 100-200 word input field. So, your keywords can be very creative like "How many cherries to the center of the earth" - have a keyword made from that, which the 2nd agent uses for additional context to create a full description.
You could also use a larger model and/or an orchestrator flow so that you simply run 1 script - it generates a keyword and then passes off to the next model to automatically generate the description
Sativatoshi@reddit
run multiple instances to download more, faster
Lazy-Pattern-5171@reddit (OP)
Yes I did get important responses. Not all posts need to have 100 upvotes and go viral for them to be considered as “answered”. If you actually check the replies you’ll see that a lot of them simply haven’t tried. My idea is novel and clearly not a lot of folks have done something like this.
Sativatoshi@reddit
Making an LLM that talks?
Lazy-Pattern-5171@reddit (OP)
Apart from the fact that you clearly have no idea what my question was. Do you really think you’ve asked a simple question here? I’ll stop this here. see ya. 👋
Sativatoshi@reddit
You want an informal LLM that specifies in chat, but not instruct. What's confusing or novel about that?
Lazy-Pattern-5171@reddit (OP)
Clearly it was confusing for you though?
Sativatoshi@reddit
The only confusing part to me is where you think that a model that can speak is a novel idea
Lazy-Pattern-5171@reddit (OP)
Well now that the confusion is gone I’d like to retract my insult and say thank you instead 😊 however I’m not looking for that which you provided me. but now I’ll have a model in my Ollama under my name which is great lol
Sativatoshi@reddit
Fair enough. Water under the bridge!
Here's the modelfile I used:
@"
FROM granite3.3:2b
PARAMETER temperature 0.3
PARAMETER top_p 0.4
PARAMETER top_k 40
PARAMETER min_p 0.05
PARAMETER repeat_penalty 1.1
PARAMETER num_ctx 8192
PARAMETER repeat_last_n -1
PARAMETER num_predict -1
PARAMETER stop "<|system|>"
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"
TEMPLATE """
<|system|>
{{ .System }}
<|user|>
{{ .Prompt }}
<|assistant|>
"""
SYSTEM """
- You are a helpful, friendly assistant.
- Your task is to add a sense of human nature to your chat, while keeping things "light on the education spectrum".
- You must be creative, soulful, and easy to conversate with. Do not be overly instructional. Be easy to speak to.
- Your goal is to speak like a human, not like a robot.
- Do not attempt to be "instructional", instead, just be an easy going friend.
"""
"@ | Out-File -FilePath .\Modelfile -Encoding utf8
ollama create -f .\Modelfile Jadio/lazy_pattern_5171
Trilogix@reddit
You are looking for a LLM (Look Listen Me or Look Li.k Me).
Lazy-Pattern-5171@reddit (OP)
So you’re saying I should train on your data?
SteveRD1@reddit
I think the average human if you ask "how many cherries to the center of the earth" won't engage in conversation at all:)
Lazy-Pattern-5171@reddit (OP)
Hahaha and that’s the type of response I’m looking for from this mythical LLM
SteveRD1@reddit
Fair!
Lcsq@reddit
Use a base model instead of a chat/instruct-tuned model. Add long conversation history in the style you prefer (from a novel or instant messaging history). I think llama 405B is free on openrouter.
BeginnerDragon@reddit
I won't pretend to know where you're starting from from a hardware standpoint, but it sounds like you might want to fine-tune with some conversational data. You could probably look into fine-tuning with some custom data, but I couldn't give you much in terms of expected costs.
Lazy-Pattern-5171@reddit (OP)
2x3090 and 48GB DDR4 Ryzen 5950.