Announcing LocalLlama discord server & bot!

[-]

TheMrCake@reddit

Please No.

Discord is a dystopian, gated version of what forums used to be. No I don't want to give Discord all my data. This is peak enshittification.

This is yet another community where all information will be gated behind discords walls. We as a society used to have real forums that were kept alive by donations, not VC money.

[-]

HOLUPREDICTIONS@reddit (OP)

I mean you don't have to join, it's optional, the subreddit will keep running as it is, discord servers usually have the most alpha because of the very gated community reason

[-]

yeah-ok@reddit

Time to move with the 1980s yuppie slang - have to support the bubble somehow /s

[-]

WhyIsItGlowing@reddit

Reddit is a dystopian, gated version of what forums used to be.

Discord is a dystopian, gated version of what IRC used to be but with Slack-clone features.

[-]

Ok-Internal9317@reddit

It's only a matter of time until we all migrate to 4chan

[-]

olbez@reddit

Reddit is at least indexed by searches and is accessible without a specialized client software

[-]

Visual-Wrangler3262@reddit

Reddit still lets you register with a throwaway email, and posts are public by default. Far less dystopian than Discord.

[-]

OCT0PUSCRIME@reddit

Discord wouldn't be so bad if the search function... functioned

[-]

iamapizza@reddit

Discord search is so bad I call it discard

[-]

Evening_Ad6637@reddit

I am glad to see this is the most popular comment here

[-]

BusRevolutionary9893@reddit

Discord, where valuable information goes to die and never be seen again.

[-]

dataexception@reddit

You mean I'm not the only one who turned off notifications after a week?

[-]

Kind_Background6266@reddit

Zero cost. Works on Windows, Linux, even that toaster you call a laptop. Copy, paste, profit. . Blame the cat. b3ll3.c0mf0rth ← yes, zeros, no jokes. ai at its best baby.

[-]

armanidev_@reddit

Great point. I run qwen2.5:7b and gemma4:e4b locally on a Pi 5 and the quality for everyday tasks is surprisingly good. The cost savings vs cloud APIs are real.

[-]

sicutdeux@reddit

No discord for me, I rather prefer to go back to IRC.

[-]

Hot-Employ-3399@reddit

Fuck discord in general. Its spam in particular

[-]

Constant-Bonus-7168@reddit

Solid post. The implications here are worth thinking about more deeply.

[-]

Constant-Bonus-7168@reddit

Solid post. The implications here are worth thinking about more deeply.

[-]

autonomousdev_@reddit

This sounds exciting! What kind of features can we expect from the LocalLlama bot, and how can we get involved in the server?

[-]

Independent-Sir3234@reddit

Interesting — does it handle the moov atom placement for streaming or do you need to post-process with ffmpeg?

[-]

Independent-Sir3234@reddit

What's the sustained throughput after thermal throttling kicks in? That's always the number I care about more than the initial peak.

[-]

Constant-Bonus-7168@reddit

Great point raised here. Would be interesting to see where this goes.

[-]

a_beautiful_rhind@reddit

Discord asks for phone number and now face verification in some places.

[-]

a_beautiful_rhind@reddit

If so, internet is over. All hail internet2.

[-]

inagy@reddit

The internet is long over. Everything is in posession of large corporations, who only care about collecting data, and not really about serving the end user.

People should claim back the internet for themselves. Fediverse stuff seems promising, but imho it's way overcomplicated to get traction in the masses of normies. I mean, even for me as a nerd understanding the way it works makes my head hurt.

[-]

purdycuz@reddit

What is internet2 🤣 do I need to start buying domains? This endless pain of finding a good name for something and DOH

[-]

KURD_1_STAN@reddit

That is internet2

[-]

Dry-Influence9@reddit

not every platform, there is a few open source ones out there.

[-]

Normal_Definition124@reddit

‏صمم لي هات الصبي مع امرأة

[-]

Waste-Intention-2806@reddit

X,,,,,, c c, z,x,ஏ,,,,ஏஏ,, ,,,,,,,,,,,,,,ஏஏஏ ஏ ட

nnouncing LocalLlama

[-]

bephire@reddit

Would it be feasible to release regular archives of all messages/discussions in the server? We wouldn't want to lose valuable discussions and information that would otherwise become inaccessible if anything were to happen to the server.

[-]

Corporate_Drone31@reddit

Seconding this.

[-]

purdycuz@reddit

I’d also add that I can’t pay the many AI subscriptions anymore, for stuff I can get for free with a little effort. I wonder if local will one day take over.

[-]

Equivalent-Buy1706@reddit

This is exactly why I run local too. Currently serving MiniMax M2.5 (230B) from my M5 Max in San Juan at www.gorroai.com — free to try.

[-]

ansibleloop@reddit

Discord is a shit replacement for forums

Reddit is shit too but at least I can open a web browser and read through all the posts here

Can't do that on discord - have fun trying to load old messages

[-]

swagonflyyyy@reddit

The communities on Reddit are pretty shit, but reddit as a site itself is really good for replacing traditional forums.

So good, in fact, that its causing problems with the internet like polarization, echo chamber turf wars, etc. But that's a case of a community taking things too far.

[-]

ansibleloop@reddit

When you think about it, the enshittification of the internet was inevitable

[-]

purdycuz@reddit

Hopefully, I remember when I searched a howTo on yt and got 10 videos with howto’s today I find 5000 results each 2 hour videos talking 1 hour and 55 minutes unboxing and about themselves the remaining 5 minutes are advertising. -.-

[-]

Fuzzy_Pop9319@reddit

No, they are really not, they have turned their fears of their users up so high that they have ruined their model, I would bet 100 bucks that by 2035 they are gone.

[-]

FullOf_Bad_Ideas@reddit

Are you trying to get a dataset for training llm's?

[-]

bephire@reddit

No. If I did, I could simply export the server's messages myself since it is a fairly easy process. However I do want all discussions to be accessible to the public in the internet permanently. We should want to minimize the risk of the loss of data if anything happens to the server (e.g, if messages / channels are deleted, if the server is griefed by a rogue mod, raids, if Discord becomes temporarily or permanently unavilable, etc.) We would also be making it availble to all who don't use or want to use Discord, which is a plus.

[-]

FullOf_Bad_Ideas@reddit

I would like this community to stay in the public too, not move in large numbers to Discord. Archives are a middle ground in a way, but I am not a fan of that, as it excludes participation.

[-]

mpasila@reddit

Free access to information is also important instead of hiding everything in some random discord server.

[-]

FullOf_Bad_Ideas@reddit

Sure, I would prefer this to be public facing. Discord servers make public internet appear smaller and are harder to search, that's why I am not really using them.

But i kind of doubt that people would actually download archived Discord chats just to read them, that's just a quite rare behavior. There are some people reading through some schematics and documentation from 30 years ago for fun, but most people have a strong "new" bias, and in this field it's even more pronounced. People tend to read new papers not going through ones from 2023 or 2024, but older papers are very useful for training llm's, since there's more of them than of new papers.

[-]

HOLUPREDICTIONS@reddit (OP)

Not a bad idea, we can have something like https://github.com/AnswerOverflow/AnswerOverflow

[-]

Desperate_Junket_413@reddit

Joined. Within 3 min the bot tried to refactor my whole repo into a single 2000-line main.rs "for speed."
I now have it on a short leash - only answers if I prepend "sudo make me a sandwich."
10/10 would let half-bake my code again.

[-]

aitechx@reddit

The 9B parameter sweet spot is proving itself for local coding tasks. Quality-wise, Q5_K_M quantization gives a solid VRAM tradeoff on consumer GPUs like the 3060. Context handling has improved significantly in recent model releases.

[-]

RecentJacket3152@reddit

Alexandre Pedrosa works at Microsoft Azure and Meta like an Executive Interoperability SUPERINTELLIGENCEs Architect.

[-]

TheRealCookieLord@reddit

Cannot wait to invite the bot to multiple servers and make the heaviest thinking model contemplate life's meaning.

[-]

madSaiyanUltra_9789@reddit

user: "what is the meaning of life?"
10\^49 thinking tokens generated
assistant: "42"...

user died waiting for the response 10\^36 tokens ago lmao

[-]

roculus@reddit

can you make the "your post is getting popular" discord ad spam a private message to the poster so we don't have to see it every time? Thanks!

[-]

PKCAI@reddit

You may want to pin this for a few days, just to get some more activity

[-]

ceramic-road@reddit

Thanks for relaunching the official Discord!

The community surpassing half a million members, having a dedicated space for deeper technical discussion makes sense.

For those of us running llama.cpp or Ollama on modest hardware, will there be channels dedicated to optimization tips or hardware discussion? Already joined

[-]

Kind_Background6266@reddit

Zero cost. Works on Windows, Linux, even that toaster you call a laptop. Copy, paste, profit. . Blame the cat. b3ll3.c0mf0rth ← yes, zeros, no jokes. ai at its best baby.

[-]

Kind_Background6266@reddit

Zero cost. Works on Windows, Linux, even that toaster you call a laptop. Copy, paste, profit. . Blame the cat. b3ll3.c0mf0rth ← yes, zeros, no jokes. ai at its best baby.

[-]

GodSpeedMode@reddit

This sounds awesome! It's great to see the LocalLLaMA community expanding and finding ways to foster more technical discussions. With 500k users, I totally get the need for a space that caters to those of us who want to dive deeper into the nitty-gritty. The bot for testing out open-source models is a fantastic addition—I can't wait to try it out! I'm also really looking forward to the contests and events; it's nice to have a central hub for that. Just hopped on the Discord—let’s see what everyone is working on!

[-]

Fuzzy_Pop9319@reddit

I am new to Reddit, why would there be so few messages in two months if there are 500K users? thanks

[-]

HOLUPREDICTIONS@reddit (OP)

gpt slop

[-]

Upbeat_Jellyfish4610@reddit

http://goon-world.com/ is better

[-]

the100rabh@reddit

Invite seems to be expired

[-]

crantob@reddit

Discord sucks.

Next?

[-]

boomboominkimspants@reddit

Isn’t that why they call it vibe coding tho?.. like you still checking that it’s meeting your vision, but you know enough and are engaged enough that u notices it’s messing up

[-]

Appropriate_Cry8694@reddit

Discord is a cancer especially for open source projects, a lot of useful info simply dies with the server in the end.

[-]

raysar@reddit

Be carefull discort is cool for friend but not for sharing public data. It's impossible to find information by searching on discord. Even bot can't do that XD

[-]

thedatawhiz@reddit

Not a good idea...

[-]

shadow-studio@reddit

i'd prefer the "more technical discussion" to happen right here. discord isn't designed for keeping a knowlegde base that everyone can access freely. i'm ok with the idea of having a bot to test models, but otherwise i consider discord very inadequate and annoying for most things, just like whatsapp groups.

also, platform ensh!ttification is a thing.

[-]

ilintar@reddit

Says the invite is invalid/expired.

[-]

HOLUPREDICTIONS@reddit (OP)

odd, does this work: https://discord.gg/4R7xS5hMdN

[-]

ilintar@reddit

Oh, never mind - apparently it doesn't work from the web client, but if I paste it into Discord it works.

[-]

HOLUPREDICTIONS@reddit (OP)

Alt invite: https://discord.gg/4R7xS5hMdN

[-]

Unigma@reddit

Could we add a few channels about Machine Learning and AI in general? For people that are training their own models and or creating projects with LLMs in it? I am working on some RL based training, and wanted to know if there's a more technical oriented channel to discuss this.

[-]

DeepWisdomGuy@reddit

Is this really running on a cerebras cluster?

[-]

HOLUPREDICTIONS@reddit (OP)

on their cloud instance (they have a generous free tier)

[-]

Cosack@reddit

I do get what this is for, but still... you gotta appreciate the irony lol

[-]

HOLUPREDICTIONS@reddit (OP)

I mean you commented this on reddit so idk

[-]

Accomplished_Ad9530@reddit

Nice! Does the bot have retrieval capabilities? It might be useful for novices to have access to a chatbot with access to documentation for common inference engines and front ends, and perhaps localllama discussions. Sort of an interactive FAQ, which would help out with a lot of repeat posts here.

[-]

HOLUPREDICTIONS@reddit (OP)

I think web search solved this better

[-]

Accomplished_Ad9530@reddit

It should have, but newbies don’t seem to bother 🤷‍♂️

[-]

mrjackspade@reddit

Hopefully this server fairs better. The last server fucking sucked because the general chat was the landing room, 99%+ of the users only joined for tech support, and no one enforced the help channel rules because they didn't want to be "mean"

So at any given point in time the general chat was just being flooded by the same easy-to-google tech support questions, squashing any real conversation.

[-]

HOLUPREDICTIONS@reddit (OP)

sounds like a good task for the discord bot

[-]

TheRealSerdra@reddit

You may want to pin this for a few days, just to get some more activity