I wrote a comprehensive guide to NATS — the messaging system that replaces Kafka, Redis, and RabbitM
Posted by Jainal09@reddit | Python | View on Reddit | 21 comments
I've been working with Kafka and aiokafka in production and kept running into the same limitations — partition rebalancing, watermark commits, DLQ as an afterthought.
NATS with JetStream solves most of these at the protocol level. This guide covers the full mental model with Python examples using nats.py throughout — pub/sub, JetStream pull consumers, per-message acks, graceful shutdown with asyncio, and the new 2.11/2.12 features.
paperlantern-ai@reddit
Honest question - how does NATS handle the situation where your consumer falls behind by hours or days? With Kafka the retention model means you can just rewind and replay. With JetStream I've seen mixed reports on how well it handles large backlogs. Curious if anyone's stress tested this in production.
Academic-Vegetable-1@reddit
Most teams looking to replace Kafka with something else didn't need Kafka in the first place.
hl_lost@reddit
ngl i've been curious about nats for a while but every time i look into replacing kafka at work i get the "if it ain't broke" talk from the team. the partition rebalancing pain is real though — we lost like 3 hours debugging a consumer group issue last quarter that turned out to be a rebalance mid-deploy.
how's the observability story? that's usually the thing that kills adoption for us. kafka's ecosystem around monitoring/tracing is pretty mature at this point and getting buy-in without good dashboards is rough.
Aggressive_Pay2172@reddit
totally agree on Kafka complexity though
it’s powerful but comes with a lot of operational overhead
NATS feels much lighter in comparison
which is appealing for smaller teams
benevolentempireval@reddit
Are these comments all AI slop?
TheRealStepBot@reddit
You’re ai slop like the article.
TheRealStepBot@reddit
Article goes back and forth on this first saying per message commits are good, the apparently more recently jet stream is starting to actually understand the point of Kafka in a streaming context and introduced batch commits. Per message commits are highly undesirable in many situations.
Which is to say that there is not atomic read write batch commits is still a massive reason to not even look at jet stream as a Kafka replacement even in a theoretical sense.
I think the bigger concerns though is that it’s a young project with a precarious governance and maintenance picture that is still trying to work up a supporting ecosystem for itself essentially in terms of compatibility with external tools.
I think the choice to not shoot for Kafka api compatibility was an odd one that will hold back growth and adoption as well.
The_Ritvik@reddit
What’s a use case like say boom I’m a junior developer and I’m vibe coding and I got no clue what’s going on and I got a simple automation task, and my boss tells me to look into NATS and I crack my knuckles and am ready to vibe code the heck outta this??
Jainal09@reddit (OP)
Vibe coding is easy unless it gets hard. General rule of vibe coding at work -- If you don't know wtf u are doing, know it before u vibecode.
No the working, concepts and terminology. You cant vibe code and present a demo and don't know how underlying thing works. TBH - The whole reason of this blog was to understand NATS from a kafka pov (coming from a kafka bg)
U can go through the sections of the blog that u feel are matching to ur use case / requirements and then vibe code saying hey build me a jetstream consumer with a persistent k/v store
Rather hey build me a NATS project in python that reads from NATS
The_Ritvik@reddit
Gotcha, but I guess I’m still a little confused. What is NATS? Is it pub sub? If so, why would I use it over a cloud based subscription service, like say SQS or SNS for example? Just trying to understand the use case here. Also, I got no clue what’s Kafka is.
AustinWitherspoon@reddit
It's not a black box, the code is all publicly available on GitHub.
Nats was made way before vibe coding, the use is whenever you need a message queue, and you want to go with a free self hostable option.
They have good docs on their website that will walk you through how to get started, but it's not meant for 5 year olds, it's meant for programmers
Jainal09@reddit (OP)
Thats a more ask your manager question. Some companies wanna go a non vendor lockin cloud agnostic self hostable version. Some companies serve private air gapped on prem customers.
To asnswer your question - Nats in a tldr is both pub / sub (non persistent) and persistent queue with a lot of features included
gfranxman@reddit
How does NATS stack up against mosquitto?
sudonem@reddit
In some ways NATS could be thought of as MQTT on steroids.
Basically all of the benefits, but the traffic can be on-demand, and you have the ability to scrub backwards or forwards in time at the messages.
Really interesting stuff.
Jainal09@reddit (OP)
NATS actually supports MQTT natively so you can connect Mosquitto devices directly to a NATS server. Different tools solving different problems: didn't cover this in the post but worth exploring separately.
andrewcooke@reddit
i've just been setting up a nats cluster in aws and writing a demo client so that a customer can play around with it. it's been very impressive - simple, easy to use tools. i've used nkey for auth and ssl certificates to secure connections. it was surprisingly painless. seems simpler than their current activemq layer.
ac130kz@reddit
I've been using NATS JetStream for a few projects, and it feels like Kafka (especially with KIP-932 Kafka Queues) is much better suited for reliable message delivery with a decent amount of custom handling code. On the NATS side there are: governance problems, reliability concerns from various 3rd party reviewers, client code is mostly focused on Go (nats-py is quite a bit out of shape, and doesn't have even simple features, such as batch publish, and the modern replacement isn't fully ready yet), some features are a bit too automatic (let's say one wants to submit an ack via an unreliable network connection with maybe custom batch processing, you can't, there will be a redelivery, which you can't really track the count of it, MaxDeliver counter is bumped with every node it touches). For anything easy to start though, maybe an MVP, yeah, I found NATS JetStream to be quite enjoyable.
4xi0m4@reddit
Fair point on the nats-py situation. The async story is definitely rough right now, and the Go-centric design of the server shows through the Python client. The nats-core rewrite looks promising though, and once it stabilizes it should close that gap considerably. For now though, for basic pub/sub the current client works fine, and JetStream pull consumers are usable even if the ergonomics could be better. The Jepsen analysis is a good read, btw. It puts NATS in a reasonable light for most use cases even if it is not CP.
CrackerJackKittyCat@reddit
Might be good to mention that the NATS server per-connected-client outbound message buffer is a user-space buffer, separate from the kernel-side socket buffer. That it happens to be 64k made me think of the kernel-side socket buffer first and needed to research to clarify.
CrackerJackKittyCat@reddit
I think one of the best things NATS has going for it over Kafka would be the 'simple client' approach allowing for quick feature parity across clients / client languages. That Kafka was born of java and with a java-fat-client-first (or Confluent librdkafka) mentality then became a weight around its neck.
Jainal09@reddit (OP)
agrred!