System design tip: Intentionally introducing and enforcing constraints produces simpler, more powerful systems
Posted by rrrodzilla@reddit | programming | View on Reddit | 22 comments
The instinct when designing systems is to maximize flexibility. Give every component every capability, and developers can build anything. This is true, but it's also why most event-driven architectures are impossible to reason about without reading every component's source code.
The alternative is to deliberately remove capabilities. Decide what each component is not allowed to do, enforce that at the boundary, and see what you get back.
A few examples of how this plays out in practice:
If a component can only produce data and never consume it, you know it has no upstream dependencies. You can reason about it in isolation. If a component can only consume data and never produce it, you know it can't create unexpected downstream side effects. If the only component that can do both is explicitly labeled as a transformer, the config file that declares these roles becomes the complete system topology. You don't need to open any source code to understand data flow.
Lifecycle ordering stops being a configuration problem. If you know which components only produce and which only consume, the correct startup and shutdown sequence is derivable from the roles. Event sourcing becomes trivial when all messages route through a central point because components can't talk to each other directly. Language independence falls out when components are isolated processes with constrained interfaces.
None of these are features you design in. They're consequences of the constraint. Remove the constraint and you have to build each of these capabilities explicitly.
I applied this thinking to an event-driven workflow engine I built in Rust and wrote up how it played out: https://www.rodriguez.today/articles/emergent-event-driven-workflows
Obvious-Treat-4905@reddit
this is such a clean way to think about system design, instead of adding complexity, you’re removing it and letting structure emerge naturally, those producer or consumer constraints make the system way easier to reason about, feels like more systems should start from limitations, not flexibility
TadpoleNo1549@reddit
this is such a clean way to think about system design, constraints forcing clarity instead of adding more abstraction layers makes a lot of sense, feels like this approach would pair really well with tools like runable where orchestration and boundaries matter a lot
viditjn02@reddit
this resonates with something i've noticed in practice. the best APIs i've worked with were the most restrictive ones. when you can't do everything, you end up doing the right thing by default. constraints force you to think about your design upfront instead of bolting on features and hoping it works
heavy-minium@reddit
What the heck is code that reads data and never produces any data?!
SecretaryAntique8603@reddit
It’s the software equivalent of a project manager
Chemical-Fault-7331@reddit
So true.
hartmanners@reddit
LOL
rrrodzilla@reddit (OP)
Agreed. It could be more clear. Code that reads data and never produces any is a system sink. It likely has side effects, like writes to a database, logs to a file, or something similar.
Pinball-Lizard@reddit
It is producing (or changing) data then, just not directly or in the view of the caller?
64Rounds@reddit
Maybe they meant a service which has no downstream consumer in the system?
heavy-minium@reddit
Yeah, speaking of upstream and downstream dependencies would make more sense.
aatd86@reddit
The simple principle of "rails"... just one word for what people often explain is so many tokens.
Computerist1969@reddit
"The instinct when designing systems is to maximise flexibility"
Who does that? I design systems to solve a problem, not to imagine a bunch of other non existent problems and solve them as well while I'm at it 😂
instantviking@reddit
Reminds me of what someone smarter than me thought architecture could be considered: the set of (artificial and voluntary) constraints you will impose on your system in order to attain some value or property.
nyibbang@reddit
It's true for any software, at any scale or scope.
The more you specify, the more specific you make it, the simpler and the more robust it is.
Software is like starting from a boundless universe. If you don't bind it, if you don't constraint it, you cannot reason about it : it is absurd, it is random, it is chaos.
You aren't gonna need the pseudo flexibility or genericity. The only ones who need flexibility are your users, and that flexibility comes from constraints and rules on your side.
tdammers@reddit
In a way, this is what (typed) functional programming is all about. No side effects, all dependencies are explicit, and the types provide a harness of things your code is supposed to be allowed to do, so that you don't need to reason about anything else.
I think it's also a good idea to turn the mindset around from "decide what your code should not be allowed to do, and put up constraints to prevent it" to "decide what you code should be allowed to do, default to disallowing everything, and then surgically punch holes into that to allow exactly what you need".
So instead of making a constraint that says "this code should not be able to access the filesystem", make one that says "this code cannot perform any side effects other than sending data on this network socket", or "this code cannot perform any side effects, it can only take exactly one immutable event and a list of mutations as its arguments, and return an immutable list of mutations".
Just like with infosec, starting with "nothing is possible" and then adding in the bare minimum of possibilities is easier to reason about than starting with "anything goes", and then trying to whack the undesirable paths one by one.
rrrodzilla@reddit (OP)
Yeah I like this framing a lot. That's basically how the three primitives work in practice in the code. A sink isn't 'a component that can't publish'. It's 'a component that can subscribe and nothing else'. The SDK types only expose the the methods that role is allowed to use. Everything else doesn't exist. Same idea as infosec, start with zero capability and grant only what the role requires.
daltorak@reddit
Sounds good in theory, but in reality, no such components exist.
Components inevitably have to have dependencies like a file system, a web API, or a device.... which in turn implies the existence of dependency on configuration settings.
The only exception here might be a locally-connected user input device.
rrrodzilla@reddit (OP)
Yep. you're right that every component has external dependencies. A source still reads from an API or a sensor or whatever. I could have been more clear in my writing because the constraint is more narrow than 'no dependencies at all'. It's that within the system's event topology, a source has no upstream message dependencies. It doesn't subscribe to other component's output. Doesn't react to other events from the pipeline. That's what makes it possible to reason about in isolation. You don't need to understand the rest of the topology to know what a source will do. Its behavior comes from the outside world, not the system it feeds into. The API it reads from will have it's own upstream inputs and so will that and so on and so forth. If you don't define your system boundary at sources that produce events that feed into your system it becomes turtles all the way down.
andsbf@reddit
Reminds of the great separation that CQRS brings
rrrodzilla@reddit (OP)
Yeah I like that parallel. CQRS separates read and write models, this separates ingest, transform, and output. Same underlying idea: a system becomes easier to reason about when you stop letting one component do everything.
Technical_Camp_4947@reddit
this is basically why go doesnt have generics for so long and everyone hated it but the code was actually readable