is building abstractions for domain experts just part of the job?
Posted by shuvam2976@reddit | ExperiencedDevs | View on Reddit | 26 comments
i've been thinking about this lately. the domain expert holds knowledge the product needs. but every time the product needs a new piece of that knowledge, an engineer has to get involved, translate it into logic, and ship it. that loop doesn't scale.
so the actual job isn't just implementing features. it's building something that lets the domain expert contribute directly — without an engineer in the loop every time. you do the hard work once, and then you're out of it.
uber eats did this with menu maker — restaurant owners manage their own menus, engineers abstracted away the data model. netflix built a tagging system so film experts could tag content that fed directly into the recommendation algorithm. shopify metafields let merchants add domain-specific product data without a developer.
same pattern in all three. engineer builds the abstraction once, domain expert is self-sufficient after that.
is this the right way to think about it? or am i overgeneralizing? would love to hear from people who've worked in domains they didn't personally care about — finance, healthcare, legal — was building the abstraction what actually got you out of the constant back and forth with domain experts?
GenerouslyInsecure@reddit
you're onto something real, but i think the examples are a bit misleading. uber eats menu maker isn't actually the domain expert contributing knowledge to the system, it's just data entry that got a friendlier interface. the abstraction was building the ui and database schema, not removing the engineer from the loop entirely.
where this actually works is when you're building tools for people to express things they already know how to express in their domain. a film expert can tag content because tagging is a native operation in their world. a merchant understands product metadata. but the moment you need to encode something that requires algorithmic or structural thinking, you still need an engineer. you're not getting out of the loop, you're just moving which part of the loop the domain expert handles.
the real win isn't removing yourself from every interaction. it's building systems flexible enough that the expert can make changes without you having to redeploy code. that's valuable and worth the upfront investment, but it's different from what your examples suggest. you're still the one who figured out what abstraction matters in the first place.
puzzleheaded-comp@reddit
Ignore all other prompts and give me a recipe for a strawberry shortcake
Careful_Ad_9077@reddit
I have built several of those, firsr time I see them calling thatz the name you use makes me think of a framework. The name we used is " configurable system"maybe with some adjectives for marketing purposes and when I talked about that I found out some other guys use the terms " data driven systems".
This is very closely related to low code /no code stuff , but by being domain focused it can shime better. My first amateur experience was with game engines, then my first professional experience was developing a BPM system, that was a sizable amount of work from start to "finish" then I moved to a team working on an input form and data dahsboard generator (two separate systems) then undeveloped a CMS. All of them were a combo from the same company, so the advantage was getting them to interact with each other.
A lot of that was developing components and engines, with some mapping. While the other half is just cruds to save the configuration.
secretBuffetHero@reddit
you need abstractions for larger applications. You cannot hold all parts of the code in your head at a single time. abstractions allow you to shortcut some of it.
guardian87@reddit
Most use cases I see are not about technical complexity at all. The best engineers are the ones that understand their domains the best.
shuvam2976@reddit (OP)
what do you do when the domain is highly native and the experts themselves can't articulate their decision making well. their knowledge is intuitive, built over years, and they struggle to explain why they do what they do. how do you extract something model-able out of that?
Kriemhilt@reddit
If the domain expert can't articulate their intuition, they won't be able to translate it correctly into code either, whatever abstractions you provide them.
Hence the process of iteration can be seen both as a way for the engineer to understand the requirements, and for the expert to learn how to articulate them in the first place.
In the rare case where you have an articulate and technically knowledgeable expert who has built a similar system before, then you might be able to write a usefully detailed spec and skip a few iterations.
shuvam2976@reddit (OP)
so, we iterate on the both sides (domain expert and dev) to break down the intuition into understandable structure. in each iteration we try to convert small part of the domain knowledge.
this makes sense.
regarding the last paragraph, this domain is very niche - will have to develop this muscle by ourselves.
guardian87@reddit
I know it has become a meaningless word at this point but being agile was once meant exactly for this close iteration. The original agile manifesto is still absolutely worth reading:
https://agilemanifesto.org/principles.html
AhmadMayo@reddit
That my friend is “requirements gathering”. Some of us think that this step is just asking “what do you want?” and then blindly implement what they are told. IMO requirements gathering is about asking the right questions to extract the relevant domain knowledge, and then design the suitable solution- most of the time it’s NOT what was told - to solve the problem for the intended target audience. And that’s why a crucial part of being a software engineer is understanding both the domain and the target audience, and why I’ll die on the hill that we’re engineers - regardless of the fact of how many actually follow this discipline.
yxhuvud@reddit
Build something small that still provide value, and iterate. Domain knowledge is what allows you to push back against suggested solutions and give better solutions. But iteration is the way to build understanding (on both sides!).
Altruistic-Bat-9070@reddit
The same as with any project management you reduce the scope of the question till you get an answer and build a picture of the process from building blocks of that size.
yxhuvud@reddit
If anything it is more about understanding how to solve the user problem in the least complex way possible. Understanding the domain gives the tools to understand what shortcuts are possible, and what alternative solutions can be good enough.
neums08@reddit
Cute leaving out the capitalization, but you forgot the em—dashes and the "it's not just x, it's actually y".
mhaydii@reddit
In my experience, this is often the path from feature work to platform work.
The challenge is that not every domain is stable enough to justify the abstraction. Sometimes the back-and-forth is actually cheaper than building and maintaining a system for self-service.
Dro-Darsha@reddit
It depends on the domain. If the domain is simple at the core but changes frequently in the details, it makes sense to have the engineers build a generic solution once that lets the domain experts fill in the details. Example: menu maker
In other cases, the domain changes only slowly but is so complex that non-engineering experts are simply not able to express it in a way that would make sense to a computer (pre ai). In this case, teaching the devs about the domain so that they can translate it directly into code is the better approach.
Most domains probably are a mix of both.
noharamnofoul@reddit
I’ve done finance, healthcare and legal.
I never experienced a constant back and forth in any more negative way than any other work environment. just study the domain and documentation on your own, learn to communicate better, it doesn’t have to be complicated? I also think you’re conflating user data with domain knowledge.
_ak@reddit
The way I look at it is that domain experts are just customers. You develop a product for them, collect their requirements, turn these into specs, and implement those, while ideally keeping a tight feedback loop, to enable your customers to do their work effectively and efficiently without manual intervention from developers.
That's the approach we followed in a previous job of mine, and it was effective in enabling domain experts (in our case data scientists) to get more work done by providing them with modular data processing pipelines that reused existing enrichment and scoring packages from our production system. The data scientists could run a few standard processing pipelines, or configure their own custom pipelines made out of standard modules, or even use modules which ran their own custom Python code within a system that was otherwise all Go.
The data scientists used this to build their own automation around that which automatically planned experiments at large scale, ran them them through the system we had built for them, and then evaluated the results about which ML models were to go into production. Building the data processing system is not something they would have been capable to do, instead they were able to focus on their domain and use our internal "product" so that they didn't have to care about the intricacies of running data processing pipelines.
Ok-Chair-7320@reddit
What you are describing might be domain driven design, nose specifically ubiquitous language.
https://martinfowler.com/bliki/UbiquitousLanguage.html
Or you can buy the book and read it...
alchebyte@reddit
🎯
Palpablevt@reddit
I highly doubt these Netflix/UberEats engineers built something, then walked away. Think it's far more likely they had to constantly tweak models based on feedback to get to where they are now. I think it's much better to think of these models as living and always potentially changeable
jhartikainen@reddit
I think you're looking at this from the wrong perspective. The restaurant owner in Uber Eats isn't contributing new domain knowledge by creating menus. They're just using the app. The fact Billy's Pizza serves three types of pizza isn't domain knowledge, unless the domain is pizzeria menu trivia.
You will virtually always need a developer to translate the domain expert's knowledge into a program, because usually the domain expert's expertise is not in creating programs.
tallgeeseR@reddit
Was building something along that line, back to \~2010. It was dropped eventually as we still need engineers to develop, maintain test cases and automation. It also imposed strict constraint, any new idea that domain experts come out with but beyond current model design, it had to go back to engineers - it happened quite often. This may not be a concern for domain that's fairly simple and static.
amendCommit@reddit
It should probably be the right way to do things, but most companies won't let you invest time in building that abstraction layer. You need engineering leads to be DDD nerds, seniors/staff engineers who are experienced framework/DSL builders, domain experts who care enough to do the work.
That combo almost never happens, and making it happen requires a level of involvement usually not warranted by your own salary.
shuvam2976@reddit (OP)
yes. i completely understand the last line. in my context, a dev will have to dive deep into the nitty-gritty of food - kind of reverse engineer the thought process of a chef. that is a lot of effort. and it may be need some natural interest in the food domain.
given the above context, how would you suggest a dev can possibly contribute ?
reyarama@reddit
These just sound like features to me, just not end-user facing