Honest question to devs about build vs buy
Posted by DropKicck@reddit | ExperiencedDevs | View on Reddit | 44 comments
I work in Data Product Management (think internal tools and platforms for data scientists), and I’m struggling with some engineering partners who refuse to evaluate vendor solutions and instead want to build.
I know there’s advantages, but their adamancy to not budge is very confusing to me. My only guess so far—and I’m open to more ideas—is that they would lock themselves into a lot of job security if they are the builders / experts of this tool / platform?
There’s at least two cases of that at my company in which the two highest paid and most tenured engineers are deemed “indispensable” do their their institutional knowledge.
codescapes@reddit
The question is always whether the vendor product will actually do what you need. People dislike the idea of recurring cost but even $1000/month is nothing compared to a full time developer salary to maintain your custom solution.
We like to build because we're engineers but if there's a vendor product that isn't garbage and will work then it's frequently the better choice.
WaitingForTheClouds@reddit
The advantage is also control and ownership which is much harder to put a price on without hindsight. When a core system of you product depends on a feature that gets deprecated and removed from new versions of a library you don't own, you're in a world of trouble. I've seen this first hand, it was open source so they tried maintaining a fork with that feature backported but they essentially had to have someone spend a ton of time just understanding and merging updates, the advantage of "not having to pay someone to maintain" went out the window. Eventually it got so incompatible they just said fuck it and now never update that library and hope it keeps working otherwise they are majorly fucked lmao as the entire codebase depends on various things from that fucking lib, not to mention they can't make use of new features.
fuckoholic@reddit
This. Worked on one like that. It's already years since no new updates came. I wish we built it ourselves, but I wasn't there when decisions have been made.
snorktacular@reddit
Forks are really the worst of both worlds. All the responsibility of maintenance but you still depend on a third party.
T_D_K@reddit
If the product is 1000/month and does something useful and non trivial, then hell yeah thats great.
Usually though, these contracts are 6-7 figures. After the initial dev costs, the back end maintenance costs are pennies on the dollar, plus the support is in house.
OpeningMysterious739@reddit
Ownership counts for a lot.
If it’s built in house, there’s no risk of losing support if the maintenance isn’t paid or the vendor goes OOB.
Of course now you have to maintain it yourself and, in my experience, this often falls on the shoulders of one person. If they leave the company, you’re in the same boat. Except you have the source.
A software vendor I used to work for had a ‘better built here’ philosophy, but they were aiming to lick customers in to their platform.
AbbreviationsFar4wh@reddit
I hate building b/c it means more shit to maintain.
Dry_Author8849@reddit
Let me give you another angle on this.
Are you trying to buy tools that they will use? Have you considered it can sound as imposing a tool to use instead of letting them choose the tool to do their jobs?
Just consider that POV. Even if you have the power to do this, it's wise to hear why they don't want to use tool X, or you will be buying something nobody will use.
Anyways, an example of what tool you are talking about can make things clear. Data Science is one area where you have a lot of custom work to do in order to get the result that you want.
So, to your question, yes there are a lot of devs that want to build their own tools. If they are doing that for job security and get away with it, it's a management problem. You need to take the issue up.
If you have the power to make the call, it's your call then. If building doesn't make sense to you, tell them why that won't happen and make them participate in the process of evaluating and choosing the tool or tools you will buy. Let them make the choice.
Cheers!
MCPtz@reddit
This sounds like nothing is being pushed on to them, although it's possible OP worded that poorly.
They need to make the engineers dig in to some of the details as to why some existing solutions won't work.
valence_engineer@reddit
Depends on the company politics and history. I've often seen platform teams pay a lot of money for solution that engineers said were slightly useful and then try to force the engineers to use them. If that has happened before to these engineers at this company then I see why they stonewall since anything else means the solution will be forced on them.
valence_engineer@reddit
In my experience in the data space the issue is that solutions often claim to solve 100% of a data scientist's problem in an area but in practice only solve 80% with no hope of bridging the last 20%. And without that last 20% the solution is barely useful in practice to data scientists. Then the company blocks any further attempts to build an internal version that solves 100% since they're already paying for the external solution. Trying to explain this to those not deep in the technical data weeds is nearly impossible as to them the 80% looks like 100%.
Material-Smile7398@reddit
It really depends (sorry) on the product/vendor.
I would get as many samples of their input/outputs as possible, then add up the cost of the software/support from them. Then factor in the inevitable support that your devs are going to have to do, for example staging/adapting data to integrate existing systems, and factor in the vendors SLA times vs your teams for bug fixes and changes. That's all you can do really.
Unless the product is incredibly complex, I tend to go for build, simply because of mis-selling and poor support that I've seen time and time again from vendors.
snorktacular@reddit
I tend to lean buy over build, but a lot of people have been burned by buying after getting the rug pulled out from under them. Maybe it's a SaaS company that shuts down and your team needs to migrate on short notice, or the company changes their licensing and finance doesn't want to pay for a subscription so you're stuck on a legacy install, or their support team isn't responsive to issues, or they release a new product and the one you're using becomes abandonware. Or maybe your company's procurement process is so ridiculously painful that people would rather be on call for the version they built than do the paperwork for buying.
Still, NIH syndrome is often much more painful than most of those things. The opportunity cost of spending engineer time solving problems that aren't core to the business is huge. Many great products have come out of internal tools that the engineers didn't actually need to build (Slack is the quintessential example) but it's rare to be able to pivot like that. The advice is usually to buy anything outside your core business, and then if you want to expand then build in areas adjacent to your core business. Carve it out incrementally.
Soccham@reddit
This is the biggest factor lately. Rug pulling from SAAS providers is becoming increasingly common and unsustainable
Nyefan@reddit
Including extreme price hikes. Just this month, a service we were paying $80/month for with on-demand pricing is going to $72000/year at the end of the month. I have real work to do and detest getting pulled into this emergency migration garbage every time someone gets acquired or decides they aren't profitable enough.
blisse@reddit
$80/mo probably means you're not profitable for them so it's likely intentional - depends on the product. SaaS pricing is a bummer because sales negotiation can be pretty arbitrary in the early days.
30thnight@reddit
Often times, it’s a mix of underestimation + a way to escape work they don’t enjoy doing.
GoblinOfMars@reddit
It’s always a balance. In 2022 we were going nuts with “buy don’t build”, mostly because we already had a massive “build only” legacy codebase. However, this year two of our paid solutions went out of business and we had to scramble. Also integrating a lot of those vendor solutions didn’t make things easier. These days I’d focus on simplicity above all else. What’s the easiest to integrate and maintain in the current stack. Also whatever moves responsibility off of my team. For example, please pay for the expensive Netsuite integration instead of having my team build it. And by the way, finance needs to be owners of that feature set, not us.
Wide-Pop6050@reddit
Don't assume its this job security thing - that's a bad angle to approach this from.
Also, ask them why.
One common issue is whether the product actually does what it says it will. Often they're not customizable enough.
I think if its core to your business, you need to build so you have control over it. You also shouldn't only have a couple people who know it, that's bad management. If it really covers all the use cases (which I think is part of your job to determine) and its not your core product, sure buy.
But go in with a positive, problem solving attitude, not "the devs are trying to pull one over me"
Mountain_Sandwich126@reddit
Mostly ego, we always think we can do a better job. And underestimate the maintenance cost.
Usually if it's not your core business or your defrienciator, you probably would want to buy.
You should be able to do a cost to build then 30-40% ongoing maintenance. Or a rebuild ever 5 years because no one bothers keeping the software production ready
waffleseggs@reddit
This is the right way to think about it. Core business should almost usually probably be made in-house, and the rest can be externalized. It's often easier if you conceive of your externalized systems in commoditized ways. If you are doing ecommerce but not using ecommerce concepts to model your database and api operations, you'll have a much harder time.
Complex subsystems are also good candidates to buy. Don't build your own Databricks or Tableau when you'll never do as well as those teams.
Domains that have complex legal requirements (SOC2, FINRA, ...) are also common targets too. Lots of people plug into Stripe, Braintree, or Plaid for this reason.
There's a lot of art in making these choices. Sometimes you can bring in an agency specialized in some focused area to build a high quality implementation that the in-house team maintains.
There can be a lot of competition in markets--VCs tend to see the same opportunities at the same time and new vendors and vendor featuresets are coming out constantly. There's often the associated question of conforming to standards these companies put in place (ie. MCP vs. DIY with AI agents now)
szank@reddit
We as the devs are currently trying to convince the management that building out own survey tool (like typeform ) is a very bad idea and we should just use intercom instead.
So its not always the devs ego.
RobertKerans@reddit
We've been trying to convince management to use an off-the-shelf CMS + API gateway instead of the jerryrigged rails app that tries to do everything for two years, to no avail. Could have saved ourselves at least half a year's work and directed all that wasted resource towards the application core + frontends but hey there's just too much work to do
Mountain_Sandwich126@reddit
That's awesome, but it sounds like the team has done proper cost benefits analysis, not refused to do the evaluation like OP said.
In the case where they refuse to do the due diligence, I suspect there might be some bias.
donalmacc@reddit
I don’t think it’s safe to assume that just because devs have asked for something they’ve done a proper cost benefit analysis, but if management have asked then they havent.
I’ve been responsible for requests where the same dev will put in asks for an entire suite of software just because it’s an option on the form. Think c++ developers requesting a python IDE.
Thommasc@reddit
I agree with the fact that if it's not your company core business you should rely on a tool instead of maintaining it.
I'm going to play devil's advocate here but there are also tons of blind costs when picking a third party tool. I won't list them here but it's easy to guess.
As usual in IT the answer is... It depends.
And I would argue that in many situations there is very few difference between solution A and B after enough time. As you train your team to the new solution they will become more efficient. So in the end it didn't matter as much as you would think.
The most important aspect is ownership. Whether you pick or build a tool, if whoever lead the project is leaving quickly it will be hard to maintain, no matter what you do.
trwolfe13@reddit
Yeah, we got blindsided with a £50,000 bill from Pulumi Cloud just for managing ~20 services and associated config in Azure. Took us a couple of weeks to shift it over to hosting our state in blob storage, now it costs us like £0.05/month.
bear-tree@reddit
For me, the most important question is “does build improve our core competencies?” Sometimes it’s not a completely straightforward answer but it will often highlight when you are building to build. An example:
You work for a firm that builds boutique marketing web sites. Clients can login and upload assets blah blah. It makes sense to build the client login and assets management etc. it does not make sense to build out your hosting solution. You are not in the devops business.
thekwoka@reddit
It's quite common to find that vendored products don't really do what you want or offer the adjustments you need to do what you need it to.
MonochromeDinosaur@reddit
Are you technical enough to engage with the engineers and figure out why they don’t want to do it?
Technically don’t need to convince the engineers just the executives.
Depends on the company but my measurement has always been.
How complete the purchased solution is for your use case
And
How many developer salaries am I spending and saving purchasing it
If I can get a managed platform that covers 80-90% of my use case with enough extensibility to get that other 10-20% that would otherwise take 3-4 engineers to maintain if I put together a bespoke solution at the cost of a single engineer salary then that’s 1/3-1/4 the cost of a bespoke solution. Then I can negotiate hiring more engineers.
Using standard solutions also breaks that institutional knowledge issue, which was a real issue at my last job.
Two jobs ago I was able to use the above logic to convince the executives to migrate from unreliable bespoke garbage to databricks for all DE/DA/DS needs.
Breklin76@reddit
I see no issue with turnkey solutions. There’s always some customization that occurs to make it meet spec.
failsafe-author@reddit
I always am biased toward build over buy, and this is something that puts me at odds with colleagues.
I have been bitten by buy before, and never by build. The value of being able to have exactly what I need the way I need it is usually worth it to me, unless it’s a very complex problem. Or a very simple one.
That is, I’ve never written my own ORM because I know its not with the time to reinvent, and I’ve never written an IOC container because existing ones work fine and it’s just not a hard problem to solve.
All that being said, I respect it when my colleagues tell me, and there are times I’ve gone with build over my instincts that were definitely for the best.
originalchronoguy@reddit
I always look at what is best for the company. It makes no sense to have 5 developers build a product for a 5 users; costing millions of dollars when a $200 product is available. It has zero value.
Northbank75@reddit
Developers always want to build, I feel this temptation often but I’m not going to build a better CRM than is available and the guy in my job before me always wanted to just keep adding shit into an ever growing ERP.
These days I do a Google for open source products that can take pressure off my team, deliver the requirements needed, and the value of basically free products that can be deployed almost immediately…. It’s easier to make a winning argument when you have the alternatives ready to go. We’re rocking OSTIcket on our help desk, Mantis for some Bug Tracking and a few other things I’m now glad I didn’t have to build.
Learning that as Developers that we need to learn to say ‘we can but there is value it evaluating x because it already does this and you can have it running tomorrow’ is a thing, it helps us, and I helps the business.
03263@reddit
buying can get very expensive when you need the vendor to do custom development to meet your needs, then you may wish you had just built. so evaluate any off the shelf solutions carefully.
aqjo@reddit
How much do the tools they build and maintain cost the company?
Compare that to training costs, onetime costs, and ongoing subscriptions for the vendor solutions.
On the other side of the coin, what are the costs if an engineer is “hit by a bus?”
Are they the only people who know critical systems?
How much of a lurch would that leave the company in?
jmartin2683@reddit
It’s because owning things is important. There is a ton of risk associated with being married to some third party tool. Depending where it slots in, it could range from catastrophic to just very inconvenient.
Materially, the freedom to define the product and process that you give up isn’t worth the trade off (usually ‘it’s easy’). The truth is that it’s not easy.. at least not easier than just writing it yourself, bespoke to your purpose.
Sensitive-Ear-3896@reddit
In a lot of cases vendor tools are just pretty wrappers around what the underlying tool already does. In many cases they’re already tried the tools and they were no better than rawdogging
Antsolog@reddit
It depends.
I try to give my team the freedom to evaluate if a vendor solution will be applicable but if the tool is used as part of the core use cases then usually it needs heavy customization and config, leading to potentially a hard time if we go into the problem without fully understanding the problem space, requirements, or tool. This is usually the case as our sector is still fairly nascent and the company is still young with less experienced employees.
It’s why I’ll usually let people prototype and then during the rewrite ask if we can replace parts of it with tools or what tools exist that we can use from OSS or vendor stuff.
Right now observability is a big project my team is taking on and we’re basically just using the grafana Loki Prometheus stack to some success. I asked if we should just look at grafana cloud but pricing can be steep for what we need and the project is fun enough that people are interested and invested.
In the end its experience which will be helpful even if we move onto grafana cloud later because we will have gone through the hard problems of making the smaller decisions around the stack and know how we could configure grafana cloud to be successful for our specific use cases.
Izacus@reddit
Did you ever had to do an allnighter weekend because your "buy" vendor decided to screw you over and stopped providing a service your business is relying on? Or where vendor figured out your business lives or dies based on them and then price hiked you so much that you had to layoff folks?
That's why.
canibanoglu@reddit
In my experience, engineers who refuse to use other products and instead force their own solutions are problems.
You might be right that their aim is to entrench themselves deeper but it could also be due to simple lack of awareness, which is more common in my experince. A lot of people, not just engineers, tend to underestimate what it takes to build something. They tend to think along the happy path of where they only see the most common 2-3 use cases and think they figured it all out. Most of the time reality hits back hard and the company who listened to these will find themselves adding teams and sprints just to iron out issues/build features on a service that wasn’t even meant to be a part of the company’s activities.
Moreover, outright refusing to weigh options is a huge red flag. For one, it literally is part of their job. Second, even if they were hellbent on building it themselves, they still have to evaluate what the others are doing. It sounds more like they are trying to dig their feet in against you.
A good rule of thumb in paying vs building is to assume that people selling a product know a lot more than someone considering to build a similar product because they don’t want to pay.
If there’s something out there that does what you need and it looks like it would work in your system and the price is within your budget, most of the time it’s better to just pay for the service rather than sinking many many hours into an inhouse solution.
HaMMeReD@reddit
Could just be straight up narcissism. I.e. the strongly held belief that they know better.
I.e. I used to take smoke breaks with this guy. He wasn't on my team but he'd just brag non-stop about his home physics engine for his game, or the new C++ database he was building that was 10,000x faster than the commercial offerings. The thing is, nothing the guy said ever actually made any sense, it was all techno-babble and nothing he ever did came to fruition. It was just years and years of the same brags over and over.
He was always adamant that the things available were never good enough. Using something someone else built was blasphemy because he's the only "smart one" in the room. But the power of technobabble somehow was all he needed to convince management and execs that he was "high value".
canibanoglu@reddit
That’s fair, it could also be a reason.
PolyPill@reddit
I agree with others, although I often see rather trivial needs being over sold and spending huge amounts of money on 3rd party solutions that could have been a rather small and simple job. So there’s always details that need consideration. Basically my rule of thumb is if the configuration of the 3rd party system is more complex than the solution to the business needs, then build, otherwise buy.