Why Engineers Can't Be Rational About Programming Languages | spf13

[-]

AceLamina@reddit

Probably the best article I've read or listened to in a while

[-]

thiagomiranda3@reddit

The article has an AI image and feel like the text used it too. So I won't bother reading if even the author didn't bother writing it

[-]

Maybe-monad@reddit (OP)

How can you feel the text is generated by AI? I read it twice and I didn't feel anything.

[-]

cube-drone@reddit

"anyways, here's a 9-part framework I authored that is in no way intended to influence you to choose Go"

[-]

My company chose mongodb as their db even though we have very hierarchical and normalized data, require cross document validations and consistency (eg orphans, fan outs), all bc the CEO and CTO heard mongodb was cool.

[-]

zoqfotpik@reddit

https://m.youtube.com/watch?v=b2F-DItXtZs

[-]

Maybe-monad@reddit (OP)

I've been through that, it was terrible

[-]

chasemedallion@reddit

Very much agree with this article, it’s a great observation.

One rule I have for language selection is that we should expect that the first thing our team/org builds in a new (to us) language will be full of design mistakes, performance bugs, and other technical debt.

While not 100% deterministic, my experience is that engineers who get excited about technologies through articles and hobby projects vastly underestimate how much more there is to master when it comes to building larger systems and operating them in production.

So the question isn’t just “is this language the right tool for the project” but also/rather “can we afford for this particular project to be burdened with a ton of technical debt so that we can build experience and operational maturity for other projects in the future?”

[-]

ArtOfWarfare@reddit

I (team lead) was exposed to Kotlin a few years ago via a senior engineer on my team choosing it for a 3 day hackathon.

I love it. The tool we built during that hackathon is still used in the build pipeline for several of our actual projects. I’ve been advocating for using it in bigger projects since then.

But I think a Hackathon is a perfect low-stakes time to try introducing new languages.

[-]

cmpthepirate@reddit

Can confirm

[-]

Prod_Is_For_Testing@reddit

I agree. I always roll my eyes when people say “good developers can learn a language in a week or 2”. Maybe you can get the basics but actually using it properly takes months or years depending on how expansive it is

[-]

chasemedallion@reddit

Part of the issue is that good developers can learn quite quickly as part of a team with existing expertise to build upon in the form of colleagues as well as established codebase patterns.

It’s a different story when the team are all novices.

[-]

elmuerte@reddit

While not 100% deterministic, my experience is that engineers who get excited about technologies through articles and hobby projects vastly underestimate how much more there is to master when it comes to building larger systems and operating them in production.

When it comes down to production quality: boring is good

New things are fun to experiment with. Once the tech becomes boring, it is ready to become part of your castle.

[-]

Ok-Craft4844@reddit

Problem, like with kiss and obvious: one person's boring is another ones exciting.

I had to defend things like .filter or .map one time to often to think and adjective alone is a good metric

[-]

PaperMartin@reddit

You still gotta develop it though, else it won’t reach the "boring" state

[-]

matthieum@reddit

Yep.

There's a good reason that the recommendation for adopting a technology in a company is to start with preferably internal non-critical projects/services. Stuff that you can afford being down, or malfunctioning.

There's quite a few steps between Hello World and a full-blown application with non-trivial communication patterns with other apps & databases.

[-]

coderemover@reddit

Rust and Go are both relatively easy to build and deploy, and the difference is so minor that it likely doesn’t matter because you configure it once and then it just works. Go has a minor edge in compilation speed and cross compilation setup (which many don’t need) but at the same time it’s not so nice when you have to link to any foreign language like Python or C APIs, and build them together whereas in Rust integrating with other languages is a breeze. So both have their strengths and weaknesses.

I think the problem is people don’t start with a set of must-have requirements for their project before even looking into languages.

Like - if I really need extremely fast edit-compile-test cycles - this would rule out Go and Rust and any compiled language in favor of something interpreted like Python or PHP.

Or if my must have is being able to fully control what the program is doing down to the assembly level, because you’re building a game or database engine, then Python, Java or Go are no-go and you look at C, C++, Zig or Rust.

[-]

loup-vaillant@reddit

Like - if I really need extremely fast edit-compile-test cycles because I’m exploring and don’t know yet what I’m building -

Then dynamic typing would automatically be ruled out, because the round trip time of the edit-test cycle is much longer than the round trip time of the edit-typecheck cycle. Plus, the type checker reports are a lot closer to the root cause of the bug than test failures.

The thing with exploratory programming, is that I don’t just want to explore, I want to get to the good stuff. Which in part, means not wasting my time with the bad stuff. Static typing is amazing for that: it culls swaths of bad lands for me, so I can focus on more promising solutions.

Granted, static typing will also cull some good stuff: a type system cannot be both sound and complete, so there’s gotta be some false positives: correct and interesting programs that will be rejected nonetheless. Realistically though, what are the chances the program I really want is among those?

Given this, I personally have no idea, all other things being equal, how anyone could be more productive with dynamic typing than they are with static typing. But I have some ideas why they think they are: all other things are not equal. They’re not equal at all.

The most widely used static type systems are pretty bad. C, C++, Java before it got generics… Few people have been exposed to Hindley-Milner type systems (ML, Haskell…)
There’s a weak correlation between dynamic typing and garbage collection. And garbage collection does improve productivity, at least in cases where we have enough spare cycles — or ~~CPU~~ user time is cheaper than dev time.
The most widely used dynamically typed languages often have huge libraries. Of course it’s all easier when most of the work had already been done.

I have no doubt there are some (many?) people out there who would actually be more productive with dynamic typing than they’d be with a good static typing, even if the language and environment were otherwise the same.

I just never could understand how they think.

[-]

chaotic-kotik@reddit

There are languages that allow you to express your ideas more easily and those that add "safety" by limiting what can be expressed. I'm way more productive in C++ or even in Go than in Rust. Sure, Rust is safer than C++ but so does any garbage collected language out there. And in Rust I can't express every logically valid idea but I can do it in C++ or Java easily. The same is true for dynamic languages. They allow you to make mistakes in a new way but they also allow you to express ideas which are impossible to express in C++, Go or Java. Like, try to make a function decorator in C++ for instance.

[-]

loup-vaillant@reddit

I get your point, and for the languages you mentions I mostly believe you.

You’re correct that for any static type system, there are ideas you can’t express, that you could have with dynamic typing. In practice though, what I found the most useful, was the ability to put stuff of various types in the same collection (be it a list, map…). And for that, all you need is tagged unions.

In fact, everywhere you need different types, and you can know those types at edit time, you can write the corresponding tagged union. This doesn’t handle all cases, but it does handle a whole lot of them.

Like, try to make a function decorator in C++ for instance.

Like, Python decorators? Noting that such decorators need to be defined where the original function is defined, it seems to me it’s pretty easy to do: the decorator is a function that takes the original function as a parameter, and returns a function of the same type. In C++ I would try something like this:

template<typename Return, typename... Args>
std::function<Return(Args...)>decorator(std::function<Return(Args...)>f, Args... args)

It’s mighty cumbersome, but I think we can make it work. (Also, the concept would be a lot easier with truly first class functions like we have in ML and Haskell).

Don’t sell static typing short. It’s more expressive than many people think.

[-]

chaotic-kotik@reddit

You can't use this decorator on a method.

With python and other dynamic languages you can do things impossible in C++ like generating serialization code automatically, or generating types and code.

[-]

coderemover@reddit

And I’m way more productive in Rust than in Java, Go or C++. So this is highly subjective. Rust may be limiting in some areas, but it shines in other. Eg enums, pattern matching, traits, generics, macros are so much more expressive than what Java/Go have to offer that it’s just no competition IMHO.

[-]

chaotic-kotik@reddit

Algebraic data types are nice but Rust lacks some other important bits. For instance, I'm using a smart pointer type in one of my C++ projects which is based on intrusive list. It has much better performance compared to normal ref-counted smart pointer in C++ because different pointers are not updating the same goddamn ref-counter so no false sharing. It also doesn't require any allocations. It can't be implemented in Rust as easily and as efficiently. You will have to use RefCell / Rc etc. Or you will use unsafe. The code will look stupid.

The whole ergonomics of Rust is bollocks. Everything returns either result or option and you have to use that wonderful pattern matching quite a lot. Unfortunately, the language doesn't have god damn function/method overloading so you have to do a ton of stupid things like converting mut Option to Option and stuff like that.

The whole async story is not great either. It looks like tokio won so most dependencies assume it. The AWS SDK uses tokio. No way to write generic async code that doesn't depend on some runtime.

[-]

Absolute_Enema@reddit

the round trip time of the edit-test cycle is much longer than the round trip time of the edit-typecheck cycle.

Nonsense and typechecks /= tests.

[-]

loup-vaillant@reddit

typechecks are not comparable to tests in terms of the quality of the feedback.

They’re not indeed, and I use both. But what the type checks does much, much better than tests, is telling me where is my error. That’s the biggest factor to shortening my edit-check cycle.

Also there is a very simple way to keep testing failures close to the root cause, which is building things bottom-up and composing them once their behavior is well established.

And I do that too. But last time I tried writing an Earley parser in Lua, I couldn’t write 50 lines of code (this is not an exaggeration) without getting incomprehensible "can’t add functions" or "can’t call numbers" errors, that I simply couldn’t trace back to my error. And I also didn’t know how to break my code down even more, it was a fairly simple recursive search.

That’s when I knew why some people need TDD: be extra disciplined so your compiler doesn’t have too. From this day forward, I gave up on dynamic typing.

You can eventually encode domain information in types, but that requires thorough knowledge to not create brittle castles that don't generalise.

That’s what I do: try to encode some (not all) domain information in types, see what can be typed before I write any actual code. It’s basically design up front, checked by the compiler. And I can make a couple designs before I decide which design to try and code.

Again, I find it much easier to encode acquired knowledge within well-tested functions, without necessarily caring about the exact shape of the data I'm dealing with.

By the time I get to well tested functions, I know exactly the shape of data I’m dealing with. Before that I’m just prototyping, measuring, playing around… but there’s got to be a point where I know what data I’m dealing with, I can guess how much data I’m expected to handle, I have an idea of the relative frequencies of the various cases… then I go to production and write the well tested stuff.

And while we’re at it, though my implementation may mutate, I’ll try my best to get the API on my first try. Or at least get close. I need a stable test suite I can trust.

the ecosystems and cultures of these languages tend to be hermetic

Yeah. That’s kind of a problem. Though in ML and Haskell specifically, my guess is that it is filled with people who did not flee from high school maths. People who see programming as applied mathematics, people who are more inclined to use mathematical jargon to think and communicate.

To someone who got into programming because the maths class was horrible to them this is a PTSD inducing nightmare. I understand why those people went to dynamic languages: these are where you can put off mathematical rigour until the very last moment (failure at runtime), and whose communities are least likely to dump math jargon on you.

And as you say yourself, ecosystems matter; it's why I've settled on Clojure […]

Makes sense. Me, I’m increasingly settling on C, because it’s the only thing I’m sure will last. But God I miss proper parametric polymorphism. And I need something better than those horrible macros. And I hate how much Undefined Behaviour it has. One of those days I’ll have to make my own language that compiles to C.

[-]

Maybe-monad@reddit (OP)

Like - if I really need extremely fast edit-compile-test cycles because I’m exploring and don’t know yet what I’m building - this would rule out Go and Rust and any compiled language in favor of something interpreted and interactive like Python or PHP.

Go builds almost instantly for small and medium sized projects

[-]

coderemover@reddit

Rust is also instant for small and medium projects. It compiles ~70k lines per second on my laptop in debug mode and recompiles in 0.5-2s after a change.

But anyway, compile time is not the main for an interpreted language. Interpreters often allow you to edit the app without stopping it. Or stop at a breakpoint, fix a bug, reset the frame and continue. Another thing is REPL.

[-]

loup-vaillant@reddit

Interpreters often allow you to edit the app without stopping it.

Some game programmers do that in C. Compiled C on Windows. They edit their code, hit compile, and then reload the dll into their running program, with all runtime state preserved.

It’s not there by default. One needs to set it up. But it’s a striking counter example to the idea that this is somehow exclusive to interpreters.

Another big thing is REPL.

Okay, that one is exclusive to interpreters… since it is one. It is not however exclusive to interpreted languages. Not to mention, strictly speaking, talking about "interpreted languages" is a category error: all languages, no exception, can be interpreted. Even C. And all languages, no exception, can be natively compiled. Even JavaScript.

And in practice look at OCaml: statically typed, compiles to native code and bytecode, has a REPL. Haskell is similar.

REPLs are freaking cool. I want one for every single language out there. It can be done, it’s just a question of time and effort.

[-]

coderemover@reddit

Game programmers often use Lua and other scripting language embedded in the game, exactly to make gameplay development faster.

[-]

loup-vaillant@reddit

Yeah, you don’t want to reset too much of the program state if you want to keep the edit-test cycle short. I’d agree there that a scripting language makes sense there.

Though if it were me I’d make myself a type checker. I tried Lua, I can’t write 50 lines of dynamically typed code without losing myself in the runtime type errors.

[-]

loup-vaillant@reddit

That’s the thing people forget about compilers: when properly engineered, they’re fast. A debug build can easily compile 100K lines of code per second on a modern machine, more if you actually multi-thread it.

GCC and Clang set a pretty terrible example for that. (That, and C++ itself is pretty horrible to compile.)

[-]

Zbojnicki@reddit

I think the conclusion should be about almost opposite of the one that author makes. PHP or Perl? Go or Rust? In the end it does not matter all that match, but if your new CTO demands full rewrite in another language, just fire him immediate

[-]

loup-vaillant@reddit

if your new CTO demands full rewrite in another language,

Then you’re probably screwed.

The CTO is supposed to be the most technically skilled in the entire company, at least when it comes to the big picture. Good luck firing him Mr founder/CEO, he’s better than you. If he says the company needs to switch languages, you’ll most probably believe him.

Unless you’re better than the CTO at his own job, but then why aren’t you CTO?

In the end, I’m afraid it all comes down to luck. We need at some point to work with people who do stuff you need better than you do. Which person should you pick? Unless the domain is easy to asses from a layman’s perspective, you can do little more than pray they really are up to snuff. When everyone is, it can make for a good company. When some aren’t… I hope you didn’t pour all your savings into this.

[-]

Labradoodles@reddit

That’s not the CTOs job btw. And they should respect the people on the grounds opinion because they will be driving what needs done to completion.

https://discord.com/blog/why-discord-is-switching-from-go-to-rust

Discords switch is because they needed to avoid the gc. I would bet the cto was informed of the decision and prototypes but that work was all the ICs. CTO just made room for them in the org to do the right thing

[-]

loup-vaillant@reddit

That’s not the CTOs job btw.

Checking Wikipedia… OK. Oops.

[-]

Maybe-monad@reddit (OP)

I'd give him the task of doing the rewrite himself with an impossible deadline to ensure he burns out before he gets fired.

[-]

SpringDifferent9867@reddit

Heh. CTO. If he even cares or knows about the languages used, the company probably doesn’t have even 10 programmers and his role is probably more in line with what a senior programmer would be doing. No, a proper CTO is always away from his office and the teams, arguing with CIO for more resources 🙄

[-]

Absolute_Enema@reddit

The best language is ~Lisp~ the one you want to program in.

[-]

FlyingRhenquest@reddit

Languages don't matter! Because I'm going to write a lisp interpreter in whatever language you pick, anyway! Lol! :-D

[-]

Maybe-monad@reddit (OP)

I'll pick Lisp

[-]

FlyingRhenquest@reddit

Gotcha covered!

[-]

qckpckt@reddit

I like the line about surnames coming from professions because of how important they are to our identities. My surname means swamp

[-]

s0ulbrother@reddit

Last project I was on was a conversion and improvement from a node backend monolith to a go microservice. The problem was it was really inefficient.

The problem was when they rewrote it they essentially mirrored the code from one to the next and it was doing like 10-20 reads/writes to the database per thing logged and there could be like 6-7 of those per request. So I rewrote them to group reads and writes together to handle it in bulk but their logic was rewrite the code to go to make it better.

[-]

Maybe-monad@reddit (OP)

You can call that resume driven optimization

[-]

foodandbeverageguy@reddit

This is a well written article, thanks for sharing

[-]

Maybe-monad@reddit (OP)

You're welcome

[-]

0xbenedikt@reddit

Almost every professional swears by at least one of their tools. Handymen, musicians but also developers.

[-]

FlyingRhenquest@reddit

I think most of the project-level YAGNI and bad language decisions I've seen at companies stems from this. The Principal was bored and wanted to try his hand at this newfangled DSL thing he's been reading about or was looking for an excuse to learn a new language. I've come across a couple of projects that had licensing measures baked in using methods like encrypted DLL loaders or Java classes compiled and stored in database tables. These projects were realistically never going to be licensed or even have much visibility outside that tiny team but they went ahead and did that. And funnily spent no effort on making the project easy to deploy because writing installers is boring. But literally before they had a working product, they wrote an encrypted DLL loader. Hmm. Or how about that one engineer who comes in halfway through the project and suggests that the entire code base be torn down and rewritten in a "safe" language (IE: Rust.) Those are always fun. Most of the time they have absolutely no experience with the language they're pitching.

On the other side of this coin, it's also not uncommon to find an engineer who spends most of their time solving already-solved problems. That'd be either endlessly dicking around with some library from elsewhere in the company that they have minor API complaints about and therefore feel the need to tear down and rewrite the library from scratch or something. Anyone else ever run across this? I've seen it a lot. And they'll spend months doing nothing but that. It's generally negative productivity since they insist that their version of the library be used on the product and will make unpredictable API changes (without testing) multiple times a sprint for what seems like an endless chain of sprints.

I think a lot more projects fail due to these behaviors than people realize. Agile processes and a halfway decent manager can kind of rein this sort of thing in these days, but it's still surprisingly common in the industry.

[-]

NotSoIncredibleA@reddit

I think this is absolutely true that programming laguages can be tied to the identity of the developers.

I am just not sure that:

it can be changed
it can be boiled down to an economic decision.

If a mid-size has their identity tied to PHP and there are shinier and better frameworks out there, it still does not matter, because said team is getting a boost from it. Same goes for python.

I personally have realized that (as a fan of Kotlin) I am absolutely incapable of writing PHP or python. My mind breaks down that I cannot have language features expressing the problem concisely and safely and I have to type convoluted code.

Typescript is tolerable for me, but still not ideal.

So I’d argue that language identity can actually be rooted in deeper identity, like the need to write defensive code, the need for expressiveness and how the code maps to our own thoughts in our heads.

[-]

Deto@reddit

I think framing it as an 'identity' makes it sound like more of a psychological problem (that people just need to 'get over') and not a practical one.

There's a kind of myth out there that if someone is a good developer they can easily pick up a new language. But in reality, every language has its warts and gotchas and syntax (that you need to have memorized) and common dependencies (that are also useful to have memorized) and design patterns and build systems and it just takes time to be proficient.

Everyone wants to be providing value, and if you're a python developer and the project lead wants to switch to java then it's not a threat to your identity, it's a threat to your role on the team and your career. You're going to go from productive to not-very-productive for a while. Eventually you'd learn java enough, but in the meantime your python skills would have atrophied. Also having to shift your focus and attention to low-level things like syntax for a while may come at a cost at developing your skills in other areas.

So if you don't have an intrinsic motivation to learn a new language, maybe you'd be better off finding a different job that lets you continue using the language you know. But then, it's such a hassle to have to leave a company and interview, and find a new job and if you like the people you work with already, it's risky as the new situation might not be as good. So what do you do? Argue like hell against any change.

I like how the article emphasizes the costs of switching languages, though, rather than just the merits of the languages themselves. If you have a PHP team, then switching to Perl will probably produce an undesirable outcome. You can do almost anything with any language, and sure some are better suited for a specific task, but a system using 'sub-optimal language X', created by experts in language X will probably be better than a system using 'optimal language Y' created by amateurs in that language.

[-]

shoot_your_eye_out@reddit

I think this example of the cto changing languages is a problem, but it doesn’t have anything to do with either php or perl. That’s just an amateur move regardless.

And no, the choice of language is not “the most important decision.” It certainly matters, and there are absolutely better and worse and catastrophically bad choices, but there are many, many other factors I would personally consider more important.

[-]

loup-vaillant@reddit

So, you’re saying that the rules that governs the text your programmers read and write all day are not the most important thing about their work?

What’s even more important?

[-]

shoot_your_eye_out@reddit

What actually determines whether an engineering team succeeds is everything around the code: clear problem-definition, good communication, realistic scope, sane processes, feedback loops, and the ability to collaborate without stepping on each other. It doesn’t matter what language is chosen if a team builds the wrong thing.

[-]

loup-vaillant@reddit

I should have added in my question, besides obvious stuff like showing up at work and trying to do a good job. When you go that route, of course there are more important things than the choice of programming language. But few of them are choices at all:

clear problem-definition, good communication, realistic scope, […], feedback loops, […]

That’s not a choice. All non-trivial projects should have those as a basic requirement. The fact that so many do not is an abject failure.

sane processes (including testing, QA, deployment processes, etc.),

If we confine ourselves to the processes that allow feedback loops and don’t destroy communication, they are all sane. Some will be more annoying than others, but the variability, to me, seems much less important than the variability induced by the choice of programming language.

[…] and the ability to collaborate without stepping on each other.

On human terms, that’s a given, same as the rest of your list. On technical term, that comes from the architecture, for which there is indeed a choice.

So, OK, I was wrong. Architecture comes before language. Now what comes above architecture, that isn’t the obvious stuff.

Now what choice could be more important than architecture? The only one I see is which problem to work on. Realistically though, what to work on is often a constraint more than a choice.

[-]

igouy@reddit

He was protecting an identity he’d spent a decade building.

He was justifying his own remuneration.

[-]

v-alan-d@reddit

The identity and bias part is a fair point; however, on the points in the economic decisions link:

Why are we treating programming languages like frameworks or libraries, judged mainly by compiler behavior, tooling, or familiar user bases? Why do we rarely consider the intrinsic qualities like information density, inambiguity, the cost of linking distant words, or the complexity a single statement can carry?

Shouldn't we think: assuming mastery, which languages move information faster? Economically, the sooner we collectively master quality languages without ruin in the process (i.e., incremental and sustainable), the sooner we can communicate effectively and efficiently.

We did evolve from signal-based communication to using complex sentences.

[-]

gjosifov@reddit

It all comes down to these factors that nobody cares to evaluate

a good programming language will have these properties

- Good diagnostic tools
- Debugging tools
- Rich ecosystem for every use case

and nobody cares about these because maintaining software doesn't look good at you CV

Inside Deutsche Bank's "dysfunctional" IT division

https://www.efinancialcareers.com/news/2018/04/technology-at-deutsche-bank

The bank can operate with only 8-10 IT system, but because every new manager needs to show "We are innovating" the number of IT systems is 38

To maintain software is hard and overlook job and it isn't cool
and who wants not look cool ?

At least for those that maintain software there will be a lot of clean up and rewrite jobs in years to come

We need CV driven developers, because they create the economic cycle for the next generations of software developers

[-]

matorin57@reddit

Like the general premise, but I dont think his conclusion really solved it lol.

Sure fair, it’s not really an easy problem to solve. But the conclusion was written as if it had one.

[-]

levodelellis@reddit

I didn't really like the article. The first story was about a rewrite which has nothing to do with language choice, and the second story seems right that go was the better pick, but the story should have been about people picking tech they like rather than picking boring tech

[-]

Jabes@reddit

This is interesting. I speak to many people that have objectively made the wrong choice with the programming language or framework -- for where they are now, at least.

But it may have been the right choice originally (even if just velocity of the original core team).

It's very rarely the right thing to do to change entirely, at least not in any non-incremental (strangler-fig) way