Seriously, how do you learn to engineer software correctly?
Posted by Famous_Cranberry452@reddit | learnprogramming | View on Reddit | 66 comments
I am decent at programming, albeit not the best. I also know enough theory in CS due to almost being done with a degree. So on that front I'm all set.
But I actually never figured out how to correctly engineer software.
I have some private projects under my belt and school has also had enough projects that I roughly know my way around.
But everytime, it looks the same:
-
Haphazardly designing something that sounds solid
-
Plan on using git correctly and document everything meticulously
-
Abandon a lot of your ideas half way through
-
Getting sloppier and sloppier with your code architecture
-
Your clean git pushes and branching turn into a giant mess
-
In the end you get something that is barely working on a codebase that looks messy and your 'engineering principles and design' went missing
Oh and testing is more like "yeah I think there are supposed to be unit tests so lemme just write some test classes. Works? Great." and then never touch it again.
It's all just very chaotic.
How can I change this and learn software engineering properly without having all the projects just turning into a mess?
I need some tips on how to learn a systematic and correct engineering workflow and design good software.
CrazyHardFit@reddit
It's a common lament that CS degrees do not teach software engineering.
Many CS grads have no idea how to design and build software, often all you learn is to drink a lot of coffee and pull all nighters to hack your code into working to meet your homework deadline.
I suggest getting a solid textbook on software engineering. I favor model based design methodologies. Software engineering is very similar to systems engineering, the tools are often almost identical. I honestly think CS should be taught as part of the systems engineering school.
a_cute_tarantula@reddit
A lot of others have mentioned that working on a project with a team for a long time is the best way to become a better engineer and I agree with that.
I do think the best software designers study some key concepts though. I would suggest you learn how to pick informative names, write functions that “do one thing and do it well” and understand why functional programming advocates for immutability and no side effects.
Just those four things take a while to master but IMO are the most important in good software design.
Perfect-Campaign9551@reddit
You get good by doing. More coding, more experience
Honest_Pepper2601@reddit
You build a lot of software. Ideally, you build software as part of a larger group of people, many or most of whom are better than you at it. Even more ideally, they review your code.
Massive_Beautiful@reddit
42 which is a programming schools known for the incredible level of the programmers that graduate, the path goes as follows: Learn Bash before anything else, knowing how to use your shell for anything will be of immense help. It is truly handicapping to not master it. After that, learn C by reimplementing the standard library (string function such as strcpy/strncpy/strlcpy) and after that implement printf from scratch. Reimplement a very small Posix Shell (pipes, parentheses and redirections). Once you’re there, you will already be very far, almost at the level where you will be able to reason through most coding problems. After that it will be just about learning new languages, frameworks and stuff like that. This is not my personal opinion, but this path has provably worked for a lot of people with 0 programming experience, after 2 years almost 100% of the students find a very good job and demolish technical interviews.
AhaoYin@reddit
Have you checked out Cursor yet? It's an AI Code Editor that can seriously boost your productivity when it comes to coding. I totally get where you're coming from with the struggle of engineering software properly. Working with experienced devs on open source projects, like those tagged "good first issue," can be a game changer. Don't rush the process; becoming an expert takes time. Trust me, being part of a team of skilled engineers will teach you more than any textbook. Keep learning and growing, you'll get there! 😊
grantrules@reddit
I think a lot of it comes from working on a team with more experienced developers so you can see how they're doing it and they can set you straight before you go off down wrong path.
If you think you're fairly competent, find an open source project you can contribute to.
Famous_Cranberry452@reddit (OP)
But isn't this a bit of a bottleneck?
"The only way to learn is by shadowing a wise man?"
I actually did have that thought as well but I can't find one where the entry isn't absurdly high.
A lot of open source projects of interest to me are so massive and complicated that it would take me another half a year to a year to even familiarize myself with the codebase, let alone contribute.
I actually tried to read through the issue tracker of a smaller project, a notepad type of software I use and I was overwhelmed because I did not know the codebase nor was there any documentation on how the code works overall.
Smart-Waltz-5594@reddit
I think you are trying to rush a process that takes a lifetime to master. Don't underestimate how hard it is to be an expert at this
____candied_yams____@reddit
I read so much about software on my own. But it was just soooooo useful for me to join a team. It's not the shadowing of an individual that was so beneficial.
grantrules@reddit
Find projects with issues tagged with "good first issue"
https://goodfirstissue.dev/
Find an issue you think you had do, then add a comment, like "hey I'd like to start contributing but I'm fairly new to all this, this seems like an issue I can tackle.. can you perhaps give me some advice on how I should begin?" .. maybe there's a discord or something with a dev channel you can join.
You don't need to sit there and study the code and understand the entire codebase. You just need to find the bits that you're working on and then look over what will be affected.
Even professional developers don't know everything about a project.. big projects are built by teams and you'll have domain experts who know one part of the project really well and can answer questions for other developers trying to deal with it, and those domain experts will go ask questions to someone else if they're dealing with something out of their domain.
So just start by solving small issues, and you'll learn about the software along the way
cheezballs@reddit
This is the answer. I feel like I've learned everything I feel are good software principles from my current job by seeing all the other code that's out there in the real world.
Alarmed_Expert_1089@reddit
So much this. Working with a team of experienced engineers is invaluable.
When I got my first real programming job, I brought a lot of habits from personal and solo projects to my code and the first review was just savage. There were so many comments and things I had to fix. I learned so much just from that one code review.
PhysicalTwin@reddit
At least you learned.
Some people get butt hurt or didn't learn anything and keep making the same mistakes.
Alarmed_Expert_1089@reddit
I’ve tried to focus my career on learning rather than being right. It’s taken me in some cool directions.
PhysicalTwin@reddit
One day when you become a senior on the other end of those pull request reviews, you may learn that juniors may not code the same way as you and to help them build confidence it's better to not nitpick the implementation as long as there aren't security or legitimate performance concerns. So while your peers may have reviewed you hard, hopefully they let you write the code in your own style enough to also build confidence in yourself. There's many ways to skin a cat as they say.
Alarmed_Expert_1089@reddit
Definitely. I am the senior now and try really hard to extend the grace I got back then to the newbies we bring on.
PhysicalTwin@reddit
Everything's about balance haha
v0gue_@reddit
This is my experience. My internship, and then my first real job after, taught me so much about software engineering that you just don't get in a CS degree. Half of the battle in software engineering is scalable architecture and fault tolerance, both of which have very little to do with programming. Without having a decently robust system to work in, I don't think I would have ever learned how to build something substantial from the ground up.
Screakkkk@reddit
You start by understanding that 95%+ of professional programming jobs use literally zero CS concepts and there is no actual engieneering involved.
What the majority of people call "software engineering" is nothing more than best practices and guidelines, and you're free to follow or disregard them as you see fit. It's not like electronic circuits, or civil infrastructure, or any other rigerous design; The overwhelming majority of software is produced via trial and error until it works, and once it works, that's usually good enough.
If you think your architectures is a mess, then redesign it. If you think your VCS layout is a mess, then reorganize.
There is no "correct" way to do things.
BlazzingBlocks@reddit
As a civil engineer, my rigorous design methodology for civil infrastructure usually revolves around “ah screw it, just give it more rebar and we’ll be fine”!
ValorQuest@reddit
Pattern recognition and application.
For every problem I look to see if someone else had a similar problem and how they solved it. If that can't be found, I see if AI can crunch some ideas. I'm building and curating my own software development library in tandem, building my experience from doing the above, rinsing and repeating about a bazillion times, and liking it all. That's software engineering.
TrueSonOfChaos@reddit
It is actually exactly like electrical engineering I wouldn't try to discount that factor in "a programming philosophy." Encapsulate and account for all possible logical states of your encapsulation (i.e. account for errors and "extreme cases" for your encapsulation). Then move on to the next encapsulated logic.
Feeling_Photograph_5@reddit
I start with architecture and go from there. I also write unit and integration tests as I go. And sure, it's tempting to get impatient, especially when your tests take longer to get working than the methods they support, but I stick with it.
My motivation comes from having worked on poorly designed and maintained software. I know how much easier tests and logs can make my life.
And it's a huge difference. I've had projects of a size that you'd think it would take a small team to maintain, but I did it on my own because I coded defensively and used good logging. That saved me so much money.
So, yeah. Pick an architecture and stick to it, and keep good test coverage. Or else pay the price.
Cybasura@reddit
...do engineering?
Like what do you think engineers (as in the idea of an engineer) do?
Engineer things - create, make stuff?
So, with that in mind - what do you think "be an engineer, not a frameworker" mean?
Blando-Cartesian@reddit
Doing the right thing continuously is possible only in two cases:
MundaneWiley@reddit
Plot twist. No one actually correctly engineers software
Venotron@reddit
Start with understanding the engineering method.
https://sites.tufts.edu/eeseniordesignhandbook/2013/engineering-method
Don't confuse engineering principles and programming principles.
Clean code, DRY, documentation, patterns, etc. are programming principles, not engineering principles.
Engineering is about focusing on the problem to be solved, then designing and delivering a solution for it.
The design and development stages in software engineering will include architecture and considering programming and language specific principles to include in the design, but the engineering design process is iterative, and that's where you're getting hung up.
Instead of throwing the design out the window and winging it when you inevitably run into issues, the idea is to go back to the design and change what needs to be changed to solve the problem, and if that means abandoning some principles that aren't fit for purpose, abandon them. Just replace them with something better.
This is important to learn to do because in the real world you're probably evebtually going to be trying to find a solution to a problem that hasn't been solved before and there won't be a known "correct" solution.
Whsky_Lovers@reddit
Decent is relative. Only once you join a team will you really start to learn how to develop. The kinds of puzzles and projects you find in college are not really all that translatable usually.
Patterns are one of the most important things to learn. You go from trying to figure out how to do it to how, has this been done in an efficient way and, how do I adapt that to the situation at hand.
Start researching different architectures like MVVM vs MVC vs MVP
Then eventually you move past the paradigms and can see efficient ways this particular project should be laid out.
ResponsibleTown1105@reddit
you dont
basdit@reddit
You learn how to write software properly when working with other engineers who want to write proper software. Add in guidance of a good architect and you will develop your skills together.
For your personal projects I would advice to set up an automated pipeline, just like in the professional world. Have it run on every commit, check code quality with SonarQube. Set test coverage to 100% code branch coverage. This will be like a colleague reviewing your PRs to keep you sharp. With a clean code base you can focus on implementing design patterns.
frobnosticus@reddit
By burning yourself by doing it wrong over and over.
That's the thing nobody ever wants to say: For every thing you know how to do, you only know it's the right way because you've tried it damn near every OTHER way first.
Grounds4TheSubstain@reddit
Answer does not have enough upvotes. I'm self-taught and mostly work alone. I've written over a million lines of code. I have written some abominations in my day, and I've learned how those mistakes made my code harder to maintain and understand. So I steer myself away from the things that produced outcomes that sucked. Ultimately I want my code to be nice because I'm the one working with it. Sometimes patterns stick, but only as long as they keep producing good results.
frobnosticus@reddit
/me nods.
I'm retired. In a little over two years I'll be able to say I've been writing software for half a century.
You only learn by carving out the negative space until you understand why the principles of the greats are there.
Most people don't even know they're cheating.
s'okay. It's how it always goes.
Thistleknot@reddit
Book on design practices?
I've bought a few
My programming has gotten a little better for simply knowing about them
JesusAleks@reddit
By getting a job. The only way to learn it "properly" is to get help and code review from people that have been software developing longer than you.
CatolicQuotes@reddit
funny you ask, I started reading this course and I think you might find it interesting: https://web.mit.edu/6.031/www/sp22/
spacepopstar@reddit
you are gonna get a lot of “stick in the industry and vibe it out” answers, but this guy has a different view you might like https://youtu.be/uyLy7Fu4FB4
imagei@reddit
Uh… it seems you put a lot of burden, expectations and structure on yourself upfront and can’t keep up with the requirements. Documenting from the get go? 😳 I’d never write anything if I did that, same with detailed upfront design, using git branches in the project infancy etc. I grow software, and evolve it as it grows.
I start with a clearly defined MVP and identify the key operations/tasks that must be performed. I write down a couple of core data structures that will travel across the code.
Then I start writing components performing those operations, usually beginning with a small set of tests, just enough to outline the component API. After I have 2-3 components I make them talk to each other and perform a simple but complete task. From there I keep adding functionality.
Sounds like utter chaos, right? Here’s the trick: at literally every step if I see anything not right I relentlessly and ruthlessly refactor until it passes the dumbass test and I can easily test everything. It sounds like a lot of work but really isn’t if your code is well-structured, simple and testable at every step.
Of course, doing this efficiently requires a good intuition of what is a good architecture and a fair bit of experience, but you can’t get those without writing bad code and making yourself improve it. I say, start small, always keep it simple and tidy (within reason), fail fast, be agile, work it until it feels good, then you will know how “good” looks like next time.
And don’t reinvent the wheel, read up on design patterns, architecture etc.
Also, once you feel the liberating feeling of having everything robust and tested and the freedom to turn everything upside down and experiment because you know it’ll either work or you’ll have a test tell you where you screwed up, you won’t turn back 😀
( what is the dumbass test? It’s when I ask myself a question: would a forgetful dumbass like me instantly understand what’s going on without seeing this code for 12 months 😅 )
joonazan@reddit
Write the tests as you are developing a feature. Instead of manually testing, you should have a unit test that lets you know that your feature is working.
However, not all code is easy to test. A sorting algorithm is pretty testable, as you can just check that the output is in the correct order. Sometimes you can make the code testable but not always. In that case you need to write the code in a way that you can trust.
My recommendation for learning to write trustworthy code is learning to understand formal proofs really well. If you can prove to yourself that each part works correctly in all circumstances without having to look at the whole, the code is well structured.
Hopeful-Sir-2018@reddit
That's because contrary to what people everywhere say - testing is similar to security - it's often a simple afterthought.
Look at every book you can buy. There's usually one chapter, often towards the end, that gives you the minimal bare bones amount of it but also says "code coverage for testing is important, you want to cover as much of your code as you can in tests". But it's just half assed thrown out.
This is the nature of engineering mostly anything. All too often theory doesn't mesh perfectly well with practice - even in real life engineering such as archtecture or even mechanical. What they see in their head doesn't always exactly map to the drawings / cad files. I've seen it IRL. 250k of destruction because "just do what I put in the drawings" instead of listening to the "monkey" (floor guy who "just" turns wrenches).
Same in software development. First off, engineering the first time around perfect is super rare in and of itself - even with experience. Secondly, with changes comes the cluster fuck of design problem because even if you had the "perfect" design - the changes tend to fuck that up nine ways from sunday. Third and finally, time. You're always have the "we'll do it later" in management. You'll worry about security later. You'll worry about testing later. You'll worry about documenting later. You'll worry about everything but release, which gets them paid, later. And once it's released... doing the "later" things never comes because there's no inherently profit in security, testing, etc. New features are profitable. This is why people say do it all right the first time. But it's nearly impossible to do it all right the first time when you're on a moving platform shooting at a moving target.
Practice. Keep going over your code and trying to figure out how to do it better.
Be cautious not to fall into the "One True Way" mentality. There can only be loose suggestions. Learn to be flexible in ideologies, languages, and frameworks.
mxldevs@reddit
As with coming up with the design of your software, you also need to come up with the design of your software process.
Various software development life cycle processes have been formulated, and generally you can just pick one and incorporate that into your process.
Waterfall, iterative, and agile are some examples you can follow.
For the actual coding, you might consider things like test driven development so that you become disciplined with coming up with test cases before you write code.
Lot of people scoff at the idea of test driven. They would ask what there is to test if you haven't written any code.
But the tests are derived from the specifications. If the tests are successful, that means the code you've written meets the requirements of the project.
It's something you have to actively think about while developing, especially if you're involved in different parts of development.
CantaloupeCamper@reddit
You learn by failure.
RevolutionaryCrab452@reddit
When having a problem, use Design Thinking methods to validate your ideas and it will help you in getting to software features/modules required to solve the initial problem and then you can think about user journey and then create a small prototype before you start coding.
wallstop@reddit
This is the path! Even the most experienced, regimented team has some of these issues plague them as their projects grow organically. Learning how to apply techniques to bring whatever you're not happy about up to whatever bar you want it to be at is part of the process. You learn by fixing these growing pains. Over time you'll build up a catalogue of "this is common stuff that sucks about complicated software projects" and you can employ practices to ensure there aren't even situations where they can happen! Things like using a code formatter on save, static analyzers, trunk based development / git flow, squash commits, whatever.
Even then, you won't get it right 100% of the time. But you'll know what things to avoid and what tradeoffs to make, which is just a fancy way of saying you'll be more experienced.
armahillo@reddit
Learn design patterns and when to use them AND mot use them.
Write tests and write testable code.
Learn iterative development — make it work, then make it better
PeteMichaud@reddit
It often comes down to discipline. You can get away with this on smaller solo projects, but if it gets too big or you have to collaborate, all this causes productivity to screech to a halt.
ZestyHelp@reddit
Dunning–Kruger effect In action here. You’re giving yourself way more credit than you deserve and you inadvertently admit that in your post.
PeteMichaud@reddit
Unnecessarily rude.
Automatic_Visit_2542@reddit
True. Idk why it's down voted
baubleglue@reddit
Base concepts of iterative development are very helpful to avoid the chaos. Of the top of my head (without google)
2, Starting from minimal set of features
3, Frequent feedback from the client. Or at least, define who the client is. Same thing for requirements, if there is none - make it up (if have a client, talk to them).
Short iterations with specific goals
At least some high level design: defining actors, main components, data flow...
Code decomposition, encapsulation of features with may change, API before implementation.
I think adopting some standard project management software (like JIRA) is important. Unit testing is probably overrated when it comes to end-to-end application, but if you develop a library (any reusable code), it is better to have it, otherwise it is almost impossible to change the library code without breaking something. Some strategy of testing should exists - even if it is manual tests (like executing set of the basic app workflows).
MrKnives@reddit
What do you mean by decent in programming? If you are and you are working alone, then it's just a self discipline issue.
If you are work in a team, you are kind of forced to work with good practices. I follow good practices because it makes it easier for me in the future and my peers would deny my PR's. I also try to set a good example for juniors and mid.
I've also worked in projects with no tests and every single time people have regretted it so never again
Famous_Cranberry452@reddit (OP)
'decent' as in I'm confident that I can solve some fairly tricky problems using my coding skills. I may not be the fanciest programmer or the fastest but I took a class where the project was to write a program optimize the traffic flow of a model small town setup in a lab via sensors and camera recognition.
So I can solve real problems.
But despite this, I feel like I have not an engineer workflow, the projects always turn very messy, the code starts off good and then just gets so convoluted, the git repo looks clean but then turns crazy with lots of weird merges between branches and rebases and all that stuff.
None of the projects feel tidy, it always seems like I just clobbered together some shit that does work but is neither organized nor structured.
I guess so but even with other people, it was sometimes a mess. Other students were often just as messy as me.
And even with those that had some experience writing proper software in their internships, it often ended up being that us as a team sort of got the project running but then they ended up restructuring and correcting the entire project on their own so that we could hand in a somewhat clean repo.
Pantzzzzless@reddit
That is a result of the blind leading the blind. Or no leading at all is more likely.
When you have a strict schedule of release dates mapped out for the entire year, contracts that are followed system-wide, and people who know better and who care, reviewing PRs, then the chances of your codebase unravelling into madness are much much lower.
And that is fine. Unless there is a real logistical need for that level of structure, you don't really need it. (Unless you just want to build habits for the future)
If you want to prevent this from happening in your personal projects, then set weekly/monthly release schedules and stick to them. Create a new release branch for each release cycle. Once that version is released, you don't push any new code to it unless you find a bug there, then you release a patch version.
Write up the code structure standards docs to your liking, and stick to them as well. It just comes down to holding yourself accountable to the standards you set.
Professional_Key6948@reddit
We're in the same boat
dldl121@reddit
Try test driven development and dynamic coding. Remember the idea should be to abstract the smallest portions of your overall goal and then make these pieces work together (unless you’re doing functional programming or have limited resources.) you need something that supports this workflow too, I recommend visual studio 2022 with C#. Comes with everything you need to write and run tests in IDE.
alkatori@reddit
Pain.
ElusiveTau@reddit
Engineering is a process of refinement: you try something, measure its performance, analyze how it can be improved, decide whether it's feasible/worthwhile to improve (in relation to an end goal), and if so, you try something else and repeat the cycle. No one writes software and calls it complete because requirements can change.
No, unit tests/testing isn't pointless. It just feels that way because you're misusing it - either your use case is too simple and isn't worth writing a test for or you aren't using it as a way to enforce specifications for a design i.e., which could be because your design wasn't good in the first place.
I sense the problem with your process is not understanding why it's done and what good execution looks like: a design that has a clear goal and detailed enough to be implemented tested, unit tests written such that it actually detects potential problems, documenting not for 100% coverage but for 90%+ comprehensibility.
ElusiveTau@reddit
Also, version control/documentation/tests are only useful to those who don't already know about the project. It's material meant to be consumed by unwitting minds - meaning other people or by you when you've forgotten about the code.
Flacid_Fajita@reddit
Software is chaotic, that’s just part of the job.
At the core of the dilemma is that products and requirements change. This applies to personal projects too. You write some code and it makes sense at that point in time, but as the demands placed up the code evolve and grow over time, the job of the software engineer is to know when and where to invest time in cleaning up the mess.
Beyond that, staying organized is just something you need to commit to. It can be easy to stray from the correct path when working on personal projects- after all, no one else is judging your work as you write it. At a company though, your coworkers should be judging your work, and you should be judging theirs. This sounds pretty easy in principle but getting 5-10 people to adhere to a rigid set of standards across time is actually a pretty monumental task, and something that requirements constant maintenance.
Just know that there is no magic bullet. Software is an art not a science. The process of getting people (or even just yourself) to follow any kind of structure is always going to be an ongoing battle, but the people and teams that are most successful are the ones that are able to commit to it, and are willing to reflect honestly on their own processes.
chocolateAbuser@reddit
how much did you study the problem and thought on what to do before start coding?
herocoding@reddit
Read books/blogs about SW-engineering, using UML (use-cases, activities, states, sequence, component, deployment, classes), think about testing and simulating, mock-objects, digital-twin, logging/tracing.
Muhammad_C@reddit
You’ll never really be able to complete plan out a project that doesn’t change while implementing it and/or users actually using it.
To an extent you should be accepting that your planned out design will change during the development life cycle and after release.
Note: Update the documentation so it’s up to date with the actual design of the system
RazarTuk@reddit
For example, I built an entire library from scratch at my previous job, and I think I went through about 3-4 iterations before settling on a class structure I liked. And even then, to get to your third point, I still had to go back and add more methods I hadn't planned on, because I completely missed a use case
Consibl@reddit
Step 1. Choose a paradigm — Functional Programming or Object Oriented.
Step 2. Follow the principles of that paradigm.
Step 3. Use TDD and MVC (or another model) to break down responsibilities.
Step 4. Always be refactoring. Always be DRY.
bobaduk@reddit
Join a team who will hold you accountable for the small stuff, then spend years making dumb mistakes until you're a walking mass of scar tissue.
That's why people are junior engineers having completed a degree. Unfortunately, you have to learn by doing, and messing things up when there are meaningful consequences. If you figure out a shortcut, you can make yourself rich and famous, but I'll not hold my breath :)