the architecture decisions that aged best were the ones i made after running real traffic, not before. without production data, you're just optimizing for imaginary scale. ship the ugly thing first, then fix what actually breaks.
I agree but that is usually not acceptable for many people. In the ideal world we assume we will have to iterate after release but first impressions matter a lot, one should care about giving it their best shot if time is available.
I agree but that is usually not acceptable for many people. In the ideal world we assume we will have to iterate after release but first impressions matter a lot, one should care about giving it their best shot if time is available.
Just consider things like cars. Let's say you "ship the ugly thing first". Now your brand gets bad reputation from all the reviews which - with good reason! - call your car a piece of shit and you don't sell any, resulting in massive financial loss and your competitors edging out your brand.
I get frustrated when my PO wants me to start every new feature with a detailed plan and architecture. All of that is just fantasizing essentially. I need to actually code and mess around a little bit to explore the solution space before doing that.
Yep. Anything serious enough to require a highly skilled dev is serious enough that it's hubris to think you are just going to sit down and write it all up and then magically make it so. For those types of things, unless you'd done it five times recently with the same tools, language, paradigms, etc... it's always going to be a process of discovery.
And, though the process is more chaotic, it'll generally result in something that stands the test of time better, and thus save time and effort down the road.
I see a lot of myself in this. So many times I get started on an idea only to be frustrated by the tooling, so I pivot to fixing the tooling, only to get frustrated by the more fundamental tooling, so I pivot, only to find I'm now frustrated by some library that has 90% of what I want except the key feature, so I pivot, only to be... on and on and on. Gah!
Sometimes I wonder if the whole industry is just doomed to plough forward building hacks and fudges ontop of wobbly bases because no-one cares to take the time to sort out the fundamentals. Then I wonder if it's just me.
Sometimes I consider starting a community for this. Focusing on doing the whole stack "right" on a scale that works for individual developers. It makes sense why things have gotten the way they are, and sometimes I wonder if we can take all the lessons learned so far and make a stack where each layer is easily understood by one engineer by making the right sacrifices
For me the motivation is enabling people to back away on whatever interests them and do novel things easily, and I've found things like C compilers that sacrifice a lot of optimizations to keep the code base manageable inspiring
As a programmer of 20+ years with diagnosed but untreated ADHD this is me on pretty much every project at the start. So many times I've gone and fixed or added something to our cmake library to do something with tooling...
Or worse I architect out the entire thing in my head then get super frustrated because now I am thinking about implementing everything at once, even if I'm not the one who will do that work.
Turns out I am actually an amazing systems engineer though so that's fun when it matters. Oh like that idea? Here's where it's gunna fail 3 steps after you stopped planning! The number of times I get called a spoilsport is not fun.
Yes! Exactly like that. The worst ones are when the pivots cycle around and you find yourself needing to fix the very thing you originally started with 🫠Â
I think it applies to architecture as well, what is a problem is creating an architecture without a good enough understanding of the problem. That is knowing what is certainly not a solution, even if it works on the simple case, is a good place to start.
Because you can't predict what the solution will have to become, what challenges will be. But understanding the problem means that there'll be a core part of the system that will never change as long as you're still solving the same problem.
So there's a value in exploring different solutions, and how they succeed or fail, but then after the exercise is done, you choose the simplest, most direct architecture that best solves the problem. Because that's the most resilient one, the one that will be easier to evolve, change, etc. The exercise of complexity was to better understand what paradigms are inherent to the problem and what perceived complexity is actually accidental. Basically you want to know the most trivial thing you need to solve the most simple version of the problem. Scale, challenges, things that are unrelated, DSLs, etc? You'll get there, but you start with a simple thing. It's ok to have some wires exposed in the first iteration.
So on to the author. We can see the exercise as one of understanding what is the problem they need, the scope and reality. I do believe that they'll eventually have to go and understand why difftastic didn't do what they wanted. I wouldn't be surprised that difftastic basically diffs the whole ast themselves, but this doesn't always match. A simple tool where instead we first split the ast into abstract "textobjects", where we identify them by name (there'll be a problem when you rename entities, but lets deal with that problem when we get there, all we know is that "identifying the equivalent entities across both versions" should be an encapsulated problem, our arch is already showing), then we can diff the text-objects in broad motions (if they moved across the AST) and then we can compare the ASTs of the objects themselves, ignoring everything else. Though by making the text-objects small enough we might get away with just doing a standard text-diff at that level and get what we want. This would imply that the reason difftastic fails is because itself it's doing something different than what the author wants. They want to identify small text-objects and work on them, independent of the file (though file and text-objects being moved and renamed should be handled too I guess).
So at this it becomes clear what the architecture should be:
Traverse the AST and "cut" it at a certain layer of abstraction (e.g. types, functions and globals). Lets call these "text objects".
Diff the tree at this level of abstraction, ignoring the differences in the content (we could show a version of the text with contents collapsed), but just in the objects themselves, where new ones added, old ones removed, and some edited in place?
Let us trasverse each object, which we do by triggering a diff only on the text contained within them. Initially a text diff should suffice.
This is simple enough that allows us to add complexity. What if we want AST diffing inside the textobjects themselves? We just switch the diff to be difftastic or something like that. What if we want to further understand the diff in terms of more detailed text objects? (e.g. identify that we changed how a variable is used and/or defined, with the definitions and uses being semantically diffed). Well we can reuse the algorithm above (though identifying the equivalent text-objects after change is going to be harder) which means we just make it recursive. What if we want to support renames? Separate concern, in how we identify text-objects (maybe a few signatures could be used to imply if they're related or not). Do we need to handle definitions being moved? Or is it easier to handle it as an add or delete. We can't know until we see these problems, but the simplest architecture that is fully mapped to the actual problem is the one that is easier to solve.
Dramatic_Turnover936@reddit
the architecture decisions that aged best were the ones i made after running real traffic, not before. without production data, you're just optimizing for imaginary scale. ship the ugly thing first, then fix what actually breaks.
reivblaze@reddit
I agree but that is usually not acceptable for many people. In the ideal world we assume we will have to iterate after release but first impressions matter a lot, one should care about giving it their best shot if time is available.
Dramatic_Turnover936@reddit
What are talking about? No one cares about your stuff unless they need it, and if they really need it they will overlook the rough edges...
kubectlanxiety@reddit
and the rough edges usually tell you more about what to fix next anyway, my is proof of that or
reivblaze@reddit
Thats just too vague and general. There is more nuance in technical / executive / sales decisions than that.
SkoomaDentist@reddit
Just consider things like cars. Let's say you "ship the ugly thing first". Now your brand gets bad reputation from all the reviews which - with good reason! - call your car a piece of shit and you don't sell any, resulting in massive financial loss and your competitors edging out your brand.
Badstaring@reddit
I get frustrated when my PO wants me to start every new feature with a detailed plan and architecture. All of that is just fantasizing essentially. I need to actually code and mess around a little bit to explore the solution space before doing that.
Dean_Roddey@reddit
Yep. Anything serious enough to require a highly skilled dev is serious enough that it's hubris to think you are just going to sit down and write it all up and then magically make it so. For those types of things, unless you'd done it five times recently with the same tools, language, paradigms, etc... it's always going to be a process of discovery.
And, though the process is more chaotic, it'll generally result in something that stands the test of time better, and thus save time and effort down the road.
Horror-Primary7739@reddit
Sr Dev: You sent the entire data set to the client? At once? Jr. dev: Yea it made it super simple. I didn't want to over think it.
dgkimpton@reddit
I see a lot of myself in this. So many times I get started on an idea only to be frustrated by the tooling, so I pivot to fixing the tooling, only to get frustrated by the more fundamental tooling, so I pivot, only to find I'm now frustrated by some library that has 90% of what I want except the key feature, so I pivot, only to be... on and on and on. Gah!
Sometimes I wonder if the whole industry is just doomed to plough forward building hacks and fudges ontop of wobbly bases because no-one cares to take the time to sort out the fundamentals. Then I wonder if it's just me.
gplgang@reddit
Sometimes I consider starting a community for this. Focusing on doing the whole stack "right" on a scale that works for individual developers. It makes sense why things have gotten the way they are, and sometimes I wonder if we can take all the lessons learned so far and make a stack where each layer is easily understood by one engineer by making the right sacrifices
For me the motivation is enabling people to back away on whatever interests them and do novel things easily, and I've found things like C compilers that sacrifice a lot of optimizations to keep the code base manageable inspiring
Murky-Relation481@reddit
As a programmer of 20+ years with diagnosed but untreated ADHD this is me on pretty much every project at the start. So many times I've gone and fixed or added something to our cmake library to do something with tooling...
Or worse I architect out the entire thing in my head then get super frustrated because now I am thinking about implementing everything at once, even if I'm not the one who will do that work.
Turns out I am actually an amazing systems engineer though so that's fun when it matters. Oh like that idea? Here's where it's gunna fail 3 steps after you stopped planning! The number of times I get called a spoilsport is not fun.
ShinyHappyREM@reddit
dgkimpton@reddit
Yes! Exactly like that. The worst ones are when the pivots cycle around and you find yourself needing to fix the very thing you originally started with 🫠Â
Physical-Sign-2237@reddit
Maybe YAGNI applies to software but in architecture you need to predict a lot of stuff. Architectural decisions stays for long
lookmeat@reddit
I think it applies to architecture as well, what is a problem is creating an architecture without a good enough understanding of the problem. That is knowing what is certainly not a solution, even if it works on the simple case, is a good place to start.
Because you can't predict what the solution will have to become, what challenges will be. But understanding the problem means that there'll be a core part of the system that will never change as long as you're still solving the same problem.
So there's a value in exploring different solutions, and how they succeed or fail, but then after the exercise is done, you choose the simplest, most direct architecture that best solves the problem. Because that's the most resilient one, the one that will be easier to evolve, change, etc. The exercise of complexity was to better understand what paradigms are inherent to the problem and what perceived complexity is actually accidental. Basically you want to know the most trivial thing you need to solve the most simple version of the problem. Scale, challenges, things that are unrelated, DSLs, etc? You'll get there, but you start with a simple thing. It's ok to have some wires exposed in the first iteration.
So on to the author. We can see the exercise as one of understanding what is the problem they need, the scope and reality. I do believe that they'll eventually have to go and understand why difftastic didn't do what they wanted. I wouldn't be surprised that difftastic basically diffs the whole ast themselves, but this doesn't always match. A simple tool where instead we first split the ast into abstract "textobjects", where we identify them by name (there'll be a problem when you rename entities, but lets deal with that problem when we get there, all we know is that "identifying the equivalent entities across both versions" should be an encapsulated problem, our arch is already showing), then we can diff the text-objects in broad motions (if they moved across the AST) and then we can compare the ASTs of the objects themselves, ignoring everything else. Though by making the text-objects small enough we might get away with just doing a standard text-diff at that level and get what we want. This would imply that the reason difftastic fails is because itself it's doing something different than what the author wants. They want to identify small text-objects and work on them, independent of the file (though file and text-objects being moved and renamed should be handled too I guess).
So at this it becomes clear what the architecture should be:
This is simple enough that allows us to add complexity. What if we want AST diffing inside the textobjects themselves? We just switch the diff to be difftastic or something like that. What if we want to further understand the diff in terms of more detailed text objects? (e.g. identify that we changed how a variable is used and/or defined, with the definitions and uses being semantically diffed). Well we can reuse the algorithm above (though identifying the equivalent text-objects after change is going to be harder) which means we just make it recursive. What if we want to support renames? Separate concern, in how we identify text-objects (maybe a few signatures could be used to imply if they're related or not). Do we need to handle definitions being moved? Or is it easier to handle it as an add or delete. We can't know until we see these problems, but the simplest architecture that is fully mapped to the actual problem is the one that is easier to solve.