The 72-hour window: Why JIT logistics is a total house of cards.
Posted by Substantial_Drop_576@reddit | collapse | View on Reddit | 93 comments
Been going down a massive rabbit hole lately with how our supply chains actually work, and honestly it's kind of terrifying.
Most people still think the country runs on physical trucks and highways. That’s a totally outdated way to look at it. Everything now runs on digital ledgers. A trucker can't even pump diesel without a fleet card pinging a corporate server somewhere. If that server goes down, the pump doesn't work. Period. We literally saw this happen with the Colonial Pipeline hack. The actual physical pipes were perfectly fine! They had to shut the whole thing down because their billing software got locked up. No digital approval means no physical movement.
And don't even get me started on the power grid. It's got a C-minus rating from the ASCE and it's almost entirely managed by automated software, not manual switches. If there's a real, sustained outage, the emergency protocols are brutal. Hospitals and critical infrastructure get power first, then dense city centers. The suburbs? We're basically at the absolute bottom of the priority list.
I've been doing the math on a cascading failure scenario, and it almost always points to a 72-hour window before residential areas hit what emergency planners call "managed scarcity." Which is just a polite bureaucratic way of saying you're completely on your own.
I got so obsessed with this that I ended up editing a short 9-minute video breakdown just to visualize the exact sequence of how it collapses—from the payment systems freezing up to the suburbs getting completely cut off. (I'll drop the link in the comments if anyone wants to check it out, don't want to spam links in the post).
But seriously, if digital payments just stopped working tonight, how many days do you honestly think your neighborhood would last before things get ugly? Everyone around me acts like the local grocery store is this magical infinite food glitch, but the reality is it's just a 48-hour illusion.
Annasach@reddit
About 15 years ago I was editing a post-apocalyptic novel and part of that was fact checking. Went down the rabbit hole and I’ve been a collapse prepper ever since. Taught myself how to grow food (that took 10 years), can and store food, basic electronics, plumbing, building, and textiles. Have a good collection of how-to books for dummies, including foraging and medicine, but still need to work on the library. People don’t want to see what’s happening, so I don’t talk about it with anyone but, yeah…
daywreckerdiesel@reddit
Why would you want to live in a post-apocalyptic world?
Substantial_Drop_576@reddit (OP)
Man, you are light-years ahead of 99% of the population. It's a lonely road when you see what's coming and everyone else is asleep at the wheel. The fact that you actually put in the work to learn those skills is incredible. Massive respect.
Substantial_Drop_576@reddit (OP)
Here is the link to video breakdown i mentioned for anyone interested : https://youtu.be/QDlHyaWEdQA?si=GghzkWMHBb0Z38UG
daywreckerdiesel@reddit
I will literally never watch a video with that AI slop voice, what a waste of your time.
bluetrevian@reddit
What are your thoughts about permissionless, decentralized, anti-fragile, distributed, digital ledgers?
Substantial_Drop_576@reddit (OP)
It's the only logical path forward. The problem is we're still running 21st-century digital currency on top of a 20th-century physical infrastructure that has a C-minus rating. You can't have a decentralized system if the power grid itself is centralized and fragile. I actually touched on that exact weak point in the full breakdown.
bluetrevian@reddit
What do you think about solar Meshtastic nodes?
https://meshtastic.org/
https://github.com/BTCtoolshed/MeshtasticBitcoinCore_Bridge
https://github.com/eddieoz/btcmesh
StarlightLifter@reddit
Thanks for the breakdown but I ain’t doing that AI stuff.
Substantial_Drop_576@reddit (OP)
Totally fair man, I get it. I just used it to visualize the research since I'm working with zero budget, but it's definitely not for everyone.
AngusScrimm---------@reddit
Actually, it's good content. Reminded me right away of Joseph Tainter's: The Collapse of Complex Societies (1988), "Tainter examines why civilizations such as the Maya, Chacoan, and Western Roman Empire experienced rapid collapse. He proposes that societies naturally become more complex as they attempt to solve problems, creating additional layers of bureaucracy, infrastructure, and specialized social roles. This complexity requires substantial energy and resource inputs..." The book is a classic, accessible free if you look for a minute or two. We are, to say the very least, incredibly vulnerable.
Substantial_Drop_576@reddit (OP)
That's the perfect way to put it. Tainter's book is a classic for a reason. We’ve become so complex that the system can't even withstand a single digital hiccup without the whole thing coming undone. It's like we built a skyscraper on a foundation of sand.
Middle_Manager_Karen@reddit
Six sigma removed one thing, resilient systems.
The optimization of profit gates the idea a a redundant component because every person, pipeline, resource has a cost to maintain so consultants came in and said
Remove this! It’s extra and costs money to keep on the books.
20 years later your 72 hour hypothesis is highly accurate very few teams and systems can sustain a component going missing.
Substantial_Drop_576@reddit (OP)
You absolutely nailed it. "Resilience" became a dirty word because it couldn't be quantified on a quarterly earnings report. The consultants sold the C-suite a fantasy of a perfect, frictionless world, and they bought it hook, line, and sinker. Now we're left with a system that has zero margin for error.
I actually visualized the exact moment this "zero margin for error" system breaks down in the full documentary—it's even faster than you'd think: https://youtu.be/nprhIfvKxLM?si=1VlsQ2XHXzqvqUNR
mityman50@reddit
Ohhh shit you’re a bot
Alita-Gunnm@reddit
Please don't make AI slop. If you have something to say, just say it.
TryptaMagiciaN@reddit
and the best part is that we dont even need a human failure. the sun could "do a funny" and throw us all into that exact scenario.
Substantial_Drop_576@reddit (OP)
Man, don't remind me. That’s the ultimate wild card. We’re so optimized for efficiency that we have zero slack for a natural event like that. It’d be a total reset.
TryptaMagiciaN@reddit
The slack are called the meek otherwise known as the amish, or mennonites in the US. at least where I'm from haha. Or what remains of some the indigenous cultures of america.
The reason it doesn't feel like "we" have zero slack is because we aren't a part of their "we".
And it isn't like they just disregard advancements. They spend time learning how to adjust for climate change and more. And I suspect if they needed to immigrate between states due to disruptive ecological scenarios, they would have people ready to take them in and help them.
How many of us can say we have that? Strangers in other places ready and willing to help us should we lose everything?
Interesting times make for interesting reflections.
I was never raised that way. I grew up with all the tech and internet. Still learned how to work, but not how to know what I desired or how to follow and trust in myself.
Took a decade to realize that (for me at least) it feels like it comes from a close community/family because it will only come from within you once you are ready to do for others.
And I'm not a member of any religion or group. I like to imagine myself as a passive observer of others and myself, and those were the conclusions of my reflections which involved (in my mind) most of the people whom I had grown up with / had raised me.
And total reset is interesting. we can create a small box to shield a solar panel, small computer with a large ssd, and a battery.
When it all settles. You get it out and begin working.
What terrifies me are the generations of learned helplessness and reliability on these broad systems. People will get scared and react. So many will die from completely preventable causes, mostly violence. And a violence that has been fomented amongst us, commom citizens, by a small group of very wealthy peoplpe preparing for any sort of collapse scenario.
This is the time to set aside pride and shame and really look at all we fear to lose. And just hold those feelings of loss and pain. See if you can get through a week holding it. Then get back to work.
We have to be whatever comes next if we want there to be something. And the wealthy believe that. They have prepared for that.
The resource in question is our own selves. Our personality, our integrity, our psyche. That is the logistical question. How many of us can put fear aside to work together when it all crashes down. That's the only real fight. And you can participate at any time from anywhere just by reflecting on your experience and the world around you and sharing that openly with others. Worst case you cause someone an anxious night, but we all need to be more anxious, and then we need to regulate and get to work at building what needs to replace the systems that got us here.
And as much as I hate to say it, it won't be a socialism or a communism. It will have to be something none of us have done before. Something we haven't died for. And most importantly something that not only prevents or regulates, but eliminate the need for economic systems. (which is why I'm no longer a socialist or communist or any other kind of ist, Im just a fkn dude)
And while that's all insane, should there be some major collapse where billions perish.. it will have seen like the obvious only choice.
That is the real power of reflection by the way. Not to see what you did before, but to see why so you can see what your future self will be doing. What sort of things are motivating you from within that your are not aware. Then we get to make the obvious choices instead of watch it all collapse like a bunch of silly geese. 🤷♂️
ttystikk@reddit
I'm in advanced indoor agriculture and of course the trend is always towards more computers, more automation, more control systems. But what happens if something, even something as simple as a thermostat, fails? Today's systems fry the crop and render it useless! There is no fault tolerance or resilience built into the system anywhere!
It's insane and I'm developing systems that need a minimum of control to work, precisely to build in resilience and robust fault tolerant operation, where there are no single points of failure and failure modes protect the crops rather than destroy them.
You'd think this is an easy sell, but noooooo....
Alex5173@reddit
One thing I don't understand about modern agriculture is how exact it is. X plant needs Y moisture in the soil and A,B,C nutrients at L,M, and N levels; and if the wheat grains are too moist they're worthless.
500 years ago we just put it in the ground, hoped it rained enough, and a few months later we harvested. What happened?
GreatPlainsFarmer@reddit
As a grain farmer, that’s an odd take. Modern hybrids are far more tolerant of a wide range of temperature and moisture conditions than the ones from 20-30 years ago. You can screw things up pretty badly and still get 80% or so of normal yields. All the tweaks and exact timing are all to squeeze out that last 10-20% of production. 80% of maximum yield might be a financial disaster for the farmers, but it’s not famine level crop failure.
500 years ago you had to leave ground fallow every few years because it simply couldn’t produce at all continuously. Famines were common.
Now the markets consider a 10% overproduction to be a burdensome thing that should be punished with low prices. The concept of a true widespread crop failure doesn’t exist anymore.
ttystikk@reddit
Yield maximization.
Also, we have a lot more mouths depending on every farmer and every acre under cultivation. That requires ever more careful allocation of water, nutrients, pesticide and of course diesel fuel, maintenance and labor time.
The increased productivity is awesome but it comes at the cost of fragility and ever longer supply chains.
This is a deep, deep dive but I hope that gives you a glimpse of the situation- and what happens if, for example, a war over the Strait of Hormuz drives the prices of diesel fuel and fertilizer up or cuts supplies altogether.
TryptaMagiciaN@reddit
It can be, but really it isnt. All we really did was enable the growth of economies to support billions using system management that nature cannot sustain. That means there isn't a magical equation.
And the biggest issue with people adopting indoor farming, is that farmers farm because they LOVE being outside. They cannot imagine life indoors.
and that doesnt make me against the indoor model. I just think it may contributr to the difficulty you may be having.
granted, I'm of the mindset of a hybrid model. I like large greenhouse designed to function somewhat like an indoor system but growing directly into the land beneath.
And the reality is we may need to be doing both things.
Because we need focused land/soil restoration and we cannot do that with feeding billions. at least not at where we are at currently in the system.
If there are models that can feed billions by scaling indoor farming (maybe even automate it) while we focus all attention outdoors on restoring the land. And that is something you can get farmers into. They just want to feed their families. And if they were paid for every acre of soil they restored from the profits generated by indoor farming... that would be something.
Then maybe 100yrs from now, we can have healed the environment to the point where we can support billions without the need for indoor farming.
The future is agrarian, the question is whether farming at these scales is agrarian. Or just destructive. We don't know. we havent done it long enough to really know. But outlook is poor and our gut says we need serious restructuring.
ttystikk@reddit
You raise a lot of very interesting and insightful points in this comment.
I'm not so optimistic about the future; while we have so far been able to use technology to stay ahead of increasing food demand as the planet's population explodes, I don't think it's a trick we can keep pulling off.
We're already breeding plants and animals far beyond any semblance of viability in the wild. Genetic engineering is just the next level but it is the opposite of resilience.
We've pretty much run up against the limits of arable land, even while we are busy paving, polluting and washing it away. Sea level rise will also chip away at habitable land and force mass migration. Climate change is already making vast areas less habitable for people, animals and crops. There is now only a thin rind of land left over for wild plants and animals and ecological collapse is all but inevitable. Water scarcity and drought will go from inconvenient to life altering.
Humanity already profits from warfare and there will be much more of it. Food scarcity is just one more weapon in the military arsenal.
Indoor facilities offer the promise of being able to grow food in inhospitable places, independent of local climate. They recycle well over 90% of their own water. As long as they get sufficient energy and fertilizer, such facilities can produce food year round, very consistently at very high production levels. This tech won't replace any, let alone all other forms of farming but it will find a niche. One niche will surely be survival bunkers.
Substantial_Drop_576@reddit (OP)
We traded common sense for data sheets. Instead of working with nature, we tried to turn farming into a chemical equation we could control. We got higher yields for a while, but lost all the built-in resilience that nature provides.
TryptaMagiciaN@reddit
we wrecked the climate and pumped tons of stuff into the sky that affects these cycles.
Also, how we plant. Monocropping = awful.
500yrs ago you had bugs and animals and other plants in your field and you didn't till the entire thing every fkn year either.
what happened? people who worked slower, were able to observe better.
everything we do is too fast
TryptaMagiciaN@reddit
I think all of us that desire the transition your speak of, are not wealthy enough to adopt your product.
Poor people have fear over expending their resources but would actually care about this product.
Wealthy people are greedy and don't want a good thing like your describe if they already have something that "works" and if your product cannot create more value for them rather quickly then they don't care.
I'm curious about your product though. Ill be moving back to farm country this fall, is there anywhrre I can look into your system?
ttystikk@reddit
DM me! I'm still in the startup stage.
No_Mall_2885@reddit
Well said
Environmental_Art852@reddit
Have you seen the upcoming CME? Push by the wind of solar storms? I've found it on 3 web sights
TryptaMagiciaN@reddit
just looked it up https://www.sidc.be/here
firekeeper23@reddit
What is Six Sigma?
mityman50@reddit
A system of statistical analysis and a management philosophy which aims to produce 99.99966% good parts in manufacturing, or where variance of up to six sigma from the target is still a good part.
There’s no relationship between it and reducing redundancy. That first sentence is nonsense.
Alita-Gunnm@reddit
I think they meant "lean".
firekeeper23@reddit
I have no idea what those words mean but I thank you.
mityman50@reddit
The management philosophy side of it is called DMAIC, and is a lot easier to digest than the stats side (I hated the stats side.. failed intro stats the first go around lol).
firekeeper23@reddit
Ok. Thank you. Sounds interesting.
mityman50@reddit
Six sigma has fuck all to do with reducing redundancy
Spunge14@reddit
You either don't understand Six Sigma at all, or you're calling "ruthless capitalistic optimization" Six Sigma.
Six Sigma absolutely allows for redundancy for safety reasons. You set your tolerances before you optimize. In a manufacturing process that cannot reach a specific error rate without a redundancy (or any other parameter for that matter), Six Sigma doctorine doesn't dictate that you just say "fuck it."
daviddjg0033@reddit
How did we get here? Coming out of WWII we had a manufacturing base that had some adequate redundancy but quality was far less than six sigma. The times were great we promised countries with bad histories we would defend them so no military was needed like Germany with 40k troops and Japan. Beginning of the 70s had muscle cars and we ended up with a Ford Pinto. In that decade you would look at a US plant and see half finished vehicles lying around. The cars were more costly to produce than Japan and were of less quality (six sigma) than Germany. Honda or Toyota developed fuel efficient vehicles for less money using the JIT just in time inventory system where the part would arrive just in time 72 hours ideally to be assembled into the vehicle.
There was a delusion that Japan would somehow take over the world by the early 1990s. That was the top of the Japanese stock market that had not been surpassed until recently. American companies could not compete on price or six sigma or one in a million defective rate quality with Japan. I can speculate about semiconductors and Taiwan here. COVID-19 broke out in Wuhan China in 2019 and suddenly one out of every ten barrels of the 100M bbl/day we use were not needed and oil went negative. Stimulus was dispersed that could have been used to make supply chains redundant. This would not happen under a pure capitalist libertarian agenda. The kind of change only a blend of capitalism and socialism that the moderates in the US advocate for. We subsidized fuel efficient vehicles and appliances. But asking the ones who lost a job or half the income or morevduring these times to upgrade was an issue I faced. War broke out in Europe and the US started to drain the SPR. Cars that were 15y old were selling for $20,000 with a 10%+ loan to subprime borrowers. New cars continued to get larger while a Cybertrunk was unveiled. Jevon's paradox applies to cars as well. Now we are looking at one in five barrels taken offline but adding back say one in ten draining our fuel reserves. Japan has no energy sources and abandoned a nuclear program that was not ideal compared to Germany where nuclear power plants make more sense or France because they are not placed on the Ring of Fire earthquake zones. Nobody is claiming Japan will take over the world.
Santzes@reddit
Also the timelines of metrics is completely messed up. No CEO will get bonus for trading current stock price for long time resiliency. Or politician getting voted to office because they fixed a problem 10 years down the road.
The whole system is built around rewards based on timelines way shorter than the problems we face
Substantial_Drop_576@reddit (OP)
Spot on man, they literally optimized the survival right out of the system just to save a few bucks on paper.
Substantial_Drop_576@reddit (OP)
Vegetaman916@reddit
Watched about a minute, just to not kill your analytics with an immediate dump. But the AI voice and canned B-roll is a hard pass from me.
Although I am pretty sure I agree with everything you were about to present in the video. Cascading failure has been my personal baby for years now.
Audience notes: You audience is the collapse-aware, or collapse-adjacent, and also the prepper types. None of these folks are big fans of AI. Without considering the other issues, at the very least AI is a huge contributor for water scarcity issues, power consumption issues, information security issues, and a whole host of other environmental and economic concerns the circle the drain if collapse.
And that's without the Terminators, lol.
People won't watch it. Not in that niche. They want authenticity. I would probably have watched all the way through if it was just you sitting on the couch reading your script from a tablet. This is collapse, which means it is educational, not entertainment. You don't need to grab with drama ir cinematic music, you just need facts and data.
My two cents which, due to inflation, will now cost you $17.43
Substantial_Drop_576@reddit (OP)
Appreciate the honesty. Still learning the ropes
Ghostwoods@reddit
Vegetaman is spot on. As usual. AI content still more or less works for tech enthusiasts, horny teen guys (so long as voice/avatar is a hot woman) and lowest common denominator fluff content, but if you want traction with any other viewers, even stick figure sketches and an "English as a third language" accent will genuinely get you much more interest.
Vegetaman916@reddit
Sure thing. And I only said it because I wish someone had been a dick with me about some stuff when I first started YT, lol.
tropical58@reddit
Buy locally produced wherever possible. Produce what you can, it takes practice. Most important of all, connect with your neighbors.
the_pwnererXx@reddit
Ai generated post to promote an AI generated video, but the skeptics and haters of /r/collapse arent able to notice?
Substantial_Drop_576@reddit (OP)
Haha, I'll take that as a compliment on my late-night writing style. The script and research are all me, man—just used some modern editing tools to put the visuals together. The real question isn't who made the video, but whether the data in it is wrong. Is the grid not rated C-minus? Did the Colonial Pipeline not shut down over billing software? Let's focus on the message, not the medium.
the_pwnererXx@reddit
literally an agent ^
Such-Rent9481@reddit
Literally. Jfc
jbond23@reddit
Its way past time to start building for resilience, not just quarterly profits. Supply shocks like the Ukraine and Israel-US-Iran war should be the wake up call.
Vegetaman916@reddit
Man, you guys are both wrong.
DivaExMachina666@reddit
The End of Days novels by John Birmingham explores a scenario where the US logistics system is brought down in a cyber attack and how that plays out. If anyone is into post apocalyptic fiction is struck me as a fairly realistic depiction of how that would go down.
Substantial_Drop_576@reddit (OP)
Definitely checking that out. Thanks for sharing
xThomas@reddit
AI assisted post? Sorry if not AI, but I have that very faint suspicion.
Substantial_Drop_576@reddit (OP)
Interesting! Adding that to my reading list.
Substantial_Drop_576@reddit (OP)
Haha, not AI. Just way too much coffee.
lowrads@reddit
Just in time inventory means all the time headaches for operations staff.
C suite ghouls will cause endless amounts of turmoil just to make the troublesome metric in their sector be smaller. It's the same as when one department streamlines their workload by dumping it on another department.
Overall, it's a consequence of democracy and consensus being excluded from the workplace. Having just a few people in charge means that organizations are rarely efficient or sustainable over extended periods of time.
Substantial_Drop_576@reddit (OP)
You've perfectly described the disease of modern management. We've created a system where the people making the decisions are completely insulated from the consequences of those decisions. The C-suite gets a bonus for "streamlining," while the operations staff deals with the chaos. It’s a complete disconnect from reality.
lowrads@reddit
Most people that are socialists are so because they understand they and others are not getting a fair share of what work yields, and are treated poorly as a result. I am a socialist because even honorable work is treated poorly.
Livid-Rutabaga@reddit
Oh I knew this back when everybody was switching to JIT. Back then my father used to hide parts without putting them in inventory to avoid equipment being down waiting for a part. He worked on hospital equipment, and he said a patient would never have to be delayed as long as he could avoid it. When something broke he would magically have just what he needed to fix it.
I worked for a sewing machine manufacturer, we switched to JIT, went through ISO and SIx Sigma training, etc., I can't believe it's worked this long, really I thought this system would have fallen apart by now.
Substantial_Drop_576@reddit (OP)
Your father is a legend. That move hiding parts to bypass the new "efficient" system for the sake of a patient is exactly what's been lost. He understood that resilience is more important than a perfect inventory sheet. It's insane that we've let the spreadsheet-gazers and consultants dismantle that kind of common-sense thinking. Massive respect to your dad.
Livid-Rutabaga@reddit
He was a legend, the hospital dedicated their catheterization lab to his name. I wish I knew if his name is still on the door.
Management made it a condition of renewing their contract that he be permanently assigned to their facility. So yes, even then the recognized the value of the old fashioned inventory.
marioncrepes@reddit
I keep attempting to explain this with my folks and they do not seem to get it. They are convinced because they've made it everything is golden, like most Americans, despite being progressive and open minded, I guess it just feels like another ME war. They're both watching the Daily Show and read the news too, it's not like they don't know the severity of the situation, it baffled me
Substantial_Drop_576@reddit (OP)
Man, I feel this in my bones. It’s the "normalcy bias" in its most powerful form. It's almost impossible to explain the fragility of the system to someone who has only ever known it to work. For them, it’s just another headline, another distant problem. They can’t visualize the day the ATM just says “insufficient funds” for everyone at the same time. You’re not alone in feeling baffled.
GagOnMacaque@reddit
Just in time delivery is basically a scam promoted by Business consultants who just want to prove that they're doing something. The fact that businesses no longer stock up on inventory has severe consequences when shortages happen.
Substantial_Drop_576@reddit (OP)
It's the ultimate "looks good on a PowerPoint" strategy. The consultants sell a fantasy of perfect efficiency, collect their check, and disappear before the first supply shock hits. They privatize the profits from the efficiency gains and socialize the losses from the fragility they create. It's a perfect scam.
firekeeper23@reddit
They seem less like delivery chains and more like delivery teaspoons... one.. at. .... a ..... time.... and no more. And don't get me started on getting prescriptions filled out ahead of time.. or any access whatsoever for antibiotics unless you scam the system with spurious infections.
The UK is the worst for any kind of forward planning..
Substantial_Drop_576@reddit (OP)
"Delivery teaspoons" is the perfect way to describe it. One little disruption and the whole thing grinds to a halt. It's especially bad with anything medical. The system is designed for perfect, sunny-day conditions and has zero tolerance for any kind of disruption.
firekeeper23@reddit
Absolutely 100%
UncleBaguette@reddit
I really recommend "Blackout" by Marc Elsberg for a perfect literary description of "house of cards crumbled" situation in EU-Wide power outage...
Substantial_Drop_576@reddit (OP)
Ah, an absolute classic. Elsberg's book should be required reading. He did a terrifyingly good job of showing how quickly the veneer of civilization peels away once the power goes out. Thanks for bringing it up!
oesness@reddit
The real point to observe also is the overall de-skilling of modern society. When we examine historical collapse scenarios something to note is that in every one of them you had a baseline of people that could plant a seed and end up with bread. The modern human by and large couldn't even identify a wheat seed let alone grow it to being to envision bread from it is a distant myth. When things go this time you are going to have panicky talking monkeys making poor decisions en masse.
This topic alone assumes the very normalcy bias that will ultimately be the biggest threat. The entire organization of the neighborhood will be focused on holding it together till the lights come back on instead. That lag time will be their undoing in the end. The ones that survive any aftermath will be the ones proceeding from the surety that the lights are not coming back and growing food instead of surviving on MREs and waiting.
We are modeling so many things assuming there is a skill floor to fall back upon. Theory is not application....one can understand the life cycle or a plant but not grasp that they like the weather have moods. Essentially if one is not learning everything there is to know about growing potatoes and doing that right now they are about to have very bad time.
Substantial_Drop_576@reddit (OP)
You've hit on the most terrifying part of this whole thing. It’s not the system's fragility that's the real threat; it's our own. We've outsourced our survival skills to the point where "self-reliance" is a foreign concept. The real collapse won't be the grid going down; it'll be the mass panic of millions of "talking monkeys" who can't even grow a potato. It's a terrifying thought
weyouusme@reddit
maybe drop the dumbass music and why is this guy sound so dramatic
Substantial_Drop_576@reddit (OP)
Appreciate the feedback. The goal was to create a cinematic, documentary-style feel to match the gravity of the subject. It’s definitely a specific vibe and not for everyone. Thanks for watching.
abstrakt42@reddit
Former sysadmin and multi-certified cloud security professional checking in. Resilient systems are a core component of modern infrastructure, there isn’t just one pathway to one system that can break down, it’s much more diverse than you might thing. At this scale, organizations are running so much redundancy it would make your head spin - there are multiple pathways to multiple geo located data centers, each data center has multiple redundant internal data pathways, redundant power systems, load balancing and high availability, and each processing node has been split out across multiple purpose built containers each of which can drop and be replaced in seconds with near instant failover to additional working systems.
While it’s not impossible to force these systems to fail, it’s quite difficult. The engineering principle behind them is that it’s not if, it’s when things break. That’s part of the design and automation principles in place. The self healing workflows are immense and powerful.
So, while the systems themselves are very difficult to disrupt (and trust me, threat actors are hard at work 24/7/365 attempting to just just that), this real issue relates to collapse is how many resources it takes to run such an operation. We should be concerned about power, water, manufacturing, mining, climate impact, jobs, and a slew of other issues that are absolutely related to risk here. But the systems themselves will be just fine - trust me, the billionaire owners make sure of it.
Cyberdogs7@reddit
On the flip side, as someone that user to be a Sev 1 resolver at Amazon, most of the redundancy doesn't work because of Human planning failure.
Example: AWS had/has 2 data centers in Japan and all amazon teams had redundancy built and deployed to both datacenters. However, they never actually implemented a switch over, so when data center 1 went down, all of Japan when down, even though Amazon had been paying millions per year for redundancy.
abstrakt42@reddit
That’s a shocking oversight. Yikes. I’m hoping this was a miss due to some failure in foreign team synchronization or localized issue. Or that this story took place in the relative early days of what’s now full cloud supremacy - the ramp up was a little painful, I’ll admit. But since the increase in IaC and time for teams to be brought up to speed on the new tech, this sort of example should be an extreme outlier.
I still believe and have experienced that “most” of the time these things are built so the failovers have failovers with failovers.
Cyberdogs7@reddit
This was core Amazon teams and this was in the 'relative' recent past. It happens all the time. I mean look at the last 3 major AWS outages. When people are trained to look for the complex problems, they tend to overlook the simple ones.
abstrakt42@reddit
Let me guess. DNS issue 😂
Cyberdogs7@reddit
I never got to the root cause, my job was to mitigate the impact and get back online as quick as possible, then issue 'Cause of Error' tasks for the responsible teams to report up to their VP with a root cause and solution.
FastPraline3322@reddit
I also think both you and OP are underestimating the amount that goes into physical JIT logistics. I work in logistics related to food distribution and this shit is held together with shoe strings and bribes. All the IT bullshit in the world won't save us if that breaks.
abstrakt42@reddit
I agree. Ive been following the looming fuel crisis (diesel, jet fuel) closely, and the potential systemic impact, it’s horrifying. I was addressing just the digital ledger aspect of the post which I assumed to be OPs main point.
But yes, things are precarious as fuck right now. It doesn’t look great.
vinegar@reddit
$2/mile or $2/gallon?
Substantial_Drop_576@reddit (OP)
Appreciate the insider perspective. You hit the nail on the head—the system is an absolute beast when it comes to self-healing, but it’s a beast that needs to be fed constantly. That’s exactly where my concern lies: the physical supply chain and energy footprint required to keep all that 'resilience' running. When the cost to maintain hits a breaking point in the physical world, even the best failover in the world won't save the network
Fire_Shin@reddit
I watched the video. I know that the just in time model can easily go sideways and I found your video to be a good visualization if what could happen.
I'm interested in your sources for the claims you make in the video. Could you provide links, please?
I'm asking because you present a compelling argument with your video, not because you don't.
Thanks!
Hour-Stable2050@reddit
As a Rail Traffic Controller I often questioned the wisdom of the JIT model. Factories would shut down and send everyone home because a train carrying parts for it derailed or got in some kind of accident. The railway would be blamed when it was their JIT working model that was to blame.
Fire_Shin@reddit
Yes, it's not a very good system if things go wrong. But what could possibly go wrong? /s
Dense-Security-5571@reddit
6 years ago
https://www.youtube.com/watch?v=oktrr6I3DY0