It’s always firewall?

[-]

At the bank I work for, "It's always InfoSec" is right 95% of the time. SSL inspection, random firewall changes, blocking shit with their overzealous AV. If we didn't have to deal with that BS, we could actually have time to fix the 300 vulnerabilities they want us to fix.

Reply

[-]

charmingpea@reddit

Can't be the firewall if the firewall is always off! If DNS is 'off' it's definitely DNS...

Reply

[-]

scratchduffer@reddit

This guy does DNS.

Reply

[-]

ComfortableProperty9@reddit

A security vendor friend joked that their product secured your data. If the VPN wouldn't connect then technically your traffic is still secure since no one can get to it, not even you.

Reply

[-]

StaffOfDoom@reddit

I could when I first started where I'm at now...previous network admin did a migration from a different brand appliance and didn't do basic clean-up/testing after so some 60 rules in place, only about 7 actually handled any traffic during the life of the device.

Reply

[-]

dracotrapnet@reddit

"The network is down for everybody, nobody can work" - no, you kicked your network cable out of the wall socket. Did you even ask your neighbor if the network was down? "Email is down" - not this time, you need to enter your password and MFA code once a year. Sometimes Microsoft does an oopsie but not this time. "The server is down for everybody, nobody can work" - no, you need to connect to the internet then log into VPN first. "The firewall is blocking X website" - no, that's a 500 response, their backend broke or that's a 400 response, the link you clicked from that email 7 years ago is outdated. "The firewall is blocking twitch" - you're right for once! Your boss had me block it.

Reply

[-]

BoltActionRifleman@reddit (OP)

The amount of calls like this we receive is staggering, once in a blue moon it is actually a wide outage, but most times it’s not and is just one user preaching doomsday (for some reason). My first question when I get these calls is to pick a coworker of theirs who I know is usually sitting right next to them and have them ask if theirs is working. That usually makes them change their tune in a hurry.

Reply

[-]

acniv@reddit

Standard trouble shooting order: My downloads are slow. 1. Desktop tech: Must be the network 2. Network tech: must be security 3. Security tech: must be the server/cloud provider 4. Server tech: must be networking/security/pc issue 5. Start at 1 and repeat until user’s supervisor calls IT VP complaining it’s been 2 days and no one has called/updated/resolved users issue

Reply

[-]

stratospaly@reddit

Smart firewalls can fuck your shit up. But it's always in the logs. DNS can fuck you up if a fairy flaps it's wings the wrong way.

Reply

[-]

BoltActionRifleman@reddit (OP)

Fair point, but to be fair I’m currently stuck with Cisco FTD. Sometimes I swear there are little digital Gremlins living inside the firmware.

Reply

[-]

awstott@reddit

We ran into this when we upgraded Snort - It just started randomly blocking traffic. Wreaked havoc on VOIP - it took us far too long to figure out it was the Snort upgrade that borked things. Thankfully we had everything logged to Graylog so it wasn't hard to find the evidence once we started connecting the dots. Downgraded Snort and the issue went away. We've since moved to a different firewall vendor, but that just happened so the jury is out on the new product at this point.

Reply

[-]

someguy7710@reddit

We had some Cisco ASA's with the Firepower modules. It would randomly start blocking all traffic. It got to the point where I had to put it in monitor only mode. Thank god we replaced them. Of course this was during covid and we were all working remotely. I setup a backdoor firewall (VM) so I could remote in and reboot them.

Reply

[-]

flems77@reddit

LOL - if a fairy does what?!? :) :) :) But I agree 100 percent. The irony being, that DNS has been around since forever, and is about as basic at s it gets. For the main part, it's nothing but 4 types of records, stored in a distributed system. It is kind of ironic, that we can't wrap our heads around it to an extend, so "it's always DNS" is a thing :) That said - we have actually disable most of our firewall logs. DOS and DDOS is blocked by the firewal, saving the life of our servers. But resulting in the firewall dying due to the massive log activities. That's kind of ironic as well.

Reply

[-]

GhoastTypist@reddit

Can't wrap our heads around it because people don't have a good analogy for it. When I learned about packets and how data flows, it was taught to me like the mail system. Each component required for data flow was explained. Like you have your post office, your trucks, your mail system database which shows where each person lives, your mail routes for which driver goes where, etc. Or its like an apartment building, DNS is the board on the wall which tells you what floor the person you are looking for lives at and their apartment # so you can contact them to let you in.

Reply

[-]

flems77@reddit

It's actually a really good point. Best comparison I can come up with, is the good old phonebook... It's all there... But a lot may have happened since it was printed. Or knowing the exact company, department and employee you would like to reach - but you to call the local receptionist, becuase you forgot the number. Either he/she remember the phonenumber from any previous calls (and it may or may not be valid) - or he/she has to patch you through to the department (who in turn will have to patch you further down the line). Not sure it's a useable analogy - but it's a try :)

Reply

[-]

GhoastTypist@reddit

I use the front receptionist analogy as well when I use the apartment concept.

Reply

[-]

Hel_OWeen@reddit

I always explain that it's basically the Internet's telephone book. And the reason for both is that most humans remember names better than numbers. You want to call Aunt Edna, you look up her phone number in the phone book. You want to "call" www.domain.tld, you look up (actually your browser does it for you) its IP address/number on the DNS server. Wrong number in the phone book: "Sorry, couldn't reach number" or simply "toot, toot, toot". Wrong IP address: "404 site not found" Sure, there's ofc more to it, but most laymen get it that way.

Reply

[-]

skylinesora@reddit

You'd hope it's always in the logs but not always the case. You don't log every interaction and you don't always have firewall traffic for long durations of time. One company I worked at, you'd be happy to get 2 hours of logs.

Reply

[-]

nickcardwell@reddit

With a NGFW (Layer 7 FW) Could be IPS, IDS, Application control, AV or some other filtering update that causes issues.

Reply

[-]

gummo89@reddit

Yes or your licence expires and it fails closed, disabling all internet 👌🏻

Reply

[-]

nickcardwell@reddit

oh yes! Seen that in action!!

Reply

[-]

mello-t@reddit

Its “the network”

Reply

[-]

Ubermidget2@reddit

Layer 4 Firewall is super binary - The rules are set correctly and everything works, or you can't telnet to the port. If you have NGFW doing location, IPS, AV Scanning, AI Threat Detection and Web Filtering, then I imagine thing get a lot more gray

Reply

[-]

jeremydallen@reddit

https://preview.redd.it/02hl2cs6ypbb1.png?width=1151&format=pjpg&auto=webp&s=776ea53640c721b7f4a76a0d89f5dc47b0fe2eb3

Reply

[-]

sysad_dude@reddit

yuuuuuuuuuuuuuuuup

Reply

[-]

vishybear@reddit

I've had some "intersting" conversations with firewall guys in the past. I swear a LOT of them somehow blagged the job and are terrified of actually touching it Even when the issue is blatently firewall...like 2 dns servers not being able to replicate with each other across sites ; it's still "reboot your servers" Even when I want access to the logs to prove that it IS the firewall blocking traffic. Nah you can't have that. And every time is HAS been the firewall. These days I tend to have access to them so I can at least go through logs. Bloody annoying though

Reply

[-]

Acrobatic_Wedding340@reddit

If it's not DNS it's the firewall. But it's always DNS.

Reply

[-]

ComprehensiveRest386@reddit

As someone who has to play firewall admin as one of my roles I often hear "It's the firewall" and then I have to spend a lot of time explaining to people with very impressive sounding job descriptions why it is not the firewall, so I don't relate but it definitely makes me feel something :D It's always DNS actually proves to be true in my experience, like a lot.

Reply

[-]

indygoof@reddit

usually its always the f5 https://isitthef5.com

Reply

[-]

PrudentPush8309@reddit

"Network"... It's network. It's always network. Network includes firewalls, as well as routers, switches, hubs, bridges, cabling, WANs, MANs, LANs, the works. Now, even if it's not network, it's a ride network. Let's say yes are having a Windows Kerberos authentication problem, or a DNS problem, or whatever it is... All those services run over the network. Therefore, I blame network. If they manage to prove that it's not network then, I blame antivirus. - an old Windows engineer 😜

Reply

[-]

Fred_Stone6@reddit

It was DNS to day, well sort of, some one forgot to add them on their static Ip address

Reply

[-]

sirimnotadoctor@reddit

It is sometimes the firewall, but I'll tell you what, when you're the guy responsible for it, and anyone in your department knows you've logged in to it in the last two weeks, then they'll certainly think it's always the firewall

Reply

[-]

Interesting-Yellow-4@reddit

In my 15 years doing this it's never DNS, always firewall. So I can relate. Btw, not talking about client configurations (windows firewall oe whatever).

Reply

[-]

mitspieler99@reddit

Since NG firewalls with layer 7 app detection this is kind of true.

Reply

[-]

_RexDart@reddit

Don't all those excuses roughly translate to "sysadmin" in the end?

Reply

[-]

mkosmo@reddit

Sysadmins who run firewalls often don’t block the things they should…

Reply

[-]

LaxVolt@reddit

Shhhhh!

Reply

[-]

DankNanky@reddit

Firewall can’t resolve URL to categorise content. It’s DNS.

Reply

[-]

mkosmo@reddit

That’s not true anymore. NG firewalls (and the rest of the cyber tooling you can stack) can do all that and more.

Reply

[-]

skylinesora@reddit

DNS doesn't categorize content either... DNS just resolves. You can perform content filtering based off of URL categorization, but that's not DNS doing it, that's another function.

Reply

[-]

maxlan@reddit

It depends where you are and how good the teams are. Where I was, it was always routing. And we ran the routing (and switching and firewalls and DNS). But that was during commissioning and at the routing level DNS is not involved. Once we'd figured out the routing, it'd generally not go wrong again. But building all the other config for a change would "just work". Routing would always see glitches with out/back taking different paths or routes going the wrong way for complex reasons. These days, people shouldn't be just changing things and breaking stuff. So do your firewalls keep failing or are people making uncontrolled changes? If they keep failing : replace them or get some HA. Or both. If it's uncontrolled changes, then put rules in place and fire someone when they break them.

Reply

[-]

_RexDart@reddit

It's always sysadmin :o

Reply

[-]

jamesaepp@reddit

It's definitely more often firewall at my employer than DNS. Which is a good thing.

Reply to Post

45 Comments