Using router commands without knowing their purpose...
Posted by Inside-Finish-2128@reddit | talesfromtechsupport | View on Reddit | 27 comments
First, the situation: I was working for a Cisco reseller doing professional services in the early 2010s. Got sent to one of the local school districts to help them rearrange their Internet routers and deploy a "Firewall Services Module" in a Catalyst 6509. (IYKYK) My main contact from the school is stingy and won't share credentials with me, so I'm stuck walking around the datacenter and "sliding the keyboard over" many times.
We get the new "router triangle" running and to me, things are looking good, but alas I'm just doing the router CLI thing. Meanwhile, my contact has RDPed into his desktop (back at his office) and says to me "I have zero Internet at my desk". Hmmm...Internet routing is fine on the triangle, lemme go look further down the path and check the router at his office building. Check the default route (it's learned from OSPF) and it's coming from some random router somewhere else amongst their WAN. What?
I get logged into that router, and sure enough under the OSPF process there it is: "default-information originate always". What? Why? (For those unaware, that's basically telling that router to tell all of its OSPF friends that it has a default route, or a route out to the Internet, all the time. Hint: it doesn't.) I remove that command and my contact was jumping for joy, saying "WHAT DID YOU DO? THIS IS THE FASTEST WE'VE EVER SEEN IT!" Um, OK, I removed a misconfiguration from the router at site XYZ. He was in disbelief..."No, really, what did you do? How was that command bad?" So I explain it to him. As I'm finishing my explanation, a few of his coworkers who were on site that evening for some other work are tracking him down..."what did you do to the Internet, it's SMOKING fast."
That's when the "cover story" was born. He decided this needed to be kept on the down-low, so "we made a few adjustments to the dynamic routing and after about thirty minutes, those optimizations all came together".
Next morning, I head to the IT offices first thing for "day one support", just in case we missed anything the night before. No problems, just everyone jumping for joy with how fast things were working. I said "so, could that command be lingering anywhere else?" He helped me log into a few other sites and sure enough, it was EVERYWHERE. He finally gave me the creds, and I spent the next hour or two logging into every single router (one per school, probably 60-70 schools in that district and growing by 5 a year) to remove that command. Any time anyone came in to ask why the Internet was so fast, he jumped in front of me and rattled off the same line. He also did the right thing and told his team what we fixed, and said "be sure you take that out of any templates you have".
You guessed it...later that summer, the whole district went off the Internet because one of his crew brought a new school online and yep, forgot to take that command out of his template. Ah, the joys of using commands without knowing why they matter. (And I'll be honest, I'm surprised the whole network wasn't melting down every day or every week prior to fixing it.)
Chocolate_Bourbon@reddit
I cannot count the times I've painted by numbers. Staring at coffee stained pages in a 3 ring binder, just following the recipe. I still do that today, but now it's a virtual document. And I now routinely create these instructions for others too. Funny how time is.
At least we've gotten past the point where people will call you and ask about any key. "It says now I need to press any key. I've looked on the keyboard and there isn't a key called 'any.' What should I do?"
Wise_Use1012@reddit
Which key is the any key?
Langager90@reddit
Depends on the date.
1st of the month is always "A", 4th will be "D", 12th is "L", etc.
"But what about the 27th on?!?" I hear your cry - just use numbers for these. "1" for the 27th, "3" for the 29th, up to "5" if you get that far.
It's a very intuitive, down-to-earth system. For exactly 3 people on the planet. The rest of us just press any button on the keyboard.
infostud@reddit
For use earlier in this month there are stickers you can put on keys.
faithfulheresy@reddit
It do be like that.
Thanks for the great story.
JaschaE@reddit
I know the cliche of doctors and lawyers being endbosses in tech support, is there a similar one about those responsible for school networks actively sabotaging things (actively, not necessarily)
Know a guy who worked under the citys school network. The 30 or so schools had 10Gbit to each other, and a singilar 1Gbit connection to the outside world. Not to mention the terrible cyberterrorism that made headlines here (in the hastily implemented days of online homeschooling during the lockdown, the shiny new system was victim to a DDOS attack. Who would have thought that on the first monday morning of homeschooling the singular server they had comissioned would have to handle 30schools worth of login requests? Those evil cyberterrorists! Yes, there was a press release ro that end)
DrHugh@reddit
Next up...
ATZ
ATDT...
ThunderDwn@reddit
ATZ
AT&C1&D2&R1&S1&M5
ATDT.....
BBS dialups for the win!
DrHugh@reddit
I even ran a BBS on my Atari computer back in the 1980s. It was a wild time.
ThunderDwn@reddit
I ran one on a C64 and then C128 for a while before I moved to an IBM clone. Fun times for sure!
PanoptesIquest@reddit
"AT&T is a modem test command."
ThunderDwn@reddit
AT&T caused many modems to be tested with their shit lines, that's for sure...
Dom_Shady@reddit
For those OOTL, what do these router commands do?
TinyNiceWolf@reddit
The reason the commands all start with AT is due to baud rates. The modem operated over a serial line, and needed to know the baud rate the computer at the other end of the line had been set to use, to make sense of the incoming data and turn it into characters (and to know how to send characters back to the computer).
Rather than have a dial or switch for the user to select the desired baud rate, Hayes decided the modem should auto-detect the baud rate.
But to do that, they needed to have a standard sequence. The modem was programmed to know what bit pattern and timing it would see if the computer sent "AT" at 300 baud or at 1200 or 2400, etc. So it watched the incoming line's raw bits for a pattern it recognized, then switched to the corresponding baud rate.
CrudBert@reddit
Those are modem commands from waaaaaayyyy back. ATZ ( reset), ATDT ( call using dial tone - instead of clicks like a rotary phone). This was called the “AT” command set ( AT meant ATTENTION and was the start of many/most commands). This was originally known as the “Hayes Command Set” as it first came out with the HAYES 1200 baud modem ( I had one, VERY expensive). It was copyright, and other manufacturers that started using it were sued. After some years of ramble ( in which other companies continued to use it anyway) it was settled - presumably $$$ money spread to Hayes to settle the matter. Later on, with modems at 28K and 33K speed ( blinding speed in my opinion) they had the “Extended AT Command Set. This was to handle new features on the newer modems. All in all, they were very, very nice to use!
TinyNiceWolf@reddit
Hayes actually introduced the Hayes AT command set with a 300 baud modem, the "Smartmodem", in 1981. It was $299. The following year they introduced a 1200 baud version for $699.
Hayes was sued by another company which had patented the use of an escape sequence in a modem to switch modes, and in 1983 paid $2 million to license that patent.
Hayes's own patent, for using an escape sequence that included guard time (as the +++ sequence would only be recognized if no other characters were sent for 1 second before and after it), was granted in 1985, and they began trying to license it to other manufacturers using the AT command set for 2% of the retail price of each modem.
ThunderDwn@reddit
They're not router commands - they're commands from the Hayes dialup modem command set. "AT" is "Attention"
ATZ - reset to saved configuration
ATDT - dial the following number using DTMF (tone) dialing. ATDP dials using pulse dialing.
guitpick@reddit
We were too cheap for touch tone growing up, so I'm familiar with ATDP. I think I might have had a modem that would do pulse at double speed, but I may be conflating that with a phone I had that could do that (20pps instead of 10).
ThunderDwn@reddit
There was no such thing as tone dialing when I first started using a modem.
Hell, the first one I used was an acoustic coupler where you had to dial the number manually on the phone, then plug the handset into it when the other end answered.
300 baud. I could type faster than it could send! 🤣
guitpick@reddit
You're ahead of me by a bit. I got a 300 modem cartridge for my CoCo when everything was on clearance and used it more out of curiosity than need (this was around '95). I also picked up some weird Tandy modem at a yard sale that I didn't have the manual for (and it wasn't Hayes compatible) and never quite got it working 100%. I also paid 75 cents for a Kyocera acoustic coupler from a surplus electronics store, but never used it for anything. It looked like a nice one, though.
ThunderDwn@reddit
By '95 I was running a multi line BBS. I'd been playing with modems since something like '79 or '80 - my dad brought home a Silent-700 terminal from work, and I went through three rolls of very expensive thermal paper playing with BBS's in the first weekend before he banned me from using it and I went out and sorted the gear to plug a modem into my Vic-20.
Those were the days!!
DrHugh@reddit
Oooh, as I recall, AT was a generic attention, and the Z was a modem reset for an acoustic modem. ATDT meant you were going to dial a number which would follow. There was also ATH for hangup. Let's see...
Here you go, the Hayes modem commands.
jimmy66wins@reddit
AT&B2 for the win
Fixes_Computers@reddit
It's been so long since I've had to construct a modem initialization string.
Just this week, someone posted a comment with three pluses in a row. I responded that it took my modem offline. I got a single knowing response which was all I needed
SlimJimPoisson@reddit
I was going to say the same thing!
lucky_ducker@reddit
When I was an I.T. manager before retirement, the one guy on my team that knew Cisco CLI was VERY well treated. I certainly didn't know it, and was not inclined to learn it.
Inside-Finish-2128@reddit (OP)
I hear you. In this context though, my personal opinion is that two separate mistakes were made here:
1) the 'default-information' originate command didn't belong everywhere - it only belonged on any router(s) that had direct Internet connections. "I have a way out, and everyone should know it."
2) even if it did belong on a particular router, the 'always' option shouldn't be there. If the underlying default route breaks, it's broken, so why would you share a broken route with the rest of the network? Let all of the other routers know that the route is gone so if anyone hops on any router to troubleshoot, they'll see "OMG the default route is gone. I should go look at the spot where that's supposed to come from!".