Background: I work tier 1 help desk for a big restaurant chain, and I've been on the job for roughly six months, half of that on my own/out of training. Anyways, we're rolling out a big project where we're upgrading all the tech in our restaurants with new network switches, servers, battery backups, it's a project and a half. We on tier 1 help desk are only involved in that whenever a technician calls, we run through a basic checklist to make sure everything is working and send the tech on their way. This is all a process of around five days for the record.

So, it's 12pm, I had just logged on for my shift and I get one of these tech calls for a store that had just finished their day and setting up everything, I want to say it was day 3? So I run through my checklist and noticed I couldn't remote into the main server to check out all the restaurants terminals, uh, what?

I tell the tech I can't remote into the main server and he proceeds to wire the network switch directly into the server.

Still nothing...

Since our servers use Dell hardware/software, I pull up the iDrac and was able to remote into the server that way and sure enough, the server didn't have any network connection despite the network switch having link lights and the GB1 port on the server also having link lights....

So I pull log into the administrator account for the server (idk about y'all but the idrac for us is slow af sometimes) and proceed to look into all the network information for the store to make sure all the IPs, network settings, etc matched up with another restaurant, and sure enough, it all did. I proceeded to run through some other basic troubleshooting steps like changing network ports, cables, restarting the server, all came back with the same result.

Now out of options, I reach out to my team to see if anyone else can help me out since I don't even know what the heck is going on at this point. The team lead then hops into the server with me and starts poking around and becomes just as stumped as I am, so, it's time to bring out the big bois! And that being, Tier 2

Tier 2 remotes into the computer and is able to mess with some settings, does some PowerShell commands, stuff a new tier 1 scrub like me would have no way of understanding what tf is going on, and so far, nothing seems to be working still even after rebuilding the port from the ground up.

At this point, Tier 2 loops in our network operations team and tells us to pull the mac address for the server, which we do and we all come to an astute realization, it was pulling a mac address from the incorrect VLAN. Netops then proceeds to check the VLANs for the new switch and realize it was configured incorrectly and that's why the servers or none of the point of sales terminals were working.

So after a reconfiguration of the VLANs, setting some of the IPs on the server back to normal, the server reconnects to the internet, the terminals come back up like normal, and everything goes back to how it should just the day before.

This took three hours, we finished this TWO minutes before the store opened, and I'm pretty sure I lost at least five years of my life. My boss sent me a little congrats message for getting through this call and hanging in there, and I will say, I sure as hell learned a lot from it lol