Saved the day
Posted by anderson01832@reddit | sysadmin | View on Reddit | 112 comments
10+ sites reporting internet outages.
Boss calls when I'm on lunch asking to drive to one of the locations
Boss blasting angry emails to our ISP
I get to the site (5 minutes from home), as soon as I got there my phone trying to connect Wi-Fi got me to a splash screen saying internet is misconfigured, we have Meraki, so I logged on and bang! Network disabled for non-payment
Feels good
thefpspower@reddit
Meraki disables your network for non-payment? That sounds dystopian
sweetroll_burglar@reddit
I understand the hate for cisco and meraki specifically but man, wifi administration has been a breeze ever since we replaced our old EoL aruba setup with meraki. I almost never have to look at it. And when I do, it's usually a radius issue. đ¤ˇââď¸ (small org)
pdp10@reddit
What was difficult about it when you had Aruba? Were you in there every day tuning your channel widths and 802.11u parameters?
sweetroll_burglar@reddit
ugh. I didn't need flashbacks, thanks! lol. Or fine-tuning our ssids because Apple specifically never played nice with roaming from ap to ap. At least we've since moved away from apple ipods for ordering/warehouse scanning.
The_Original_Miser@reddit
This is why I'd never purchase Meraki.
Disable non critical features? Sure!
Brick the whole thing? No.
DobermanCavalry@reddit
This is why I pay for things I buy.
The_Original_Miser@reddit
This is why I, for the most part, despise "forever rent". Everything does not need to be a service.
DobermanCavalry@reddit
Most enterprises orgs are paying every year for licenses and advanced features on their firewalls because places like Cisco, Palo Alto, Fortigate, etc lock their best features behind renewables. Not much different with Meraki. You are effectively forever renting regardless.
If you are a home gamer or small business, i agree maybe the yearly cost is not appropriate.
DrDew00@reddit
I'm okay with locking firmware updates and certain features behind a support contract like FortiNet does but to make the device itself non-functional when the contract expires is just stupid. I should be able to block my device from ever reaching their servers if I want.
The_Original_Miser@reddit
I'm okay with renting advanced features that doesn't brick your entire enterprise if you don't pay the bill.
I work in the mon profit sector and renting certain things is out of the question. Sometimes what bills to pay first is a reality (not currently, but you never know) and I certainly don't want to decide between Internet and the inside phones working, for example.
Gloomy_Stage@reddit
I had an organisation (education) that ran the legacy version of access points licensing (free). They decided to pull this and wanted to start charging and gave 2 months notice.
We already had plans to replace the WiFi with Aruba so brought this forwards. While Aruba setup and licensing was still cheaper than the Meraki license.
And Aruba will still work and can be configured locally even without Aruba Central licensing so thatâs one step better.
Zomgsolame@reddit
You have 30 days after the end of your license before it turns off.
TommyVe@reddit
How can a business "forget" to pay for something this crucial. It's to be automatic or with a bazillion of reminders at the very least.
ConcealingFate@reddit
My company's first troubleshooting step for internet issues was emailing accounting to check if the bill was paid.
kg7qin@reddit
Unfortunately, this is a lot more common than people realize.
It is even more fun when accounting just pays everything by check and sends via regular mail.
TommyVe@reddit
Kekw
How many people was employed there?
ConcealingFate@reddit
Around 2-2.5k accross the US.
RunningAtTheMouth@reddit
Used to work for a company where the purchasing manager controlled AP. Would refuse to pay some bills because "Eff you". One site was regularly offline. They always called me. I just called accounting and got them to pay by credit card to get the site online again.
I cannot express how happy I am that I'm not there anymore.
StudioDroid@reddit
The billing goes to the head of IT's company card and her email is the contact address. She gets a new gig and departs the company. The team bungles the handover and her card and email are turned off.
This is why we use tactical email addresses for subscriptions.
TEverettReynolds@reddit
Accounts and payment methods can change even at good companies.
Tymanthius@reddit
Yep. Meraki is Firewall-as-a-Service.
You don't have a license? it's just a brick.
kg7qin@reddit
You can also refer to this type as Hostage As A Service (HaaS), since you are held hostage and locked out if you don't pay.
Joe Pesci's scene from Goodfellas comes to mind here:
Fuck You. Pay Me.
The most extreme being: Have a SIP phone system and someone needs to call 911 for an emergency?
Stuck in an elevator, and the building owners/IT decided or had to switch from copper. Instead of going cellular they decided using an ATA insteqs so they could "leverage" their exisitng VoIP phone system and the cheap SIP trunk to dial emergency services when the emergency button is pressed?
Fuck You. Pay Me.
How many places that use HaaS have though of this?
ManagerActive3188@reddit
This guy knows Accounting depts!
CatoDomine@reddit
Ray Liotta*
TechieSpaceRobot@reddit
ArchdukeTrout@reddit
I figured this out when they first started selling them and you could "get one free" just by going to their sales meeting. Free meant the brick, not the service. Never , ever use meraki unless it is for a company you hate.
vmxnet4@reddit
Fortinet used to do similar for their business partners. They totally scrapped the program last year. Before then, you just had to get a couple self-paced online-based free certs ... foundation level ones, and then send them the Credly link to the badges. Then, they'd send you some form to fill out and send back to them. Couple weeks later, you'd get not-quite-a-brick low end FortiGate in the mail (unless you picked the virtual appliance option in the form.) It still worked, but due to not having a license for any of it, it would just barely work for basic stuff.
Tymanthius@reddit
I think that the free one, as long as it's just personal and never more than 1, you can use. At least that's what I've heard.
equityconnectwitme@reddit
I'll never buy Meraki for that reason.
Pickle-this1@reddit
We used to do this at an MSP I worked at. For some reason the AMs let the customers not pay or chase payments, so the business setup essentially a debt collection team.
The amount of times I got billing calling me (was a TL) asking if we disable X meraki service will it stop their internet.
It was a two person problem at my place A: customer for not paying, B: AM for not making sure they pay.
RayG75@reddit
Yes, itâs horrible. I am proud that I was able to divert about 25 large multi-location companies from getting âHe Who Must Not Be Namedâ and avoid this and a huge list of other issues.
Memlapse1@reddit
We had a provider who manages our server, backups and end point protection. They set up an appointment to come in and replace our older Cisco firewall with a Meraki (we paid about 6K I think). While they were configuring the system they mentioned they set us up with a three year subscription. Curious I looked up the Meraki site and found out that at renewal time if we chose not to continue that the firewall would stop all outside connections (for our safety).
We changed providers this year and pulled the Meraki ones out. Not going that route again.
creamersrealm@reddit
My favorite was when Meraki did this to us and we had proof we paid and they refused to acknowledge it and said legal said to turn it off. That didn't go quite so well for them in the end.
JMejia5429@reddit
Yup. My company wanted to go all Meraki. Did a demo site and although we were paid up, it didnât sit well with me. Pulled the plug 9 months into it and put back the 9300 switches.
llDemonll@reddit
You have 30 days and a lot of notices that go out. Itâs not a surprise unless you go out of your way to disable all those things.
GhostOfRandomUsrName@reddit
You have to pay yearly for the license.
thefpspower@reddit
I know that but the devices I deal with usually don't outright disable your network if you forget to pay.
For example just today we had a client's Arista firewall expire because whoever received the quote saw the email but forgot. The client was still working, the basic firewall still functions, IPSEC still up but lost some features and packet filtering.
That is how these things should be handled, not just turn it off and ask question later, just sounds more like blackmail to me.
ITrCool@reddit
I mean, this is CISCO weâre talking about. Theyâre ruthless in everything, including their predatory pricing. I can see them just disabling a customerâs network for non-payment.
Ron-Swanson-Mustache@reddit
As the saying goes; Cisco's console cables are blue from the tears of CFOs.
OrangeDartballoon@reddit
Brilliant ,đ, haven't heard that one before.
ITrCool@reddit
So true
psiphre@reddit
they're ruthless in everything, not just their predatory pricing. lol.
demglassesshitinnit@reddit
Meraki definitely gives you a 90 day grace period too. Even if your license is expired, you're good for 90 days. Not that I would rely on that fact.
ShadowSlayer1441@reddit
Still sending angry emails and threatening suit/aggressive negotiations makes way more sense. What if your licensing server shits the bed and suddenly you're on the hook for lawsuits surrounding lost business.
Rabid_Gopher@reddit
It's a good thing Cisco puts a lot of effort into making sure their licensing works flawlessly.
laughing hysterically
farva_06@reddit
Most devices just lock you out of making changes to it, but still continue to function. At least for some sort of grace period.
Ron-Swanson-Mustache@reddit
Palo Alto's the same way. Didn't pay? No more updates for you! But we're not going to bork your systems.
ManWithoutUsername@reddit
that normal, but stop working is a bad "joke".
Anyway the problem is buying that shity brands
mdug@reddit
I worked at a startup that was running mostly on VC money and got various reasons was circling the drain so was pretty low on cash. So the Meraki renewal was one the leadership decided not to pay. CEO called me and asked why the WiFi wasn't working in the office. I reminded him that they had decided that they didn't pay to keep it working. This was also in 2021 and the office had been more or less abandoned but he wanted to use the printer.
On the one hand, yeah, we didn't pay to keep it working, on the other, I'll never work with a vendor again that will effectively brick equipment because a license wasn't paid. I'll pay for support, software updates etc, quite happily (ok, maybe grudgingly), but just disabling the gear is shitty.
mike9874@reddit
So the firewall keeps allowing traffic but with less protection? Perfect!
thefpspower@reddit
Less user protection but not less outside intrusion protection which is what matters because for user protection you should still have your AV layer.
jake04-20@reddit
I get that, but most software doesn't runtime restrict you for compliance issues, or at the very least they give you a chance to true up. I have a network friend that swears by meraki and is trying to get my org to switch off cisco, but this is a negative for me.
BatemansChainsaw@reddit
It sure does. It's also why I refuse to deal with subscriptions for critical infrastructure. I know some here would rag on Ubiquiti but I've run sites with 1000+ individuals and at least five times that number in connected devices and services without a hitch.
I'd willingly run a network with 10x that many people and endpoints with Ubiquity gear.
nope_nic_tesla@reddit
Not all subscriptions work this way though. With RHEL subscriptions for example if you stop paying then your servers don't stop working, you just won't be able to pull updates from Red Hat anymore (and you can still install them from upstream sources if you really want)
Disturbed_Bard@reddit
That's how most other vendors run, Sophos, Fortigate etc.
Which is great
Hell the Sophos gear once it's expired, you could flash with Pfsense, so your hardware isn't a brick it's still usable.
Meraki you can't do shit with their hardware without a licence
Zero-Cool-1507@reddit
Hell yeah they do. They also disable your network if you register a single device over your license count.
fire_over_the_ridge@reddit
I canât believe the EPA doesnât fine them for manufacturing e-waste.
SystemGardener@reddit
Ciscos done this shit for years.
pdp10@reddit
Could be worse. Like the Meraki users inside Russia, who were disallowed by sanctions to pay.
jaydizzleforshizzle@reddit
Welcome to Cisco, this alone makes me refuse to work with them, I can understand some features but Cisco bricks your shit.
JesterOne@reddit
Yeah, I think if there isn't a "service contract" in place, everything stops working.
MIS_Gurus@reddit
That is interesting, i know they used to do that but I believe they got in trouble for this practice. I know I've had network recently where licensing was not renewed and they did not go down. Sounds like some more testing is in order.
The_art_of_Xen@reddit
We used to have a critical application that required licenses renewed every 3 months. The vendor offered a longer model but for some reason accounting declined. No clue why but the business refused to budge.
Every 3 months the calls would come in, staff would lose their shit, people would blame the vendor, vendor would kindly advise the bill wasnât paid (even after they started giving 14 day extensions to avoid this nothing changed), accounting would apologise and say âdonât worry, weâll definitely make sure this doesnât happen again!â
Rinse and repeat.
dinominant@reddit
Don't deploy network equipment that requires 3rd party remote services to continue working.
Broadcom raised prices 10x for vmware. If Cisco does the same, you have a serious problem.
What if there is some other problem that prevents those devices from remotely confirming your subscription? You still have a major problem.
sorderon@reddit
hence why meraki has been taken over by someone else - damn good hardware, useless business model.
green_hawk1@reddit
This reminds me of a call we got from a local veterinarian clinic. Not one of our customers but they were desperately calling around for local IT help.
Turns out it was the same issue. They had a Meraki firewall and it was shut off for non payment.
They wanted to sign on as a customer so we replaced the Meraki and then business as usual...until a year later when we fired them for non payment....
BigMikeInAustin@reddit
Way back in the day, company phone lines went down. I was just a lowly worker. Had to use a payphone to call the phone company.
I barely made it past my first sentence telling the phone company how important the phone lines were for the business, and I needed a priority repair.
They said it was failure to pay the bill.
Used a dime to call the big boss, who wasn't in the office.
And I enjoyed a few slow hours until the phone bill got paid.
Fr0gm4n@reddit
Had a CFO that would pull crap like that. He even got us pushed out of a facility when the property owners decided to not renew the lease due to him usually being late paying until they hounded him for it. I was glad he didn't come over when the company was bought.
BigMikeInAustin@reddit
Yikes! That's really bad!
Big-Penalty-6897@reddit
As soon as I saw "Meraki" I figured that was the problem. One of my associates has been having Meraki hardware make his work life a living hell. I'd have put those damn things under a hammer and setup PFSense boxes.
Unable-Entrance3110@reddit
Same. It's not like their aren't options in the same market segment. I have learned today that we will never be entertaining these guys as a vendor.
I understand licensing security but to make basic functionality fail? That's a hard no. Mistakes happen and sometimes invoices don't get paid for one reason or another despite the best of intentions.
Ace417@reddit
They send multiple emails and there is a red banner at the top of your dashboard. Itâs not hard to stay on top of it. They can also extend that timeframe if you talk to your account manager. I work in government where things donât happen fast and weâve never had an issue here.
DobermanCavalry@reddit
Most people here in r/sysadmin just like to bitch about things.
The same people who wouldnt dare run a piece of Palo Alto, or insert brand X here without advanced licenses activated and renewed every year, will scream absolute bloody murder that they have to renew a license or their equipment stops working.
Ace417@reddit
I just donât understand it though. If you use common sense and some forward thinking, this doesnât happen.
pdp10@reddit
Put them in a box for the OpenWrt crowd.
QuietThunder2014@reddit
Boss was sending messages to ISP and sent you onsite when 10+ sites were down and didnât bother to open the Meeaki management portal to diagnose? Dude.
My biggest issue with Meraki is theyâll put 50 devices on individual non co-termed contracts then if one goes out of service they shut down all of them. Not just the one that fell out of renewal. Itâs bullshit and itâs why I moved all our shit off Meraki. They do give you a 30 day grace period though so clearly someone should start checking this critical infrastructure more regularly.
DobermanCavalry@reddit
So convert them to a co-term expiration?
QuietThunder2014@reddit
Iâve tried and Meraki support always makes it such a pain in the ass and usually you end up losing time as a result.
DobermanCavalry@reddit
Yeah, pay your bills.
Whats amazing is someone got through their licensing period AND grace period without realizing their license was never renewed. Thats negligence. It shouldnt be a surprise when a license is renewed.
Dont blame poor management and planning on Meraki. Its part of the package, its not like Meraki springs this on anyone, its well known.
CaptainZippi@reddit
Iâm with the supplier here - which might put me on the unpopular side, but Iâm ok with that.
Business decides to buy a service - for any number of reasons, but Iâm imagining moving capex to opex for at least one.
This requires a commitment from the business to fund it. If not funded for whatever reason, you donât get the service. Theyâre not a charity, and youâre (probably) not a charity case.
DatManAaron1993@reddit
What is the "Timeout" or is it instant?
North-Plantain1401@reddit
A month, then you can request extensions from support if you have a reason to not renew it
We moved from an mx84 to dual mx105s last fall and they let us ride for 3 months while our ISP got their rack on line. The meraki sucks in a lot of ways for sure, but as long as you pay your bills it'll work ;)
fys4@reddit
Had something similar when we added a new device and tried to get Meraki to co-term it. They issues us something like three different enablement codes over multiple months before they stumbled on the magic combination that did what we'd ask them to do
It's networking for "The children of the magenta line"
DatManAaron1993@reddit
Thatâs more than enough time lol
lowNegativeEmotion@reddit
MeRAPEee
BankIOfnum@reddit
The techs at the ISP must've been rolling their eyes so hard at you guys lmao.
alec_at_home@reddit
The admin portal will have been screaming about this for months. Your mailbox will have been full of notifications if you'd set it up right. This isn't the huge win you're selling it as.
way__north@reddit
One of my collegues is a huge Meraki fan from his MSP days and use to tease me sometimes with switching to Meraki.
I say "never" I dont want a network that can be externally disabled with a flick of the switch, if say accounting messes up one payment
Watsonwes@reddit
This actually happened to me at my first it job but it took us longer to find out comcast disabled us for non payment
tryfor34@reddit
The sad part about this is Meraki emails the shit outta you when it's approaching. Someone ignored emails for the last week.
Bubba_Phet@reddit
That happens at my work from time to time. I enjoy it far more than I should.
bhillen8783@reddit
Oh man that happened to us once and the failover to our other ISP wasnât configured correctly. The network team had a scramble for a little while failing over the connections manually and re-engineering the config but it wasnât too bad.
tunaman808@reddit
I have a client site where the users RDP into virtual machines. One user has a tricky VM that sometimes hangs when booting. I need to make him a new one (and will soon), but the client has had several meetings with a new POS vendor.
Anyway, the user's VM hung rebooting after this week's Patch Tuesday. However I was able to get him working again... in my comfy king-size bed in my hotel room in the north Atlanta suburbs, after coming to town for a concert the night before.
fonetik@reddit
And that's how a "P1 - Emergency" becomes a "P4 - Awaiting purchasing department" as the manager quietly sweeps this under the rug.
Capt91@reddit
That's why you never go Meraki
Tymanthius@reddit
Who's the person that is responsible for keeping that contract up to date?
IroN-GirL@reddit
Yeah, I wish I could see his face when he was told. Hopefully the screaming boss was the one supposed to have paid it.
anderson01832@reddit (OP)
I think he was because if was fixed 2 minutes later lol
jcleme@reddit
In which case it had been paid for and someone had forgot to apply the license key.
dansedemorte@reddit
or was told that the purchase had gone through.
Terriblyboard@reddit
Ahh thats great... had a site go down TWICE because the AP department didnt pay the bill on time. Felt great when I told them both times.
pdp10@reddit
It also feels good to hand out those USB-to-Ethernet dongles that AP also complained about, to the AP users who need them to pay the bill in order to fix the WiFi outage.
North-Plantain1401@reddit
This whole thread should be on r/shittysysadmin
MrVantage@reddit
Wow thatâs ridiculous! I get disabling non critical features (I.e IDS/IPS) and removing management capabilities, but completely disabling everything is a joke. Glad we are riding the Ubiquiti train.
bgatesIT@reddit
thats happened here at our org before, on my first week here, and i was the only it guy in the office that day.... that was a fun couple phone calls to my boss
chefnee@reddit
They made it easy. The error message has the answer! And itâs Friday. Two Wins.
anderson01832@reddit (OP)
And Monday is a holiday
chefnee@reddit
Forgot about that. Three Day weekend!!
MeatPiston@reddit
Shit as a service claims another victim.
ADynes@reddit
Meraki is the worst networking brand I've ever experienced. I understand charging a license fee if you don't pay for the hardware but the fact you pay for the hardware and a license fee and if that license expires it breaks your device is completely ridiculous.
Years ago we were switching out some old Microsoft TMG software firewalls and after lots of convincing by Meraki salespeople we decided to go with them. Within the first 30 days I had six different support tickets open all for things that the sales people said would work that simply didn't. Some more features that were simply broken and others were features that didn't exist that I was told did but it turned out they were coming in a future release. I then requested a full refund, sent them back, bought Sophos XG firewalls, and I've never looked back.
Meraki's sales people still call and send me emails and every single time I reply with if my license expires does my device stop working? And when they say yes I tell them we will never do business with them.
And don't get them confused with Cisco which even though they are the "same" company is apples to oranges.
Prestigious_Wall529@reddit
Agreed. Over-engineered junk. Wouldn't take it for free.
Creative_Onion_1440@reddit
At least Extreme only disables your management dashboard.
Network will still work, as long as you don't need to change anything.