Been having frequent slow network speeds and outages
Posted by glazed_pottery@reddit | sysadmin | View on Reddit | 67 comments
Pretty new to my IT job, been having to deal with frequent internet outages, checked the router logs and router crashes every once in a while. Company’s main router is an asus rt-ax55 which from what I’ve read is pretty bad when we have 250 connected devices, ram is pretty much hovering around 87% at all times and spikes in cpu usage. Should we switch to a more robust router? Any suggestions would be appreciated, more budget friendly options are preferred since our dept budget is tight. Thank you!
Ikelley317@reddit
I can help get you set up with a way better infrastructure. Please shoot me a Dm! Budget friendliness is my goal always!
Capable_Noise5543@reddit
Yeah, 250 devices on something like an RT-AX55 is definitely pushing it that router is more for home/small setups, not that kind of load. The high RAM and CPU usage you’re seeing is basically the router hitting its limit, which explains the crashes and outages. At that scale, you’d want something more business-grade or even a setup with multiple access points or a mesh system. Wi-Fi 6/6E routers or dedicated business routers handle higher device density much better and are designed for stability under load.
Also, consider separating traffic (VLANs or guest networks) and possibly load balancing if your setup grows. If budget is tight, even upgrading to a stronger mid-range/business router or splitting the network can make a big difference. And if it gets too complex, some people bring in services like Geeks On Site to help redesign and stabilize networks like this.
glazed_pottery@reddit (OP)
I’ve tried suggesting setting up vlans previously but was rejected because depts wanted file sharing between them and having management try to buy managed switches when the current setup works kinda fine is a tall order. This is pretty much my first job but even I can see how messy the network is, a bunch of hubs and cat5 cables still being used, and with the IT dept being just me and my supervisor fixing the network is a nightmare while minimizing downtime. Thanks!
INSPECTOR99@reddit
FIRST, replace all the HUBs..........
doglar_666@reddit
Is there a reason your supervisor hasn't pushed to improve the quality of the equipment? Even if 100Mbps speeds are acceptable for client devices, a router that can't handle the total amount of client traffic is a core business issue. Any competent business owner will understand that X number of tickets raised about the same isue over Y number of months/years = Z dollars lost due to downtime, even if they aren't "Computer" people. If you can somehow get a monetary value for each outage, you can frame that value vs the cost of a better class or router or firewall. Even Users affected x Hourly wage x Length of outages is a start, without including lost sales due to delayed emails/outbound comms. Given your company seem super cheap, making it about money left on the table might be what gets through to them.
glazed_pottery@reddit (OP)
My supervisor wants to play it safe and not mess up the network further, main worry is buying new equipment and having to deal with integration pains which would make the outages worse in the short term.
Framing outages as lost revenue and efficiency is a great idea, might be what convinces management to finally invest in new equipment, might also include future scalability and potential audits with our network infrastructure. Thanks!
VA_Network_Nerd@reddit
You don't need VLANs in a small business environment until security becomes a topic of meaningful discussion.
protogenxl@reddit
Pull a desktop install a Intel dual nic card and run https://opnsense.org/
Moontoya@reddit
Windows 11 uses a shit load more connections/sessions than 10
250 devices is absolutely hammering that kit, looks to be mesh WiFi too.
You want something chonkier asap
Also, disclose what your internet connection is rated for, for example draytek 286x cap out around 400mbit throughput , so you'll never see the full rate if it's a half GB or full GB line, need to step up to the 29xx range for that.
glazed_pottery@reddit (OP)
The internet plan is for 1gbps, and the throughput on some devices is limited due to a cheap 100mbps unmanaged switch lol
Moontoya@reddit
even a cheap gb switch will provide a BIG boost to the network, like lol hyueeegely big
100mb is sorta kinda ok for Voip handsets, but damn all use for more than _maybe_ 2-3 users.
250 devices into 100mbit crippled network segments = holy saturation batman.
Yyou can build an opensource router out of a Raspberry pi or an old desktop running linux with a cheap additional NIC. (opensense), it`ll handle static assignments or ppoe etc connections as well as give you some fine granular control on the insite of your network, filtering, vpns and more.
minimal investment in %localcurrency%, (some investment in research and implementation time - heck get a reconditioned "enterprise" router off Ebay and teach yourself its setup (lotsa howtos on youtube and various forums).
glazed_pottery@reddit (OP)
Would rather management hire external consultants to fix the network rather than me making a router which while might be doable would be way above what I can handle since I’m really just fresh out of college with little actual experience lol
Moontoya@reddit
Homelab it, play with it, screw it up with no consequences, learn to take backups before you screw around, fail, fail better, it works but it breaks something else , it works !
Build experience for yourself in parallel!
Don't be afraid of failure, everyone here has screwed up many times along the way.
pdp10@reddit
Sockets are opened by applications. If there are more sockets opened, then
lsofornetstatwill show which executables are responsible.Moontoya@reddit
Hunt, teams / i365, copilot, AI helpers
gakule@reddit
This is the perfect time to step back and gather some quotes from MSP's/consultants. Frankly this is something that is really above what I would have an expectation for you to execute.
You've identified the issue, but designing the solution is better suited for someone that has experience in this arena. You'll be able to learn a lot, and you'll have someone else you can lean on to be accountable for executing the project.
If nothing else, getting some quotes and approaches will help you make a better decision on whether it's something you can handle on your own or not.
I'd also see if your ISP has any managed solutions they offer. In my experience they're hit or miss, but they might be competitive on pricing as well.
glazed_pottery@reddit (OP)
If only they would, management doesn’t really get tech and is fine with something that kinda works so management approving such a big expenditure is kinda hard. Been fiddling my way around my job since the IT dept is just me (recent hire and my first actual job) and my supervisor, having an experienced mentor would be great and would make my job a lot easier. Thanks a lot!!
gakule@reddit
I understand what you're saying, but I think what you're missing is making this into a business case.
Are these outages affecting productivity? Are they causing any business-related losses? Are they introducing unacceptable downtime?
While this is a technical solution, you need to present your case to management in a way that makes sense to them - how many dollars is this costing us to not address, how many dollars is it going to cost to address?
Every (especially young) technical professional really needs to learn to lean in on and speak the language of the other side of the aisle (maybe your supervisor can help?) which is Return On Investment (ROI).
If every outage costs $500 in productivity, for instance, how many times does it happen in a year? "Only" 20 instances of losing $500 is $10,000 a year.
What would it cost to have a solution put in place that has an actual network stack and access points to serve all 250 devices?
What is the expected lifetime of the solution?
Even if the solution only lasts 3 years (probably a lot longer, but just for example), spending $7,500 (example number, no clue on actual cost/specs) with my imaginary number of $10,000 a year will turn a profit in under a year.
Do your homework, build a business case, compare the total lifetime costs.
glazed_pottery@reddit (OP)
This seems like a good way to present investing in new equipment, potential loss of revenue and productivity might just convince management. Will need to research more on this to make a pretty solid proposal. Maintaining such an outdated and cobbled together network is a nightmare. Thanks!
gakule@reddit
There you go, you've got it. Happy to give you any advice if you need, feel free to DM me!
doglar_666@reddit
If I were in this particular situation, I'd be half tempted to take a lighter to the inards of the Asus and then claim I found it that way. You'd be shocked how quickly money can be found for a more robust replacement when the business is losing X amount of preferred fiat currency per day.
OP, if your budget really is so tight you cannot swing for an appropriately spec'd device, I wouldn't bother trying to replace the Asus. It's likely to make your life worse, as you will have the same problem, even less money to throw at it, and management questioning your judgement. In this instance, proper kit replacement is the only solution.
glazed_pottery@reddit (OP)
I figured after reading a bunch of helpful comments, the entire network is a mess of unmanaged 100mbps switches and old cat5 cables
doglar_666@reddit
This doesn't come as a surprise, given your company is running the Asus router. It sounds like you've been dealt a shitty hand, infrastructure-wise. If budget is tight, you'd be best served decoupling a lot of the functionality the Asus is responsible for and moving it onto other devices. Namely, Firewall, DHCP and DNS. It won't improve the "Internet" speed clients experience, but if you're able to upgrade to 1Gbps LAN and lower the router's overhead, you might gain enough stability to buy you time to put together a proposal for a better grade of hardware. Any temporary mitigation you put in place is just kicking the can down the road, should suitable hardware not be purchased.
glazed_pottery@reddit (OP)
Trying to maintain the network is a nightmare, would try and setting up the firewall, dhcp, and dns on old mini PCs while I cobble together a decent proposal and see where it goes from there. Thanks a lot!
Nonaveragemonkey@reddit
Soon an smb is using consumer grade toys and wondering why the internet sucks... I'd ask about the internet provider.. don't be surprised if it's a residential plan...
hihcadore@reddit
Why? So the ISP can charge them 600 bucks a month for 500 MB vs 70 dollars a month residential for 1 GB?
Nonaveragemonkey@reddit
Because there usually a consistency requirement in a business contract with penalties that can make some serious difference in how fast they move to fix issues.
Like you ever had spectrum or at&t have techs show up at 2am because business dependent networks are down as a consumer? Nah, they shown up 2 weeks later and take 3 trips to decide that the pretty roots showing on the tree the city just ripped up or the drunk hit are important lol
hihcadore@reddit
OP’s home router is choking to death on 200 users. I’m sure they’re not running anything that critical.
I’d go residential with LTE backup backup for them and spend the 6k it saves on a nice firewall / office chair / mini fridge / and in house arcade
glazed_pottery@reddit (OP)
It isn’t haha, internet is mostly used for emails and zoom meetings as well as some downloading but having a bunch of annoyed calls everytime the network is down is such a hassle
hihcadore@reddit
I bet! WiFi will get you during calls too if you’re WAPs aren’t handing the traffic off correctly.
Might be worth it to upgrade your router and a few WAPs depending on the size of your office.
glazed_pottery@reddit (OP)
It’s not like I’m the one who planned the whole network out, I’m just a recent hire with little experience in network management trying to make my job a bit easier lol
mods_are_lame1@reddit
For fuckssake, buy a proper firewall and some access points.
glazed_pottery@reddit (OP)
Not like I don’t want to fix the shit network architecture but I’m just a new hire with little experience lol
pdp10@reddit
How many devices connected over WiFi, how many over wired Ethernet? Do you have no other Ethernet switches or WiFi APs?
Is this academia? Is the department in question, the IT/ICT department? Normally, the whole organization or site pays for the whole organization or site's connectivity.
glazed_pottery@reddit (OP)
Around 100 wifi devices, the rest being wired. 4 APs and like 5 unmanaged switches.
It’s for the whole company but management is fine with something that kinda works so funding requests for something better is difficult.
poizone68@reddit
When selecting a new router, you'll want to check not just raw throughput speed, but also what throughput you will have with VPNs and IPS enabled. This can have a dramatic effect on perceived speeds and stability.
glazed_pottery@reddit (OP)
The network is super simple and doesn’t have vpns setup, management would rather settle with something that ‘works’ than forking out money for new equipment.
VA_Network_Nerd@reddit
Yes.
It's difficult to answer that without a better understanding of your environment.
This should be a good place to start:
1 x UCG-Fiber (30W)
1 x USW-Flex-2.5G-8-PoE (196W)
1 x UACC-Adapter-AC-210W
3 x U7-Pro
glazed_pottery@reddit (OP)
Will be looking into this. Thanks!!
Y-Master@reddit
Hi, This Asus is a very simple home router, clearly not made for a company of 200+ users
For a small company, you have 2 way : - if you want something simple but with more capacity, go with Ubiquiti. - if you want something more advanced but a bit more complex, go with Opnsense installed on a robust hardware.
For ubiquiti, I would advise a UCG-Fiber or UDM-Pro. For Opnsense, check the D740 on their store, it's a robust board pre-installed with Opnsense. You can also install Opnsense on any spare pc/server with 2 network card and check if you like it.
I forgot to ask : do you need wifi or do you already have some Access point?
glazed_pottery@reddit (OP)
Been seeing ubiquiti being reccomended quite a bit, will be bringing this up to my supervisor and proceed from there. Thanks!
Ferretau@reddit
That's a home router not really suitable for the number of devices you are running. You should replace with a commercial grade unit.
glazed_pottery@reddit (OP)
Are there any models you would recommend to look into? Thanks!
Ferretau@reddit
It depends on the exact interface requirements and features. If you're looking for a pure router or a firewall devices that also routes like a fortigate, juniper etc. Tp-link have the Omada's can't say if they are any good, Mikrotik is another vendor that may have something suitable. You need to work out the requirements and work from that.
glazed_pottery@reddit (OP)
Will look into this more, thanks a lot!!
SimpleSysadmin@reddit
That is 100% a good idea. If you have a decent amount of devices on wireless that’s probably also limiting you there
glazed_pottery@reddit (OP)
Lots of devices are connected via wifi, and the coverage range is not great, recently had to set up two APs just to have somewhat decent coverage. Are there any routers you would reccomend? Thank you!
Moontoya@reddit
Unifi WiFi Man tool is helpful in diagnosis, freebie android app (probably apple too,)
Can build heat maps, show you channels in use, speed test, show roaming values etc.
We use a mix of drayteks and unifi kit across 200+ clients , our biggest install has a 3920, driving usw Poe switches and 17 access points, serving 300 staff in a manufacturing facility that's about the size of 4 football fields.
Had to step them up to 39xx range from the 2865ax they had as , frankly the router couldn't keep up on a GB line with that many devices.
I like drayteks but I'm a weirdo that way :)
glazed_pottery@reddit (OP)
Thanks a lot! Will try and using this to find out a more optimal setup for ap bands, this seems like a good start in trying to improve the situation.
SimpleSysadmin@reddit
Bang for your buck would be unifi, you just need to make sure you get a router that includes the Network controller function, otherwise you have to run it on a server and it’s a bit more complex for your setup. You can then also consider upgrading to their access points to get much better performance and coverage when ready
UptimeNull@reddit
Over lapping radio frequencies is not great. You could just set up roaming and see where the gaps lie.
PhroznGaming@reddit
Yes absolutely 100% percent
hihcadore@reddit
And at times it seems 40% or less
SevaraB@reddit
Wireless brings back a couple of problems from the old Ethernet hub days before we switched over to… well… switches:
The wireless radio is a shared broadcast domain and a shared collusion domain. It doesn’t look like it in a packet capture, because the NIC software tells it to ignore packets that are addressed to another client and doesn’t have a way to understand active RF interference that isn’t WiFi, but there’s a lot of it, especially in the 2.4GHz range. Put another way, wired Ethernet is to WiFi as fiber optic Internet is to cable Internet- one is a dedicated circuit that will get you a consistent link speed, and the other will have you constantly wondering if your broadband is just too crowded for you to get much out of it.
Unless you’ve got (expensive) ultra-high density access points meant for large events, you really shouldn’t be connecting more than 50 devices to one AP. So, yes, you need better APs, and yes, they should be better quality than TP-link. Not knocking TPL, I use it at home, but I don’t have a network SLA for my cell phones and laptops.
pdp10@reddit
50 is pushing it pretty seriously, in the real world, on non-specialist hardware. Bearing in mind that enterprises usually benefit from more-numerous APs, tuned to the lowest practical power, then an optimum number of simultaneous clients is often closer to 10.
Universities and event spaces have traditionally been the pioneers in high-density, high-demand WiFi. They sometimes push the limits, but then they're also often using rather specialist equipment like special high-density Xirrus (now Cambium) APs when large open spaces don't allow for numerous smaller APs.
VividVigor@reddit
87% cpu on consumer chipset is basically cooked.
HP Aruba Instant On is my first and last recommendation for small business. Replace full stack at the same time. New secure gateway firewall, PoE gigabit switches and standardize on same Wi-Fi 6 access points. Probably run you $5 to $7K depending on wired and wireless device counts and office space. Plan for 25% more AP’s than you think you need so you have the horsepower to handle client density during big meetings and sales demos.
If you can swing $30 to 40K then get a quote from Fortinet or CDW for HP Juniper kit. You need to know what you are doing with this gear, but you will not be disappointed come audit time or cyber insurance renewals.
Avoid Cisco Meraki. It is garbage. AP’s are good but the Meraki ecosystem is children toys.
$85k can get you Cisco Catalyst with a $70K annual renewal due in three years.
I would not pick up a brand new Ubiquity UniFi junk if it was on the floor. Ubiquity is not even a good door stop.
Mikrotik is a closed-source hassle. It’s a fantastic router/firewall UTM built by Latvians or aliens. Maybe a joint partnership.
Netgate makes decent, open-source pfSense UTM appliances but good luck explaining your compliance to a soc2 auditor.
UptimeNull@reddit
Why would you route through that device op? Doing some of what this commenter is recommending in my home lab while getting rocked at work so slow baby steps I guess.
I have Aruba but why would you choose Aruba on ? Cloud based? Especially after mentioning compliance.
I’ve played with pfsense. They can audit out.
Please explain.
I guess I could have googled it but want to hear your Y ?
VividVigor@reddit
I think a fortune 100 company can pass compliance. No. I have not done the vendor risk or privacy assessment on HP or the Instant On platform. I’m willing to bet HP will pass.
Route through what device? OP needs more than a new $200 Best Buy gaming router if his network is running an Asus broadband router. I recommend to replace the entire network stack in his office with consistent, and purpose designed equipment. A real NGFW firewall like Cisco Firepower, Juniper SRX family, Palo Alto Networks NGFW series or Fortinet Fortigate will be out of OP’s budget for certain. The Instant On secure gateway will be 100x better than the Asus.
UptimeNull@reddit
Also we can tell your great at engineering with guessing as a resolution.
“I am willing to bet” lmfao.
Apparently you don’t do security ya?!
VividVigor@reddit
Ahh yes. Ya got me. Super agent cyber guy right here.
Akctually. Let me go and perform an exhaustive assessment for a fortune 100 company that made $50B last year and has been in the game for 40 years so that I can reply to the post about some shitty Asus router.
Get back to Fortnite. The adults are talking.
UptimeNull@reddit
Go throw some more acronyms around bud. Have a day!
VividVigor@reddit
Ok. My bad. AP is Access Point. UTM is Unified Threat Mamagement. HP is Hewlett-Packard. Or more specific to enterprise networking is HPE “Hewlett-Packard Enterprise”. Let’s see. Cisco. Is just Cisco. Wi-Fi is Wireless Fidelity but that is a weird one. I can see how you would be confused. Palo Alto Networks NGFW is a family of next-generation firewall appliances. Not to be confused with Palo Alto Networks Cloud NGFW, Strata or Prisma family. FortiGate is a portmanteau of Fortinet and a secure, NGFW Gareway. I hope that helps.
SchemaAndShell@reddit
Wi-Fi does NOT mean wireless fidelity though.
UptimeNull@reddit
And you delete comments lol
UptimeNull@reddit
Go read your own response. wtf you talking about.
UptimeNull@reddit
Also enough with the ngfw. I assume in this thread we all kinda know what next gen firewalls can do.
UptimeNull@reddit
I was asking op why he is routing through that device to begin with. Chill bud