Datto appliance firmware updated disables ICMP
Posted by tentjib@reddit | sysadmin | View on Reddit | 13 comments
So we recently acquired a customer that uses datto backups with an on premise box that replicates to the cloud . Fantastic solution and so far we have had zero complaints.
Until today we noticed the Ubuntu on prem box hasn’t checked into prtg (onboarding mode was enabled - 100% my fault and a good spot from my colleagues)
Spent an hour or so troubleshooting the basics , and in the process decided to reboot it to see if that would help ( 90% of problems are fixed by turning it off and on again amirite)
So we see a handful of pings during what we assumed was the reboot then nothing .. weird … really weird
I’ll save you the saga of us checking things like firewall rules which quite frankly we knew were not the problem as we hadn’t changed them
We ended up giving their support a call and was basically told yeah , no more icmp and no your not getting it back . Big sad
In all honesty I get it .. just annoying that I know have to figure out monitoring for these backups that does not rely on email and I was quite happy to leave this thing as a set and forget device considering how good the rest of the system is as a whole
TLDR: datto on prem device firmware update has disable icmp pings and it wasted a few hours of my day
mattyice6@reddit
Not sure if this applies to all Datto devices, but mine run on Ubuntu. I was able to SSH into them and add an iptables rule to allow pings from my monitoring server.
sudo iptables -I INPUT 1 -p icmp -s xxx.xxx.xxx.xxx -j ACCEPT
ExceptionEX@reddit
It's a last ditch effort that generally probably causes more harm than good.
Giving up a lot of diagnostic functionality to prevent something compromised in a local network from seeing it just seems a bit backwards.
To me this should really be a choice at a network level not at an appliance level. But whatever
thebigshoe247@reddit
I'm glad my Datto server is now a hardened repository for Veeam.
Jealous-Bit4872@reddit
They might just be sick of “disable icmp when unused” showing up in every penetration test report ever made and disabling it to make the auditors happy.
i-void-warranties@reddit
I'm guessing this was an anti ransomware thing to make it "stealthy" but I bet there are a bunch of known ports listening and the bad guys know the profile. Happy to be proven wrong...
ledow@reddit
"Yeah, we turned off our doorbell, now nobody will ever know our house exists!!".
The ping nonsense is really the worst security excuse I've ever heard. Always comes up on our pentests etc.
"YOU ACCEPT PING?!!??!!!?!?!?!"
Yeah. Like almost every Internet-connected service in the entire world. You've never pinged Google to see if you're connection is up? Or your ISP default gateway?
"But it could take down your computer?!?!?!"
What damn shite OS are you using? Put it back in the cupboard with the rest of the 1980's junk. And how is that different to any other TCP/UDP/etc. service anyway?
"But attackers will know your machine is online!!!?!?!??!?!"
It has a dozen ports open to offer the services that it's literally there to offer. What kind of hackers only ever pings instead of just using nmap anyway?
"But it could be used in DDOS?!?!?!"
Really? Because traffic-amplification is far easier with a bunch of other stuff that's REALLY common and I don't see you worrying about that.
Such nonsense.
tentjib@reddit (OP)
Yeah that’s basically what they said which we totally anticipated .. I’m not even mad I just wish I had the option to enable it. If someone can ping a device from a local IP I have way bigger problems I would have thought (I am still a junior so please correct me if I am wrong 😁)
i-void-warranties@reddit
Based on a 5 second search, yeah there are known listening ports. This is a waste of time so they can tell unitelligent buyers "you can't even ping us, we are ransomware proof!". Again, happy to be proven wrong...
malikto44@reddit
This is a little rant. Yanking ICMP does nothing for security. The bad guys are just going to find it via
nmapanyway. It removes a useful tool and healthcheck, especially if the app layers of the appliance are down, but the OS is okay.If I wanted to sell a "stealthy" appliance, I'd have a "stealth mode" in some place out of the way in the config which details that ICMP gets shut off, but it wouldn't be a default.
I have been working on a "ransomware appliance", just for grins in the homelab. Pretty much, took Minio resurrected for the S3 server, and it drops data on a ZFS array. The OS boots with a TPM (I do have a manual LUKS code to enter if that goes south), and it is on TailScale. Definitely not ready for prime time, as it needs a good web UI, but if some attacker gets my desktop box, they can't pivot to the OS of the appliance. From there, MinIO's object locking is good enough, and the appliance uses Borg Backup to snapshot stuff offsite. Not marketable yet, but it is a hedge against ransomware.
thesals@reddit
I had a lot of issues with the Datto BCDR appliances back when I was at an MSP... They'll eventually get in a state where support can't fix it without wiping your local and cloud backups...
I then researched and found another company that does the same thing with a lot more reliability and much cheaper monthly cost. I highly recommend Axcient x360, they do require an MSP partner agreement, but they're an awesome system. And depending what model Datto appliance you have, you can actually install their appliance OS on it.
tentjib@reddit (OP)
Oh don’t get me wrong this is a system we inherited and need to support until the contract is up but so far I’m kinda liking the system . Can you link me to any examples of them wiping local and cloud backups as that would be a good selling point for our solution !
thesals@reddit
I didn't have any saved examples of that, it was 6 years ago and I'm long moved on to another role at another company.
cjchico@reddit
Good old security through obscurity. Do an nmap and see what's open