Best (mesh) network for multiple 10.000 devices?

Posted by Psychological-Ice964@reddit | sysadmin | View on Reddit | 28 comments

I am the lucky IT guy, who got the task to find a solution for a wireless mesh style network. It will be a network of ad-hoc devices, which will be in a urban circle of about 500m. The environment cannot be predefined as it may change from event to event, and each urban environment may look differently. But it can be assumed, that the devices are somehow even distributed about the rural area. So a mesh style networking could solve the issue: * support for multiple 10k devices in the mesh * low energy consumption (since on battery) * thus a low range of distance for the radio signals * low bandwidth requirements (>1MBit/s would be great, but 10kbit/s are also okay if reliable and low network collisions). But of course if being able to utilize 10MBit/s or more, I am not unhappy... * no preinstalled infrastructure wanted (e.g. like wifi accesspoints) * optionally a infrastructure could be installed to make the network more stable, but it shall be optional only! The devices would be have like sleeping for minutes and receiving commands and sending collected data in batches of lets say every 60 seconds. Here the faster we can transmit, the less the air-time will be in milli-seconds. Any advices, which technologies are even able to solve such needs?

Reply to Post

28 Comments

[-]

ElevenNotes@reddit

Just use 4G/5G.

[-]

Psychological-Ice964@reddit (OP)

I would like to use 4G/5G, and it would solve everything. But it is sadly not always available. And if connecting on small 4g/5g infrastructure 10k+ devices, it will (afaik) kill the network So would like to have something which is working without any infrastructure requirement. So the devices would connect to the local mesh (self managed/maintained). On the network, which I would later define, some devices would get master and would use a existing 3g/4g/5g network or any other preexisting networking to share the collected data. I am adding your good point to my initial post

[-]

JustSomeGuy556@reddit

So just to think this through... You acknowledge (properly) that tens of thousands of devices will kill a 4-5G network, a network that people who are the absolute best in their field spend a vast sum of money build and maintain, and you expect to do this with battery powered devices, limited external connections, and (probably) a fairly small budget, and have this be some sort of semi-portable system? Do you see the issue here? Your list of technologies reads like a list of buzzwords from marketing, not with any serious understanding of even wifi at scale, and some of those techs are just *wildly* the wrong choice. I'm not trying to be a jerk here, but you are in so far over your head on this.... You aren't even really sharing what you are trying to do here, and while I understand that might have it's reasons, when talking about this scale it matters. You don't seem to understand what "infrastructure" means in this context. You don't seem to have a good understanding of how IP works. Honestly the parent post here is correct. Your only real option would be 5G, and accept that this isn't going to work in a 4G space.

[-]

Psychological-Ice964@reddit (OP)

u/JustSomeGuy556 let us start to discuss which of the named technologies are even possible to handle the named amount of devices. I think that will narrow the possible set of solutions quickly down. So what are your suggestions for the device count?

[-]

JustSomeGuy556@reddit

Without more information about the application, I can't even begin to answer that. If I had to answer, it would be "5G".

[-]

Psychological-Ice964@reddit (OP)

if not possible to use 5G (implies 3G, 4G too), which alternative would be able to cope with that amount of devices?

[-]

JustSomeGuy556@reddit

3G and 4G won't handle that density. None of the mesh technology stacks will handle that device count. The LPWAN stuff matches your need *in theory,* but most of those products are still pretty early in the development cycle, or have really only been used in pretty bespoke applications, and almost always at much smaller scale. The closest I can think of here are things like smart meters (for things like water and power) but those don't have the density, and are usually one way communications (though some are two way). But I don't think the densities are anywhere near what you are talking about, and these are largely fixed networks with actual infrastructure backing them. Looking at a company called xylem, it looks like they have products that do that sort of thing, but they tend to report every few hours, not once a minute. (And again, density concerns apply) If this is a for real requirement, not some good idea fairy thing, Be prepared to spend seven figures, and start calling places like that, and Motorola and Honeywell, You might be able to build on an existing tech stack, but it's probably going to be proprietary, and expect custom development work for both hardware and software. You'll also certainly need licensed frequencies for this and probably FCC approval. (though that cost will be inconsequential) I suspect that you got tasked to do this by somebody, but I don't think that somebody has *any fucking clue* just how big an ask this is. You are way into the weeds here... I'm looking around and finding academic papers on theories to deal with stuff that touches on this, but this sort of scale for what you are talking about is basically new ground. Nobody has done this. There's no tech stack you can ride on, no existing product.

[-]

Psychological-Ice964@reddit (OP)

thank you for your insights. LPWAN was also one of my candidates like NB-IOT. If the LTE is available it could work out of the box. But if LTE is not present, there probably will be required to install some infrastructure to provide the communication option. The things is, I cannot find any specs about max amount of devices. So neither if any provider based infrastructure is present nor for the case to implement own NB-IoT infrastructure. Do you have some info about this? Based on the above, I was looking for a second alternative. So the general idea was to find a self organizing peer-2-peer mesh solution. It shall be able to manage own mesh-cells with one or a few active/active or active/passive masters. Each master itself could so communicate with the other masters. And these masters itself can be the router for all nodes in a mesh-cell to hand over the data to a upstream node, which could have its own IP-based internet connectivity. The assumed principle here would be the basis how BGP / routing in the internet it working. So my question would be, are the solutions which could handle such an requirement? Also having issues on my understanding: - how to defined bands on which the mesh-cells communicate, to avoid interference with other cells - how to identify a solution, which copes with such requirement to self define a mesh network - how to identify a solution, which copes with the cell infrastructure Any ideas or hints?

[-]

JustSomeGuy556@reddit

Some of the NB-IOT stuff looks promising (which is a form of LPWAN), but all that is essentially carrier based. So call your local carriers and talk to them, I guess. NB-IoT isn't universally deployed and you'll need to coordinate with them... and probably have to pay. And that's not going to be a mobile setup. At this point, I really don't have anything further to suggest. There are no solutions, in terms of commercial off the shelf products that will do this. There are no existing tech stacks that would do the job without substantial custom work. At a (wildly) smaller scale and density, this is probably a graduate student level project but at the scale you describe is a whole different level. You are still asking for a tech stack that supports this, and I'm not aware of any that do, outside (sortof) of carrier based solutions. Not at this density and scale. You've re-asked the question a few different ways, but that's not going to change the answer. You aren't quite at the level of being asked for warp drive, but that's the context you need to understand. *The technology you are asking for doesn't really exist yet.* Maybe you can hire people who can build it, but that's going to be a very, very big lift with associated spend and time and development cycles to optimize hardware and write new software that can do this. You have some ideas, but ideas are easy. Implementation is hard. You need to tell whomever came up with this that if they are serious, they need to be prepared to spend serious money and have very high level meetings with important people who control large amounts of industry that can build this. This isn't a thing you find a solution for on reddit. If they can't deal with that, I'm strongly suggest finding a new job.

[-]

Stryker1-1@reddit

So you want to deliver wifi without the wifi access points? 🤔 You need some kind of infrastructure to deliver what you want.

[-]

Psychological-Ice964@reddit (OP)

you refer to wifi, which is one of the possibilities. Even wifi is providing multiple solutions for different cases. WiFi mesh, Wifi HaLow and so on. There is also LoRa and many more options. So I updated my question to be more clearer

[-]

Cozmo85@reddit

What exactly are you connecting to this system?

[-]

Psychological-Ice964@reddit (OP)

small IoT devices, if you mean system==network

[-]

Cozmo85@reddit

Okay so they are all Wi-Fi devices?

[-]

Psychological-Ice964@reddit (OP)

my quesiton is more generic to all possible networkings. But let us go the path with wifi. What do you have in mind with that path?

[-]

Crenorz@reddit

Starlink (runs off 12v car connection) + Cisco AP's (as they use PoE = low power like \~15w each). Call Cisco sales, they will sort it for you with more details.

[-]

Psychological-Ice964@reddit (OP)

starlink was also a idea, but the antenna is clearly - as of now - a required infrastructure. And also not clear how many devices they can eat on a small area like this. Cisco APs would be an option if we could decide for infrastructure. Sure could call some bigger vendors, but my question is more a general question to understand which technologies are even possible to handle this. And cisco will for sure not be the only vendor capable of solving it

[-]

JustSomeGuy556@reddit

So... exactly what are you trying to achieve here? Because honestly, this feels like some sort of "good idea fairy" thing. This is really not a "wi-fi" application.

[-]

Psychological-Ice964@reddit (OP)

I have updated my initial post to be more clearer

[-]

Tymanthius@reddit

> but the antenna is clearly - as of now - a required infrastructure. So your bosses want magic? b/c ANYTHING will require AP's of some sort, which is infrastructure. Or do you mean that you can't be tied to a single ISP? Which I guess is another ? - do you need an internet connection, or just a closed network?

[-]

Psychological-Ice964@reddit (OP)

i have updated my initial post to be more clear and to get down to the initial fruitful discussion for startlink

[-]

JustSomeGuy556@reddit

tens of thousands of devices on a mesh? On battery power? With all these devices in a 500m area? Presumably outside? In an urban environment with all the signal management issues that come with it? (You say urban twice and rural once so???) This seems like a recipe for failure. Like, total and complete disaster. I'm not saying it's not achievable at all, but at the very least it's going to require very expensive solutions made by people who probably aren't on reddit. And even then... Honestly, no. Just no.

[-]

Psychological-Ice964@reddit (OP)

thank you for your note about urban/rural. I ment urban in all cases. I understand that you have high objections, but anyway what is the nearest possible solution in you opinion? And I am not asking for help here to implement a solution I am asking for possible technologies and solutions. Based on that the solution needs to be tested and afterwards implemented. So you refer to the both later. I am asking about possibilities. Hope I could clarify

[-]

JustSomeGuy556@reddit

I think that anything with tens of thousands of client devices is going to mean some sort of bespoke solution, especially in an urban area. Based on what you've written here, LoRa Mesh would be the tech I *might* start with, but you are almost certainly looking at a pretty custom implementation. I'd maybe start calling people like Motorola. And prepare to spend *substantial* money.

[-]

amgeiger@reddit

There's got to be a vendor that specializes in something like this. If this is for any type of PoS, wifi is going to be a nightmare.

[-]

Psychological-Ice964@reddit (OP)

updated my initial question. Do you know a vendor and for what technology?

[-]

Cozmo85@reddit

My advice is to hire a consultant

[-]

RGB-128128128@reddit

If you find yourself asking a question like this on reddit, you're fucked. Part of this industry is knowing when to bring in specialists, and this is pure specialist territory.