Networking applications should not be opening sockets
Posted by Public_Being3163@reddit | Python | View on Reddit | 8 comments
From my first development project involving networking I was hooked. I also found some areas of networking software a bit unresolved. There was some strong modeling for people who make networking components but that seemed to peter out after the sockets library. Nobody seemed to have a good compelling way to bundle all that block I/O, byte framing, encoding/decoding, message dispatching etc into something that was reused from project to project.
I finally did something about this and have produced a software library. I also wrote a discussion paper that is the first link in the readme of the following github repo. The repo contains demonstration modules that are referred to in the other readme links.
Networking is not about sockets
Is there anyone else out there that has thought along similar lines? Has anyone seen something better?
thisismyfavoritename@reddit
Core code is not on git, yet there are pip packages. Sus.
Also skimmed through the examples a bit, to me it looks dreadful. Id use low level sockets any day over this
Public_Being3163@reddit (OP)
Skimmed ... looks dreadful ... [prefer] low level sockets.
You havent responded to any of the technical points raised for discussion. The only point you offered was about the optics? Checked and can confirm that wasnt in my list of requirements.
yvrelna@reddit
I think this is pretty obvious. Sockets are just the basic building blocks.
In real life, you want to just be using HTTP frameworks, or building on top of ØMQ, or some sort of RPC protocol, or Websocket, or basically other higher level abstractions than raw sockets.
I've been building complex networked applications for over a decade, and had never really had to actually do sockets programming directly outside toy examples.
arthurazs@reddit
I work with specific protocols on the data link layer and some tcp/ip protocols that are not http based, so we end up using raw sockets
thisismyfavoritename@reddit
or just if you work with companies that have their own protocols and there exist no open source implementation
Public_Being3163@reddit (OP)
I guess you mean that the title of the post is stating the obvious.
To a developer who has spent time in this space I would mostly agree. To everyone else - not so much. I have also worked with very experienced and competent developers who will still write custom sockets-based IPC for a very small, focused task. Because every IPC toolset out there just involves "too much distraction" (their words not mine). These are people who design, implement and manage complex cloud services while also pushing through bug reports on the linux kernel.
Where is the "out-of-the-box" IPC solution that is also based on object ids (or in OSI terms, application ids)? The discussion document linked in the github readme went to great pains to paint the full reason for the title of the post, i.e. HTTP, zeromq, RPC, Websocket, et al do not have the same goals and are not a solution.
If you have been building complex networking software for a long time, without dipping down to sockets level and you are satisfied with the toolsets available to you, then I am a little envious. The last project I was involved with adopted the best-of-breed for everything - golang, node, angular, neo4j, graphql, protobuf, HTTP, etc - and i would describe it as networking hell.
arthurazs@reddit
https://sans-io.readthedocs.io/
Public_Being3163@reddit (OP)
Thanks for this link. Huge amount of work that I was not aware of. Had to take the time and truly read the material.
While I respect the work in its entirety, there are elements that are tangent to my own goals with networking. There is the general preservation of synchronous thinking when I believe that networking needs to be fundamentally async. There is also nothing about a networking framework based on object ids (discussed in the first link off the github repo). This is a substantial body of work focused on ensuring the quality implementation of protocols that do not solve the issues highlighted in that link.
Their approach is consistent with a general preference for sync in the development community, it is intended for integration with python async primitives and it should result in fewer and better implementations of the listed protocols. But these were not the issues I was trying to solve.
Thanks.