Why doesn’t Python have true private variables like Java?
Posted by PalpitationOk839@reddit | Python | View on Reddit | 38 comments
Hey everyone
Today I was learning about encapsulation in Python and honestly I got a bit surprised
In languages like Java we have proper private keywords but in Python it feels like nothing is truly private
Even with double underscores it just does name mangling and you can still access it if you really want
So I was wondering why Python is designed this way
Is it because Python follows a different philosophy or is there some deeper reason behind it
Also in real projects how do developers maintain proper encapsulation if everything can technically be accessed
Trying to understand how to think about this in a more practical and runable way
Would love to hear your thoughts 👍
authorinthesunset@reddit
Because it's not Java.
grismar-net@reddit
Private variables are not there for security, they're there to let developers know that they are not intended to be modified or accessed externally. This helps prevent situations where external code only works if it knows the value of the internal variable, which will cause trouble if the internal workings of the class change.
Python's name mangling makes accidental access unlikely; editors, IDEs, and linters surface this through warnings or inspection hints. If you choose to ignore those, that's hopefully for good reason, but there is no point in Python making it any harder. If you want to ensure that this simply is not an option for your team, you can add tools that will catch it.
Java makes it much harder, and it has gotten harder still since Java 9, but it is still possible. It just signals a different attitude towards reliance on developers making the right choices, and whether it is sufficient to be clear about a mistake, or whether it should be hard to make them.
mikeblas@reddit
Java's access control isn't for security, either.
grismar-net@reddit
Agreed - it never is, but Java does make it a lot harder to access these variables, so it's not just about signalling either. It's more about enforcing a rule within the language, instead of leaving it up to external tooling.
Any_Salary_6284@reddit
Technically speaking, in Java it is possible to access private variables and methods using the reflection API, although it is discouraged and considered an anti-pattern. So actually, sort of like Python.
Any_Salary_6284@reddit
(Cont…)
In my experience with scripting languages (Python, JS), the only way to have truly private variables is via functional closures, where the private variables are declared in the outer function’s scope, and therefore only visible to inner functions defined within that scope.
That should make the variables completely private from the perspective of the scripting language. However, even with that, it might be possible to use a lower-level API to inspect raw memory data directly, but you’d need to reverse-engineer how the interpreter structures the data, which could vary by implementation or even runtime oddities.
If it’s genuinely critical (I.e a security concern) to make the data truly private to other code in same process, then you probably should not be running that code in the same process. Have a separate process (or container/server) with the sensitive data, so it is truly isolated.
Brian@reddit
You can technically inspect them without dropping down to raw memory access. Given the closure function, you can access
f.__closure__to get the cell variables, and use the.cell_contentsattribute to get or modify their current value.biohoo35@reddit
It’s more of a cultural developer contract. Rather than outright preventing private methods from being used, we just use signaling with underscores.
If there was a valid reason for preventing someone from using a method, then using the underscore convention will also prevent importing, even using the * convention
headmaster_007@reddit
Using underscore does not prevent importing right? Yes, only the * convention does not import names starting with underscore but you can still do: from module import _var
oldendude@reddit
I think that the real reason is that Python objects are basically hash tables. E.g.
class C(object):def __init__(self, x):self.x = xc = C(5)print(c.__dict__)This prints {'x': 5}.
A hash table has no concept of the what code is accessing it. If 'x' is a key, (i.e., if self.x has been assigned), then access to that dict's 'x' key will work, regardless of context. I supposed a compiler could block access based on scope, but then access to the object's __dict__ would provide results with that access enforcement.
max123246@reddit
Sure, but I'm sure they designed the language's class fields first and then decided how to implement it.
Ha_Deal_5079@reddit
double underscore isnt privacy its just name mangling so subclasses dont clobber your attrs. _single is the real dont touch convention
HwanZike@reddit
Encapsulation is a convention after all, not an actual hardware or compiler limitation thing. In Java you can also access private variables via reflection, it just takes more steps. Its a matter of convenience, python is just more flexible. Just like typing, having everything public by default makes it a bit more unstable in a way but faster development (at least in the short term) and more expressiveness is the tradeoff. As for the exact reason why, I can't tell why python decided against having private in classes but the convention is leading underscores iirc.
acdha@reddit
I’d also add that the Java culture comes from Sun’s experience of operating system development before circa 1990 where any exposed variable became a potential support obligation customers will demand must be supported forever. This wasn’t unwarranted —apps like Microsoft Word and Excel famously had tons of crashing bugs caused by serializing C structures and breaking if, say, someone added a field or changed the size of something—but it’s not where we are as an industry now, or even in 1991 when Python, which is older than Java, started with a higher level abstraction. Also, the internet caught on and so you stopped having things be effectively unfixable on a timescale not measured in years.
Once you didn’t have things like a field changing from int8 to int16 breaking everything, or were using real serialization formats, most of the strong reasons for hard blocking access faded and the remaining issues ones were mostly the hubris of thinking you got all of the API perfect so callers would never need private variables.
I’ve been using Java since 1.0 and Python since 2.0, and contributed to open source in both, and I don’t think I’ve ever seen a case where hard enforcement of private variables would have helped. I have seen tons of architecture theater and performance problems created by creating accessors solely to gatekeep access and also more than a few Java developers using reflection to access private variables anyway so I’m solidly in the camp of saying anything more than a suggestion isn’t worth the effort unless you actually are building the Windows APIs. If what you’re working on doesn’t have hundreds of outside developers with support contracts, it’s just not worth it.
visicalc_is_best@reddit
As a fellow Java 1.0 user at the time (as you may tell from my username), I hope you’re staying up to date on your various screenings. Early detection saves lives!
No-Cranberry1547@reddit
yeah the convention thing is pretty spot on. python really goes with "we are all consenting adults here" philosophy instead of trying to protect you from yourself
in my experience working with different codebases the underscore conventions work fine for most teams. when someone accesses _private_method they know exactly what they doing and usually have good reason for it. sometimes you actually need that flexibility when working around bugs in third party libraries or testing internal state
the name mangling with double underscores is more about preventing accidental conflicts in inheritance rather than real privacy anyway. like if you subclass something and accidentally override __some_method you wont break parent class
SearchAtlantis@reddit
Python doesn't enforce matching signatures for abstract types why would you expect private variables.
Keith@reddit
Early on in Java I needed to use a field or method in a library. I could see it, it was documented, and it did what I wanted, but I couldn't use it because it was marked private, because Sun had interns write their class libraries. Access modifiers are like child safety locks you impose on yourself because you think you're an idiot and can't be trusted. If something is an implementation detail and shouldn't be touched, document that fact, but sometimes you need that shit and the language shouldn't block you out from what's possible.
victotronics@reddit
"you think you're an idiot and can't be trusted." Well, aren't you?
No but seriously. If your code misbehaves because a variable gets changed that should have not, you'll kick yourself for not declaring more things "private" and (in C++ only?) "const".
acdha@reddit
Don’t you have tests or use linters? Lots of things need testing for major updates and this only addresses a single uncommon failure mode.
knobbyknee@reddit
Python is designed for writing programs in small teams, with the assumption that you know what you are doing, and you know what people working in the same module are doing.
As long as you don't monkeypatch your imports, you have enough isolation for things to work very well in practice. There are usecases for the monkey patching too, so making cross module monkey patching impossible would reduce the usefulness of the language. (The most important use cases are fixing problems in third party modules and in mocking for unit tests.)
Java is designed for enterprise programming, where hordes of programmers who don't know what they are doing, are producing code according to specifications made by someone else. They are too many to properly coordinate development that touches the same code as some other part of the organisation is concerned with, making the isolation directives a necessary evil.
Tucancancan@reddit
The linter will catch if you access anything prefixed with an underscore. But at the end of the day you are right, nothing stops you from doing dumb things so you just have to be responsible and not do it.
The hackiest hack I ever pulled was using patch in production code to overwrite a private variable several layers deep in a package because they didn't expose it as a parameter lol
GoofAckYoorsElf@reddit
Oh yeah! Back in my younger years I did something I would call heresy nowadays. I overloaded the 'import' command to be able to handle optional dependencies itself without the need of wrapping the imports into try except blocks everywhere...
I was so young and so naive. It felt cool however to learn that you can do that. You shouldn't. But you can.
jeffrey_f@reddit
Variables created/initialized within a function are private. Those variables live and die within a function and are considered "private"
unlikely_ending@reddit
It's a language for use by consenting adults
Zenin@reddit
Java is full of horrifically bad design choices that the community around it likes to preach as some kind of divine best practice handed down on clay tablets. The insane levels of protection it puts around the internals of classes is very much one of them. Folks coming from such a stick up the you know what community often are shocked at the cultural shift, especially if they've spent more time in academia than the real world.
Python takes a philosophical clue here from Perl (yes) which I believe Larry Wall once described like this:
Python thankfully takes a very similar philosophical approach. This simple choice allows much more direct problem solving in the real world, with far less frustration, and countless reduction in awful workaround kludges.
Yes, reaching around a class's published spec to access or tweak something that the author didn't publish puts the user well into "may break unexpectedly in the future" territory, but that's squarely on the user to decide not the class publisher to enforce.
In the real world it's not at all uncommon to see Java classes internally forked (if code is available) or literally reverse compiled from bytecode and then forked internally, simply to un-private something. This happens on the regular in enterprise environments working in Java. Yes, it's a horrible practice, but that's the point: Java's paranoia about users doing small wrong things has forced those very users to do some of the biggest wrong things imaginable.
Penguinase@reddit
https://www.artima.com/articles/the-making-of-python
from that interview i think it was probably because he had a C background and the initial design days were in the mid '80s.
ahferroin7@reddit
Making variables private is actually more resource intensive than not doing so. It’s only very recently that any kind of mechanism for this sort of thing has made it into hardware, but all of the options for it are designed for a very different type of use case (mostly protecting memory in virtual machines from the hypervisor, or protecting a block of memory in a process from other things on the system) and would still impose some pretty nasty performance penalties. So there are performance benefits to not having enforced private scoping of things.
Python is also built more around a philosophy of developers being responsible and knowing what they’re doing, instead of assuming you need to be protected from yourself.
The standard in Python is that things prefixed with underscores are functionally not part of the public API. In practice:
x, you can get the name of the class of that object withx.__class__.__name__).This is all partly enforced by the data model as well actually. Wildcard imports
from foo import *ignore any class, function, or variable names in the module that are prefixed with an_, and there are a couple of other places they get explicitly ignored as well. You can also configure most linters to warn about violations in a codebase, but in practice a vast majority of it is just developers agreeing to follow the convention.AlmostSignificant@reddit
Does it feel to you like the language was designed from first principles?
dbell@reddit
Just slap an underscore in front of it. Everyone will respect it.
liquidpele@reddit
Because private variables are stupid. I’m using the code just let me fucking do shit with my own fucking code.
CranberryDistinct941@reddit
Python's philosophy on this is "we're all consenting adults"
ArtOfWarfare@reddit
The person using a library may fully understand everything their project is supposed to do. It is impossible for the library author to understand everything their library will be used to do.
It therefore makes no sense that the library author would be able to dictate what it can and can’t be used to do.
Library authors make suggestions. And the library user may or may not follow those suggestions, in both languages. In Java you use reflection or decompilation and hot-patching when you know better than the library author. In Python you just use underscores.
Interesting-Town-433@reddit
Because Java is trash
ironmaiden947@reddit
Python has a different philosophy. If you want the variables to be private, just don’t use it.
Designer-Ad-2136@reddit
It's philosophy. If somebody really wants to use private variables, theyll just edit your code anyways. Why not simplify the whole process?
Empanatacion@reddit
You can muck about with private variables in java using reflection, it's just harder. Python doesn't enforce the access restrictions, but there is still encapsulation if you follow the conventions around naming and don't directly access variables that are clearly meant to be private.
wannasleeponyourhams@reddit
dont need it