Best practices for using Python & uv inside Docker
Posted by ashishb_net@reddit | Python | View on Reddit | 111 comments
Getting uv right inside Docker is a bit tricky and even their official recommendations are not optimal.
It is better to use a two-step build process to eliminate uv from the final image size.
A two-step build process not only saves disk space but also reduces attack surface against security vulerabilities
0x256@reddit
The linked security issue is a bad example. If an attacker can use
uvin your container, they could also download and run whatever executable they want and do not need to exploit bugs inuvfor that. With very few exceptions, CVEs in unused executables in containers are almost never an issue, because if the attacker already has shell access to be able to use them, they won't gain anything from those bugs.ashishb_net@reddit (OP)
Yeah. Best to avoid having uv in the final image.
leafert@reddit
No. uv isn't the problem in the final image. If an attacker gets shell access to your container then not having uv isn't what stops them.
ashishb_net@reddit (OP)
Keeping final image lean reduces the attack surface.
AstroPhysician@reddit
You are incapable of admitting you’re wrong or listening
Dangle76@reddit
I think you’re missing the point
catecholaminergic@reddit
Some people think they are their ideas, so when they get told they're wrong, they're going to come back with something no matter how bonkers because the ego's not going to not defend itself.
JorgiEagle@reddit
But this is coming down to the basics of risk management.
Is having uv a larger overall benefit over the increased risk of keeping it in.
It’s not just “this is more risky so take it out”, it’s “is the risk worth it”
ashishb_net@reddit (OP)
What's the benefit of uv in the final image?
JimDabell@reddit
You’re skipping past the first solution they offer, which is the more efficient distroless solution. You can literally just copy the standalone
uvbinary directly into your image, you don’t need to base your entire image on theirs.This takes ~43MiB, not the 77MiB you cite.
scaledpython@reddit
43MB for a pip-like installer is insane.
JaguarOrdinary1570@reddit
that's rust binaries for you
Proper-Ape@reddit
In this case the fully contained binary makes it possible to have such a minimal distroless image.
There are drawbacks and benefits to this approach.
LLoyderino@reddit
but it's blazingly fast 🚀
ashishb_net@reddit (OP)
Yeah that works as well. Slightly different approach with same effect.
ihavenoname143@reddit
No, same effect would be identical sizes.
ashishb_net@reddit (OP)
The final size of the image is identical as neither will contain
uvbinary.ArgetDota@reddit
You can also mount the executable during image build for a duration of a specific RUN instruction
stibbons_@reddit
Always use virtual env even in docker. I now make apps with frozen dependencies to avoid a dependency update on pypi to break existing app (on docker rebuild). And if 2 apps has imcompatible dependencies, you have to have venv per app.
RepresentativeFill26@reddit
How would a requirements file with versions specified break an exiting app?
stibbons_@reddit
App A depends on lib X. Requirements tells to take X in version >=1,<2 For some reason, X is released in pypi in version 1.3 that beak something. It happens. Life. Now, you rebuild your docker image for some reason and your perfectly working app in X version 1.1 is reinstalled in version 1.2. With uv tool install it is even worst. It is important to freeze all dependencies for application to the version you have validated
RepresentativeFill26@reddit
Im unfamiliar with uv but can’t your export the exact versions to a requirements file? I used to do this with conda and poetry
aidandj@reddit
You have no hash locking with a pinned version requirements file. Leaving you open to supply chain attacks.
stibbons_@reddit
Yes and Python does not provide a native way to release a tool with all dependencies locked and its library version (its api) in loose version, without having to have 2 projects
stibbons_@reddit
Yes but if you have several such app you can’t install them in the same venv because each will have different dependency versions
RepresentativeFill26@reddit
Why would you run several apps in a single container?
stibbons_@reddit
What is your app use several tools ? I mainly use docker image to bundle ready-to-use environment with several internal tools preinstalled for our developer
usrlibshare@reddit
Why would I use
uvinside docker in the first place?The tool is for managing environments, and direct deployments. In a python docker container, I simply install my built package natively:
Don't get me wrong, I love uv and use it everywhere else, including for the management and building of projects I deploy via docker ... but inside a container, it's just not necessary.
ashishb_net@reddit (OP)
How do you build docker images for say a Python-based web server then?
TedditBlatherflag@reddit
What? He is saying you just create the package to install as a build artifact outside Docker. Inside Docker the wheel can be directly installed. This would work for any system, web server included.
ashishb_net@reddit (OP)
And how do you build the wheel?
Inside Docker? outside Docker?
TedditBlatherflag@reddit
As long as it’s a py3-any wheel it can be built anywhere you want.
Build it inside a container and use docker cp to get it out.
Build it in your CI/CD and put it on a private PyPi repo.
Build it on metal or a VM or a container or a pod or whatever.
It’s just a portable artifact.
The same is actually true if it’s a platform specific wheel with compiled extensions, as long as your platform includes the correct tools for the target platform.
Personally, what I do is make a multistage image and make a base with the system dependencies which is also tagged and versioned, so it can be quickly pulled or is cached locally.
On top of that the next stage is a dependency only install which creates either a venv or site-packages artifact. It is also tagged, versioned, and published and as long as neither the system deps or python deps change it’s stable.
Separately I have a build tools stage which is used to create the dist/whl - it shares the system stage but only includes the necessary build dependencies, which we may cache into a published image if they’re significant. This stage is typically what builds every run since the code is changing. But importantly it’s only ever doing “uv build” and and producing the wheel artifact.
The next stage brings the venv/packages image back and installs the wheel into that same location.
The final stage is based off another cached image which only includes linked libraries (not -dev headers), and an ultra minimal OS required for Python to function, where we then bring in the fully built venv/site packages and set the runtime commands etc.
Basically for any normal CI run we’re doing a “uv build” and a “pip install” (with all deps already installed) and just copying that into a secure final image, which is fast and repeatable.
RepresentativeFill26@reddit
Just wondering, why do you want to use a virtual env in a docker container?
ArgetDota@reddit
You don’t. You can install your packages from the lock file with uv without creating a virtual environment. Just set UV_PROJECT_ENVIRONMENT to point to the system environment of the image.
https://docs.astral.sh/uv/reference/environment/#uv_project_environment
Luckinhas@reddit
UV_SYSTEM_PYTHON=1is a bit clearer in the intention and simpler to use. Maybe more portable too.ArgetDota@reddit
This work with the project interface (uv sync).
BelottoBR@reddit
Did not find that option in the documentation
Mr_Again@reddit
It belongs to the uv pip command
Luckinhas@reddit
https://docs.astral.sh/uv/reference/environment/#uv_system_python
RepresentativeFill26@reddit
Sound like the right option! Allows for fast local dev with uv an easy CI option.
thrope@reddit
The venv part is a means to an end here. It’s about having a perfectly replicated environment in production based on the uv.lock file which specifies precise versions.
RepresentativeFill26@reddit
Why can’t you create a perfectly replicated environment in a docker container?
thrope@reddit
What Python tooling would you use for that?
BogdanPradatu@reddit
Docker is itself a virtual environment, so unless you need multiple python environments in your container, just create an image with the right python version and packages. Voila: python virtual environment in docker and you don't need to do it everytime you run the container, you just do it once at build time.
captain_jack____@reddit
uv also locks versions so it would always install the exact same packages. How would you install the requirements from the uv.lock file?
DerShokus@reddit
Just gen requirements txt and install it globally in the containers
runawayasfastasucan@reddit
Or just use uv so you have the same setup no matter if you are in or outside docker?
Mr_Again@reddit
I think their point is you can use the --system flag with uv to avoid using a venv? but tbh these tools all expect a venv so I don't really see the point in trying to be clever
BogdanPradatu@reddit
If you are already using uv, sure, but I wouldn't adopt it just for setting up docker images.
Kryt0s@reddit
Which does not lock sub-dependencies...
Kryt0s@reddit
Then you would either have to install your packages globally for development or develop inside the container, which is a pain.
RepresentativeFill26@reddit
You create a requirements file from your local venv, copy that into the container while building and then run pip install
Kryt0s@reddit
Which does not lock sub-dependencies...
HommeMusical@reddit
No Python tooling, it is not needed. Think of docker as a venv for the whole computer.
You create a Docker environment just to run a single program and then install the packages you need into the single system version of Python inside the Docker with
python -m pip. Normally that is a bad idea but in this case it is good practice - there's no need for a Python virtual environment when there's only one and it only ever runs a single program.thrope@reddit
Going to disengage now because I’m not sure if I’m being trolled or if this sub is now full of reply guys who have never used Python.
HommeMusical@reddit
Personal insults make the world a poorer place. Please refrain.
I have used Python for over 21 years. Here's my Github repo: https://github.com/rec
I don't use Docker at all these days in my work, but I did extensively in the not-impossibly distant past.
If you have a technical refutation of my claims, lay it out. I am perfectly prepared to be wrong, but I won't be insulted.
thrope@reddit
Requirements.txt is not a lock file and is difficult to manage, especially cross platform. Before uv there was poetry, but uv is much easier to use, faster had hence has had really fast adoption. The issue of in a venv or system python is missing the point of the functionality uv offers - which is cross-platform lockfile and better dependency management. Maybe this is useful https://www.reddit.com/r/Python/s/qJXbFwjpFM
https://docs.astral.sh/uv/
HommeMusical@reddit
Man, I feel unpopular today! :-)
I'm extremely familiar with
uv: I dumpedpoetryforuvearly in 2024 and never looked back.I do not use Docker in my development, because I'm mainly a Python developer. Other developers have different needs and using only Docker for all their environmental needs and not using any other package manager is a perfectly viable and solid solution, even though I never do that.
thrope@reddit
The post is about using uv in docker, so you are being a reply guy
HommeMusical@reddit
I am answering the question above this comment in the thread: "Why can’t you create a perfectly replicated environment in a docker container?"
You're blocked. I hope you don't treat people in the real world this way.
Wyndegarde@reddit
Think the point being made is that if you are using uv for development you will create a uv.lock file that has all dependencies in it. If you want to use that to build your docker image you need to use uv and create a venv via uv tooling. Generating a requirements file from the lock file is an extra manual step and also removes the ability to leverage uv’s benefits when building the image
HommeMusical@reddit
Yes, I develop much that way myself, except without the Docker.
People who develop entirely inside Docker, which is not me, have no need of the
uv.lockat all. The Docker environment is the single source of truth. The only way to develop is to spin up this Docker and run from within it, but on the other hand, they have a simple unified environment that just works for everyone.Again, I don't develop this way, but I do understand the people that do.
Please remember that a lot of Docker users are not Python experts but people packaging a lot of programs in many languages together. One of the advantages of doing this is that they don't have to learn about poetry, uv, node, or whatever packaging system some specific application and language uses.
HommeMusical@reddit
Sure, I develop in much this way myself.
However, people who work entirely inside a Docker have no real need to create a uv.lock file. The Docker itself is the source of truth, and since they always live in this hermetic environment, it works perfectly well.
runawayasfastasucan@reddit
You can but why not use the best tooling for that, which would work just the same outside docker?
RepresentativeFill26@reddit
Because a best tool that is superfluous is still a superfluous tool.
runawayasfastasucan@reddit
Not really, why have a different way to set up your project whether its inside a docker container or not, why not have one way to set up.
ahal@reddit
Depending on the image you're using, there could be system packages included already.
yakimka@reddit
For moving between stages
RepresentativeFill26@reddit
Well, you can use the docker image between stages right?
Yablan@reddit
This is the relevant question here. There is no need at all for a virtual environment within a docker container.
Huberuuu@reddit
It is still best practice to
Yablan@reddit
No it's not. I'm a full time backend Python dev since at least 13 years now, and was a Java dev before that. We use docker for everything at work, we deploy a lot of different projects internally, and we never use virtual environments inside containers. No need at all.
And I was a consultant before my current employment, and never worked on anything where we had virtual environments inside docker containers.
MasterThread@reddit
You reduce final image size, reduce build time, and ofc reduce ci/cd costs. It’s bad not to develop yourself for 13 years. It was ok not to use buildx 13 years ago, but now - industry changed.
Yablan@reddit
I disagree. Not worth the effort. Following the KISS principle is very important. No need to overcomplicate things unless you really have build times and image size problems. YAGNI. Premature optimization is the root of all evil.
MasterThread@reddit
That's sick you don't see your Ci/Cd takes more money than it could. Your CTO/CEO which gives money to devops department wont see that either. You cannot overcomplicate it since it takes 10 more rows in Dockerfile, but you get 10x slimmer image.
Yablan@reddit
Hmm.. I might actually give it a try after all. I have reconsidered and will have a look at it soon. Thank you for your candor. :-)
xaraca@reddit
Do you build and publish your python packages to somewhere and then install from that somewhere in your container image?
I'm just getting started and the easiest thing to do seemed to be just checkout, sync, and run the app in our dockerfile without bothering to build it.
Yablan@reddit
At work we have a full pipeline, with tags for releases and jenkins building to sn internal registry, and then we deploy to environments using rancher.
But for my private projects, I simply use docker-compose based projects, and then run them within docker compose even during local development. And then I have one script on the root project that builds, starts, stop the docker projects etc.
So on my VPS, I just git clone the project there too, and then simply git pull and then run the scripts to start the projects. So I use git for versioning, but also for deployment.
I did not understand what you meant with uv sync.
My dockerfiles usually copy the sources and the requirement files into the image, and then install the dependencies, and then start the web service. And on the docker-compose, I mount the source code too, and then I run the web service inside the container in dev mode with hot reload. And my IDE has docker integration, so I can start and stop and even debug my project that is running inside docker.
RepresentativeFill26@reddit
The pip install step in your dockerfile is cached so unless you run docker build with —no-cache or change a dependency the while dependency installation is cached anyway and completes directly.
HommeMusical@reddit
It's unnecessary cruft.
RepresentativeFill26@reddit
Why would that be? Dependencies should be resolved during local dev and when you don’t have multiple apps running in a single container I can’t really think of a reason to use is.
lukerm_zl@reddit
uv does have lightning fast installs, so it might be a build-time thing. Just guessing.
RepresentativeFill26@reddit
Speed would be a good argument if you don’t cache the results of the build stage or frequently (no idea when that would be) changes in the dependencies
lukerm_zl@reddit
The implementation is razor fast. I think even with the cache mount (which is somewhat of a hidden gem), you stand to gain time on first (especially) and subsequent builds.
I'm not pushing that argument, just theorizing. Would love to know what OP's opinion is!
RepresentativeFill26@reddit
Just wondering. How could UV be faster than a cached layer in docker?
james_pic@reddit
In this case, it looks too be because uv already creates a venv in the builder image, and copying this across is the most straightforward way of bringing the app dependencies into the final image without pip or uv in the final image. I'm not sold on that being a worthwhile goal, but that looks too be the reason.
More generally, putting venvs into Docker images isn't a "you should always do this" thing, but it's sometimes a useful technique for solving specific problems, for example if your application sometimes calls Python problems provided by the base distro and you don't want to mess with their Python environment.
PickleSavings1626@reddit
this is the 3rd time today i've read that uv in docker is a pain or tricky. is there a misinformation campaign going around? it took me maybe 10min to get it working. a single COPY of the binary and life went on.
yes, it's always better to use a two-step build process. that's dockerfile 101. if the app doesn't need uv to run, it shouldn't be in the image.
Ambitious-Kiwi-484@reddit
Something I'd love to know a way around is that, since pyproject.toml contains my project's version number, it's not possible to bump the version without invalidating all further docker cache layers - which leads to slow builds since all deps are getting pulled again. This seems like an unavoidable caveat of copying pyproject.toml in an early layer. But there must be a workaround.
bublm8@reddit
Stumbled into this myself recently:
https://github.com/fslaktern/parcellocker/blob/main/src%2Fapi%2FDockerfile
ashishb_net@reddit (OP)
You don't have uv.lock file and that makes the build non-hermetic afaik.
bublm8@reddit
Yep, should've added it along with the pyproject.toml
Huberuuu@reddit
Wouldn’t the UV copy mode made the size bigger, not smaller? I understood that UV used hardlinks, so aren’t you duplicating packages on disk here?
1010012@reddit
No, the cache is mounted by docker only during the build, so not in the final image.
thrope@reddit
What's wrong with the official example? I use the standalone example here (which has multistage build and doesn't include uv in the final image).
ashishb_net@reddit (OP)
It depends on pyproject.toml when uv.lock will suffice
ashishb_net@reddit (OP)
The official example copies pyproject.toml when uv.lock would suffice and it is better to rely just on uv.lock.
thrope@reddit
Pyproject is not only the dependencies though, it includes other project metadata and configuration options for other tools?
ashishb_net@reddit (OP)
If you are building a python library then that's useful.
If you are building a docker image for d to deployment then there is nothing in that file worth mounting or copying.
Conscious-Ball8373@reddit
I don't use uv for this, but I find the packaging process rather painful. I end up with cryptography as a dependency quite often on a platform where mypi doesn't have a wheel for it. The build dependencies are huge and the runtime dependencies are non-trivial. I usually end up building a wheel for it in one stage and using it in another but I'm realizing I could avoid that by constructing a venv and copying that. Hmmm. Thanks for provoking the thought.
Fenzik@reddit
This is what you should be doing indeed
thrope@reddit
Don’t use uv for what? The question is about uv so I don’t follow.
Conscious-Ball8373@reddit
Never mind, just me missing out loud about my own situation, thoughts you prompted.
BelottoBR@reddit
Can we use pip to real the toml file? If so, we could use uv on local development and when deploy , just use pip. Can we ?
ashishb_net@reddit (OP)
Not really.
Uv handles various convoluted scenarios like installing only pytorch GPU/cpu version based on the underlying OS.
_squik@reddit
I've always looked at the official multistage Dockerfile example which has Astral's best recommendations.
ashishb_net@reddit (OP)
The official solution suggests copying pyproject.toml which I believe is a bad idea.
You only need uv.lock.
zacker150@reddit
I'm guessing you didn't scroll down to this part?
h4l@reddit
This and also use a cache mount to give uv access to previously-downloaded packages to speed up the install and also prevents the cache files remaining in the image:
Glathull@reddit
Thank you again for adding more examples to my TED talk about why no one should ever look to Reddit for best practices. It only is your example absolutely terribly implemented and doesn’t do what you said you wanted it to do, the things that you said you wanted it to do weren’t correct to begin with.
You are basically the worst of Reddit. Congrats, bro.
sonik562@reddit
Are you referring to OP? You are calling him the worst (a bit heavy don't you think?), but give no argumentation as to what is the recommended approach. Or why what he is asking is wrong.
Fluid_Classroom1439@reddit
Nice article! Entry point still says poetry for the example instead of uv