AI desktop agent that controls your OS (opensource, crossplatform)
Posted by IlyaZelen@reddit | Python | View on Reddit | 5 comments
https://github.com/777genius/os-ai-computer-use
What This Project Does
Local AI agent that lets control your entire desktop: mouse, keyboard, drag-and-drop across any application, with built-in vision of what's on the screen. Python backend + Flutter UI, runs fully on your machine.
Target Audience
Developers and users experimenting with computer-use AI. Functional MVP, actively developed.
Comparison
Browser agents (Browser Use, Playwright-based) only work inside browsers. OS AI operates at the OS level - automate Finder, Photoshop, System Settings, or any native app. Cross-platform (macOS/Windows/Linux), provider-agnostic architecture, no Docker required.
Built with Python. Provider-agnostic architecture - currently uses Anthropic, but designed to support OpenAI, Gemini and others. Plans: offline mode, execute cli commands on request. Your support motivates to develop the project ❤️
Amazing_Upstairs@reddit
Does it support local ollama?
IlyaZelen@reddit (OP)
This is currently under development to enable the model to be run locally with a focus on privacy.
The current clean architecture allows to easily integrate any model.
Amazing_Upstairs@reddit
Not local then is it
IlyaZelen@reddit (OP)
docs.anthropic.com/en/docs/build-with-claude/computer-use is currently used.
This is the smartest model for this purpose currently available. And the advantage is that this cloud-based solution doesn't overload the computer.
Sourabh7747@reddit
Great! Besides use of local LLM limits the task handing, highly hardware dependant, that's the tradeoff for privacy. Wish you all the best!!