About Self-Operating Computer
Self-Operating Computer is a framework that enables multimodal models to operate a computer. It uses the same inputs and outputs as a human operator: the model views the screen and decides on a series of mouse and keyboard actions to reach an objective. The README says it was released in November 2023.
It installs with pip and starts with the operate command. By default it uses OpenAI's gpt-4o and asks for an OpenAI key, and other modes switch to models such as o1, gpt-4.1, Gemini Pro Vision, Claude 3, Qwen-VL and LLaVA. LLaVA can run locally through Ollama.
Beyond the default mode, the README describes voice input for the objective, an OCR mode that gives the model a map of clickable elements by coordinates, and Set-of-Mark prompting with a small YOLOv8 model trained for button detection. On a Mac, the terminal app must be granted Screen Recording and Accessibility permissions.
Key features
- Model views the screen and chooses mouse and keyboard actions
- Works with several multimodal models
- Installs with pip and starts with operate
- Voice input for the objective
- OCR mode maps clickable elements to coordinates
- Set-of-Mark prompting for visual grounding
- Local models via Ollama
Good fit for
- Experimenting with computer-use agents
- Comparing multimodal models on computer operation
- Voice-directed computer tasks
Self-Operating Computer: questions and answers
- What is Self-Operating Computer used for?
- Self-Operating Computer is a Python framework that lets a multimodal model view the screen and operate a computer with mouse and keyboard actions. It is a good fit for experimenting with computer-use agents, comparing multimodal models on computer operation and voice-directed computer tasks.
- Is Self-Operating Computer open source?
- Yes. Self-Operating Computer is open source under the MIT licence. Its source code is on GitHub at OthersideAI/self-operating-computer and is written mainly in Python.
- Is Self-Operating Computer free?
- Yes. Self-Operating Computer is open source, so the software itself is free to use.
- What is Self-Operating Computer an alternative to?
- Self-Operating Computer is an open-source alternative to Writer. Other open-source alternatives to Writer include Magic and Onyx.
- Is Self-Operating Computer actively maintained?
- Yes. The most recent commit to Self-Operating Computer was on 19 October 2026, and the latest release is v1.5.8, published on 28 February 2025. The project has 10k stars on GitHub, with 17 added in the last 30 days.
Open-source alternatives to Self-Operating Computer
See all
Magic
AI Tools
Magicrew. The first open-source all-in-one AI productivity platform (Generalist AI Agent +
OSSvs Lindy★ 5.1k
Onyx
AI Tools
The open-source AI enterprise search platform that helps teams find information across all
OSSvs Glean★ 33k
Kiro Crew
AI Tools
A persistent workspace for development work that self-improves and continues beyond one se
Apache-2.0vs Devin★ 4.9k
Skill Seekers
AI Tools
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with a
MIT★ 15kExcel MCP Server
AI Tools
A Model Context Protocol server for Excel file manipulation
MIT★ 4.3k
Nanobot
AI Tools
nanobot is an ultra-lightweight, open-source, self-hosted personal AI agent for chat apps,
MIT★ 50k
SaaS alternatives to Self-Operating Computer
See all
Writer
AI Tools
Enterprise generative AI platform with style guides, agents and company knowledge
SaaS
Anyword
AI Tools
AI copywriting platform with predictive performance scoring for ads and emails
SaaS
Copy.ai
AI Tools
AI workflow platform for sales and marketing content and go-to-market automation
SaaS
DeepSeek
AI Tools
AI chat assistant and API built on DeepSeek reasoning and language models
SaaS
Grammarly
AI Tools
AI writing assistant for grammar, tone and clarity across apps and browsers
SaaS
Jasper
AI Tools
AI content platform for marketing teams with brand voice and campaign tools
SaaS
