GitHub Added Computer Use to Copilot CLI

The public preview allows Copilot agents to interact with desktop applications on macOS and Windows via UI automation.

Updated on Oct. 2, 2026 in Artificial Intelligence

Isometric editorial illustration of a mechanical cursor navigating a translucent lattice of abstract digital cubes representing software interface automation. — GitHub has launched public preview features for Copilot CLI, enabling AI agents to automate mouse and keyboard actions within desktop software environments. AI Illustration. Upload story photo >

Live Poll

Would you trust an AI agent to control applications and sensitive data on your computer?

GitHub has launched computer use capabilities for its Copilot CLI and desktop app in public preview. The feature allows AI agents to read, click, and type within desktop applications that lack APIs, functioning by interpreting the operating system accessibility tree and screenshots.

Why it matters

This capability enables AI to perform tasks in software environments that previously required manual human input or integration through code. As workflows move toward agentic interaction, GitHub expects users to navigate potential stalls or task repetition when handling dynamic interfaces.

The system processes visual context via screen captures and parses user interface elements through the operating system accessibility tree. This enables the agent to interact with desktop environments, though GitHub recommends utilizing direct tools when available for more predictable outcomes.

The players

GitHub

A Microsoft-owned platform providing cloud-based software development tools, version control, and AI-powered coding assistants.

OpenAI

A research laboratory and developer of large language models that provides the foundational models for many AI coding tools.

The details

The tool enables agents to manipulate desktop environments by simulating mouse and keyboard actions, including clicking, typing, scrolling, and dragging. On macOS, the system requires specific Accessibility and Screen Recording permissions to function. GitHub notes that while this extends control to legacy or closed-source applications, users may experience performance variability in dynamic workflows where interface elements change rapidly.

April 2026: OpenAI added computer use capabilities to the Codex model.
October 1, 2026: GitHub launched computer use in public preview.
October 22, 2026: New enterprise policy for unconfigured features takes effect.

The Tech Race

This development follows the trajectory established by OpenAI’s computer use integration for Codex earlier this year. GitHub is now scaling these agentic capabilities into its primary developer ecosystem, positioning its tools to interact directly with the desktop environment rather than just code files.

Developers can now enable the feature through the Copilot CLI or desktop app settings to automate repetitive tasks across various desktop applications. Users should note that settings are shared across devices, and enterprise administrators maintain the authority to block these permissions.

The takeaway

The move marks a shift toward AI agents that operate as general-purpose desktop users rather than just code-generation engines. Keep watch on the October 22, 2026 policy deadline to see how enterprise governance impacts the adoption of these agentic tools in professional settings.