Copilot Can Operate Desktop Apps, Moving Work Forward on Screen

On October 1, GitHub released a preview of a feature that lets Copilot operate desktop apps. It can read the screens of apps a user has authorized and click, type, and scroll. This lets it work through the interface of older apps that have no programmatic way to control them.

AI Clicks Through an Expense Report

GitHub demonstrated Copilot using Safari to navigate an expense reporting screen. An AI assistant for writing code can now reach into browser and desktop interfaces as well. The example points to a way of handling tasks such as reading information in one document and entering it into another app from a single request.

The feature is in public preview in Copilot CLI and the Copilot app on macOS and Windows. It is off by default and asks for the user's permission before operating an app. Organization administrators can also disable it.

From Development Screens to Everyday Work

Technology media in Japan and Europe covered the announcement. Comparable engagement figures on X, YouTube, and other platforms could not be verified, so its level of global popularity remains unclear. The concrete change is that Copilot can now operate beyond the development environment.

It can check the results of its actions on screen and proceed within the bounds a person has authorized. Requests to AI are moving closer to “see this task through” than simply “write code.”

github.blog ↗ ↗

AI Can Now Carry Work Forward Across App Screens.

What caught my attention in today's Copilot announcement is that AI is taking on the role of someone using an app.

Read an expense reporting screen. Enter information. Move to the next screen.

AI can now follow tasks across app screens that people previously had to move between themselves. GitHub also announced a way to define workflow steps in code on the same day, suggesting another approach: set out the process in advance, then let AI carry it out.

It feels like AI's workplace has expanded.

People can set the boundaries of what they authorize while connecting steps across the apps they already use. Even at this preview stage, that way of working is coming into focus.

AI is becoming more than a place to get answers: it is becoming a way to move work forward across screens.

Claude Code / Anthropic

Medium impactMedium signal
Anthropic Plans to Train Engineers to Bring AI into Businesses
On October 2, Anthropic announced Claude Frontier Academy to train engineers who will lead AI adoption at companies. Its goal is to train 10,000 people by the end of 2027; that is a target, not a number already achieved.
anthropic.com ↗ ↗
Medium impactMedium signal
Anthropic Publishes Research on Work Robots Could Perform
On September 30, Anthropic published research comparing robot capabilities with the tasks involved in different jobs. The analysis is intended to help assess the current scope for automation; it does not report how many robots have actually been deployed.
anthropic.com ↗ ↗

OpenAI / ChatGPT

Medium impactMedium signal
OpenAI Publishes Practical Guide to Choosing a GPT-6 Model for the Job
On October 2, OpenAI published a guide to choosing among GPT-6 models based on use case, speed, and cost. It also covers how to manage long-running tasks and measure performance in production.
openai.com ↗ ↗
Medium impactMedium signal
Chatham Financial Uses AI for Trade Confirmation Work
On October 2, OpenAI published a case study describing how Chatham Financial built an internal tool using Codex and GPT-5.6. The reported reduction in trade confirmation time from 30 minutes to under four minutes comes from that case study.
openai.com ↗ ↗

AI Agents / Coding

Medium impactMedium signal
GitHub Copilot Can Follow Workflow Steps Defined in Code
On October 1, GitHub introduced “Dynamic workflows” as an experimental feature in Copilot CLI and the Copilot app. Users can define steps and checkpoints in code and leave parts that require judgment to AI.
github.blog ↗ ↗
Medium impactMedium signal
VS Code Copilot Adds Support for Scheduled Tasks and Post-Review Fixes
On October 1, GitHub summarized its September Copilot updates for VS Code. It is testing features that run recurring tasks automatically and help address review feedback.
github.blog ↗ ↗

AI INSIGHTS / KOHEI TORIGOE

Kohei Torigoe

AI Product Architect

Over 25 years in UI/UX design, product development and business development. Founded and led a company with a team of around 100 people.

Currently planning, designing and developing new products using AI.

Through AI Signal, I share discoveries from developing new products and the possibilities AI opens up for the future, while trying the latest AI technologies firsthand.

Profile, work and contact ↗

AI Video Generation

No new article was verified in this category today.

AI Music Generation

No new article was verified in this category today.

Foundation Models / Research

Medium impactMedium signal
Google Explores a New Way to Train AI While Protecting Data on Devices
On October 2, Google Research announced an approach to distributed learning using data on devices that aims to provide privacy guarantees that can be verified externally. The research moves some computation to servers to improve speed and support more devices.
research.google ↗ ↗

AI Automation / Business Use

No new article was verified in this category today.

AI Insights