A developer has built a system that watches your computer screen using Gemma 4 - Google's latest open-source language model - and automatically generates reusable "skills" that any AI agent can later execute.
The system works by having Gemma 4 observe your on-screen activity, then convert those actions into structured skill definitions: step-by-step instructions stored in a shared library. Any compatible agent can access and run those skills, including agents that had nothing to do with the original recording session.
How the Self-Improvement Loop Works
The self-improvement mechanism is where the concept gets interesting. Agents can refine these auto-generated skills over time based on outcomes, creating a feedback loop where the library improves without you rewriting anything manually. This differs from fine-tuning - which means retraining a model on new data, an expensive and slow process - because skills are just structured text instructions, not modifications to the model itself.
Think of it like macros for AI agents. If you regularly run a research workflow - open tabs, scan sources, summarize findings - the system captures that sequence, packages it as a named skill, and lets any agent replay or adapt it later.
The entire stack runs on local hardware via Gemma 4, so screen data stays on your machine. For anyone handling sensitive documents or proprietary workflows, that's a concrete advantage over cloud-based approaches that require sending screen content to an external API.
The implementation is early and rough around the edges - this is a community project, not a polished product. The immediate audience is developers already running local AI agents. But the broader concept - agents that accumulate capabilities by observation rather than explicit instruction - is one of the more practical paths toward AI tooling that actually adapts to how a specific person works, rather than just responding to prompts.