I Built a Free AI Agent That Controls My Browser Hands-Free

What started as a 12-year-old’s science fair project at Starport became the backbone of my latest automation experiment. I found a way to control my browser, execute tasks, and manage entire workflows using nothing but hand gestures, voice commands, and completely free AI tools. No $200 monthly subscriptions. No sending my data to third-party servers. Just open-source software running locally on my machine.

Key Takeaways

• I built a hands-free browser control system using MediaPipe, free LLMs, and the Hermes AI agent — total cost: £0
• MediaPipe (Google’s free computer vision tool) tracks hand and eye movements through my webcam to control the browser
• I tested 5 free LLM models and found open-source alternatives scoring 75.6%-76.4% on benchmarks, rivaling paid models at $200-300/month
• The Hermes AI agent runs 24/7, manages Kanban boards, answers messages, and executes complex workflows across macOS, Windows, and Linux
• My complete setup: MediaPipe for gesture control → free LLM for reasoning → Hermes for task execution

How I Discovered This 12-Year-Old’s Project

I stumbled across this while browsing for new automation ideas. A young student at Starport had built a science project that used computer vision to control a computer browser. The simplicity struck me immediately — here was a child solving a problem that enterprises charge thousands to address.

I decided to rebuild and expand it. My version works two ways: I can control everything with hand gestures visible to my webcam, or I can simply speak commands and watch the system execute them. When I say “go to YouTube and reply to comments,” the browser opens, navigates, and handles the task. The system hears my voice, understands the intent, and performs the actions using AI models I’ve configured in the background.

Step 1: Setting Up MediaPipe for Gesture and Eye Control

The first component is MediaPipe, Google’s free computer vision framework that Google Developers has maintained since April 3, 2023. I’ve linked the GitHub repository in the comments section of this post.

MediaPipe divides your face or hand into specific tracking points, allowing camera-based browser control. The system recognizes where I’m looking, tracks my hand position, and translates those movements into cursor actions and clicks. Google has documented support across Android, Web, Python, iOS, and other platforms — worth reading if you’re building something similar.

For my setup, I run MediaPipe locally. Nothing leaves my machine. That’s a deliberate choice I make with all my automation tools now.

Step 2: Choosing the Right Free LLM (I Tested 5)

This was the research-heavy part. A follower had asked which free LLM performs best, so I treated this as a direct comparison test. Here are my findings:

Ornit — My Top Recommendation

I covered Ornit in my video two days ago (check my YouTube channel for the full setup walkthrough). It’s performing at levels I didn’t expect from a free model. Benchmark scores show it hitting 75.6% to 76.4%, placing it competitively against paid alternatives. I walked through the installation step-by-step in that earlier video.

GLM 5.2 — The Close Second

GLM 5.2 scores 81% versus Anthropic’s 85% on comparable tests. That’s a 4-point gap with a $200 price difference. I saw reports suggesting GLM may become paid soon, though I’m not certain. I’ll update my channel if that changes — another reason to subscribe for automation updates.

Timi 2.7 — The Coding Specialist

For code-writing tasks specifically, Timi 2.7 has become my go-to. I’ve featured it in multiple YouTube videos already. The open-source models available here handle scripting and development workflows better than general-purpose alternatives in my testing.

Here’s what surprised me most: when OpenAI released their 4.8 model at 75.1% and Anthropic’s comparable model scored 74.4%, the gap was less than one percentage point. I leave the math to you — is that worth $200 monthly?

Step 3: Installing the Hermes AI Agent

Hermes is the execution layer. While hundreds of browser agents exist, Hermes has proven the most reliable for my workflows. It’s completely free, runs 24/7, and operates across all three major operating systems.

I installed it directly on my machine (terminal installation is also available for command-line users). Hermes takes my instructions and performs the actual browser operations: opening tabs, navigating sites, generating images, answering messages, and advancing whatever task I’ve defined.

The critical requirement: you must know exactly what you want done. The agent executes precisely what you specify — clarity in your instructions determines your results.

My Complete Workflow Configuration

Here’s how the three layers connect in my daily use:

Input layer: MediaPipe captures my hand gestures or eye movements through the webcam, or I speak commands directly.

Reasoning layer: One of my tested free LLMs (usually Ornit or GLM 5.2) processes the instruction and determines the required steps.

Execution layer: Hermes receives the structured plan and manipulates the browser, applications, and backend systems to complete the task.

I’ve configured specific skill sets within Hermes, and it now runs 24/7 autonomously improving in the background. The Kanban board feature lets me track progress visually. When I want direct interaction, I use the built-in chat interface to converse with the agent and refine operations in real-time.

What This Actually Costs Me

Nothing. The LLMs are open-source and free. MediaPipe is Google’s free framework. Hermes is free. I downloaded everything directly to my browser and machine. My data stays local — I’m not forced to share information with external services.

This matters because I’ve watched the AI education space evolve since I first launched AI training programs in Turkey. The same information I share freely here gets repackaged and sold for thousands elsewhere. I built this blog and my YouTube channel specifically to remove those barriers.

FAQ

Do I need coding experience to set this up?

Basic technical comfort helps but isn’t required. I provide GitHub links and step-by-step installation commands in my video descriptions. The terminal installation for Hermes is a single command. MediaPipe has pre-built web implementations. If you can follow a tutorial, you can build this.

Which free LLM performed best overall?

In my testing, Ornit delivered the most consistent results across general tasks with benchmark scores of 75.6%-76.4%. For coding specifically, Timi 2.7 outperformed. GLM 5.2 sits between them at 81% but may transition to paid soon. Your choice should match your primary use case.

Is this system actually secure for business use?

Running locally provides significant security advantages over cloud-dependent tools. Your data never leaves your machine for LLM processing. However, standard security practices still apply: keep software updated, verify GitHub repositories, and test thoroughly before deploying on sensitive business systems.

Can this really run 24/7 without supervision?

Hermes can operate continuously, but I don’t recommend fully unsupervised deployment for critical tasks. I use the 24/7 capability for background research and content preparation, with review checkpoints before any publish or purchase actions. The Kanban tracking helps me monitor what the agent has queued.

Final Thoughts

This experiment confirmed what I’ve suspected: the gap between expensive AI services and capable open-source alternatives is narrowing faster than most realize. A 12-year-old’s science project, combined with mature free tools, produced a browser automation system that would have cost thousands in subscription fees just two years ago.

I’ve now applied this same stack across eBay, Amazon, Facebook, and software development workflows — over 25 different business models in my own operations. The tools are here. The information is free. What separates those who benefit from those who don’t is simply the willingness to build rather than buy pre-packaged solutions.

“The right question isn’t whether free AI can match paid alternatives. It’s whether you’re willing to spend 3 hours setting it up instead of $200 monthly forever.”

I’ll continue testing and sharing what I find. The landscape changes weekly, and my early access to training and beta systems here in London means I can usually get hands-on before these tools hit mainstream awareness. Subscribe to the channel, check the video comments for all links, and start building.

Watch the full video (in Turkish — English subtitles available):

Tools & Community

• TurkoLister — the AI listing tool I use to turn Amazon products into optimized eBay UK listings in about 60 seconds (from £4.99/month, £1 one-week trial).
• AI & E-commerce Community — my Turkish-speaking community ($19/month) with weekly live sessions.
• Subscribe on YouTube — new experiments every week.