I Tried to Build a GPU Teleprompter App With AI Agents. Four Generations Failed Before One Worked.

I’m a graphics programmer, and since COVID I’ve used a teleprompter for recording - which, if you’ve never set one up, is just a second monitor showing your content mirrored so it reads correctly through the beam-splitter glass. The commercial options I tried were either broken, bloated, or so heavy on the CPU that they’d stutter mid-take. So I did what I now do with most problems: I handed it to an AI coding agent and asked it to build me exactly what I wanted.

It took four generations of agents to get there. The failures are the interesting part, because they map almost perfectly onto where AI coding agents are strong and where they still fall down on genuinely hard systems work.

Quick answer

Building a GPU-heavy desktop app with AI agents is very doable in 2026 - but only if you treat the agent like a senior engineer working on your real machine, not a snippet generator. The task that finally succeeded needed the agent to diagnose why the previous attempt failed, rebuild from scratch, and verify its own math before handing it over. That only works when the agent has full access to your desktop’s toolchain, which is exactly what driving agents through MobileVibe gives you.

The problem was harder than it looked

“Mirror a monitor” sounds trivial. The real specification was not. I wanted:

• GPU-only processing - zero meaningful CPU load, so recording software and PowerPoint could run smoothly alongside it.
• A perspective warp, not just a horizontal flip, so the image could be corrected for a screen mounted at an angle.
• 60 frames per second, rock steady.
• PowerPoint compatibility, including presenter view.

Delivering that meant stitching together several low-level Windows graphics technologies that do not naturally cooperate: DXGI Desktop Duplication for capturing the screen at the GPU level, Direct3D 11 for the perspective warp, DirectComposition to present the result as an overlay, a pile of Win32 window-management calls, and the Magnification API to handle the cursor. Each one has sharp edges. Together they’re a minefield.

Why the first four agents failed

The early attempts all failed the same way, and it wasn’t for lack of knowledge about any single API. Each agent could explain DXGI or Direct3D fine in isolation. They failed at architecture - the way the pieces had to fit together across process and GPU boundaries.

An agent would get desktop duplication working, then bolt the warp on in a way that forced a round-trip through system memory, killing the frame rate. Or it would present the overlay correctly but lose the cursor. Or it would pass a quick visual check and fall apart the moment PowerPoint entered presenter mode. These are the classic failure modes of generating plausible code without holding the whole system in mind - each layer looked right, and the integration was quietly wrong.

This is the honest state of the art: on a well-bounded task, a good agent is superb. On a task where correctness lives in the seams between five subsystems, weaker agents produce something that compiles, runs, and is subtly, fundamentally broken.

What changed on the fifth attempt

The generation that succeeded - I was using Fable 5 through MobileVibe - did three things the others didn’t, and they’re worth naming because they’re the difference between a toy and a tool:

1. It diagnosed the previous attempt instead of ignoring it. Rather than starting cold, it read the broken version, identified why the architecture couldn’t hit 60fps, and used that as the design constraint.
2. It rebuilt from scratch rather than patching. Once it understood the flaw was structural, it didn’t try to bandage it - it re-architected around a GPU-only path with no system-memory round-trips.
3. It verified its own perspective math before declaring victory. The warp is just linear algebra, but getting the homography wrong gives you a subtly skewed image that looks almost right. The agent checked the math independently rather than trusting that the code “seemed fine.”

The result captures a full monitor, perspective-warps it, and re-presents it at a steady 60fps using essentially no CPU. It’s called Teleprompter Screen Warp, and I gave it away free.

Proof it’s all on the GPU: a full monitor captured, warped, and re-presented at 60fps while the app sits at essentially zero CPU.

The lesson: give the agent a real machine

None of that fifth-generation success is possible if the agent is reaching through a thin cloud sandbox. Diagnosing the old build meant running it. Verifying the warp meant executing real Direct3D against a real GPU. Hitting 60fps meant profiling on actual hardware. This is systems programming - it lives and dies on having the true environment, not an approximation of it.

That’s the core reason I drive agents on my own desktop and steer them from my phone rather than using a browser-based coding sandbox. The agent has my full toolchain, my GPU, my Windows APIs - everything it needs to build and check low-level graphics code. I just supply the intent and the judgment, and approve the steps that matter. If you’re curious how that setup works in practice, the MobileVibe docs walk through pairing a machine and running your first agent.

What you can take from this

If you’ve tried to build something ambitious with an agent and watched it produce confident garbage, the problem may not be the agent - it may be the task shape and the environment. Two things reliably move hard builds from failing to working:

• Make the agent diagnose before it generates. Point it at the broken version and ask why it fails before asking for a fix. You’ll get architecture, not band-aids.
• Give it the real environment. Agents can’t verify what they can’t run. On systems work, a true desktop with the actual hardware is the difference between “looks right” and “is right.”

I went from four dead prototypes to a free, production-quality graphics app that thousands of people now use for teleprompting, rear projection, and keystone correction. Same idea, same me - the variable was giving a capable agent a real machine and the room to think.

Want to try building your own hard thing? MobileVibe is free to start - pair your desktop and hand an agent a problem you’d assumed was too fiddly to bother with.

FAQ

Can AI agents really write low-level graphics code?

Yes, but the environment matters enormously. Agents excel at bounded graphics tasks and struggle when correctness spans several subsystems. Giving the agent a real GPU and toolchain to run and verify against dramatically improves results.

Why not just use a cloud coding sandbox?

Systems and graphics work depends on the actual hardware - GPU, drivers, OS APIs. A cloud sandbox can’t faithfully run or profile that. Driving an agent on your own desktop lets it build and verify against the real environment.

What made the fifth attempt succeed where four failed?

It diagnosed the previous failure before coding, rebuilt the architecture from scratch instead of patching, and verified its own perspective math before delivery - the habits of a senior engineer, applied on a real machine.