AI & Machine Learning

The Remote Control Problem Nobody Talks About: Why AI Agents Need Better Mobile UX

A

Adil Sher

Author

Nov 11, 2026
5 min read
1 views
The Remote Control Problem Nobody Talks About: Why AI Agents Need Better Mobile UX

I was sitting in a client meeting in Lahore last month when my laptop, running Claude Code in the background, hit a permission wall. I had started an agent to refactor a database migration, left it running on my desk in Islamabad, and suddenly it was waiting for me to approve something. I checked Slack, refreshed my email, and realized I had zero way to respond from my phone without driving back home or SSHing into a messy terminal interface. That's when I started thinking about how broken this whole experience actually is.

The problem isn't that we can't interact with AI agents remotely. The problem is that we're duct-taping solutions together when we should be asking why this feels so janky in 2025. I read through the original article and had some thoughts about what's really happening here, and what we're missing.

The Remote Control Landscape Is a Fragmented Mess

Let me be direct: we have three separate solutions for one basic problem. Claude Code has /remote-control. Codex uses the ChatGPT desktop app's pairing system. And if you're running Grok or anything else, you're relegated to SSH + tmux. Each requires different setup, different apps, and different mental models.

I understand why this happened. Each company built their own stack, their own infrastructure, their own security model. Claude doesn't want to depend on OpenAI's infrastructure, and xAI isn't thinking about mobile control at all. But from a developer's perspective, this is exhausting. I shouldn't need to maintain three different apps and three different connection methods to answer the same type of question from my phone.

The SSH + tmux approach works universally, sure, but it's brutal. Typing Escape, Tab, and Ctrl combinations on a mobile keyboard while attached to a remote session? That's not a feature, that's punishment for wanting flexibility.

What Actually Works, and What Doesn't

Claude Code's Remote Control is the smoothest experience here. Once you run /rc, the session stays alive on your computer with full file access, MCP servers, everything intact. Your phone just acts as a view and a messenger. The constraints are reasonable too, it needs a higher-tier Claude plan, but if you're running agents seriously, you probably have that anyway.

Codex's approach through the ChatGPT apps is similarly practical, though it requires the desktop app setup ritual first. You're pairing devices, confirming accounts, managing permissions, it feels like the setup you'd expect from a company as large as OpenAI.

But here's where the design falls apart: both solutions stop working the moment your computer sleeps. This is the catch nobody wants to admit. You can't just leave an agent running in the background and expect to answer from your phone if your machine decides to nap. You need to actively prevent sleep, which means messing with system settings, maybe installing keep-alive scripts, definitely burning more power.

In a Karachi summer, that's not just inconvenient, that's actually expensive.

My Take: This Is a Band-Aid Over a Design Problem

I think what we're seeing here is a symptom of how AI agents weren't designed for async workflows. The entire architecture assumes a human at a keyboard, waiting for the next output. Remote control is basically us retrofitting that synchronous assumption onto a mobile, asynchronous world.

The real solution would be one of these:

First, agents should be able to queue approval requests and wait indefinitely. Not stop execution, not error out, actually wait. When you finally connect from your phone, hours later, the agent picks up context and asks its question again. That's hard because it means redesigning how agents handle state, but it's the right UX.

Second, there should be a unified remote control standard. Not necessarily a single company running it, but a protocol that Claude Code, Codex, and anything else can integrate with. Something like how SSH works, universal, secure, boring. PonyMux tries this, aggregating agents in one place, but that's still a third-party solution to a problem that shouldn't exist.

Third, mobile-first design should be built in from the start, not bolted on after launch.

What This Means in Practice

If you're actually running agents in production, here's what I'd do:

  • Use Claude Code's Remote Control if you're solo or on a small team. It's the least friction, and the UX is solid.
  • Keep your Mac awake deliberately. Don't pretend you'll SSH from your phone regularly, you won't, and you'll regret it when you try.
  • Consider whether you actually need to approve actions in real-time. Sometimes batching approvals and running agents fully autonomously (with good guardrails) is the better move.

The honest take? This feature exists, and it works, but we're all still figuring out the workflow. The tech is ready. The UX isn't yet.

What's Your Setup?

Are you actually using remote control for your agents, or have you found a different way to handle this? I'm curious what's worked for you and what hasn't. Drop a comment or reach out on Twitter if you've solved this better than I have.

Source: This post was inspired by "How to answer Claude Code or Codex from your phone" by Dev.to. Read the original article

Written by Adil Sher

Full stack developer building high-traffic platforms, AI services, and custom web applications. Explore my portfolio, learn about my background, or get in touch.

Related Articles

Why I Finally Stopped Trusting LLMs to Summarize My Documents
AI & Machine Learning Nov 8

Why I Finally Stopped Trusting LLMs to Summarize My Documents

Last month, I watched a financial audit report lose a transaction ID. A contract number just... vanished. The LLM had read it, understood it, but when it summarized the chunk, that specific fact didn't make it to the final output. My team caught it during QA, but the thought that...