argent-device-interact: a skill that lets your agent tap, swipe and type on a simulator

argent-device-interact teaches your agent to operate an iOS simulator, Android emulator or Chromium app by tapping, swiping, typing and taking screenshots.

Installs: 28K; on skills.sh
Stars: 3.1K; on GitHub
Files: 4; besides SKILL.md
Updated: Oct 2026; last repo commit

The skill sets the rules for using Argent's device tools. The same tool names work on every platform and pick iOS, Android or Chromium from the shape of the device id. The agent opens apps with launch-app or open-url, finds tap targets with describe first, and uses normalized 0.0 to 1.0 coordinates.

It triggers on tapping, gestures, scrolling, typing, hardware buttons, launching apps, screenshots and waiting for an element. It is not for TV targets. It needs the Argent tools, set up by the installer in the box above, and a booted device.

Good for

Mobile developers who want an agent to click through their app
QA engineers scripting UI checks without writing test code
Developers who need an agent to reproduce a bug on a device

Install argent-device-interact

Claude Code

Copy this prompt.

Install the argent-device-interact skill in Claude Code. Run this command:

npx @swmansion/argent@latest init

Then check that it loaded and tell me how to use it.

2
Paste it into Claude Code. It shows the change and asks before running it.

Requires Node.js.

This runs the Argent installer. It sets up the Argent tools (an MCP server) that these skills call, plus the Argent skills, in the editors you pick. Skills copied on their own with the skills CLI have no tools to call. The installer offers to add Argent tools to your editor's auto-approve list. The default Automatic skills option installs all Argent skills, not only this one. Telemetry is on by default and argent telemetry disable turns it off.

From the project README. Source.

What's in the skill

When it loads

Interact with an iOS simulator, Android emulator, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. Not for TV targets.

Sections in SKILL.md

Unified tool surface
1. Before You Start
2. Best Practices
3. Opening Apps
launch-app: by bundle ID
open-url: by URL scheme
4. Choosing the Right Tool
5. Finding Tap Targets
If describe Tool Fails
6. Tool Usage
gesture-tap: Single tap at a point
gesture-swipe: Straight-line gesture
gesture-pinch: Two-finger pinch
gesture-rotate: Two-finger rotation
gesture-custom: Custom touch sequence
button: Hardware button press

Key facts

Author: Software Mansion (Argent)
Repo: software-mansion/argent, path packages/skills/skills/argent-device-interact
Installs on skills.sh: 27,970 (as of Nov 2, 2026)
GitHub stars: 3,138 (shared repo, Nov 2, 2026)
License: Apache-2.0 for source code; some bundled binaries are proprietary
Last repo commit: Oct 31, 2026
Files: 5 files including SKILL.md: references/chromium.md, gesture-examples.md, harmonyos.md, secrets.md
Needs: The Argent tools (MCP server), set up by the installer in the box above

What it includes

gesture-tap, gesture-swipe, gesture-pinch, gesture-rotate, gesture-custom
Coordinates are fractions of the screen. Use gesture-swipe for lists and gesture-custom for long presses and drag and drop.
Tap and gesture tools
keyboard, paste, button, rotate, fold, shake
Tap a field first, then type. paste is only for things a user would paste, such as a one-time code. button presses home, back, power, volume.
Typing tools
launch-app and open-url
SKILL.md says never to tap home-screen icons to open an app.
Open apps
describe, debugger-component-tree, native-describe-screen
describe returns the accessibility tree with coordinates. Screenshots are the last resort for navigation.
Read · Find tap targets
await-ui-element and await-screen-idle
Blocks until an element is visible, hidden or has text, or until the screen stops changing, instead of polling screenshots.
Waiting
Batch several steps in one call
Allowed steps are taps, swipes, keys, typing, paste, rotate, shake, fold, tv-remote and await-ui-element. It stops at the first error.
run-sequence
Downscaled to 30% by default
Full-size PNGs are for baselines only. Points to argent-screenshot-diff for comparisons.
Files · Screenshots
{{secret:NAME}} placeholders keep passwords out of the chat
References/secrets.md says values come from ARGENT_SECRET_ variables, .argent/secrets.env, .env.local, .env or ~/.argent/secrets.env.
Read · Typing secrets
Android adb reverse, reinstall-app, Chromium tabs, HarmonyOS
Chromium adds chromium-tabs, chromium-cookies and chromium-storage. HarmonyOS needs `argent enable harmonyos` first.
Runs code · Platform notes

From the official docs

Permissions

Medium risk

It lets the agent tap, type and reinstall apps on a device, so it can change app data. Keep passwords in a secrets file and use the placeholder syntax, never paste them into chat.

What it can do

Read your data
Create and change things
Delete things
Read-only mode

Watch out for

The agent can act as you inside the app
Taps, typing and reinstalls can submit forms, send messages or change app data on the simulator. Use a test account and a throwaway device. github.com
Secret placeholders read local env and files
A {{secret:NAME}} placeholder is resolved from ARGENT_SECRET_ variables or from .argent/secrets.env, .env.local, .env and ~/.argent/secrets.env on the machine running the tool server. Put only test credentials there and gitignore the file. github.com
Installer can auto-approve Argent tools
`argent init` asks whether to add Argent tools to your editor's auto-approve list and its prompt defaults to yes. Say no if you want a confirmation on every tool call. github.com
Chromium targets expose cookies and storage
chromium-cookies and chromium-storage can read and write cookies, including HttpOnly ones, and web storage for the active tab. github.com
Android reinstall grants all runtime permissions
reinstall-app on Android installs with -g, so first-launch permissions are pre-granted. github.com
Argent telemetry is on by default
The README says telemetry is opt-out. Run `argent telemetry disable` to turn it off; the privacy notice says it collects no source code, file paths or tool inputs, and records a hashed installation identifier derived from a device identifier, never the raw one. github.com

Technical details

Talks to	Argent telemetry (on by default)
Sign-in and keys	The skill text does not use an account or API key.

Questions

What does argent-device-interact do?

It tells your agent how to control an iOS simulator, Android emulator or Chromium app with Argent's tools: open apps, tap, swipe, type, press buttons, wait for screens and take screenshots.

How do I install it in Claude Code?

Use the install box above. It runs the Argent installer, which sets up the device tools this skill calls along with the skills. The skill alone only has instructions.

Does it work on a real phone or a TV?

The skill covers iOS simulators, Android emulators and Chromium (CDP) apps, and a HarmonyOS reference if you enable it. It says it is not for TV targets and points to argent-tv-interact. paste is rejected on physical devices.

Is it safe to run?

It lets the agent tap and type in apps and read secret placeholders from local files, so use test accounts on a throwaway device. It bundles no scripts. The installer can also add Argent tools to your editor's auto-approve list, which you can decline.

How does it type passwords without exposing them?

Use a {{secret:NAME}} placeholder in the text. The value is read from the machine running the tool server and the result echoes only the placeholder. SKILL.md also says to skip screenshots of the filled field.

Why does the agent not tap app icons on the home screen?

SKILL.md says never to open an app that way. launch-app and open-url are instant and reliable.

How is it different from argent-test-ui-flow?

This skill covers the mechanics of each tool. argent-test-ui-flow is for interactive QA with a screenshot check after each action.

Information

updated Nov 2, 2026

Repository: software-mansion/argent
Folder: packages/skills/skills/argent-device-interact

software-mansion/argent skills.sh

Sources

Docs, issue trackers and discussions this page is built from.

Show all 8

SKILL.md · github.com
Argent README · github.com
Argent installer: editor adapters · github.com
Argent installer: auto-approve step · github.com
init-skills.ts · github.com
secrets.md · github.com
chromium.md · github.com
harmonyos.md · github.com