Using ScreenTap

Tools

These are the tools ScreenTap gives your agent. You don't call them yourself: the agent picks what it needs.

screenshot	See the current screen.
tap	Tap a point on the screen.
tap_text	Tap a button or label by its text.
long_press	Press and hold, for context menus and rearranging.
swipe	Swipe in any direction.
scroll / scroll_to	Scroll, or keep scrolling until some text shows up.
type_text	Type into the focused text field.
press_key	Press keys like Return, Delete or the arrows.
open_app	Open any installed app by name.
batch	Run several steps in one go, for speed.
start_recording / stop_recording	Record the screen as a video, with taps drawn on top.
transcribe	Turn audio playing on the iPhone into text.
start_listening / get_transcript / stop_listening	Transcribe live audio, like a call or a video.
status	Check that the iPhone, screen and Bluetooth are connected.
calibrate	Re-measure the pointer if taps land off target.

Audio transcription runs on your Mac and is off until you turn it on in ScreenTap › Settings.