Teach Your AI Agent to Browse the Web with Claude Computer Use

What you'll build

A Python agent that gives Claude eyes and hands inside a real Chromium browser: it takes screenshots, clicks, types, and scrolls through Playwright, looping autonomously until a multi-step web task — searching Wikipedia and reading the answer off the page — is done.

Prerequisites

Versions verified for this tutorial:

• Python 3.10+ (tested on 3.13)
• anthropic 0.120.2 and playwright 1.62.0 (Chromium 151 headless shell, installed by Playwright)
• An Anthropic API key with credit, from the Claude Console — exported as ANTHROPIC_API_KEY
• Model claude-opus-5 with the computer-use-2025-11-24 beta header

Computer use is a beta feature. Everything here runs headless on macOS, Linux, or Windows — no X11 or Docker required.

1. Understand what the API actually does

Computer use is a client-side tool: Claude never touches your browser. You declare a schema-less computer_20251124 tool, and Claude responds with tool_use blocks containing abstract actions — screenshot, left_click with a [x, y] coordinate, type with text, key for shortcuts, scroll, and so on. Your code executes each action, returns the result (screenshots go back as base64 PNG blocks), and calls the API again. That request → execute → respond cycle repeats while stop_reason is "tool_use" — the agent loop.

Anthropic's reference implementation wraps a whole Linux desktop in Docker — Xvfb, a window manager, Firefox. That's the right call for general desktop automation, but for web-only tasks a Playwright-controlled headless browser is a much smaller surface: no container, no display server, and the screenshot dimensions are exactly what you configure, which matters because Claude's click coordinates are pixel positions on the image you send.

One wrinkle for browsers: headless Chromium has no address bar, so Claude can't navigate by typing a URL into a screenshot. The API explicitly supports mixing custom tools into the same tools array, so we'll add a tiny navigate tool alongside the computer tool.

2. Install the dependencies

python3 -m venv .venv && source .venv/bin/activate
pip install anthropic playwright
playwright install chromium
export ANTHROPIC_API_KEY="sk-ant-..."

playwright install chromium downloads a ~95 MB pinned browser build; your system Chrome is not used.

3. Write the agent

Save this as browser_agent.py. It's complete — no elisions.

"""Minimal computer-use agent that drives a real Chromium browser via Playwright."""

import base64
import sys

import anthropic
from playwright.sync_api import sync_playwright

WIDTH, HEIGHT = 1280, 800
MODEL = "claude-opus-5"
COMPUTER_USE_BETA = "computer-use-2025-11-24"
MAX_ITERATIONS = 30

TOOLS = [
    {
        "type": "computer_20251124",
        "name": "computer",
        "display_width_px": WIDTH,
        "display_height_px": HEIGHT,
    },
    {
        "name": "navigate",
        "description": "Load a URL in the browser. Use this instead of an address bar.",
        "input_schema": {
            "type": "object",
            "properties": {"url": {"type": "string", "description": "Absolute URL to open"}},
            "required": ["url"],
        },
    },
]

# Claude emits xdotool-style key names; Playwright uses its own. Map the common ones.
KEY_MAP = {
    "return": "Enter", "enter": "Enter", "tab": "Tab", "escape": "Escape", "esc": "Escape",
    "backspace": "Backspace", "back_space": "Backspace", "delete": "Delete", "space": "Space",
    "up": "ArrowUp", "down": "ArrowDown", "left": "ArrowLeft", "right": "ArrowRight",
    "page_down": "PageDown", "page_up": "PageUp", "home": "Home", "end": "End",
    "ctrl": "Control", "control": "Control", "alt": "Alt", "shift": "Shift",
    "super": "Meta", "cmd": "Meta",
}


def to_playwright_keys(combo: str) -> str:
    return "+".join(KEY_MAP.get(k.lower(), k) for k in combo.split("+"))


class BrowserComputer:
    def __init__(self, page):
        self.page = page

    def screenshot(self):
        png = self.page.screenshot()
        return [{
            "type": "image",
            "source": {
                "type": "base64",
                "media_type": "image/png",
                "data": base64.b64encode(png).decode(),
            },
        }]

    def handle(self, name: str, action: dict):
        """Execute one tool call and return tool_result content."""
        if name == "navigate":
            self.page.goto(action["url"], wait_until="domcontentloaded")
            return f"Loaded {self.page.url}"

        kind = action["action"]
        if kind == "screenshot":
            return self.screenshot()
        if kind in ("left_click", "right_click", "middle_click", "double_click", "triple_click"):
            x, y = action["coordinate"]
            button = {"right_click": "right", "middle_click": "middle"}.get(kind, "left")
            clicks = {"double_click": 2, "triple_click": 3}.get(kind, 1)
            self.page.mouse.click(x, y, button=button, click_count=clicks)
            return f"{kind} at ({x}, {y})"
        if kind == "type":
            self.page.keyboard.type(action["text"])
            return f"Typed {action['text']!r}"
        if kind == "key":
            keys = to_playwright_keys(action["text"])
            self.page.keyboard.press(keys)
            return f"Pressed {keys}"
        if kind == "scroll":
            x, y = action["coordinate"]
            amount = action.get("scroll_amount", 3) * 100
            dx, dy = {"down": (0, amount), "up": (0, -amount),
                      "right": (amount, 0), "left": (-amount, 0)}[action["scroll_direction"]]
            self.page.mouse.move(x, y)
            self.page.mouse.wheel(dx, dy)
            return f"Scrolled {action['scroll_direction']} at ({x}, {y})"
        if kind == "mouse_move":
            x, y = action["coordinate"]
            self.page.mouse.move(x, y)
            return f"Moved mouse to ({x}, {y})"
        if kind == "wait":
            self.page.wait_for_timeout(action.get("duration", 1) * 1000)
            return "Waited"
        return f"Unsupported action: {kind}"


def run(task: str):
    client = anthropic.Anthropic()
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page(viewport={"width": WIDTH, "height": HEIGHT})
        computer = BrowserComputer(page)
        messages = [{"role": "user", "content": task}]

        for _ in range(MAX_ITERATIONS):
            response = client.beta.messages.create(
                model=MODEL,
                max_tokens=4096,
                tools=TOOLS,
                messages=messages,
                betas=[COMPUTER_USE_BETA],
            )
            messages.append({"role": "assistant", "content": response.content})

            tool_results = []
            for block in response.content:
                if block.type == "text":
                    print(f"[claude] {block.text}")
                elif block.type == "tool_use":
                    print(f"[action] {block.name}: {block.input}")
                    try:
                        result = computer.handle(block.name, block.input)
                        tool_results.append({
                            "type": "tool_result",
                            "tool_use_id": block.id,
                            "content": result,
                        })
                    except Exception as exc:
                        tool_results.append({
                            "type": "tool_result",
                            "tool_use_id": block.id,
                            "content": f"Error: {exc}",
                            "is_error": True,
                        })

            if response.stop_reason != "tool_use":
                break
            messages.append({"role": "user", "content": tool_results})

        browser.close()


if __name__ == "__main__":
    run(sys.argv[1] if len(sys.argv) > 1 else
        "Navigate to https://en.wikipedia.org, use the search box to search for "
        "'Alan Turing', and tell me his year of birth. Take a screenshot after "
        "each step to verify it worked.")

Four details are load-bearing:

• The viewport must exactly match display_width_px/display_height_px. Claude's click coordinates are pixel positions on the screenshot you sent; any mismatch and every click lands off-target.
• Key names need translation. Claude was trained on xdotool keysyms (Return, ctrl+l, Page_Down); Playwright wants Enter, Control+l, PageDown. The KEY_MAP bridges them.
• Failures go back as tool_result with is_error: true, not a crash — Claude reads the error and tries another approach.
• Append the full response.content back into messages. On claude-opus-5 thinking is on by default, and thinking blocks must be returned unchanged for the next turn to be accepted.

The screenshot-after-each-step instruction in the default task is lifted from Anthropic's prompting guidance — without it, Claude sometimes assumes an action worked instead of checking.

4. Run it

python browser_agent.py

Each loop iteration costs one API call carrying the conversation so far, and every screenshot is roughly a thousand input tokens, so a typical run of this task is 6–10 iterations. MAX_ITERATIONS is your cost circuit breaker. If you later move to an older model to cut cost, Anthropic's docs note that for computer use specifically, medium effort is the accuracy-to-cost sweet spot on Sonnet 4.6 and Opus 4.6 (set via output_config={"effort": "medium"}), while max adds tokens without improving UI accuracy.

Verify it works

You should see an action trace like this (Claude's exact wording and coordinates will vary):

[claude] I'll navigate to Wikipedia first.
[action] navigate: {'url': 'https://en.wikipedia.org'}
[action] computer: {'action': 'screenshot'}
[action] computer: {'action': 'left_click', 'coordinate': [583, 384]}
[action] computer: {'action': 'type', 'text': 'Alan Turing'}
[action] computer: {'action': 'key', 'text': 'Return'}
[action] computer: {'action': 'screenshot'}
[claude] Alan Turing was born in 1912 (23 June 1912, in Maida Vale, London).

Success looks like: a navigate call, at least one screenshot, click/type/key actions on the search box, and a final [claude] text line containing 1912 with no further [action] lines after it (meaning stop_reason flipped from tool_use to end_turn).

Troubleshooting

BrowserType.launch: Executable doesn't exist at .../chrome-headless-shell followed by "Looks like Playwright was just installed or updated." You installed the playwright package but not the browser, or installed it under a different virtualenv. Run playwright install chromium from the same venv that runs the script.

anthropic.BadRequestError: Error code: 400 saying tools.0.type: Input tag 'computer_20251124' ... does not match any of the expected tags. The tool version and beta header are mismatched. computer_20251124 requires betas=["computer-use-2025-11-24"] and a supporting model (Claude Opus 5, Sonnet 5, Opus 4.8/4.7/4.6, Sonnet 4.6, or Opus 4.5). Older models like Sonnet 4.5 and Haiku 4.5 need the computer_20250124 tool with computer-use-2025-01-24 instead.

anthropic.AuthenticationError: Error code: 401 - {'type': 'error', 'error': {'type': 'authentication_error', 'message': 'invalid x-api-key'}}. ANTHROPIC_API_KEY is unset, stale, or not exported into the shell running the script. echo $ANTHROPIC_API_KEY to confirm, and re-export after opening a new terminal.

Clicks consistently land in the wrong place. Your screenshots aren't the size you declared. This happens when adapting the code to a headed browser on a Retina/HiDPI display, where screenshots come back at 2x scale. Keep device_scale_factor at 1 (the headless default) or downscale screenshots to exactly WIDTH×HEIGHT before sending.

Next steps

Swap the default task for your own — form filling, price checks, multi-page research — but keep two of Anthropic's warnings in mind: content on web pages can prompt-inject the agent (the API runs injection classifiers on screenshots and will steer Claude to ask for confirmation), and anything consequential — purchases, logins, accepting terms — should require human sign-off. From here, try Anthropic's computer-use reference implementation, a Dockerized full Linux desktop with Firefox and a web UI; add "enable_zoom": true to the tool definition so Claude can inspect small text at full resolution; or bolt the schema-less bash_20250124 and text_editor_20250728 tools into the same tools array so the agent can save what it finds to disk.