# rtrvr.ai (Retriever AI) — Full Documentation

> DOM-native AI web agent platform for browser automation.
> Full documentation corpus for LLM ingestion — every page concatenated below.
> Source: https://rtrvr.ai/docs · Per-page Markdown at https://rtrvr.ai/docs/<slug>.md

## Contents

- Getting Started: https://rtrvr.ai/docs
- Web Agent: https://rtrvr.ai/docs/web-agent
- Sheets Workflows: https://rtrvr.ai/docs/sheets-workflows
- Recordings & Grounding: https://rtrvr.ai/docs/recordings
- Tool Calling: https://rtrvr.ai/docs/tool-calling
- Knowledge Base (RAG): https://rtrvr.ai/docs/knowledge-base
- Enrichment Datasets: https://rtrvr.ai/docs/enrichment-datasets
- CLI & SDK: https://rtrvr.ai/docs/cli
- API Overview: https://rtrvr.ai/docs/api
- Agent API: https://rtrvr.ai/docs/agent
- Scrape API: https://rtrvr.ai/docs/scrape
- Browser as API/MCP: https://rtrvr.ai/docs/mcp
- Shortcuts: https://rtrvr.ai/docs/shortcuts
- Triggers: https://rtrvr.ai/docs/triggers
- Webhooks: https://rtrvr.ai/docs/webhooks
- Schedules: https://rtrvr.ai/docs/schedules
- Cookie Sync: https://rtrvr.ai/docs/cookie-sync
- Permissions & Privacy: https://rtrvr.ai/docs/permissions-privacy

---

# Overview

DOM-native AI web agent platform for browser automation

[rtrvr.ai Overview](https://www.youtube.com/embed/slMJm1qo0GU)

rtrvr.ai is a DOM-native AI web agent platform. Give it a task in plain English, and an AI agent executes it in a real browser — clicking, typing, navigating, and extracting data autonomously. Built by ex-Google engineers, it uses proprietary Smart DOM Trees and Chrome Extension APIs to achieve industry-leading accuracy without the detection issues of CDP-based solutions.

## Core Advantages

#### DOM-Native Intelligence

Text-only DOM trees instead of screenshots. Avoids bot detection, handles JS-heavy SPAs, faster than vision-based agents.

#### 81.4% Benchmark Accuracy

State-of-the-art on WebBench. Competitors score 40–66%. Built with Chrome Extension APIs, not detectable CDP.

#### No Polling Required

API responses return inline. No session management or async polling loops — simpler integration, lower latency.

#### Cookie Syncing

Sync your logged-in browser sessions from the Extension to Cloud browsers. Access authenticated sites at scale.

## What rtrvr.ai Handles

#### Data Extraction at Scale

Extract structured data from dynamic websites without building custom scrapers. Handles pagination, infinite scroll, and JS-rendered content.

#### Form Automation

Navigate multi-step forms with conditional logic. Upload files, fill fields, and submit — autonomously.

#### E-commerce Workflows

Search products, compare prices, add items to cart — all through natural language.

#### Lead Enrichment

Visit company websites row-by-row, extract contacts, funding data, and tech stack info into your CRM.

#### Web Monitoring

Schedule recurring checks on competitor pricing, content changes, or stock levels with alerts.

#### QA & Adaptive Testing

Test web apps without brittle selectors. The agent adapts to UI changes automatically.

Example prompts:

"Extract all product prices from this e-commerce page, including pagination"

"Visit each company website in my spreadsheet and extract CEO name and employee count"

"Fill out this multi-step insurance form using the attached JSON data"

"Monitor competitor pricing daily and alert me via Slack when prices drop"

## Integration Options

Choose the integration method that fits your stack. All share the same authentication and credit system.

[

#### Chrome Extension

Most powerful. Uses your logged-in browser — cookies, SSO, internal sessions. Record workflows and replay them.

](/docs/mcp)[

#### Cloud Platform

Managed browser cluster. Scale to thousands of URLs. Live VNC streaming. No local resources consumed.

](/cloud)[

#### REST API

/agent for end-to-end workflows, /scrape for raw DOM extraction. JSON in, JSON out. No polling needed.

](/docs/api)[

#### CLI & SDK

Unified CLI for cloud & extension. One command to run agents, scrape pages, and configure MCP.

](/docs/cli)[

#### MCP Server

Expose your browser as tools for Claude Desktop, Cline, or any MCP client. OAuth + API key auth.

](/docs/mcp)[

#### Webhooks

Real-time notifications when tasks complete or data is ready.

](/docs/agent#webhooks)[

#### n8n / Make / Zapier

Direct HTTP integration with no-code platforms. No polling — results returned inline or via webhook.

](/docs/integrations)

Method

Best For

Auth Sites

Chrome Extension

Logged-in automation, recording workflows

✅ Yes

Cloud Platform

Scaling to 1000s of URLs

✅ With cookie sync

REST API /agent

Programmatic end-to-end tasks

Via cloud browsers

REST API /scrape

Raw DOM for your own LLM pipeline

Via cloud browsers

MCP Server

Claude Desktop, Cline, MCP clients

✅ Your browser

Webhooks

Async triggers from n8n, Zapier, Make

Depends on endpoint

## Live Streaming & VNC

Cloud browser sessions provide a live stream URL you can embed in your own dashboards or watch directly. Take over sessions manually with full VNC interactive control when the agent needs human intervention — for credentials, CAPTCHAs, or edge cases.

💡 Embedding live views

Every cloud task returns a `liveUrl` you can iframe into any application for real-time monitoring.

## Cookie Syncing

Sync your authenticated browser sessions from the Chrome Extension to Cloud browsers. This lets you run automations on login-protected sites (Gmail, Salesforce, LinkedIn) at scale using your real cookies — no credential storage needed.

Agent API with synced session

```
// Cloud tasks automatically use synced cookies
{
  "input": "Export my last 30 days of orders from the vendor portal",
  "urls": ["https://vendor-portal.com/orders"],
  "response": { "verbosity": "final" }
}
```

## Get Started

[

#### Quick Start

First task in 2 minutes

](/docs/quick-start)[

#### API Reference

Complete endpoint docs

](/docs/api)[

#### Integrations

n8n, Make, Zapier guides

](/docs/integrations)[

#### Webhooks

Async notification setup

](/docs/webhooks)

#### Use rtrvr.ai in Claude Code or Codex

Download the rtrvr.ai skill — the full docs bundled for coding agents to load on demand. Unzip into `~/.claude/skills/` or `~/.codex/skills/`.

[Download skill (.zip)](/rtrvr-ai-skill.zip)

[

Next

Quick Start

](/docs/quick-start)

---

# Web Agent

The core DOM-native intelligence engine that powers every rtrvr.ai surface — extension, cloud, and API.

# The Web Agent

[Video: Web Agent Demo](https://www.youtube.com/watch?v=BqDy6st7iGA)

rtrvr.ai is a DOM-only AI agent designed to understand, navigate, and extract data from the web with human-like precision. Unlike screenshot-based tools that rely on vision models, rtrvr.ai reads the actual page structure — making it faster, more accurate, and far harder for sites to detect.

## Agentic Architecture

When you issue a command, the Planner agent orchestrates 20+ specialized sub-agents to complete the task:

- Action Agent — Handles clicks, typing, and navigation
- Extraction Agent — Identifies and structures data from the DOM
- Crawl Agent — Manages pagination and multi-page discovery
- PDF / File Agent — Reads and fills complex forms, uploads documents

> **info**: For repeated tasks, skip the planning step entirely by using Shortcuts or Replay for deterministic, consistent results.

## Example Prompts

```text
"Go to Amazon and find the price of iPhone 15 Pro"

"Fill out this contact form with my information"

"Add everyone on this event page as a professional connection"

"Extract all email addresses from this company's team page"

"Go to ChatGPT, ask about top restaurants in SF, extract the response"
```

> **info**: The agent handles complex multi-step workflows and will ask for clarification when needed. It operates identically whether running in the Chrome Extension, on Cloud headless browsers, or via the API.

## Multi-Tab Orchestration

The agent can reference multiple open tabs simultaneously — comparing data across sites, moving information between systems, or processing dozens of URLs in parallel via Sheets Workflows.

## Deterministic Replay

Once a multi-step workflow succeeds, you can replay it to bypass the planning phase entirely. Replay produces 100% consistent results and is the foundation for Shortcuts, Schedules, and Triggers.

## Free with Gemini

Save credits by using your own Gemini API key. In the extension, type /add-gemini-key and the agent will walk you through getting a free key from Google AI Studio and configuring it automatically. The system falls back to platform credits if Gemini encounters issues.

## Files & PDFs

Drag and drop files into the chat or attach them via the toolbar. The agent has specialized capabilities for PDFs: reading content, filling form fields, and generating new documents. It can also upload attached files to web pages when prompted (e.g., "fill this job application and upload my resume").

## Context Tips

Everything in your chat — messages, files, images — becomes context for the AI. Keeping context clean leads to better results.

- Start fresh chats for new tasks — stale context can confuse the planner
- Only enable the MCP/custom tools you actually need for the current task
- Use Personal Context in settings to store persistent info (resume, company details) the agent remembers across all chats
- Click the ✨ Enhance button (Cmd/Ctrl+P) to have a specialized model rewrite your prompt for clarity before sending

> **success**: Enhancing prompts lets you co-plan with the agent. It's especially powerful for complex multi-step tasks where precise instructions matter.

## Cloud & Mobile

The Chrome Extension is where you build and test workflows. Export successful workflows to rtrvr.ai Cloud for 24/7 headless execution — no laptop required. Recordings, tools, and KB context travel with the workflow automatically.

- Export any workflow from the extension to Cloud with one click
- Scale to thousands of parallel URLs via the API
- Connect your account to WhatsApp — send text or voice messages to trigger automations and receive results on your phone
- Share your best automations via link — recipients import with one click, including recordings, tools, and all context

## Platform Availability

| Capability | Extension | Cloud | API |
| --- | --- | --- | --- |
| Natural-language planning | ✅ | ✅ | ✅ |
| Multi-tab orchestration | ✅ | ✅ | ✅ |
| Authenticated site access | ✅ (your sessions) | ✅ (via Cookie Sync) | ✅ (via Cookie Sync) |
| Real-time feedback & steering | ✅ | — | — |
| Deterministic replay | ✅ | ✅ | ✅ |
| Files & PDF handling | ✅ | ✅ | ✅ |
| WhatsApp trigger & results | — | ✅ | — |
| Share & import workflows | ✅ | ✅ | ✅ |

---

# Sheets Workflows

Parallel processing for bulk data extraction — feed a spreadsheet, get structured results.

# Sheets Workflows

[Video: Sheets Workflows Demo](https://www.youtube.com/watch?v=Sas8iWMCBA4)

Process hundreds or thousands of rows by connecting a Google Sheet. The agent opens URLs in parallel sub-agent tabs, performs extraction or actions for each row, and writes results back in real-time. It combines the dynamic capabilities of AI agents with the deterministic structure of spreadsheets.

> **success**: We only access sheets you explicitly grant via the Google Drive Picker — we use the most restrictive drive.file scope. See Permissions & Privacy for details.

## How It Works

- Upload your spreadsheet or connect an existing Google Sheet with input data
- Define the task using a natural-language prompt
- Specify which columns to use as context (URLs, names, search terms, etc.)
- The agent executes the task for each row, opening parallel tabs as needed
- Results are written back to new columns in real-time

## Example: Lead Enrichment

Input sheet with company URLs. Prompt: "Visit the website and extract the CEO name and company size". To append data from marketplace datasets like LinkedIn or Crunchbase instead of scraping each site, see [Enrichment Datasets](/docs/enrichment-datasets).

```csv
Company Name,Website,Industry,CEO Name,Company Size
Acme Corp,acme.com,Software,John Smith,50-100 employees
Beta Inc,beta.io,Healthcare,Sarah Johnson,100-500 employees
Gamma LLC,gamma.co,Finance,Mike Wilson,10-50 employees
```

## Advanced Use Cases

- Price monitoring across multiple e-commerce sites
- Contact extraction from company team pages
- Social media profile analysis and data collection
- Competitive research and market analysis
- Form submissions with personalized data per row
- Content scraping with structured JSON output

## Tool Mapping in Sheets

You can map tool calls to run for every row. The agent intelligently pulls arguments from specific columns to populate tool parameters. Include details in your prompt like: "For each row upload as contact to HubSpot using the loadContact tool."

| Column A (Input) | Column B (Agent Action) |
| --- | --- |
| user@example.com | @hubspot_lookup(email=A1) |
| admin@test.org | @hubspot_lookup(email=A2) |

## Best Practices

- Use clear column headers that describe the data
- Include example rows to guide the agent's extraction pattern
- Break complex tasks into smaller, focused workflows
- Test with a small subset (5-10 rows) before processing large datasets
- Use rate limiting for respectful web scraping

> **warning**: Keep your prompts specific and include examples of expected output format for best results.

## Platform Availability

| Capability | Extension | Cloud | API |
| --- | --- | --- | --- |
| Google Sheets integration | ✅ | ✅ | ✅ (via dataInputs) |
| CSV / TSV upload | ✅ | ✅ | ✅ |
| Parallel sub-agent tabs | ✅ | ✅ | ✅ |
| Real-time sheet updates | ✅ | ✅ | — |
| Append mode (running logs) | ✅ | ✅ | ✅ |

---

# Recordings & Grounding

Show the AI how to perform complex tasks by example. Record once in the extension, reuse everywhere — cloud, API, shortcuts, schedules, and triggers.

# Recordings & Grounding

Sometimes showing is easier than explaining. Recordings let you ground the agent by performing a task manually once so the AI can mimic the exact DOM interactions. Record a workflow in the Chrome Extension, then deploy it anywhere — cloud batches, API calls, shortcuts, schedules, or triggers.

## What Gets Recorded

The recorder captures DOM interactions — clicks, typing, navigation, selections — not your screen. This makes it more robust than screen recording (no pixel-matching failures), more private (we never see your screen), and portable across different screen sizes and resolutions.

> **info**: Privacy: We only record DOM interactions. Your screen is never captured, stored, or transmitted.

## How to Record

- Click the Record button in the side panel toolbar (or search to name your recording)
- Perform the task naturally in your browser — click buttons, fill forms, navigate pages
- Optionally narrate what you're doing (add text annotations the agent can reference)
- Click Stop when complete
- Select the recording in chat to give the AI the perfect example

## Dramatic Success Boost

Providing a recording demonstration dramatically increases task completion rates. For complex multi-step forms and interactions that are hard to describe in words, a recording often outperforms even the most detailed prompt. Record once, use forever.

> **success**: Recordings often work better than detailed prompts for complex interactions — especially forms with unusual layouts, multi-step wizards, or sites with non-standard UI components.

## Cross-Platform Reuse

Recordings are a shared primitive — they're created in the extension but reusable across every rtrvr.ai surface. When you use a recording in a workflow, that recording is automatically bundled with the workflow artifact.

| Surface | How Recordings Are Used |
| --- | --- |
| Replay | Recording is included automatically — the agent follows the same steps |
| Shortcuts | Recording is bundled with the shortcut for perfect execution every time |
| Schedules | Scheduled runs include the recording — no drift over time |
| Triggers | Triggered workflows carry the recording for consistent execution |
| Cloud | Export to cloud — recording travels with the workflow for headless execution |
| API (replay_workflow) | Replay a workflow by ID — recording is resolved server-side |

## Management & Sharing

Recordings sync across your devices when logged in. You can organize, share them via URL with teammates, or import recordings from others.

- View all recordings from the Recordings section in the side panel
- Share via URL — recipients can import with one click
- Recordings bundle with workflows when shared, so teammates get the full context

## Platform Availability

| Capability | Extension | Cloud | API |
| --- | --- | --- | --- |
| Create recordings | ✅ | — | — |
| Use recordings in workflows | ✅ | ✅ | ✅ |
| Share recordings via URL | ✅ | ✅ | ✅ |
| Auto-bundle with shortcuts | ✅ | ✅ | ✅ |
| Sync across devices | ✅ | ✅ | — |

---

# Tool Calling

Extend your agent with MCP servers, AI Subroutines, custom JavaScript functions, and recording-grounded integrations — shared across extension, cloud, and API.

# Tool Calling

[Video: Tool Calling Overview](https://www.youtube.com/watch?v=cQN-4kJkjUQ)

rtrvr.ai agents aren't limited to browsing. They can call external APIs, query databases, and run custom logic using the Model Context Protocol (MCP) or secure JavaScript tools. Tools are a shared primitive — create them once, use them across the extension, cloud, and API.

## AI Subroutines

AI Subroutines are reusable tools generated by rtrvr.ai from natural-language intent, recordings, API docs, or the current webpage context. They are ideal when you want a stable capability such as “send a LinkedIn DM”, “create a HubSpot contact”, or “submit this internal form” without rebuilding the workflow each time.

- Generate a subroutine from a natural-language request
- Generate a subroutine from a recording to capture a real interaction flow
- Point the generator at API docs or an authenticated webpage and let it infer the tool shape
- Save the resulting tool once and reuse it from chats, workflows, schedules, and API-triggered runs

## Direct Tool Calls

When you know exactly which tool to use, bypass the AI planner with the @toolName syntax. This is highly efficient for repetitive tasks and gives you direct control.

```text
@act(action="click submit")              → Direct browser action
@extractToSheets(prompt="get emails")     → Fast extraction across open tabs
@hubspot_lookup(email="user@example.com") → Call a custom/MCP tool directly
```

## MCP Servers

Connect to any Model Context Protocol server by pasting its URL. Once connected, all MCP tools become available in your chat with @toolname syntax. rtrvr.ai supports SSE (Server-Sent Events), HTTP Streamable, and OAuth-protected servers.

- Open the Tools section in extension settings
- Paste the MCP server URL (e.g., your company's internal MCP, or a public one)
- Authenticate if required (OAuth flow handled in-browser)
- All server tools appear instantly — use them with @toolname in chat

> **warning**: OAuth-protected servers require periodic reauthentication. You can manage sessions in the extension settings or via the cloud dashboard.

## AI Tool Generator

Don't have an MCP server? The Tool Generator sub-agent can create AI Subroutines automatically. Just describe what you need in plain English — or better yet, point the agent at an API documentation page, an authenticated web app, or a recording and let it build the tool for you.

```text
"Find my HubSpot API keys and create a tool to load contacts"

"Use this onscreen API docs to create a tool that searches our internal database"

"Create a tool that calls the OpenWeatherMap API with a city name and returns the forecast"

"Turn this LinkedIn recording into an AI Subroutine that opens Message, writes a DM, and sends it"
```

> **success**: Tools created in the Chrome Extension can be saved and reused in the Cloud environment. They travel with shared workflows.

### Create a Subroutine From a Recording

Recordings are one of the fastest ways to create robust AI Subroutines. rtrvr.ai analyzes the DOM interactions and nearby network traffic, then chooses the best replay strategy: semantic DOM automation when the UI flow is the stable source of truth, or network-backed execution when the captured request is clearly first-party and reusable.

- Record the task once in the extension
- Open the recording and choose to generate a tool/subroutine
- The generator uses the recording as grounding context for buttons, composers, waits, and API shape
- Save the generated tool and call it directly with @toolName or from workflows

> **warning**: For volatile sites, DOM-grounded subroutines are usually safer than replaying internal GraphQL or hashed API endpoints. Recordings help the generator decide which path is actually stable.

## Custom JavaScript Tools

Write your own tools in JavaScript or let the AI build them. Subroutines can run as sandboxed JavaScript tools or as webpage tools that execute in the authenticated browser context. You can store API keys as default parameters — they remain local to your machine and are never sent to rtrvr.ai servers.

### Two Execution Styles

| Style | Best For | Runtime Implications |
| --- | --- | --- |
| Sandbox tool | Pure computation, public APIs, data transforms, authenticated APIs with local API keys, fetched HTML parsed with `DOMParser` | Runs in an isolated extension JavaScript runtime. It can use parameters, defaults, `fetch`, and `rtrvr.*` agent helpers, but it does not read the live page's `document`, `window`, `localStorage`, or application JavaScript state. |
| Webpage tool / AI Subroutine | Logged-in websites, DOM actions, page cookies, in-page fetch, rich editors, CSRF-protected requests | Opens or reuses a browser tab, waits for load plus any settle delay, injects the runtime, then runs in the authenticated page context with page `window`, `document`, cookies, localStorage, in-page fetch/XHR, and webpage `rtrvr` helpers. |

> **info**: Default to sandbox unless the tool needs the live page. Choose webpage when the result depends on browser session state, client-side rendering, DOM interaction, page cookies, localStorage, CSRF tokens, or the app's own JavaScript runtime.

### Webpage URLs and Parameters

Webpage tools need a page to run inside. At call time, URL selection uses this priority: `urls` parameter first, then `url`, then the saved `defaultUrls`. The runtime deduplicates the chosen URLs, opens each page, waits for load, applies `pageExecution.settleMs`, and injects `url`, `urls`, and `tabId` into your code.

- `defaultUrls` can contain static URLs, one URL per entry.
- Use simple `${paramName}` placeholders when the destination depends on a parameter.
- Only simple placeholders are supported. Do not use expressions like `${handle.replace('@', '')}`, ternaries, or nested paths.
- If a placeholder is unresolved, execution fails during URL resolution before opening a tab.
- If the parameter is a full URL, make it the whole template: `${profileUrl}`. If it is an ID or handle, build the URL around it: `https://x.com/${handle}`.

```javascript
// Static page target
defaultUrls: ["https://news.ycombinator.com/news"]

// Identifier parameter: the runtime opens https://x.com/rtrvr_ai
parameters: {
  handle: { type: "string", description: "X handle without normalization expressions" },
}
defaultUrls: ["https://x.com/${handle}"]

// Full URL parameter: use the parameter as the whole template
parameters: {
  profileUrl: { type: "string", description: "Complete profile URL" },
}
defaultUrls: ["${profileUrl}"]
```

`pageExecution.settleMs` controls the extra wait after the tab reports loaded. Leave it blank to use the runtime default of `1500` ms, set it to `0` to skip the extra wait, or increase it when a single-page app needs more time to render client-side content.

```javascript
executionMode: "webpage",
defaultUrls: ["https://app.example.com/orders/${orderId}"],
pageExecution: {
  settleMs: 2500,
},
code: `
const title = document.querySelector("h1")?.textContent?.trim();
return { url, urls, tabId, title };
`
```

### How Parameters Work

When you define parameters for a custom function, they become available as JavaScript variables in your code. The system automatically converts each parameter into a const declaration before running your code.

```javascript
// Parameters defined: firstName (string), lastName (string), apiKey (string, default)
// Your code — parameters are pre-declared as const variables:

const res = await fetch(`https://api.example.com/users?name=${firstName}+${lastName}`, {
  headers: { "Authorization": `Bearer ${apiKey}` }
});
const data = await res.json();
return data.results;
```

- Define parameters with names, types, and optional defaults
- Reference parameters directly as variables (firstName, not params.firstName)
- Parameters are injected as const declarations before your code runs
- Return any JSON-serializable value — async/await is fully supported
- console.log() output is captured and shown in results for debugging

> **info**: API keys stored as default parameters = local only, never sent to servers. This is the recommended pattern for authenticated tool calls.

### Code Examples

```javascript
// Simple greeting
return `Hello ${firstName} ${lastName}!`;

// Math
return price * quantity * (1 + taxRate);

// API call
const r = await fetch(url);
return await r.json();

// Transform
return items.map(i => i.name).join(", ");
```

Parameter names must be valid JavaScript identifiers (letters, numbers, underscores — no spaces).

### Webpage Helper Functions (`rtrvr`)

Webpage tools and AI Subroutines can use the built-in `rtrvr` helper namespace to interact with the live page without dropping down to brittle manual DOM code. These helpers are especially useful for authenticated flows, contenteditable editors, CSRF-protected requests, and waiting for UI state transitions.

| Helper | Use |
| --- | --- |
| `rtrvr.find({ role, name, text, placeholder })` | Find a semantic page target and return an opaque handle |
| `rtrvr.click(handleOrTarget)` | Click a previously found handle or a semantic target |
| `rtrvr.type(handleOrTarget, value, { clear, submit })` | Type into inputs or rich contenteditable editors |
| `rtrvr.waitFor(targetOrFn, { timeoutMs })` | Wait for the next UI state, modal, composer, or control |
| `rtrvr.waitForUrl(match, { timeoutMs })` | Wait for navigation or route changes |
| `rtrvr.request(url, init)` | Make authenticated in-page requests using the page context |
| `rtrvr.requestJson(url, init)` | Same as request, but parses JSON when available |
| `rtrvr.getCsrfToken()` | Read the current page CSRF token for sites that require it |
| `rtrvr.getCookie(name)` | Read a cookie value from the current page |

```javascript
// Example webpage tool body
const button = await rtrvr.find({
  role: "button",
  name: /Connect/i,
});

if (!button) {
  return { success: false, error: "Connect button not found." };
}

await rtrvr.click(button);

const csrfToken = rtrvr.getCsrfToken();
return await rtrvr.requestJson("/voyager/api/example", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-csrf-token": csrfToken,
  },
  body: JSON.stringify({ ok: true }),
});
```

### Agent Capabilities (`rtrvr.*` built-ins)

Both sandbox and webpage tools can call the agent's internal capabilities through the `rtrvr` namespace. These helpers share names and parameter shapes with the rtrvr MCP (for example `get_page_data`, `take_page_action`, `knowledge_base_query`), so code written against the MCP translates directly inline. They reuse the user's existing Google auth and credit allowance — don't hand-roll OAuth or tree parsing yourself.

> **info**: Complete built-in surface: `getPageTree`, `pageAction`, `listTabs`, `createSheet`, `appendRow`, `appendColumn`, `readSheet`, `createKB`, `listKB`, `queryKB`, `addToKB`, `createTool`, `customToolGenerator`, `listRecordings`, `startRecording`, `finishRecording`, `act`, `extract`, and `processText`. Tool-composition helpers are documented below: `callTool`, `listTools`, and `invokeBuiltin`.

#### Page data & actions

| Helper | Returns / Effect |
| --- | --- |
| `await rtrvr.getPageTree({ tabId?, onlyTextContent?, disableAutoScroll? })` | `{ tabId, url, title, tree, elementLinkRecord, accTreeId }` — same shape as MCP `get_page_data` |
| `await rtrvr.pageAction({ tool, args, tabId? })` | Run a single system tool on a tab; mirrors MCP `take_page_action` |
| `await rtrvr.listTabs()` | List browser tabs as `{ tabId, url, title, active, windowId }[]` |

Supported `pageAction` tools include: `click_element`, `type_into_element`, `type_and_enter`, `select_dropdown_value`, `clear_element`, `focus_element`, `hover_element`, `right_click_element`, `double_click_element`, `press_key`, `scroll_page`, `scroll_to_element`, `drag_element`, `drag_and_drop`, `adjust_slider`, `goto_url`, `go_back`, `go_forward`, `refresh_page`, `open_new_tab`, `switch_tab`, `close_tab`, `google_search`, `describe_images`, `discover_and_extract_network_data`, `copy_text`, `paste_text`, `upload_file`, `wait_action`, `wait_for_element`, `solve_captcha`. See Browser Tools for argument schemas.

#### Google Sheets

| Helper | Returns / Effect |
| --- | --- |
| `await rtrvr.createSheet({ title, tabTitle?, headers? })` | Creates a spreadsheet in the user's Drive. Returns `{ sheetId, sheetTab, sheetUrl }`. |
| `await rtrvr.appendRow({ sheetId, sheetTab?, values | rows })` | Append one row (`values`) or many rows (`rows`). Returns `{ appended, updatedRange, updatedRows, updatedCells }`. |
| `await rtrvr.appendColumn({ sheetId, sheetTab?, values, header? })` | Append a new rightmost column. Optional `header` is written before `values`. Returns `{ appended, column, updatedRange, updatedRows, updatedCells }`. |
| `await rtrvr.readSheet({ sheetId, range? })` | Read an A1 range. Returns `{ range, rows }`. |

#### Knowledge Base (RAG)

| Helper | Returns / Effect |
| --- | --- |
| `await rtrvr.createKB({ displayName })` | Create an empty KB store. Returns `{ storeId, storeName, displayName }`. Mirrors MCP `knowledge_base_create_store`. |
| `await rtrvr.listKB()` | List stores as `{ storeId, displayName, documentCount, updatedAt }[]`. Mirrors MCP `knowledge_base_list_stores`. |
| `await rtrvr.queryKB({ storeId, query })` | RAG query. Returns `{ response, citations }`. Mirrors MCP `knowledge_base_query`. |
| `await rtrvr.addToKB({ storeId, tabIds? })` | Index tabs (or the active tab) into a KB store. Mirrors MCP `knowledge_base_batch_index`. |

#### Recordings & tool creation

Use recordings when the agent needs an example of the real interaction path. `recordingId` is the convenient form; `recordingContext` is the lower-level backend field. Passing either one to `act`, `extract`, or `createTool` grounds that subroutine in the captured DOM and network context.

| Helper | Returns / Effect |
| --- | --- |
| `await rtrvr.listRecordings()` | List recording metadata as `{ recordingId, recordingName, captureTimestamp }[]`; full recording payloads are not returned. |
| `await rtrvr.startRecording({ name, tabId? })` | Start recording DOM interactions and network calls. Returns `{ recordingId, recordingName, state }`. |
| `await rtrvr.finishRecording({ timeoutMs? })` | Stop the active recording and wait for upload when possible. Returns `{ recordingId, recordingName, uploadStatus, metadata?, warnings? }`. |
| `await rtrvr.createTool({ userInput, tabIds?, recordingId?, recordingContext?, generationMode?, editContext? })` | Create a reusable custom tool with the custom tool generator. Defaults `generationMode` to `"save"`; generated tools should not call `createTool` recursively. |
| `await rtrvr.customToolGenerator({ userInput, tabIds?, recordingId?, recordingContext?, generationMode?, editContext? })` | Alias for `createTool`, kept for explicit access to the generator. Prefer `createTool` for new tools; do not generate tools that call either helper recursively. |

#### Agent sub-routines

When deterministic helpers aren't enough, you can invoke the same planner-backed sub-agents the chat experience uses. These helpers mirror the rtrvr MCP `act_on_tab`, `extract_from_tab`, and `processText` endpoints.

| Helper | Returns / Effect |
| --- | --- |
| `await rtrvr.act({ userInput, tabIds?, schema?, recordingId?, recordingContext? })` | Run the Act agent across one or more tabs (navigation, clicks, form fills), optionally grounded in a recording. Returns `{ data, warnings, creditsUsed }`. Mirrors MCP `act_on_tab`. |
| `await rtrvr.extract({ userInput, tabIds?, schema?, recordingId?, recordingContext? })` | Structured extraction across tabs, optionally grounded in a recording. Returns `{ data, jsonData, warnings, creditsUsed }`. Mirrors MCP `extract_from_tab`. |
| `await rtrvr.processText({ textInputs, taskInstruction, schema? })` | Summarize or transform text inputs via the LLM. Returns `{ data?, text?, warnings, creditsUsed }`. **Especially useful for parsing structured JSON out of unstructured strings** — pass any text blob plus a `schema` and the helper returns typed `data` matching it. |

> **info**: `rtrvr.processText` is the right reach whenever you have raw text (an email, HTML excerpt, model response, chat transcript) and need fielded JSON. It runs no browser tab and no DOM extraction — just LLM-backed string-to-schema parsing — so it's faster and cheaper than `rtrvr.extract` for text-only inputs.

> **info**: Agent sub-routine helpers burn credits the same way a chat invocation does. Prefer `rtrvr.pageAction` / `rtrvr.getPageTree` / `rtrvr.callTool` when the task is deterministic and doesn't need planning.

### Cloud Scrape at Scale (`rtrvr.scrape`)

When a task needs to scrape *many* URLs — roughly 30+, and especially 100+ — or hits sites that rate-limit or bot-block local access, `rtrvr.scrape` runs the whole list on rtrvr's cloud browsers behind rotating proxies instead of in the user's local browser. It chunks large lists into batches of 10 (the per-request cap) and fans them out with bounded concurrency, so a generated plan can hand it the full URL list in one call. The same proxy-backed infrastructure powers the standalone [Scrape API](/docs/scrape).

| Helper | Returns / Effect |
| --- | --- |
| `await rtrvr.scrape({ urls, settings?, timeoutMs?, response?, concurrency?, batchSize? })` | Cloud-scrape an array of URLs of any length behind rotating proxies. Returns `{ tabs, tabCount, succeeded, failed, usageData, creditsUsed, summary }`, where each `tabs[]` entry is `{ url, title, status, content?, tree?, error? }` in input order. `tree` is the same accessibility-tree string `rtrvr.getPageTree` returns; pair it with `rtrvr.processText` for structured extraction. |

> **warning**: `rtrvr.scrape` bills cloud browser + proxy **infra credits per URL** — far more than the FREE local `rtrvr.getPageTree` / `fetch`. Reach for it only when the volume is too high to run locally or local access is being blocked; for a handful of reachable URLs, loop `getPageTree` or `fetch` instead.

```javascript
// Bulk-scrape 100+ URLs that would be rate-limited locally, then parse the results.
const { tabs, succeeded, failed, creditsUsed } = await rtrvr.scrape({
  urls: productUrls, // any length — chunked into batches of 10 automatically
});

const rows = tabs
  .filter((t) => t.status === "success")
  .map((t) => [t.url, t.title ?? ""]);

console.log(`Scraped ${succeeded}/${tabs.length} (${failed} failed) for ${creditsUsed} credits`);
```

```javascript
// Compose planner-backed sub-agents with deterministic helpers.
const { data: leads } = await rtrvr.extract({
  userInput: "Pull each visible lead's name, email, and company",
  schema: {
    type: "array",
    items: {
      type: "object",
      properties: {
        name: { type: "string" },
        email: { type: "string" },
        company: { type: "string" },
      },
    },
  },
});

// processText shines for string -> JSON: pass raw text + a schema, get typed data.
const rawEmailBody = await rtrvr.callTool("fetchInboundEmail", { id: emailId });
const { data: parsed } = await rtrvr.processText({
  textInputs: [rawEmailBody.body],
  taskInstruction: "Extract the sender intent, requested meeting time, and any cited deal names.",
  schema: {
    type: "object",
    properties: {
      intent: { type: "string" },
      requestedMeetingTime: { type: "string" },
      dealNames: { type: "array", items: { type: "string" } },
    },
    required: ["intent"],
  },
});

const { sheetId } = await rtrvr.createSheet({
  title: "Lead outreach",
  headers: ["name", "email", "company", "intent", "deals"],
});

for (const lead of leads) {
  await rtrvr.appendRow({
    sheetId,
    values: [lead.name, lead.email, lead.company, parsed.intent, (parsed.dealNames || []).join(", ")],
  });
}

return { sheetId, count: leads.length };
```

#### Tool composition

A subroutine can call **any other tool the user has** — that includes user-authored custom tools and **every MCP server tool the user has connected**. The execution environment looks the tool up by name, dispatches MCP calls through the connected MCP client, and dispatches user tools through the same sandbox or webpage runtime that powers them in chat. There is no separate plumbing to wire up — connect a HubSpot MCP server once and `await rtrvr.callTool('hubspot.createContact', { email })` works inside any subroutine.

When the tool generator builds a subroutine, the prompt is given the user's full tool list (custom + MCP, with parameter schemas and `[MCP]` / `[custom]` badges) so the LLM can wire calls correctly without you re-pasting the spec. Use `await rtrvr.listTools()` at runtime if you ever need to introspect what's available.

| Helper | Returns / Effect |
| --- | --- |
| `await rtrvr.callTool(name, params)` | Call another custom tool or **MCP server tool** by name. Resolution: the runtime looks the tool up by name, validates `params` against its schema, and dispatches — MCP tools through the connected MCP client (e.g. HubSpot, Linear), user tools through the same sandbox/webpage runtime they use in chat. Returns the callee's raw result. |
| `await rtrvr.listTools()` | List every tool the current user has available to `callTool` — both custom tools and MCP server tools — each with `{ name, description, parameters, source: 'custom' | 'mcp' | 'predefined' }`. |
| `await rtrvr.invokeBuiltin(helper, args)` | Escape-hatch that invokes any builtin by name — equivalent to `rtrvr.<helper>(args)`. |

> **info**: Recursive tool calls are allowed up to a safety depth (5) and guarded against cycles. Prefer composing existing tools over re-implementing logic — especially for repetitive auth flows, MCP-backed CRM/ticketing actions, and sheet writes.

```javascript
// MCP + custom-tool composition in a single subroutine.
// Assumes the user has connected the HubSpot MCP server and authored a custom
// tool called `fetchInboundEmail`. Both surface as `rtrvr.callTool(name, params)`
// without any extra glue.

const email = await rtrvr.callTool("fetchInboundEmail", { id: emailId });

const { data: parsed } = await rtrvr.processText({
  textInputs: [email.body],
  taskInstruction: "Extract sender intent and any cited deal names.",
  schema: {
    type: "object",
    properties: {
      intent: { type: "string" },
      dealNames: { type: "array", items: { type: "string" } },
    },
    required: ["intent"],
  },
});

// Dispatched to the HubSpot MCP server by name — the runtime resolves it.
const contact = await rtrvr.callTool("hubspot.createContact", {
  email: email.from,
  firstName: email.fromName?.split(" ")[0],
  lifecycleStage: parsed.intent === "demo_request" ? "lead" : "subscriber",
});

return { contactId: contact.id, intent: parsed.intent, deals: parsed.dealNames };
```

```javascript
// Capture the active tab's tree, write a row to a new sheet, and index in KB.
const { tree, title, url } = await rtrvr.getPageTree({});

const { sheetId, sheetTab } = await rtrvr.createSheet({
  title: "Page Snapshots",
  headers: ["url", "title", "capturedAt"],
});

await rtrvr.appendRow({
  sheetId,
  sheetTab,
  values: [url, title, new Date().toISOString()],
});

const stores = await rtrvr.listKB();
const store = stores[0] || await rtrvr.createKB({ displayName: "Page Snapshots" });
await rtrvr.addToKB({ storeId: store.storeId });

return { sheetId, treeLength: tree.length };
```

```javascript
// Drive the page via named system tools — no DOM code needed.
await rtrvr.pageAction({
  tool: "goto_url",
  args: { url: "https://example.com/orders" },
});

await rtrvr.pageAction({
  tool: "wait_for_element",
  args: { text: "Order list" },
});

const { tree, elementLinkRecord } = await rtrvr.getPageTree({});
// Pick a link in the tree and follow it by element_id
await rtrvr.pageAction({
  tool: "click_element",
  args: { element_id: Object.keys(elementLinkRecord)[0] },
});

return { followed: true };
```

```javascript
// Ask a knowledge base a question and fan the answer into a sheet.
const [store] = await rtrvr.listKB();
if (!store) return { error: "No knowledge base stores yet" };

const { response, citations } = await rtrvr.queryKB({
  storeId: store.storeId,
  query: "Summarize the latest onboarding changes in one paragraph",
});

const { sheetId } = await rtrvr.createSheet({
  title: "KB digest",
  headers: ["question", "answer", "citationCount"],
});

await rtrvr.appendRow({
  sheetId,
  values: ["Latest onboarding changes", response, (citations || []).length],
});

return { sheetId, answer: response };
```

## Choosing DOM vs Network Tools

The best AI Subroutines pick the right execution strategy for the site. In general: use DOM actions when the UI flow is the stable contract, and use network-backed execution when the request itself is the stable contract.

| Prefer DOM Actions When... | Prefer Network Interaction When... |
| --- | --- |
| The site uses volatile internal APIs, rotating GraphQL IDs, or framework-specific hidden identifiers | You have a clear first-party endpoint with a stable request/response shape |
| You need to click, type, open menus, wait for dialogs, or work inside rich editors | The action is fundamentally an API call and does not depend on visible UI state |
| The only evidence you have is a recording of buttons, composers, and page transitions | You need structured JSON back and the site’s authenticated page context can safely make the request |

- For DOM tools, prefer semantic targets such as role, accessible name, placeholder, and visible text
- Use staged waits after each meaningful action: open dialog → wait for composer → type → wait for send button
- Avoid brittle CSS selectors and volatile class names in generated tools
- For network tools, prefer `rtrvr.request()` / `rtrvr.requestJson()` over copying full captured browser headers
- Never hardcode cookies, bearer tokens, CSRF tokens, or volatile query/operation IDs from a recording

> **info**: A good rule of thumb: if a human would describe the task as “click this, wait for that, then type here,” generate a DOM subroutine. If they would describe it as “send this authenticated request and parse the JSON,” generate a network-backed tool.

## Sheets Tool Mapping

In Sheets Workflows, the agent can map tool calls to run for every row — intelligently pulling arguments from specific columns. Just include instructions in your prompt:

```text
"For each row, use the loadContact tool to upload the email in Column A and name in Column B to HubSpot"
```

| Column A (Email) | Column B (Name) | Column C (Result) |
| --- | --- | --- |
| user@example.com | Jane Doe | ✅ Contact created (ID: 12345) |
| admin@test.org | John Smith | ✅ Contact created (ID: 12346) |

## Platform Availability

| Capability | Extension | Cloud | API |
| --- | --- | --- | --- |
| @toolname direct calls | ✅ | — | — |
| MCP server connections | ✅ | ✅ | ✅ |
| AI Tool Generator | ✅ | ✅ | — |
| Custom JavaScript tools | ✅ | ✅ | — |
| Sheets tool mapping | ✅ | ✅ | ✅ |
| Tools in replayed workflows | ✅ | ✅ | ✅ |

---

# Knowledge Base (RAG)

Index web pages into a searchable knowledge base powered by Gemini File Search — then query it from the extension, cloud, or during agentic workflows.

# Knowledge Base (RAG)

The Knowledge Base lets you index web pages into a Retrieval-Augmented Generation (RAG) store powered by Gemini File Search. Once indexed, you can query your stored documents during agentic workflows or chat with your knowledge base directly — perfect for research, documentation lookup, and building persistent context the agent can reference anytime.

[Video: Knowledge Base Overview](https://www.youtube.com/watch?v=kgvhLVElI5I)

## Adding Pages

Index any open browser tab into a knowledge base. Each page's content is extracted, chunked, and embedded for semantic search.

- Click the Database (KB) icon in the chat toolbar
- Create a new knowledge base or select an existing one
- Click "+" on a store to open the Add Pages modal
- Select one or more open tabs and click "Add Pages"
- Pages are indexed and searchable within seconds

> **info**: You can also add pages during an agentic workflow — the agent can index pages it visits on your behalf.

## Updating Indexed Pages

Re-index a URL at any time to refresh its content. If you add a page that's already in the store, the old version is automatically replaced — no duplicates. Great for pages that change frequently like dashboards or documentation.

## Chat with Your Knowledge Base

Switch to Knowledge Base mode using the Agent Mode selector (bottom-left of chat input). When a KB is selected, the mode automatically switches to "KB" — all your questions are answered from your indexed documents with citations and source references.

- Select a knowledge base from the Database dropdown
- The Agent Mode automatically switches to "KB"
- Type your question — the agent searches your indexed documents
- Responses include citations linking back to source pages

```text
Example:
  Index 20 API docs → Ask "How do I authenticate with OAuth?"
  → Get a cited answer pulled from your indexed documentation
```

## KB as Agent Context

Beyond direct chat, the knowledge base serves as context during full agentic workflows. When a KB is selected and you use Auto (Planner) mode, the agent can reference your indexed documents while planning and executing multi-step tasks — grounding its decisions in your specific documentation.

## Managing Stores & Documents

- Click the Database icon to open the KB dropdown
- Click the chevron to expand a store and view its documents
- Use the "×" button on a document to remove it
- Use the trash icon to delete an entire knowledge base

## Credits & Pricing

Knowledge base operations always use platform credits — your personal Gemini API key is not supported for KB operations due to permissioning requirements.

| Operation | Cost |
| --- | --- |
| Indexing pages | ~15 credits per 1M tokens (based on Gemini embedding rate) |
| KB queries | Standard model pricing (Flash is cheapest) |

> **warning**: Your own Gemini API key is always ignored for KB operations. Platform credits are always consumed.

## Tips for Best Results

- Index documentation, articles, and text-rich pages for best results
- Use descriptive store names to organize by project or topic
- Re-index pages after major content updates
- Start with Flash model for queries — upgrade to Pro for complex reasoning over your docs
- Clean, text-heavy pages work best — interactive apps or heavily JS-rendered pages may yield less useful results

## Platform Availability

| Capability | Extension | Cloud | API |
| --- | --- | --- | --- |
| Create / manage KB stores | ✅ | ✅ | — |
| Index pages into KB | ✅ | ✅ (agent can index during execution) | — |
| Chat with KB (direct query) | ✅ | ✅ | — |
| KB as agent context | ✅ | ✅ | ✅ |
| Sync across devices | ✅ | ✅ | — |

---

# Enrichment Datasets

Append external data to your sheets and extractions using Bright Data marketplace datasets — LinkedIn, Crunchbase, Instagram, Amazon, Zillow, Indeed, and Google Maps.

# Enrichment Datasets

Retriever Enrich appends external data to your extracted results and sheets by matching them against [Bright Data's dataset marketplace](https://brightdata.com/products/datasets) — LinkedIn people and companies, Crunchbase, Instagram, Amazon, Zillow, Indeed, and Google Maps. It runs inside a code plan via the `rtrvr.enrich(...)` helper, is billed in credits at cost, and every paid call asks for your approval first.

> **info**: Enable Retriever Enrich under Settings → Labs to use it. Discovering datasets and inspecting their fields is always free — you only spend credits on a lookup or scrape you explicitly approve.

## Available Datasets

Reference a dataset in a plan with the `rtrvr.datasets` constants (e.g. `rtrvr.datasets.linkedinPeople`). The links below open each dataset's page in the Bright Data marketplace.

| Dataset | `rtrvr.datasets` key | Modes |
| --- | --- | --- |
| [LinkedIn people profiles](https://brightdata.com/products/datasets/linkedin/profiles) | `linkedinPeople` | Instant lookup + Live scrape |
| [LinkedIn people + business contact info (emails)](https://brightdata.com/products/datasets/linkedin/profiles) | `linkedinPeopleContact` | Instant lookup + Live scrape |
| [LinkedIn company information](https://brightdata.com/products/datasets/linkedin/company) | `linkedinCompanies` | Instant lookup + Live scrape |
| [Crunchbase companies](https://brightdata.com/products/datasets/crunchbase) | `crunchbaseCompanies` | Live scrape only |
| [Instagram profiles](https://brightdata.com/products/datasets/instagram/profiles) | `instagramProfiles` | Live scrape only |
| [Amazon products](https://brightdata.com/products/datasets/amazon) | `amazonProducts` | Live scrape only |
| [Zillow properties](https://brightdata.com/products/datasets/zillow) | `zillowProperties` | Live scrape only |
| [Indeed job listings](https://brightdata.com/products/datasets/indeed) | `indeedJobs` | Live scrape only |
| [Google Maps businesses](https://brightdata.com/products/datasets/google-maps) | `googleMaps` | Live scrape only |

> **warning**: Live-scrape datasets enrich rows you already have a URL for — an Amazon product page, a Zillow listing, an Indeed posting, a Google Maps place. They cannot search or discover ("find homes in 78704"). Only the three LinkedIn datasets support instant lookup by field; Bright Data's Search API rejects every other dataset id outright.

## Fields

Each dataset exposes dozens of fields. The most commonly used ones are below — call field discovery (`rtrvr.enrich({ dataset, fields: true })`, free) to get the full, live list before you filter, since wrong field names fail the job.

| Dataset | Key fields |
| --- | --- |
| `linkedinPeople` / `linkedinPeopleContact` | `id` (profile slug — the lookup key), `url`, `name`, `position` (role/title), `current_company_name`, `city`, `about` (bio), `experience`. The contact variant adds `email` and `cellphone_number` — partial coverage, most profiles return null for both. |
| `linkedinCompanies` | `url`, `name`, `website`, `industries`, `company_size`, `headquarters` |
| `crunchbaseCompanies` | Company profile fields — name, about, industries, founded date, size, region, website, socials — use field discovery for the full list |
| `instagramProfiles` | Profile fields — profile name/id, followers, posts, website, bio, verification — use field discovery for the full list |
| `amazonProducts` | `asin`, `url`, `title`, `brand`, `initial_price`, `currency`, `availability`, `reviews_count`, `categories` — 113 fields in total, use field discovery |
| `zillowProperties` | `zpid`, `url`, `address`, `price`, `bedrooms`, `bathrooms`, `homeStatus`, `yearBuilt` — 140 fields in total, use field discovery |
| `indeedJobs` | `jobid`, `url`, `job_title`, `company_name`, `location`, `salary_formatted`, `job_type`, `date_posted_parsed`, `description_text` |
| `googleMaps` | `place_id`, `url`, `name`, `address`, `category`, `reviews_count`, `open_hours`, `business_details` |

## Two Modes: Instant Lookup vs Live Scrape

Enrichment runs one of two ways. Instant lookup queries Bright Data's pre-collected index and is the default for people and company enrichment; live scrape fetches fresh data on demand for exact URLs or datasets the index doesn't serve.

|  | Instant Lookup | Live Scrape |
| --- | --- | --- |
| API | Bright Data Search API | Web Scraper API (`/scrape`, sync) |
| Speed | Sub-second | ~10–30s per input |
| Cost | 0.25 credits / returned record | 0.15 credits / successful record |
| Datasets | The 3 LinkedIn datasets only | Any scraper-backed dataset (Amazon, Zillow, Indeed, Google Maps, Crunchbase, Instagram) |
| How to call | `{ dataset, filter, fields }` | `{ dataset, scrapeUrls, fields }` |
| Best for | Bulk lookups by field (e.g. profile `id`) | Exact URLs, non-LinkedIn data, or lookup misses |
| Limits | Pass all keys in one `in` filter — batching is server-side | Max 20 inputs per call — chunk larger lists with `rtrvr.mapLimit` |

### Slow Scrapes: Pending Jobs

Bright Data serves a scrape inline only while it fits their one-minute window. A heavier job (20 Zillow listings, say) hands back a job handle instead, and the call returns `{ pending: true, snapshotId }` having billed nothing. Do other work, then collect the records later in the same plan:

```js
let { records, pending, snapshotId } = await rtrvr.enrich({
  dataset: rtrvr.datasets.zillowProperties,
  scrapeUrls: listingUrls,          // max 20 per call
  fields: ['zpid', 'address', 'price', 'bedrooms'],
});

// ...do other free work here while Bright Data finishes...

if (pending) {
  ({ records, pending } = await rtrvr.enrich({ collect: snapshotId, fields: ['zpid', 'address', 'price', 'bedrooms'] }));
}
// Never re-scrape a pending job — collecting it is how you avoid paying twice.
```

## Cost & Credits

- Instant lookup — 0.25 credits per returned record; zero matches cost nothing.
- Live scrape — 0.15 credits per successful record; failed inputs are free.
- Discovery — listing datasets and inspecting fields is always free.
- Pending jobs — a scrape that exceeds the sync window bills nothing until you collect it.
- Credits are billed at cost (1 credit = $0.01), and every paid call is estimated and approved before it runs.

## Approval Flow

Retriever Enrich never spends credits without your explicit go-ahead. For every paid run, the agent:

- Builds the entity list locally for free (from your sheet rows or extracted data).
- Estimates the credit cost (records × the per-record rate).
- Asks you to approve the spend via a prompt inside the task.
- Runs the paid lookup or scrape only after you approve, then joins the results back onto your rows.

## Example: Enrich LinkedIn Profiles

You do not have to write this yourself — ask in plain English (e.g. "enrich these LinkedIn profile URLs with name, title, and company") and the agent generates and runs the plan. The code below is what it runs under the hood.

```js
// Build lookup keys from your sheet rows — match on the lowercase LinkedIn
// profile slug (the segment after /in/), NOT the full URL.
const ids = rows
  .map(r => r.linkedinUrl?.split('/in/')[1]?.split(/[/?#]/)[0].toLowerCase())
  .filter(Boolean);

// Estimate cost and get explicit approval before any paid call.
const estCredits = Math.ceil(ids.length * 0.25);
const { answers } = await rtrvr.askUser({
  questions: [{
    key: 'enrichApproval',
    query: `Enrich ${ids.length} LinkedIn profiles for ~${estCredits} credits?`,
    choices: ['Yes', 'No'],
  }],
});
if (answers.enrichApproval !== 'Yes') return { summary: 'Enrichment declined.' };

// Instant lookup — pass ALL ids in ONE `in` filter (batching is server-side).
const { records } = await rtrvr.enrich({
  dataset: rtrvr.datasets.linkedinPeople,
  filter: { name: 'id', operator: 'in', value: ids },
  fields: ['id', 'name', 'position', 'current_company_name', 'city'],
});
// records → join back onto your rows on `id`, then write the enriched columns.
```

> **warning**: Match on the lowercase `id` slug, not the full profile URL — stored URLs vary by country subdomain and trailing slash, so URL equality matches nothing. Always pass `fields` to keep records small (full records are tens of KB each); the join keys `id`, `url`, and `input_url` are always returned so you can match records back to your rows.

## Enriching LinkedIn URLs with Emails

To add emails and phone numbers, use the `linkedinPeopleContact` dataset — same instant lookup by `id` slug, but request the `email` and `cellphone_number` fields explicitly (they are dropped unless named). Coverage is partial: this is partner-sourced B2B contact data, and most profiles return null for both, so treat any email you get as a bonus rather than a guarantee.

```js
const { records } = await rtrvr.enrich({
  dataset: rtrvr.datasets.linkedinPeopleContact,
  filter: { name: 'id', operator: 'in', value: slugs },
  fields: ['id', 'name', 'position', 'current_company_name', 'email', 'cellphone_number'],
});
const filled = records.filter(r => r.email).length;
// e.g. "12 of 60 profiles had an email" — a blank email on a matched row is
// normal, not a failed lookup.
```

> **info**: For rows the contact dataset leaves blank, the free supplement is fetching the person's company website contact/about page — the same technique the lead-generation recipe uses. There is no field named `headline`: `position` is the role/title and `about` is the bio.

## Example: Enrich Companies by Domain

The `linkedinCompanies` dataset is instant-searchable and carries a `website` field, so a column of company domains can be enriched with firmographics in a sub-second lookup — no scraping required.

```js
const { records } = await rtrvr.enrich({
  dataset: rtrvr.datasets.linkedinCompanies,
  filter: { name: 'website', operator: 'in', value: domains },
  fields: ['name', 'website', 'industries', 'company_size', 'headquarters'],
});
// Domains are stored as full URLs, so `in` can miss on scheme or a leading
// "www.". If a domain returns nothing, retry it with the `includes` operator.
```

## Example: Enrich Amazon Products by URL

Non-LinkedIn datasets are live-scrape only: you supply each product's URL and get a fresh record back. Chunk lists larger than 20 with `rtrvr.mapLimit`.

```js
const chunks = [];
for (let i = 0; i < productUrls.length; i += 20) chunks.push(productUrls.slice(i, i + 20));

const results = await rtrvr.mapLimit(chunks, 2, async chunk =>
  rtrvr.enrich({
    dataset: rtrvr.datasets.amazonProducts,
    scrapeUrls: chunk,
    fields: ['asin', 'title', 'brand', 'initial_price', 'availability'],
  }),
);
// Each result may come back `pending` — collect it with { collect: snapshotId }.
// Join back onto your rows on `url` (or `input_url`, the URL you passed in).
```

## Web Search

Plans can also search the web programmatically with `rtrvr.webSearch({ query, count?, gl?, hl? })` — one Google query per call, returning parsed organic results (`{ title, url, snippet }`) and, for place-flavored queries, `localResults` (Google's local pack: name, address, rating, reviews, category, and a canonical maps link). Phone numbers and websites are not in the pack — they come from each business's own site or listing. It costs a flat 0.15 credits per query with no approval prompt, and replaces navigating a browser tab to Google. Localize with `gl` (country) and `hl` (language).

## Recipe: Local-Business Lead Generation

The pieces compose into a lead-generation pipeline — ask in plain English (e.g. "find 100 businesses in Casablanca and Rabat that likely need a website") and the agent runs this shape:

- Fan `rtrvr.webSearch` over each industry × city (localized, e.g. `gl: 'ma', hl: 'fr'`) and harvest the local pack — business name, phone, website (or its absence), rating, maps link.
- Construct a Google Maps link per business: `https://www.google.com/maps/search/?api=1&query=Name+City` resolves to the real place page.
- Enrich businesses that have a website with a free fetch of their contact/about page (email, socials), or instantly via the `linkedinCompanies` dataset filtered on `website`.
- Qualify with one LLM pass (`rtrvr.inferSheetData`) — e.g. a "why they need a website" column and a priority score.
- Rows append to a Google Sheet as they are found, so partial progress always survives; export the sheet as CSV when done.

> **info**: Expect partial email coverage on local-business runs — many small businesses publish no email anywhere, and the strongest leads (no website at all) usually have none. Phone numbers and maps links fill far more reliably.

---

# rtrvr CLI & SDK

One CLI, three execution modes. Browser automation for AI agents and humans — across cloud, local browser extension, or auto-routed.

[npm → @rtrvr-ai/cli](https://www.npmjs.com/package/@rtrvr-ai/cli)[GitHub](https://github.com/rtrvr-ai/rtrvr-cli)

Cloud Mode

Run agents and scrape via rtrvr's managed cloud browsers.

Extension Mode

Route to your local browser extension for authenticated sessions.

Auto Mode

Smart routing with automatic fallback between cloud and extension.

CLI (global)

bash

```
npm install -g @rtrvr-ai/cli
```

Programmatic API

bash

```
curl https://api.rtrvr.ai/agent
```

The CLI provides the `rtrvr` command. Programmatic callers can use the HTTP API directly.

1Authenticate

bash

```
rtrvr auth login
```

2Run your first task

CLI

```
rtrvr run "Extract the top 10 products and prices" --url https://example.com
```

3Explore

bash

```
# Scrape a page
rtrvr scrape --url https://example.com

# Check your capabilities
rtrvr capabilities

# Run diagnostics
rtrvr doctor
```

Every command supports three execution targets. Use `--target` or the shortcut flags.

Cloud

`--cloud`

Managed browsers via api.rtrvr.ai. No local setup needed. Best for scraping, data extraction, and tasks that don't require your login sessions.

Extension

`--extension`

Route to your local browser via the rtrvr Chrome extension. Use for authenticated sessions (logged-in sites) and tasks that need your cookies/sessions.

Auto (default)

`--target auto`

Smart routing — checks if extension devices are online, tries extension first, falls back to cloud automatically. This is the default behavior.

Auto mode routing flow

rtrvr run "task"

Check for online extension devices

Found

Extension

Not found

Cloud

The response includes routing metadata showing which mode was selected and whether fallback was applied:

json

```
{
  "metadata": {
    "selectedMode": "cloud",
    "fallbackApplied": true,
    "fallbackReason": "no extension devices online"
  }
}
```

bash

```
# Positional input
rtrvr run "Find the latest pricing for each plan" --url https://example.com

# From file
rtrvr run --input-file ./task.txt --url https://example.com --target cloud

# From stdin
echo "Summarize this page" | rtrvr run --url https://example.com
```

`<input>`

Task description (positional argument)

`--input <text>`

Task description (flag alternative)

`--input-file <path>`

Read task description from a file

`--url <url>`

Target URL(s) for the agent

`--target <mode>`default: `auto`

Routing mode: auto, cloud, or extension

`--cloud`

Shortcut for --target cloud

`--extension`

Shortcut for --target extension

`--device-id <id>`

Target a specific browser extension device

`--schema-file <path>`

JSON schema file for structured output

`--json`

Machine-readable JSON output

`--no-stream`

Disable real-time progress streaming

Use the authenticated HTTP API directly for programmatic cloud runs.

### Create a Run

cURL

```
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer $RTRVR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Find latest headline and author",
    "urls": ["https://example.com"],
    "target": "auto"
  }'
```

### Client Options

`apiKey`

Your rtrvr API key (rtrvr\_...)

`defaultTarget`default: `auto`

Default routing mode

`baseUrl`default: `https://api.rtrvr.ai`

Override the API base URL

`mcpUrl`default: `https://mcp.rtrvr.ai`

Override the MCP base URL

### Unified Run

JSON body

```
{
  "input": "Find latest headline and author",
  "urls": ["https://example.com"],
  "target": "auto"
}
```

### Direct Tool Calls

CLI

```
rtrvr raw extract_from_tab --user_input "Extract all product names and prices" --tab_urls '["https://example.com/products"]'
rtrvr raw act_on_tab --user_input "Click the Sign Up button" --tab_urls '["https://example.com"]'
rtrvr raw crawl_and_extract_from_tab --user_input "Find all blog post titles" --tab_urls '["https://example.com/blog"]'
```

### Scrape

CLI

```
rtrvr scrape https://example.com
```

### Utilities

CLI

```
rtrvr devices
rtrvr credits
rtrvr profile
```

## Related Documentation

[

API Overview

REST API endpoints and authentication

](/docs/api)[

Agent API

Full agent endpoint reference

](/docs/agent)[

Scrape API

Page scraping endpoint reference

](/docs/scrape)[

MCP Docs

Browser as API/MCP protocol

](/docs/mcp)

### Ready to automate from your terminal?

Install the rtrvr CLI and start running agents in seconds.

[](https://www.npmjs.com/package/@rtrvr-ai/cli)[](/docs/api)

---

# API Overview

High-level overview of the Agent API plus recommended client patterns. For the full schema and Playground, see /docs/agent.

# API Overview

[Video: API Overview](https://www.youtube.com/watch?v=jmVgTLezDT8)

The rtrvr.ai HTTP API exposes the same planner + tools engine that powers the web agent UI. Use it to run web automations or sheets-style workflows directly from your backend or serverless functions.

> **info**: For the full request/response schema, live Playground, and language-specific examples, open the API Reference at /docs/agent.

## Agent Endpoint

Almost all integrations use a single endpoint: POST /agent. Send a natural-language input, optional URLs, optional dataInputs (CSV/JSON), and an optional response config. The planner chooses the right tools and returns a structured AgentApiResponse.

```bash
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Summarize this page in 5 bullet points.",
    "urls": ["https://example.com"],
    "response": { "verbosity": "final" }
  }'
```

## Request: Mental Model

- input — required natural-language instruction the planner reasons over
- urls — optional array of pages to open before the agent runs
- dataInputs — optional tabular inputs (CSV/TSV/JSON) converted into in-memory sheets
- tools.enableAdditionalTools — opt-in for heavier tool families (generate_docs, generate_sheets, generate_pdfs, generate_websites, ask_questions)
- response.verbosity — detail level: "final" (default), "steps", or "debug"

> **warning**: For production, keep response.verbosity = "final" unless you need per-step telemetry. Debug mode returns much more data and costs more bandwidth.

## Response: Mental Model

- result.text — human-readable summary of what happened
- result.json — structured machine-readable result, shaped by your optional schema
- output — all blocks (text, json, and in debug mode, per-step tool_result blocks)
- usage — creditsUsed, creditsLeft, and other billing fields
- metadata — trajectoryId, toolsUsed, optional references to full snapshots

## SDK & Client Patterns

You don't need a heavy SDK. The recommended pattern is a thin server-side wrapper around POST /agent, plus optional batching for fan-out. This keeps your API key on the backend, centralizes rate limiting, and minimizes per-request latency.

### Node / Next.js

```typescript
// /app/api/rtrvr/agent/route.ts (Next.js App Router)
import "server-only"

const RTRVR_BASE = "https://api.rtrvr.ai"
const RTRVR_KEY = process.env.RTRVR_API_KEY!

type Payload = { input: string; urls?: string[]; dataInputs?: any[]; tools?: any; response?: any }

export async function POST(req: Request) {
  const body = (await req.json()) as Payload | Payload[]
  const tasks = Array.isArray(body) ? body : [body]

  const results = await Promise.all(
    tasks.map((task) =>
      fetch(RTRVR_BASE + "/agent", {
        method: "POST",
        headers: { Authorization: "Bearer " + RTRVR_KEY, "Content-Type": "application/json" },
        body: JSON.stringify({
          response: { verbosity: "final", inlineOutputMaxBytes: 200_000 },
          ...task,
        }),
      }).then((r) => r.json())
    )
  )

  return Response.json(Array.isArray(body) ? results : results[0])
}
```

### Python

```python
import os, requests
from typing import List, Dict, Any

BASE = "https://api.rtrvr.ai"
KEY = os.environ["RTRVR_API_KEY"]

def execute_tasks(tasks: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
    session = requests.Session()
    results = []
    for task in tasks:
        resp = session.post(
            f"{BASE}/agent",
            headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"},
            json={"response": {"verbosity": "final", "inlineOutputMaxBytes": 200_000}, **task},
            timeout=60,
        )
        results.append(resp.json())
    return results
```

> **success**: For the complete schema, error codes, and many more code samples, open the API Reference at /docs/agent.

---

# Agent API `/agent`

The primary entry point to the rtrvr.ai planner + tools engine. Send one JSON payload that can browse the web, load tabular data as in-memory sheets, call tools, and return structured results.

[Try Playground](#playground)[Get API Key](/cloud?view=api-keys)

Full Planner Engine

Multi-step reasoning with automatic tool orchestration and browser control.

Tabular Data Inputs

Load CSV, JSON, XLSX, or Parquet files as in-memory sheets for enrichment.

Structured Outputs

Define JSON schemas for type-safe, predictable result formats.

New: Run agents from your terminal with [rtrvr CLI](/docs/cli) — `rtrvr run "Extract products" --url https://example.com`

[Agent API Walkthrough](https://www.youtube.com/embed/FOmwVh1tJw0)

### Agent API Playground

POST`/agent`

Planner + tools engine in API mode.

Try:

API Key

Get from [rtrvr.ai/cloud](/cloud?view=api-keys)

Task Description \*

URLs (one per line)

curl -X POST https://api.rtrvr.ai/agent \\
  -H "Authorization: Bearer YOUR\_API\_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
  "input": "Retrieve the top 5 article titles and authors.",
  "urls": \[
    "https://news.ycombinator.com/"
  \],
  "response": {
    "verbosity": "final"
  }
}'

Base URL`https://api.rtrvr.ai`

Primary endpoints: `/agent` (planner + tools) and `/scrape` (raw page data).

Use your API key in the `Authorization` header:

Header

```
Authorization: Bearer rtrvr_your_api_key
```

**Security:** Always keep your key on the server side (e.g. backend, serverless). Never embed it in browser code or ship it to clients.

POST`https://api.rtrvr.ai/agent`

Send a single JSON payload describing what you want. The planner orchestrates browser tabs, tools, and in-memory sheets to get the job done.

cURL

```
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Summarize the main points of this page in 5 bullet points.",
    "urls": ["https://example.com/blog/ai-trends-2025"],
    "response": { "verbosity": "final" }
  }'
```

Internally, this maps to an execution trajectory. New requests get a new `trajectoryId`; continuations reuse it.

For low-level raw page data, see the [Scrape API docs](/docs/scrape) (`/scrape`).

Both endpoints share the same browser + proxy infra but are optimized for different jobs.

Dimension

/agent

/scrape

What it does

Full agent run: planner + tools + browser + optional Sheets/Docs/etc.

Loads pages and returns extracted text + accessibility tree.

Typical latency

Higher – dominated by LLM calls and multi-step tools.

Lower – usually just browser + proxy round-trips.

Credits

Infra credits + model/tool credits.

Infra-only credits (browser + proxy); no model/tool usage.

Best for

End-to-end automations, multi-step workflows, writing back to external systems.

Feeding your own LLM/RAG stack, ad-hoc scraping, prefetching page data.

Capabilities

Planner, tools, Sheets workflows, Docs/PDF generation, `ask_user`, etc.

Extracted text, accessibility tree, `elementLinkRecord`, usage metrics.

Before using the Agent API, ensure you have the required setup completed.

### 1\. API Key

Get your API key from [rtrvr.ai/cloud → API Keys](https://rtrvr.ai/cloud?view=api-keys)

### 2\. Google Integration (Optional)

**Required for:** `generate_sheets`, `generate_docs`, `generate_slides`

1

Sign in to rtrvr.ai/cloud

Go to [rtrvr.ai/cloud](https://rtrvr.ai/cloud) and sign in with your account

2

Connect Google Drive

Click the **Sheets** tab and authorize Google Drive access when prompted

3

Verify Connection

You should see your Google Sheets listed. Try selecting one to confirm access.

**Note:** This is a one-time setup. Once connected, the API will automatically use your Google account for document operations.

### 3\. File Uploads (Optional)

To use the `files` parameter, upload files at [rtrvr.ai/cloud → Files](https://rtrvr.ai/cloud?view=files) to get Storage URLs.

### Trajectory & Phase

A **trajectory** is a stable ID for a workflow. Use it to group related phases (e.g. discovery → enrichment → reporting) and continuations.

-   Omit `trajectoryId` to start fresh.
-   Reuse the same `trajectoryId` with `continuePlanning = true` and the returned `history` to continue.
-   `phase` (default `1`) lets you structure long-running projects into multiple stages.

### Planner + Tools

You don't call tools directly. Instead, you describe the task and optionally configure which `enableAdditionalTools` to allow. Support for `tools.enableAdditionalTools` in the public API will come soon.

Under the hood, the planner can call tools like `act_on_tab`, `crawl_and_extract_from_tab`, `sheets_workflow`, `create_sheet_from_data`, and more. Only a subset (Docs, Slides, PDFs, persistent Sheets, ask\_user, etc.) is gated behind `enableAdditionalTools` to control cost and latency.

### Tabular Inputs & In-Memory Sheets

Use `dataInputs` to attach CSV/TSV/JSON, text, markdown, or binary formats (XLSX/Parquet via URL or storage). The system:

-   Infers the format from extension or content type when omitted.
-   Parses header and row schema.
-   Creates an in-memory sheet (no Google Drive write) exposed to tools like `sheets_workflow`.

The `files` parameter lets you attach PDFs, images, and documents for the agent to analyze or use. This is different from `dataInputs`which is specifically for tabular/structured data.

**What the agent can do with files:**

-   Read and analyze PDF documents
-   Process and describe images (screenshots, diagrams, photos)
-   Fill out PDF forms with extracted or provided data
-   Upload files to web forms during browser automation
-   Compare multiple documents and find differences

### ApiExecuteRequestFile schema

`files[].displayName`stringrequired

Human-readable filename shown to the agent (e.g., 'Q3-Report.pdf', 'screenshot.png')

`files[].uri`stringrequired

File location. Accepts Firebase Storage URL, GCS URI (gs://bucket/path), or public HTTPS URL

`files[].mimeType`stringrequired

MIME type (e.g., 'application/pdf', 'image/png', 'image/jpeg')

### Supported file types

PDF(application/pdf)

PNG(image/png)

JPEG(image/jpeg)

GIF(image/gif)

WebP(image/webp)

Word(docx)

Text(text/plain)

Markdown(text/markdown)

### Three ways to provide file URIs

Recommended1\. Firebase Storage URL

Upload files via [Cloud → Files](/cloud?view=files) and copy the Storage URL. This is the most reliable option.

`https://firebasestorage.googleapis.com/v0/b/bucket/o/path%2Ffile.pdf?alt=media&token=...`

2\. GCS URI

If you have files in Google Cloud Storage, use the gs:// URI directly.

`gs://your-bucket/path/to/file.pdf`

3\. Public HTTPS URL

Any publicly accessible URL. Must not require authentication.

`https://example.com/documents/report.pdf`

### Example: Analyze a PDF report

PDF Analysis

```
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Extract the key financial metrics from this quarterly report and summarize them.",
    "files": [
      {
        "displayName": "Q3-2024-Report.pdf",
        "uri": "https://firebasestorage.googleapis.com/v0/b/bucket/o/reports%2Fq3.pdf?alt=media&token=abc",
        "mimeType": "application/pdf"
      }
    ],
    "schema": {
      "type": "object",
      "properties": {
        "revenue": { "type": "string" },
        "profit": { "type": "string" },
        "growth": { "type": "string" },
        "highlights": { "type": "array", "items": { "type": "string" } }
      }
    }
  }'
```

### Example: Analyze an image

Image Analysis

```
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Describe this screenshot and identify any UI/UX issues.",
    "files": [
      {
        "displayName": "app-screenshot.png",
        "uri": "gs://my-bucket/screenshots/app-v2.png",
        "mimeType": "image/png"
      }
    ]
  }'
```

### Example: Compare multiple documents

Multi-file comparison

```
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Compare these two contracts and list all the differences.",
    "files": [
      {
        "displayName": "contract-v1.pdf",
        "uri": "https://firebasestorage.googleapis.com/.../contract-v1.pdf?...",
        "mimeType": "application/pdf"
      },
      {
        "displayName": "contract-v2.pdf", 
        "uri": "https://firebasestorage.googleapis.com/.../contract-v2.pdf?...",
        "mimeType": "application/pdf"
      }
    ],
    "schema": {
      "type": "object",
      "properties": {
        "differences": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "section": { "type": "string" },
              "v1_text": { "type": "string" },
              "v2_text": { "type": "string" },
              "significance": { "type": "string" }
            }
          }
        }
      }
    }
  }'
```

### Example: Upload file to web form

File upload during automation

```
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Go to the application page, upload my resume, and fill out the form with name: John Doe, email: john@example.com",
    "urls": ["https://company.com/careers/apply"],
    "files": [
      {
        "displayName": "resume.pdf",
        "uri": "https://firebasestorage.googleapis.com/.../resume.pdf?...",
        "mimeType": "application/pdf"
      }
    ]
  }'
```

**Limits & notes:**

-   Maximum file size: **20MB** per file
-   Files are processed by the LLM, larger files may use more credits
-   Binary files are base64-encoded internally for LLM processing
-   Use `dataInputs` for tabular data (CSV, JSON, Excel) - it's more efficient

The full request shape is `AgentApiRequest`:

AgentApiRequest (conceptual)

```
interface AgentApiRequest {
  // ─────────────────────────────────────────
  // CORE PARAMETERS
  // ─────────────────────────────────────────
  
  /** Stable ID for a workflow. Omit to start new; reuse to continue. */
  trajectoryId?: string;
  
  /** Phase index within a trajectory (default: 1). Use ≥2 for multi-stage workflows. */
  phase?: number;
  
  /** Main user instruction - REQUIRED */
  input: string;
  
  /** URLs to open in browser tabs */
  urls?: string[];
  
  /** JSON Schema describing expected result shape */
  schema?: Schema;

  // ─────────────────────────────────────────
  // FILE INPUTS (PDFs, Images, Documents)
  // ─────────────────────────────────────────
  
  /**
   * File attachments for the agent to process.
   * Supports PDFs, images, and documents up to 20MB each.
   * 
   * The agent can:
   * - Read and analyze file contents
   * - Extract text from PDFs
   * - Analyze images
   * - Upload files to web forms
   */
  files?: ApiExecuteRequestFile[];

  // ─────────────────────────────────────────
  // TABULAR DATA INPUTS
  // ─────────────────────────────────────────
  
  /**
   * Tabular data to load as in-memory sheets.
   * Supports CSV, TSV, JSON, text, markdown, XLSX, Parquet.
   */
  dataInputs?: ApiTabularInput[];

  // ─────────────────────────────────────────
  // TOOL CONFIGURATION
  // ─────────────────────────────────────────
  
  tools?: ApiToolsConfig;

  // ─────────────────────────────────────────
  // SETTINGS & HISTORY
  // ─────────────────────────────────────────
  
  /** Per-request settings override */
  settings?: Partial<UserSettings>;
  
  /** Recording context for guided execution */
  recordingContext?: string;
  
  /** Continuation state from previous runs */
  history?: {
    continue?: boolean;
    previousSteps?: PlannerPreviousStep[];
    lastToolPreviousSteps?: ToolPreviousSteps;
  };

  // ─────────────────────────────────────────
  // RESPONSE CONFIGURATION
  // ─────────────────────────────────────────
  
  response?: {
    /** 'final' (default) | 'steps' | 'debug' */
    verbosity?: ApiVerbosity;
    /** Max bytes for inline output (default: 1MB) */
    inlineOutputMaxBytes?: number;
  };

  // ─────────────────────────────────────────
  // ARTIFACT REUSE (Advanced)
  // ─────────────────────────────────────────
  
  /**
   * Control how the agent reuses existing Google artifacts.
   * Useful for appending to existing Sheets/Docs.
   */
  reuseArtifacts?: ReuseArtifacts;

  // ─────────────────────────────────────────
  // INTERNAL OPTIONS
  // ─────────────────────────────────────────
  
  options?: {
    skipToolsStorageLoad?: boolean;
    pinTools?: boolean;
    pinSettings?: boolean;
    
    /** Execution trigger context */
    trigger?: {
      type: 'schedule' | 'ui' | 'api';
      context?: ScheduleContext;
    };
    
    /** UI and VNC live view settings */
    ui?: {
      /** Enable VNC live browser viewing. Default: false */
      enableVnc?: boolean;
      
      /** 
       * Which browser sessions to expose:
       * - "root": Main browser only (default)
       * - "all": Main + all batch worker browsers
       */
      vncScope?: 'root' | 'all';
      
      /**
       * Emit progress events to Firestore.
       * Opt-in only: set true to enable progress event writes.
       */
      emitEvents?: boolean;
    };
  };

  // ─────────────────────────────────────────
  // WEBHOOKS (Async notifications)
  // ─────────────────────────────────────────

  /**
   * Webhook subscriptions for async delivery of execution results.
   * Up to 5 webhooks per request. Delivered via Cloud Tasks (best-effort).
   */
  webhooks?: WebhookSubscription[];
}

// Webhook subscription type
interface WebhookSubscription {
  /** Webhook endpoint URL (HTTPS required in production) */
  url: string;

  /** Events to subscribe to. If omitted, all events are delivered. */
  events?: (
    | "rtrvr.execution.succeeded"
    | "rtrvr.execution.failed"
    | "rtrvr.execution.cancelled"
    | "rtrvr.execution.requires_input"
  )[];

  /** HTTP method (currently only POST supported) */
  method?: "POST";

  /** Custom headers to include in webhook requests */
  headers?: Record<string, string>;

  /** Authentication config */
  auth?: {
    type: "bearer";
    token: string;
  } | {
    type: "basic";
    username: string;
    password: string;
  };

  /** Secret for HMAC signature (X-Rtrvr-Signature header) */
  secret?: string;

  /** Request timeout in ms (1000-30000, default: 8000) */
  timeoutMs?: number;

  /** Retry policy: "default" (with retries) or "none" */
  retry?: { mode: "default" | "none" };
}

// File input type
interface ApiExecuteRequestFile {
  /** Human-friendly filename, e.g. "Resume-2025.pdf" */
  displayName: string;
  
  /**
   * File location. Accepts:
   * - Firebase Storage URL: https://firebasestorage.googleapis.com/...
   * - GCS URI: gs://bucket/path/to/file
   * - Public HTTPS URL: https://example.com/file.pdf
   */
  uri: string;
  
  /** MIME type, e.g. "application/pdf", "image/png" */
  mimeType: string;
}

// Tabular input type
interface ApiTabularInput {
  /** Optional client-provided correlation ID */
  id?: string;
  
  /** Description for the sheet (used as title) */
  description?: string;
  
  /** Format hint: "csv" | "tsv" | "json" | "text" | "markdown" | "xlsx" | "parquet" */
  format?: InputFormat;
  
  /** Inline data content */
  inline?: string;
  
  /** Remote URL to fetch data from */
  url?: string;
  
  /** Backend storage reference (advanced) */
  storageRef?: StorageReference;
}

// Tools configuration
interface ApiToolsConfig {
  /**
   * Additional tool families to enable.
   * Core tools (browser, extraction, in-memory sheets) are always available.
   */
  enableAdditionalTools?: (
    | "ask_questions"      // Pause for user input
    | "generate_sheets"    // Write to Google Sheets
    | "generate_docs"      // Create Google Docs
    | "generate_slides"      // Create Google Slides
    | "generate_websites"  // Generate web dashboards
    | "generate_pdfs"      // Create new PDFs
    | "pdf_filling"        // Fill PDF forms
  )[];
  
  /** Names of user-defined tools to make available */
  userDefined?: string[];
  
  /**
   * Tool loading mode:
   * - "profile": Load user's saved tools (default)
   * - "allowlist": Only use tools in userDefined
   * - "none": No user-defined tools
   */
  mode?: "allowlist" | "profile" | "none";
}

// Artifact reuse configuration
interface ReuseArtifacts {
  /** 'off' (default) | 'auto' | 'force' */
  mode?: 'off' | 'auto' | 'force';
  
  targets?: {
    sheets?: {
      sheetId: string;
      tabTitle?: string;
      tabId?: number;
      /** 'SAME_TAB' | 'NEW_TAB' */
      tabMode?: 'SAME_TAB' | 'NEW_TAB';
    };
    docs?: {
      docId: string;
      /** 'APPEND' | 'OVERWRITE' */
      mode?: 'APPEND' | 'OVERWRITE';
    };
    slides?: {
      presentationId: string;
      mode?: 'APPEND' | 'OVERWRITE';
    };
    pdfs?: {
      templateFileId?: string;
    };
  };
}
```

### Core fields

`input`stringrequired

Natural-language task description; what you want the system to do.

`urls`string\[\]

Optional list of URLs to open. The first real URL loads full content; others default to text-only for efficiency.

`schema`Schema

Optional OpenAPI-style JSON Schema describing the desired final JSON shape. Planner and tools will try to honor it when producing result.json.

`trajectoryId`string

Stable ID for a workflow. Omit to start a new trajectory; reuse to continue or add phases.

`phase`numberdefault: `1`

Phase index within a trajectory. Use ≥2 for multi-stage workflows.

Example schema: bulleted summary

```
{
  "type": "object",
  "properties": {
    "bullets": {
      "type": "array",
      "items": { "type": "string" }
    },
    "sourceUrl": {
      "type": "string"
    }
  },
  "required": ["bullets"]
}
```

### Tabular inputs (`dataInputs`)

`dataInputs`ApiTabularInput\[\]

Optional list of tabular inputs to materialize as in-memory sheets.

`dataInputs[].description`string

Human-readable description. Used as sheet title in the UI.

`dataInputs[].format`"text" | "markdown" | "csv" | "tsv" | "json" | "xlsx" | "parquet"

Optional explicit format. If omitted, inferred from file extension or content type.

`dataInputs[].inline`string

Raw content (CSV/TSV/JSON/text/markdown) embedded directly in the request. For XLSX/Parquet prefer URL or storageRef.

`dataInputs[].url`string

HTTP(S) URL to fetch as a tabular source (works well for large CSV/XLSX/Parquet files).

`dataInputs[].storageRef`StorageReference

Advanced: backend-managed GCS object reference when clients upload to storage directly.

dataInputs example: CSV (inline)

```
# CSV inline
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Enrich each company with website and description.",
    "dataInputs": [
      {
        "description": "Companies",
        "format": "csv",
        "inline": "company\\nOpenAI\\nDeepMind\\nAnthropic\\n"
      }
    ],
    "response": { "verbosity": "steps" }
  }'
```

dataInputs example: JSON (inline)

```
# JSON inline (array of objects)
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Infer seniority and return an updated JSON array.",
    "dataInputs": [
      {
        "description": "Contacts",
        "format": "json",
        "inline": "[{\"name\":\"Alice\",\"title\":\"VP Engineering\"},{\"name\":\"Bob\",\"title\":\"Software Engineer\"}]"
      }
    ],
    "response": { "verbosity": "steps" }
  }'
```

dataInputs example: XLSX (via URL)

```
# XLSX via URL
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Summarize opportunity pipeline from this Excel file.",
    "dataInputs": [
      {
        "description": "Sales pipeline",
        "format": "xlsx",
        "url": "https://example.com/sales-pipeline.xlsx"
      }
    ],
    "response": { "verbosity": "steps" }
  }'
```

dataInputs example: Parquet (via URL)

```
# Parquet via URL
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Compute daily active users per region from this Parquet dataset.",
    "dataInputs": [
      {
        "description": "Events parquet",
        "format": "parquet",
        "url": "https://example.com/events.parquet"
      }
    ],
    "response": { "verbosity": "steps" }
  }'
```

### Tools configuration (`tools`)

`tools.enableAdditionalTools`string\[\]

Coming soon: optional list of higher-power tool families to enable for this request.

`"ask_questions"``"generate_docs"``"generate_slides"``"generate_websites"``"generate_pdfs"``"pdf_filling"``"generate_sheets"`

Core tools (browser actions, extraction, sheets\_workflow on in-memory sheets, etc.) are always enabled. Additional tools control Docs, Slides, PDFs, persistent Sheets, and explicit ask\_user behavior.

**Google Integration Required**

To use `generate_sheets`, `generate_docs`, or `generate_slides`:

1.  Sign in at [rtrvr.ai/cloud](https://rtrvr.ai/cloud)
2.  Click **Sheets** tab and connect your Google Drive when prompted
3.  Verify by checking that you can see and select your Google Sheets

Once connected, the API will use your Google account to create/edit documents.

### Response configuration (`response`)

`response.verbosity`"final" | "steps" | "debug"default: `"final"`

Controls how much detail you get back.

`"final"``"steps"``"debug"`

`response.inlineOutputMaxBytes`number

Hard cap (in bytes) for inline output blocks. Larger payloads are snapshot to storage and previewed.

### History & continuation (`history`)

`continuePlanning`boolean

Canonical flag that this call should continue the previous workflow state. Top-level value wins over legacy history.continue.

`history.continue`boolean

Legacy alias for continuePlanning, accepted for backward compatibility.

`history.previousSteps`PlannerPreviousStep\[\]

Planner-internal state from previous runs. Returned in response.history for advanced clients.

`history.lastToolPreviousSteps`ToolPreviousSteps

Tool execution state for the last tool. Used for precise continuations.

### Advanced options

`settings`Partial<UserSettings>

Per-request overrides for stored user settings (model, proxy, extraction config, etc.). Generally only needed from first-party or advanced SDKs.

`options.skipToolsStorageLoad`boolean

Internal optimization flag when all tools are provided directly. Most clients should omit.

### UI & VNC options (`options.ui`)

Enable live browser viewing via VNC. Perfect for debugging, demos, or embedding real-time browser sessions in your app.

`options.ui.enableVnc`booleandefault: `false`

Enable VNC live view for this execution. When true, you can retrieve an embeddable URL to watch the browser in real-time.

`options.ui.vncScope`"root" | "all"default: `"root"`

Controls which browser sessions are visible. 'root' shows only the main browser. 'all' includes batch worker browsers when using parallel execution.

`"root"``"all"`

`options.ui.emitEvents`booleandefault: `false`

Opt-in only. When true, execution progress events are written to Firestore for SSE/polling clients. If omitted/false, no execution event stream is written.

**emitEvents policy:** API and extension/MCP executions only emit events when`options.ui.emitEvents: true`. CLI streaming defaults on for `run`, `agent`, and `scrape` (use `--no-stream` to disable). Streamed payloads at or under 1MB stay inline; larger payloads include inline preview markers and storage references (\`outputRef\` / \`resultRef\` / \`responseRef\`) for full downloads.

**Quick Start:** To embed a live browser view, set `options.ui.enableVnc: true`, then call `POST /vnc/share` to get an iframe-ready URL. See the [VNC Live View section](#vnc-live-view) for full details.

### Webhooks (`webhooks`)

Get notified when your workflow completes via HTTP webhooks. Up to 5 webhooks per request. Webhooks are delivered asynchronously via Google Cloud Tasks with best-effort delivery and automatic retries.

`webhooks[].url`stringrequired

Webhook endpoint URL. HTTPS required in production. HTTP allowed for localhost development.

`webhooks[].events`string\[\]

Events to subscribe to. If omitted, all events are delivered.

`"rtrvr.execution.succeeded"``"rtrvr.execution.failed"``"rtrvr.execution.cancelled"``"rtrvr.execution.requires_input"`

`webhooks[].secret`string

HMAC secret for request signing. If set, requests include X-Rtrvr-Signature header with format: t=timestamp,v1=hmac\_sha256

`webhooks[].auth`object

Authentication config. Supports Bearer token ({ type: 'bearer', token: '...' }) or Basic auth ({ type: 'basic', username: '...', password: '...' })

`webhooks[].headers`Record<string, string>

Custom headers to include in webhook requests

`webhooks[].timeoutMs`numberdefault: `8000`

Request timeout in milliseconds (1000-30000)

`webhooks[].retry`objectdefault: `{ mode: "default" }`

Retry policy. 'default' retries on failure, 'none' disables retries

Webhook subscription example

```
// Webhook request example
curl -X POST "https://api.rtrvr.ai/agent" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Scrape the homepage and summarize it",
    "urls": ["https://example.com"],
    "webhooks": [
      {
        "url": "https://your-server.com/webhook",
        "events": ["rtrvr.execution.succeeded", "rtrvr.execution.failed"],
        "secret": "your-signing-secret",
        "headers": {
          "X-Custom-Header": "my-value"
        }
      }
    ]
  }'
```

#### Webhook Payload

When an event triggers, rtrvr sends a POST request with this envelope:

Webhook payload envelope

```
{
  "id": "whd_abc123",           // Unique delivery ID
  "event": "rtrvr.execution.succeeded",
  "createdAt": "2025-01-15T10:30:00.000Z",
  "data": {
    "trajectoryId": "exec_xyz789",
    "status": "success",
    "taskRef": "gs://bucket/user-tasks/uid/exec_xyz789/workflow.json",
    "responseRef": { ... },    // Full response snapshot when > 1MB
    "output": { ... },         // Inline preview payload
    "outputRef": { ... },      // Optional large output download reference
    "resultRef": { ... },      // Optional large result download reference
    "usage": {
      "creditsUsed": 0.15,
      "creditsLeft": 99.85
    }
  }
}
```

#### Signature Verification

If you provide a `secret`, verify the signature:

Verify X-Rtrvr-Signature

```
// Node.js signature verification
const crypto = require('crypto');

function verifyWebhook(payload, signature, secret) {
  const [tPart, v1Part] = signature.split(',');
  const timestamp = tPart.split('=')[1];
  const receivedSig = v1Part.split('=')[1];

  const signedPayload = timestamp + '.' + JSON.stringify(payload);
  const expectedSig = crypto
    .createHmac('sha256', secret)
    .update(signedPayload)
    .digest('hex');

  return crypto.timingSafeEqual(
    Buffer.from(receivedSig),
    Buffer.from(expectedSig)
  );
}
```

**Response metadata:** The API response includes `metadata.webhooks: { requested: number, attempted: number }` showing how many webhooks were configured and enqueued.

**Store & reuse webhooks**

Save your webhook endpoints in [Cloud → Webhooks](/cloud?view=webhooks) to quickly attach them to any execution without re-entering the URL, secret, and events each time.

Every call returns an `AgentApiResponse`:

AgentApiResponse (conceptual)

```
interface AgentApiResponse {
  protocol: {
    name: 'rtrvr.agent';
    version: string;
    compatibleWith: Array<'a2w.run' | 'a2a.task'>;
  };
  run: {
    id: string;
    trajectoryId: string;
    executionId?: string;
    status: 'success' | 'error' | 'cancelled' | 'requires_input' | 'executing' | 'requires_tool';
    phase: number;
  };

  success: boolean;
  status: 'success' | 'error' | 'cancelled' | 'requires_input' | 'executing';

  trajectoryId: string;
  phase: number;

  // Rich output blocks
  output: ApiOutputBlock[];
  // Note: in debug mode, tool_result blocks may include outputRef/resultRef
  // when large per-step payloads are stored out-of-line.

  // Convenience view of final output
  result?: {
    text?: string;
    json?: any;
  };

  // Present when verbosity !== 'final'
  steps?: ApiStepSummary[];

  usage: {
    creditsUsed: number;
    creditsLeft?: number;
    currentCreditsUsed?: number;
    expiryReason?: string;
  };

  metadata: {
    taskRef: string;
    inlineOutputMaxBytes: number;
    toolsUsed: string[];
    outputTooLarge?: boolean;
    responseRef?: StorageReference;
  };

  warnings?: string[];
  error?: string;

  // Continuation payload for advanced clients
  inputRequest?: {
    reason: string;
    questions?: PlannerQuestion[];
    expiresAt?: string;
    resume: {
      executionId: string;
      continuePlanning: true;
      phase: number;
      method?: 'POST';
      path?: string;
      expiresAt?: string;
    };
    browser?: {
      parkStatus?: 'parked' | 'reused' | 'restored' | 'expired' | 'unavailable';
      ttlMs?: number;
      expiresAt?: string;
      liveReuseAvailable?: boolean;
    };
  };
  history?: {
    previousSteps?: PlannerPreviousStep[];
    lastToolPreviousSteps?: ToolPreviousSteps;
  };
  browser?: {
    parkStatus?: string;
    expiresAt?: string;
    resumedFromPark?: boolean;
    restoredFromVault?: boolean;
    activeOrigin?: string;
    tabCount?: number;
  };
}
```

### Output blocks & result

The low-level `output` is an array of blocks:

`output[].type`"text" | "json" | "tool\_result"

Block type: final text, JSON payload, or detailed tool result (debug mode).

`output[].text`string

Present when type = 'text'.

`output[].data`any

Present when type = 'json'.

`output[].tool_result`…

When type = 'tool\_result', includes stepId, toolName, args, inline output preview, optional outputRef/resultRef, thought, etc. Only present when verbosity = 'debug'.

`result.text` is the concatenation of all text blocks. `result.json` is either the single JSON block, or an array of JSON blocks if the workflow produced multiple.

### Steps & usage

When `response.verbosity` is `"steps"` or `"debug"`, you also get `steps: ApiStepSummary[]`:

`steps[].toolName`string

Which tool ran in this step (e.g. 'sheets\_workflow', 'act\_on\_tab').

`steps[].status`ExecutionStatus

success, error, executing, etc. per step.

`steps[].duration`number

Execution time in ms for this step (when available).

`steps[].creditsUsed`number

Credits consumed by this step, useful for analytics.

`steps[].hasOutput`boolean

Whether this step produced output or an outputRef.

`steps[].hasSheets`boolean

Whether this step produced or touched tabular data.

`steps[].hasGeneratedContent`boolean

Whether this step generated external content (docs, slides, etc.).

`usage` mirrors your credit accumulator and is ideal for per-customer dashboards and server-side cost control.

### Large output handling

When the full response exceeds `inlineOutputMaxBytes`:

-   • The full response is snapshot to storage under `metadata.responseRef`.
-   • The inline `output`/`result` fields remain as preview content for UX.
-   • Debug `tool_result` blocks may include `outputRef` / `resultRef` for full per-step payloads.
-   • `metadata.outputTooLarge` is set to `true`.

Client pattern: render the preview for UX, but fetch `responseRef.downloadUrl` from your backend when you need the full payload.

### status & success

`status`"success" | "error" | "cancelled" | "requires\_input" | "executing"

Execution-level status. success implies success = true; all others imply success = false.

-   • `"success"` – Final result is available in `result` and `output`.
-   • `"error"` – Workflow failed. You still get `usage`, `steps` (if enabled), and partial output if any.
-   • `"cancelled"` – Client abort or timeout. Credits are accounted for partial work.
-   • `"requires_input"` – Planner paused because it needs human answers. Read `inputRequest` for questions, resume ids, and browser park expiry.

Responses include an additive `protocol.name = "rtrvr.agent"` envelope so A2W/A2A-style clients can map runs, messages, output, history, and input requests consistently. Legacy fields remain top-level for existing clients.

**Continuation pattern:**

1.  When you see `status: "requires_input"`, surface your own UI to collect missing info.
2.  On the next call, send `inputRequest.resume.executionId` as the same `trajectoryId`, set `continuePlanning = true`, and pass back the returned `history`.
3.  If `inputRequest.browser.liveReuseAvailable` is true, resume before `inputRequest.expiresAt` to continue in the same live browser; otherwise the agent restores cookies and the last active URL.

rtrvr supports **view-only live VNC streaming** for any execution. Watch the browser in real-time, embed it in your app via iframe, or build custom viewers using the VNC websocket URL.

**Use Cases**

-   • **Debugging:** Watch exactly what the browser is doing during automation
-   • **Demos:** Show clients real-time browser activity
-   • **Dashboards:** Embed live views in internal tools
-   • **Quality assurance:** Monitor batch jobs visually

### Key Concepts

`trajectoryId = executionId`

In VNC endpoints, `executionId` refers to your `trajectoryId`. They are the same identifier.

`vncScope`

`"root"` = main browser only (default).  
`"all"` = main + all batch worker browsers.

`shareKey`

A secret token for public/share endpoints. Anyone with it can view (not control) the session until it expires.

`Sessions`

`root` = main browser session.  
`batch` = worker sessions (when vncScope="all").

### Step 1: Enable VNC on Execution

Add `options.ui.enableVnc: true` to your execute request. **Pro tip:** Generate your own `trajectoryId` upfront so you can request the embed URL immediately without waiting for the response.

Enable VNC on execution

```
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "trajectoryId": "0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1",
    "input": "Go to example.com and summarize the homepage",
    "urls": ["https://example.com"],
    "options": {
      "ui": {
        "enableVnc": true,
        "vncScope": "root"
      }
    }
  }'
```

### Step 2: Get the Embed URL

Call `POST /vnc/share` with the `executionId` (same as your trajectoryId). This returns an `embedUrl` ready for iframe embedding.

POST`https://api.rtrvr.ai/vnc/share`

`executionId`stringrequired

The trajectoryId from your execute request

`rotate`booleandefault: `false`

Set true to generate a new share key (invalidates previous share URLs)

Get embed URL

```
curl -X POST https://api.rtrvr.ai/vnc/share \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "executionId": "0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1",
    "rotate": false
  }'
```

#### Response

Share response

```
{
  "ok": true,
  "executionId": "0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1",
  "embedUrl": "https://vnc-embed.rtrvr.ai/vnc/embed/0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1#key=<SHARE_KEY>",
  "expiresAt": 1730000000
}
```

### Step 3: Embed in Your App

Use the `embedUrl` directly in an iframe. The hosted page handles everything: loading the VNC viewer, connecting to the relay, and auto-refreshing tokens.

Embed in iframe

```
<iframe
  src="https://vnc-embed.rtrvr.ai/vnc/embed/0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1#key=YOUR_SHARE_KEY"
  style="width: 100%; height: 720px; border: 0; border-radius: 12px;"
  allow="clipboard-read; clipboard-write"
></iframe>
```

**Why the share key is in the fragment (#key=...)**

The share key is placed in the URL fragment so it's **never sent to servers** in HTTP requests by default. The embed page reads it client-side and sends it in an Authorization header when calling VNC endpoints.

### Complete Integration Example

Full VNC integration (JavaScript)

```
const API_URL = "https://api.rtrvr.ai";
const API_KEY = "YOUR_API_KEY";

async function startExecutionWithVnc(input, urls) {
  // 1. Generate your own trajectoryId for immediate embed
  const trajectoryId = crypto.randomUUID();
  
  // 2. Start execution with VNC enabled
  const executePromise = fetch(`${API_URL}/agent`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      trajectoryId,
      input,
      urls,
      options: {
        ui: {
          enableVnc: true,
          vncScope: "root"  // or "all" for batch workers
        }
      }
    }),
  });
  
  // 3. Immediately request embed URL (don't wait for execute)
  const shareRes = await fetch(`${API_URL}/vnc/share`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ 
      executionId: trajectoryId,
      rotate: false 
    }),
  });
  
  const shareData = await shareRes.json();
  
  if (shareData.ok) {
    // 4. Embed the VNC view
    const iframe = document.createElement("iframe");
    iframe.src = shareData.embedUrl;
    iframe.style.cssText = "width:100%;height:720px;border:0;border-radius:12px;";
    iframe.allow = "clipboard-read; clipboard-write";
    document.getElementById("vnc-container").appendChild(iframe);
  }
  
  // 5. Wait for execution to complete
  const executeRes = await executePromise;
  const result = await executeRes.json();
  
  return { result, embedUrl: shareData.embedUrl };
}

// Usage
startExecutionWithVnc(
  "Scrape the top 5 articles from Hacker News",
  ["https://news.ycombinator.com"]
).then(({ result, embedUrl }) => {
  console.log("Execution complete:", result);
  console.log("VNC embed URL:", embedUrl);
});
```

Full VNC integration (Python)

```
import uuid
import requests
import threading

API_URL = "https://api.rtrvr.ai"
API_KEY = "YOUR_API_KEY"

def start_execution_with_vnc(input_text: str, urls: list[str]):
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    
    # 1. Generate your own trajectoryId
    trajectory_id = str(uuid.uuid4())
    
    # 2. Start execution in background thread
    def execute():
        return requests.post(
            f"{API_URL}/agent",
            headers=headers,
            json={
                "trajectoryId": trajectory_id,
                "input": input_text,
                "urls": urls,
                "options": {
                    "ui": {
                        "enableVnc": True,
                        "vncScope": "root"  # or "all" for batch workers
                    }
                }
            },
            timeout=300
        ).json()
    
    execute_thread = threading.Thread(target=execute)
    execute_thread.start()
    
    # 3. Immediately get embed URL
    share_res = requests.post(
        f"{API_URL}/vnc/share",
        headers=headers,
        json={
            "executionId": trajectory_id,
            "rotate": False
        }
    ).json()
    
    if share_res.get("ok"):
        print(f"VNC Embed URL: {share_res['embedUrl']}")
        print(f"Share key expires: {share_res['expiresAt']}")
    
    # 4. Wait for execution
    execute_thread.join()
    
    return {
        "trajectory_id": trajectory_id,
        "embed_url": share_res.get("embedUrl")
    }

# Usage
result = start_execution_with_vnc(
    "Go to example.com and take a screenshot",
    ["https://example.com"]
)
print(f"Embed in iframe: {result['embed_url']}")
```

Full VNC flow (cURL)

```
# 1. Start execution with VNC enabled (use your own trajectoryId)
TRAJECTORY_ID="$(uuidgen)"

curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"trajectoryId\": \"$TRAJECTORY_ID\",
    \"input\": \"Go to example.com and summarize\",
    \"urls\": [\"https://example.com\"],
    \"options\": {
      \"ui\": {
        \"enableVnc\": true,
        \"vncScope\": \"root\"
      }
    }
  }" &

# 2. Immediately get the embed URL (don't wait for execute)
curl -X POST https://api.rtrvr.ai/vnc/share \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"executionId\": \"$TRAJECTORY_ID\",
    \"rotate\": false
  }"

# Response contains embedUrl - use it in an iframe
```

### Advanced: Build a Custom Viewer

For custom UIs, use the public VNC endpoints to list sessions and get websocket URLs. These endpoints require the `shareKey` (from the embed URL) as a Bearer token.

#### List Sessions

GET`/vnc-public/sessions?executionId={executionId}`

List VNC sessions (public endpoint)

```
curl "https://vnc-embed.rtrvr.ai/vnc-public/sessions?executionId=0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1" \
  -H "Authorization: Bearer YOUR_SHARE_KEY"

# Response:
# {
#   "ok": true,
#   "executionId": "0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1",
#   "sessions": [
#     {
#       "sessionId": "0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1",
#       "kind": "root",
#       "batchIndex": null,
#       "state": "running",
#       "expiresAt": 1730001234
#     }
#   ]
# }
```

#### Get Viewer Token & WebSocket URL

GET`/vnc-public/token?executionId={id}&sessionId={id}`

Get VNC token and websocket URL

```
curl "https://vnc-embed.rtrvr.ai/vnc-public/token?executionId=0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1&sessionId=0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1" \
  -H "Authorization: Bearer YOUR_SHARE_KEY"

# Response:
# {
#   "ok": true,
#   "executionId": "0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1",
#   "sessionId": "0f3f2d33-0f6a-4d79-bb3d-56c2d4d7c2a1",
#   "wsUrl": "wss://relay.rtrvr.ai/vnc?token=<JWT>",
#   "expiresAt": 1730000600
# }
```

**Token refresh:** VNC tokens expire (~12 minutes). For long-running sessions, refresh the token before expiry by calling `/vnc-public/token` again.

### Authenticated VNC Endpoints (Owner Access)

These endpoints use your API key directly (no share key needed). Useful when you don't want to expose share links.

`GET /vnc/sessions`endpoint

List sessions for an execution. Add ?withToken=1 to include wsUrl for each session.

`GET /vnc/token`endpoint

Get viewer token + wsUrl. Add ?createIfMissing=1 to auto-create the session doc.

`POST /vnc/share`endpoint

Generate/rotate a share key and get embedUrl.

Authenticated session list with tokens

```
curl "https://api.rtrvr.ai/vnc/sessions?executionId=YOUR_TRAJECTORY_ID&withToken=1" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

### VNC Endpoint Reference

VNC endpoints are split across two services. **Authenticated endpoints** live on the main API (`api.rtrvr.ai`), while **public/share-key endpoints** and the embed page are served by the embed service (`vnc-embed.rtrvr.ai`). The correct embed-service host is returned in the `embedUrl` from `POST /vnc/share`.

Endpoint

Host

Auth

Description

`POST /vnc/share`

api.rtrvr.ai

API Key

Get embedUrl + shareKey

`GET /vnc/sessions`

api.rtrvr.ai

API Key

List sessions (owner access)

`GET /vnc/token`

api.rtrvr.ai

API Key

Get wsUrl token (owner access)

`GET /vnc-public/sessions`

vnc-embed.rtrvr.ai

Share Key

List sessions (public/share)

`GET /vnc-public/token`

vnc-embed.rtrvr.ai

Share Key

Get wsUrl token (public/share)

`GET /vnc/embed/:executionId`

vnc-embed.rtrvr.ai

Share Key (in #fragment)

Hosted noVNC viewer page

### Security Best Practices

-   • **Treat shareKey like a password:** Anyone with it can view your browser session
-   • **Use fragment URLs:** The `#key=...` format prevents the key from appearing in server logs
-   • **Rotate when needed:** Call `POST /vnc/share` with `rotate: true` to invalidate old links
-   • **Use vncScope wisely:** Only use `"all"` if you need to see batch worker browsers
-   • **View-only:** Share links cannot control the browser, only view it

**💡 Tip:** Using Google tools? First connect your Google account at [rtrvr.ai/cloud → Sheets](https://rtrvr.ai/cloud)

cURL

```
# ═══════════════════════════════════════════════════════════════
# BASIC EXAMPLES
# ═══════════════════════════════════════════════════════════════

# 1. Simple page summarization
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Summarize the main points of this page in 5 bullet points.",
    "urls": ["https://example.com/blog/ai-trends-2025"],
    "response": { "verbosity": "final" }
  }'

# 2. With JSON schema for structured output
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Extract article title, author, and publish date.",
    "urls": ["https://example.com/blog/article"],
    "schema": {
      "type": "object",
      "properties": {
        "title": { "type": "string" },
        "author": { "type": "string" },
        "publishDate": { "type": "string" }
      },
      "required": ["title"]
    }
  }'

# ═══════════════════════════════════════════════════════════════
# FILE INPUT EXAMPLES
# ═══════════════════════════════════════════════════════════════

# 3. Analyze a PDF document
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Extract key financial metrics from this quarterly report.",
    "files": [
      {
        "displayName": "Q3-Report.pdf",
        "uri": "https://firebasestorage.googleapis.com/v0/b/bucket/o/files%2Freport.pdf?alt=media&token=abc",
        "mimeType": "application/pdf"
      }
    ],
    "schema": {
      "type": "object",
      "properties": {
        "revenue": { "type": "string" },
        "profit": { "type": "string" },
        "growth_rate": { "type": "string" }
      }
    }
  }'

# 4. Analyze an image
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Describe this UI screenshot and identify usability issues.",
    "files": [
      {
        "displayName": "app-screenshot.png",
        "uri": "gs://my-bucket/screenshots/app.png",
        "mimeType": "image/png"
      }
    ]
  }'

# 5. Compare two documents
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Compare these contracts and list all differences.",
    "files": [
      {
        "displayName": "contract-v1.pdf",
        "uri": "https://firebasestorage.googleapis.com/.../v1.pdf?...",
        "mimeType": "application/pdf"
      },
      {
        "displayName": "contract-v2.pdf",
        "uri": "https://firebasestorage.googleapis.com/.../v2.pdf?...",
        "mimeType": "application/pdf"
      }
    ]
  }'

# ═══════════════════════════════════════════════════════════════
# DATA INPUT EXAMPLES
# ═══════════════════════════════════════════════════════════════

# 6. CSV enrichment (inline)
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "For each company, find their website and a one-sentence description.",
    "dataInputs": [
      {
        "description": "Companies to enrich",
        "format": "csv",
        "inline": "company\nOpenAI\nAnthropic\nGoogle DeepMind"
      }
    ],
    "response": { "verbosity": "steps" }
  }'

# 7. JSON array processing
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Classify each review as positive, negative, or neutral.",
    "dataInputs": [
      {
        "description": "Reviews",
        "format": "json",
        "inline": "[{\"text\":\"Great product!\"},{\"text\":\"Terrible support.\"},{\"text\":\"It works okay.\"}]"
      }
    ]
  }'

# 8. XLSX from URL
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Summarize the sales pipeline metrics.",
    "dataInputs": [
      {
        "description": "Sales pipeline",
        "format": "xlsx",
        "url": "https://example.com/data/pipeline.xlsx"
      }
    ]
  }'

# ═══════════════════════════════════════════════════════════════
# COMBINED: FILES + DATA + BROWSING
# ═══════════════════════════════════════════════════════════════

# 9. Complex workflow: Analyze resume + scrape job posting + match
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Analyze the resume, scrape the job posting, and score the candidate fit (1-10) with reasoning.",
    "urls": ["https://company.com/careers/senior-engineer"],
    "files": [
      {
        "displayName": "candidate-resume.pdf",
        "uri": "https://firebasestorage.googleapis.com/.../resume.pdf?...",
        "mimeType": "application/pdf"
      }
    ],
    "schema": {
      "type": "object",
      "properties": {
        "fitScore": { "type": "number" },
        "strengths": { "type": "array", "items": { "type": "string" } },
        "gaps": { "type": "array", "items": { "type": "string" } },
        "recommendation": { "type": "string" }
      }
    }
  }'

# ═══════════════════════════════════════════════════════════════
# GOOGLE SHEETS
# ═══════════════════════════════════════════════════════════════

# 10. Create new Google Sheet
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Scrape all products and save to Google Sheets.",
    "urls": ["https://example.com/products"],
    "tools": { "enableAdditionalTools": ["generate_sheets"] }
  }'

# 11. Append to existing Google Sheet
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Scrape new leads and add to my existing sheet.",
    "urls": ["https://linkedin.com/search/..."],
    "reuseArtifacts": {
      "mode": "force",
      "targets": {
        "sheets": {
          "sheetId": "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms",
          "tabTitle": "Leads",
          "tabMode": "SAME_TAB"
        }
      }
    },
    "tools": { "enableAdditionalTools": ["generate_sheets"] }
  }'

# ═══════════════════════════════════════════════════════════════
# GOOGLE SLIDES
# ═══════════════════════════════════════════════════════════════

# 12. Create new presentation
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Create a 5-slide presentation summarizing key points. Include title, 3 content slides, and conclusion.",
    "urls": ["https://example.com/blog/industry-trends-2025"],
    "tools": { "enableAdditionalTools": ["generate_slides"] }
  }'

# 13. Append to existing presentation
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Add 3 slides with competitor analysis to the existing presentation.",
    "urls": ["https://competitor.com/products"],
    "reuseArtifacts": {
      "mode": "force",
      "targets": {
        "slides": {
          "presentationId": "1abc123_your_presentation_id",
          "mode": "APPEND"
        }
      }
    },
    "tools": { "enableAdditionalTools": ["generate_slides"] }
  }'

# ═══════════════════════════════════════════════════════════════
# GOOGLE DOCS
# ═══════════════════════════════════════════════════════════════

# 14. Create new Google Doc
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Create a detailed research report about this company including products, leadership, and news.",
    "urls": ["https://example.com/about", "https://example.com/news"],
    "tools": { "enableAdditionalTools": ["generate_docs"] }
  }'

# 15. Append to existing Google Doc
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Add a new section summarizing this weeks market news.",
    "urls": ["https://news.example.com/markets"],
    "reuseArtifacts": {
      "mode": "force",
      "targets": {
        "docs": {
          "docId": "1abc123_your_doc_id",
          "mode": "APPEND"
        }
      }
    },
    "tools": { "enableAdditionalTools": ["generate_docs"] }
  }'

# ═══════════════════════════════════════════════════════════════
# PDF GENERATION & FILLING
# ═══════════════════════════════════════════════════════════════

# 16. Generate new PDF
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Create a PDF invoice: 3x Widget Pro at $99 each, customer: Acme Corp.",
    "tools": { "enableAdditionalTools": ["generate_pdfs"] }
  }'

# 17. Fill PDF form template
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Fill the W-9 form with: Name: John Smith, Business: Smith Consulting LLC",
    "reuseArtifacts": {
      "mode": "force",
      "targets": {
        "pdfs": { "templateFileId": "1xyz_w9_template_id" }
      }
    },
    "tools": { "enableAdditionalTools": ["pdf_filling"] }
  }'

# ═══════════════════════════════════════════════════════════════
# WEB DASHBOARDS
# ═══════════════════════════════════════════════════════════════

# 18. Generate interactive dashboard
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Create an interactive dashboard showing sales by region with charts.",
    "dataInputs": [{
      "description": "Sales data",
      "format": "csv",
      "inline": "region,sales,month\nNorth,50000,Jan\nSouth,42000,Jan\nEast,38000,Jan"
    }],
    "tools": { "enableAdditionalTools": ["generate_websites"] }
  }'

# ═══════════════════════════════════════════════════════════════
# INTERACTIVE WORKFLOWS (ask_questions)
# ═══════════════════════════════════════════════════════════════

# 19. Enable follow-up questions
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Help me find the best flight from NYC to London.",
    "urls": ["https://flights.example.com"],
    "tools": { "enableAdditionalTools": ["ask_questions"] }
  }'

# 20. Continue after requires_input status
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Direct flights only, economy class, morning departure.",
    "trajectoryId": "exec_from_inputRequest_resume_executionId",
    "continuePlanning": true,
    "history": { "previousSteps": [] }
  }'

# ═══════════════════════════════════════════════════════════════
# MULTI-PHASE WORKFLOWS
# ═══════════════════════════════════════════════════════════════

# 21. Phase 1: Discovery
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Find the top 10 AI startups in healthcare.",
    "urls": ["https://techcrunch.com/tag/ai-healthcare"],
    "phase": 1
  }'

# 22. Phase 2: Enrichment (same trajectoryId)
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Get funding details and executives for each startup.",
    "trajectoryId": "traj_from_phase1",
    "phase": 2,
    "tools": { "enableAdditionalTools": ["generate_sheets"] }
  }'

# 23. Phase 3: Report
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Create a presentation with investment recommendations.",
    "trajectoryId": "traj_from_phase1",
    "phase": 3,
    "tools": { "enableAdditionalTools": ["generate_slides"] }
  }'

# ═══════════════════════════════════════════════════════════════
# CUSTOM TOOLS
# ═══════════════════════════════════════════════════════════════

# 24. Use custom/user-defined tools
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Find leads and add them to our CRM.",
    "urls": ["https://linkedin.com/search/..."],
    "tools": {
      "mode": "allowlist",
      "userDefined": ["add_to_crm", "enrich_lead"]
    }
  }'
  # ═══════════════════════════════════════════════════════════════
# VNC LIVE VIEW EXAMPLES
# ═══════════════════════════════════════════════════════════════

# 25. Start execution with VNC enabled
TRAJECTORY_ID="$(uuidgen)"
curl -X POST https://api.rtrvr.ai/agent \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"trajectoryId\": \"$TRAJECTORY_ID\",
    \"input\": \"Navigate to example.com and fill out the contact form\",
    \"urls\": [\"https://example.com/contact\"],
    \"options\": {
      \"ui\": {
        \"enableVnc\": true,
        \"vncScope\": \"root\"
      }
    }
  }"

# 26. Get embeddable VNC URL
curl -X POST https://api.rtrvr.ai/vnc/share \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"executionId\": \"$TRAJECTORY_ID\",
    \"rotate\": false
  }"

# 27. List VNC sessions (authenticated)
curl "https://api.rtrvr.ai/vnc/sessions?executionId=$TRAJECTORY_ID&withToken=1" \
  -H "Authorization: Bearer YOUR_API_KEY"

# 28. Get VNC token (authenticated)
curl "https://api.rtrvr.ai/vnc/token?executionId=$TRAJECTORY_ID&sessionId=$TRAJECTORY_ID" \
  -H "Authorization: Bearer YOUR_API_KEY"

# 29. List sessions with share key (public — served by the embed service)
curl "https://vnc-embed.rtrvr.ai/vnc-public/sessions?executionId=$TRAJECTORY_ID" \
  -H "Authorization: Bearer YOUR_SHARE_KEY"

# 30. Get VNC token with share key (public — served by the embed service)
curl "https://vnc-embed.rtrvr.ai/vnc-public/token?executionId=$TRAJECTORY_ID&sessionId=$TRAJECTORY_ID" \
  -H "Authorization: Bearer YOUR_SHARE_KEY"
```

### Ready to automate?

Join teams using rtrvr.ai to build playful, powerful web automation workflows.

[](/cloud)[](/docs/scrape)

---

# Scrape API `/scrape`

Low-level endpoint that reuses the same browser + proxy infra as the agent, but returns raw page text and accessibility trees. No planner, no tools—just data for your own models and pipelines.

[Try Playground](#playground)[Get API Key](/cloud?view=api-keys)

Infra-Only Credits

No model/tool credits—just browser + proxy costs for maximum efficiency.

Raw Page Data

Get extracted text, accessibility trees, and element link records.

Composable Output

Feed results directly into your own LLM/RAG pipelines.

New: Scrape from your terminal with [rtrvr CLI](/docs/cli) — `rtrvr scrape --url https://example.com`

[Scrape API Walkthrough](https://www.youtube.com/embed/QOHXXp5Q_4k)

### Scrape API Playground

POST`/scrape`

Low-level endpoint for raw page text + accessibility tree.

API Key

Get from [rtrvr.ai/cloud](/cloud?view=api-keys)

URLs (one per line)

curl -X POST https://api.rtrvr.ai/scrape \\
  -H "Authorization: Bearer YOUR\_API\_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
  "urls": \[
    "https://news.ycombinator.com/"
  \],
  "response": {
    "inlineOutputMaxBytes": 1048576
  }
}'

Base URL`https://api.rtrvr.ai`

Use `/scrape` for raw page data and `/agent` for full agent runs.

Use your API key in the `Authorization` header:

Header

```
Authorization: Bearer rtrvr_your_api_key
```

**Security:** Keep your key server-side (backend or serverless). Don't ship it to the browser.

POST`https://api.rtrvr.ai/scrape`

Agent vs Scrape

Use `/agent` when you want the full planner + tools engine, and `/scrape` when you just need raw page text + structure for your own models.

[See comparison](/docs/agent#agent-vs-scrape)

Open one or more URLs in our browser cluster and get back extracted text, the accessibility tree, and link metadata. The endpoint is designed to be:

-   **Cheap** – infra-only credits (browser + proxy), no model usage.
-   **Predictable** – stable schema for tab content + usage metrics.
-   **Composable** – plug the result into your own LLM/RAG pipeline.

Minimal scrape – single URL

```
curl -X POST https://api.rtrvr.ai/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/blog/ai-trends-2025"]
  }'
```

Each scrape uses a unified `UserSettings` profile stored in the cloud.

Relevant UserSettings fields (conceptual)

```
interface UserSettings {
  extractionConfig: {
    maxParallelTabs?: number;
    pageLoadDelay?: number;
    makeNewTabsActive?: boolean;
    writeRowProcessingTime?: boolean;
    disableAutoScroll?: boolean;
    /**
     * When true, only text content is returned from scrapes.
     * The accessibility tree + elementLinkRecord are omitted.
     */
    onlyTextContent?: boolean;
  };

  // Proxy Configuration
  proxyConfig: {
    mode: 'none' | 'custom' | 'default' | 'device';
    customProxies: ProxySettings[];
    selectedProxyId?: string;
    selectedDeviceId?: string;
  };
}
```

Two ways to control behavior:

-   **1\. Cloud profile:** configure defaults in [Cloud → Settings](/cloud?view=settings).
-   **2\. Per-request overrides:** send `settings` in your request body.

The request body is an `ScrapeApiRequest`:

ScrapeApiRequest (conceptual)

```
interface ScrapeApiRequest {
    /**
     * Optional stable id if you want to tie multiple scrapes together.
     * Mostly useful for analytics/observability on your side.
     */
    trajectoryId?: string;

    /**
     * One or more absolute URLs to load in the browser.
     * Must be a non-empty array of non-empty strings.
     */
    urls: string[];

    /**
     * Optional per-request settings override.
     * Merged on top of the stored UserSettings profile (proxyConfig, extraction, etc.).
     *
     * Use extraction-related settings if you only want text content and don't need
     * the accessibility tree + elementLinkRecord.
     */
    settings?: Partial<UserSettings>;

    /**
     * Response size control for API callers.
     */
    response?: {
      /**
       * Max bytes allowed for the inline JSON response.
       * If the full response exceeds this, tabs remain inline as preview content,
       * and a StorageReference is returned under metadata.responseRef for full payload download.
       * Default: 1MB (1048576 bytes)
       */
      inlineOutputMaxBytes?: number;
    };

    /**
     * Optional execution options.
     * Set options.ui.emitEvents=true to write progress events for SSE/polling clients.
     * If omitted/false, no execution event stream is written.
     */
    options?: {
      ui?: {
        emitEvents?: boolean;
      };
    };

    /**
     * Webhooks to call when the scrape completes, fails, or is cancelled.
     */
    webhooks?: WebhookSubscription[];
  }

interface WebhookSubscription {
    /** The URL to POST to */
    url: string;
    /** Events to subscribe to. Defaults to all scrape events. */
    events?: WebhookEvent[];
    /** Optional custom headers */
    headers?: Record<string, string>;
    /** Optional auth (bearer or basic) */
    auth?: { type: "bearer"; token: string } | { type: "basic"; username: string; password: string };
    /** Optional secret for HMAC signing (X-Rtrvr-Signature header) */
    secret?: string;
    /** Timeout for webhook delivery (default: 8000ms) */
    timeoutMs?: number;
    /** Retry policy (default: { mode: "default" }) */
    retry?: { mode: "default" | "none" };
  }

type WebhookEvent =
    | "rtrvr.scrape.succeeded"
    | "rtrvr.scrape.failed"
    | "rtrvr.scrape.cancelled";
```

### Parameters

`urls`string\[\]required

One or more absolute URLs to scrape. Must be a non-empty array.

`trajectoryId`string

Optional stable id for grouping scrapes together (analytics, observability).

`settings`Partial<UserSettings>

Optional per-request override merged on top of your cloud UserSettings profile.

`response.inlineOutputMaxBytes`numberdefault: `1048576`

Maximum inline response size in bytes (default 1MB).

`options.ui.emitEvents`booleandefault: `false`

Opt-in execution progress events for SSE/polling consumers.

`webhooks`WebhookSubscription\[\]

Optional array of webhook endpoints to notify when the scrape completes, fails, or is cancelled.

Receive HTTP callbacks when your scrape completes, fails, or is cancelled. Webhooks are delivered asynchronously after the scrape finishes.

### Webhook Subscription

`url`stringrequired

The HTTPS endpoint to POST the webhook payload to.

`events`WebhookEvent\[\]

Which events to subscribe to. Defaults to all scrape events.

`"rtrvr.scrape.succeeded"``"rtrvr.scrape.failed"``"rtrvr.scrape.cancelled"`

`headers`Record<string, string>

Custom headers to include with each webhook request.

`auth`object

Authentication for the webhook endpoint. Supports bearer token or basic auth.

`auth.type`"bearer" | "basic"

The authentication type.

`auth.token`string

Bearer token (when type is "bearer").

`auth.username`string

Username (when type is "basic").

`auth.password`string

Password (when type is "basic").

`secret`string

HMAC secret for signing. When provided, requests include X-Rtrvr-Signature header.

`timeoutMs`numberdefault: `8000`

Timeout for webhook delivery in milliseconds.

`retry`object

Retry policy. { mode: "default" } retries with backoff; { mode: "none" } delivers once.

### Example with Webhook

Scrape with webhook notification

```
curl -X POST https://api.rtrvr.ai/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/page1", "https://example.com/page2"],
    "webhooks": [
      {
        "url": "https://your-server.com/webhooks/scrape",
        "events": ["rtrvr.scrape.succeeded", "rtrvr.scrape.failed"],
        "secret": "whsec_your_signing_secret",
        "headers": { "X-Custom-Header": "my-value" }
      }
    ]
  }'
```

### Webhook Payload

Each webhook delivery is a POST request with a JSON envelope:

Webhook envelope

```
{
  "id": "whd_abc123...",          // unique delivery id
  "event": "rtrvr.scrape.succeeded",
  "createdAt": "2025-01-15T10:30:00.000Z",
  "data": {
    "trajectoryId": "traj_xyz...",
    "success": true,
    "tabs": [...],
    "usageData": {...}
  }
}
```

### Signature Verification

When you provide a `secret`, each request includes an `X-Rtrvr-Signature` header:

text

```
X-Rtrvr-Signature: t=1705312200,v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd
```

Verify signature (Node.js)

```
import crypto from 'crypto';

function verifyWebhookSignature(payload, signature, secret) {
  const [tPart, vPart] = signature.split(',');
  const timestamp = tPart.split('=')[1];
  const receivedSig = vPart.split('=')[1];

  // Recreate the signed payload
  const signedPayload = `${timestamp}.${JSON.stringify(payload)}`;
  const expectedSig = crypto
    .createHmac('sha256', secret)
    .update(signedPayload)
    .digest('hex');

  // Timing-safe comparison
  return crypto.timingSafeEqual(
    Buffer.from(receivedSig),
    Buffer.from(expectedSig)
  );
}
```

**Store & reuse webhooks**

Save your webhook endpoints in [Cloud → Webhooks](/cloud?view=webhooks) to quickly attach them to any execution without re-entering the URL, secret, and events each time.

The API response is an `ScrapeApiResponse`:

ScrapeApiResponse (conceptual)

```
interface ScrapedTab {
  tabId: number;
  url: string;
  title: string;
  contentType: string;
  status: "success" | "error";
  error?: string;

  /**
   * Full extracted visible text (when available).
   */
  content?: string;

  /**
   * JSON-encoded accessibility tree (stringified).
   * Use this if you want a rich, structured view of the page for your own models.
   * Every link node in the tree has a numeric 'id' field which is used as the key
   * in elementLinkRecord.
   */
  tree?: string;

  /**
   * Map of accessibility-tree element id -> href/URL for link elements.
   * Only present when 'tree' is present.
   */
  elementLinkRecord?: Record<number, string>;
}

interface ScrapeUsageData {
  totalCredits: number;
  browserCredits: number;
  proxyCredits: number;
  totalUsd: number;
  requestDurationMs: number;
  proxyPageLoads: number;
  proxyTabsDataFetches: number;
  usingBillableProxy: boolean;
}

interface ScrapeApiResponse {
  success: boolean;
  status: "success" | "error";
  trajectoryId: string;

  tabs?: ScrapedTab[];
  usageData: ScrapeUsageData;

  metadata?: {
    taskRef?: string;
    inlineOutputMaxBytes: number;
    durationMs: number;
    outputTooLarge?: boolean;
    responseRef?: StorageReference;
  };

  error?: string;
}
```

### Tabs & content

`tabs`ScrapedTab\[\]

One tab per URL, in the same order as the input urls.

`tabs[].content`string

Full extracted visible text when available.

`tabs[].tree`string

JSON-encoded accessibility tree (stringified). Omitted when onlyTextContent=true.

`tabs[].elementLinkRecord`Record<number, string>

Lookup table mapping accessibility-tree element id → href/URL.

### Infra usage

`usageData.totalCredits`number

Total infra credits consumed by this scrape.

`usageData.browserCredits`number

Credits attributable to browser usage.

`usageData.proxyCredits`number

Credits attributable to proxy usage.

`usageData.requestDurationMs`number

End-to-end latency for the scrape request in ms.

cURL

```
# Basic scrape using profile defaults
curl -X POST https://api.rtrvr.ai/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/blog/ai-trends-2025"],
    "response": { "inlineOutputMaxBytes": 1048576 }
  }'

# With per-request settings override
curl -X POST https://api.rtrvr.ai/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://example.com/blog/ai-trends-2025",
      "https://example.com/pricing"
    ],
    "settings": {
      "extractionConfig": {
        "onlyTextContent": true
      },
      "proxyConfig": {
        "mode": "default"
      }
    },
    "response": {
      "inlineOutputMaxBytes": 1048576
    }
  }'
```

### Ready to automate?

Join teams using rtrvr.ai to build playful, powerful web automation workflows.

[](/cloud)[](/docs/agent)

---

# Browser as API/MCP `/mcp`

Turn your logged-in Chrome profile into an agentic API endpoint, and access cloud-only knowledge base tools from the same MCP surface.

Most `/mcp` tools control your local Chrome via the extension as a remote MCP server. Cloud-only knowledge base tools like `knowledge_base_query` run without a device session.

[Try Playground](#playground)[Install Extension](https://chromewebstore.google.com/detail/retriever-ai-web-agent/jldogdgepmcedfdhgnmclgemehfhpomg)

OAuth Support

Sign in with Google via mcp.rtrvr.ai for secure, token-based authentication.

HTTP + MCP Protocol

Use the MCP URL in Claude/clients, or POST JSON directly from any backend.

Extension + Cloud

Mix device-backed browser tools with cloud-only knowledge base tools on one endpoint.

Chrome only (for now):Extension-as-remote-MCP-server currently supports Chrome. Additional browsers are planned.

Quick setup: Configure MCP from your terminal with [rtrvr CLI](/docs/cli) — `rtrvr mcp init --client claude`

## Video Tutorials

[Browser as API Introduction](https://www.youtube.com/embed/0EecnoFqiYk)

Introduction

[MCP Integration Deep Dive](https://www.youtube.com/embed/B4BTWNTuE-s)

MCP Deep Dive

### Browser as API/MCP Playground

POST`/mcp`

Control your logged-in Chrome or call cloud-only knowledge base tools via HTTP or MCP

API Key

Get from [rtrvr.ai/cloud](/cloud?view=api-keys) or your [Chrome Extension](https://chromewebstore.google.com/detail/retriever-ai-web-agent/jldogdgepmcedfdhgnmclgemehfhpomg)

Device ID (optional)

Leave blank to auto-select. Use `list_devices` to find IDs.

Select Tool

Execute complex multi-step tasks from natural language

Task Description\*

Starting URLs

URLs to open before starting

Additional Context

Max Steps

curl -X POST "https://mcp.rtrvr.ai" \\
  -H "Authorization: Bearer rtrvr\_xxxxxxxxxxxxxxxxx" \\
  -H "Content-Type: application/json" \\
  -d '{
  "tool": "planner",
  "params": {}
}'

Endpoint URL`https://mcp.rtrvr.ai`

Shared by both direct HTTP calls and MCP clients. Extension-generated MCP URLs include your **API key** and **deviceId**; cloud-only knowledge base tools do not require a device.

The rtrvr.ai Chrome Extension registers as a remote browser device and exposes a single public entrypoint at `https://mcp.rtrvr.ai`. That same endpoint speaks:

-   **MCP:** paste the generated MCP URL (includes `apiKey` + `deviceId`) into any MCP-enabled client (e.g. Claude).
-   **HTTP:** POST JSON describing `tool` + `params`. Extension-backed tools dispatch into your online browser devices, while cloud-only knowledge base tools execute immediately without a device.

"Your Chrome browser is now an Agentic API Endpoint."

Trigger complex workflows in your own logged-in browser instance from CI/CD, Slack bots, cron jobs, or backend services.

OAuth Support (Recommended)

MCP clients that support OAuth can authenticate by connecting to `mcp.rtrvr.ai` directly. This triggers a Google Sign-In flow and returns a secure session token.

API Key in URL (Fallback)

For MCP clients without OAuth support, embed your API key directly in the URL:

`https://mcp.rtrvr.ai?apiKey=rtrvr_your_api_key&deviceId=your_device_id`

For direct HTTP calls, auth can be provided via:

-   `Authorization: Bearer YOUR_API_KEY` (recommended)
-   `X-API-Key: YOUR_API_KEY`
-   Query param: `?apiKey=YOUR_API_KEY`

Header

```
Authorization: Bearer rtrvr_your_api_key
```

**Security:** Prefer OAuth or headers in production. Avoid exposing API keys in URLs/logs when possible.

Each Chrome/Chromium profile you install the extension into registers as a separate `deviceId`. This enables powerful multi-device workflows:

How deviceId works:

-   Your `deviceId` is embedded in the MCP URL generated by the extension
-   Install the extension on multiple browsers/profiles to get multiple deviceIds
-   Target a specific device by passing `deviceId` in your request
-   **Leave deviceId blank** to auto-select the most recently active online device

### Device selection rules:

-   **Explicit deviceId:** Pass `deviceId` or `device_id` in the query or body to target a specific device.
-   **No deviceId (default):** The most recently active online device is auto-selected (highest `lastSeen` timestamp).
-   **Multi-profile orchestration:** Install the extension on multiple browser profiles and orchestrate them independently by calling `/mcp` with different device IDs.
-   **Device-independent tools:** Utility tools like `get_current_credits` and `list_devices` work even if no device is online.

### Finding your deviceId:

-   **From MCP URL:** Open the extension → MCP / Remote Browser → copy the MCP URL. The `deviceId` parameter is embedded in the URL.
-   **Via API:** Call `list_devices` to see all registered devices and their online/offline status.

Discovering devices

```
// List all your devices
curl -X POST "https://mcp.rtrvr.ai" \
  -H "Authorization: Bearer rtrvr_xxx" \
  -H "Content-Type: application/json" \
  -d '{"tool": "list_devices"}'

// Response shows deviceId + online status
{
  "success": true,
  "data": {
    "devices": [
      { "deviceId": "dj75mmaTWP0", "online": true, "lastSeen": "2025-01-15T10:30:00Z" },
      { "deviceId": "abc123XYZ", "online": false, "lastSeen": "2025-01-14T18:00:00Z" }
    ]
  }
}
```

POST`https://mcp.rtrvr.ai`

The same endpoint powers both MCP and HTTP. For HTTP, you send a single JSON object describing which tool to run, which parameters to pass through, and optionally which device to target for extension-backed tools.

BrowserAgentApiRequest (conceptual)

```
interface BrowserAgentApiRequest {
  /**
   * Canonical tool name, e.g. "planner" or "get_browser_tabs".
   * Only one of "tool" or "action" is required.
   */
  tool?: string;

  /**
   * Optional alias of "tool". Use canonical snake_case tool names.
   */
  action?: string;

  /**
   * Parameters for the tool. "params" and "parameters" are equivalent.
   * Prefer canonical snake_case parameter names.
   */
  params?: Record<string, any>;
  parameters?: Record<string, any>;

  /**
   * Optional: route extension-backed tools to a specific Chrome profile / device.
   * Cloud-only knowledge base tools ignore this field.
   */
  deviceId?: string;
  device_id?: string;

  /**
   * Per-request timeout in milliseconds (default: 300000 / 5 minutes).
   */
  timeout?: number;

  /**
   * Reserved for future async modes.
   */
  async?: boolean;
  webhookUrl?: string;
}
```

### Top-level fields

`tool`string

Canonical tool name (snake\_case), e.g. 'planner' or 'get\_browser\_tabs'. Only one of tool/action is required.

`action`string

Optional alias of tool. Use canonical snake\_case tool names for reliability.

`params / parameters`object

JSON object of tool-specific parameters. 'params' and 'parameters' are aliases. Prefer snake\_case parameter names.

`params.options.ui.emitEvents`booleandefault: `false`

Opt-in only. Set true to write execution progress events for SSE/polling consumers. If omitted/false, no execution event stream is written.

`deviceId / device_id`string

Optional device routing for extension-backed tools. If omitted, we pick the most recently active online browser extension device for that user. Cloud-only knowledge base tools ignore this field.

`timeout`numberdefault: `300000`

Max execution time for this request in milliseconds. Defaults to 5 minutes.

`async / webhookUrl`boolean / string

Reserved for future async execution modes. Ignored for now.

**emitEvents policy:** API and extension/MCP executions only emit events when`options.ui.emitEvents: true`. CLI streaming defaults on for `run`, `agent`, and `scrape` (use `--no-stream` to disable). Streamed payloads at or under 1MB stay inline; larger payloads include inline preview markers and storage references (\`outputRef\` / \`resultRef\` / \`responseRef\`) for full downloads.

The Browser as API/MCP exposes extension-backed browser tools, cloud-only tools, scheduling/trigger management, and utility tools. They are grouped into free, credit-based, cloud, utility, and user-defined families.

### Free tools (no credits)

-   `get_browser_tabs` – list open tabs (filter by all/active/domain).
-   `get_page_data` – get accessibility-tree representations for specific tab IDs.
-   `take_page_action` – run system tools like click, type, scroll, etc.
-   `execute_javascript` – run JS inside a secure browser sandbox (disabled by default).

### Credit-based tools

-   `planner` – multi-step planning and tool orchestration from natural language.
-   `act_on_tab` – intelligent page interaction with optional structured schemas.
-   `extract_from_tab` – structured extraction to JSON or Google Sheets.
-   `crawl_and_extract_from_tab` – multi-page crawls with schema extraction.
-   `replay_workflow` – replay a previously executed workflow by task ID or shared URL.
-   `schedule` – create or modify a scheduled workflow on the browser extension.
-   `trigger_setup` – create or modify a trigger workflow on the browser extension.

### Cloud tools (no extension required)

-   `cloud_scrape` – scrape web pages using cloud browsers, returns accessibility trees.
-   `cloud_agent` – execute AI agent tasks using cloud browsers with structured output.
-   `knowledge_base_create_store` – create an empty private knowledge base store.
-   `knowledge_base_list_stores` – list your private knowledge base stores and their metadata.
-   `knowledge_base_query` – query a private knowledge base by `store_id`.
-   `knowledge_base_batch_index` – batch index scraped pages into a private knowledge base.
-   `check_schedule_results` – check execution results of scheduled workflows from cloud storage.
-   `check_trigger_results` – check execution results of trigger workflows from cloud storage.

### Utility & user-defined tools

-   `list_devices` – list all registered extension devices and online/offline status.
-   `get_current_credits` – fetch current plan, credits used, and credits remaining.
-   `list_recordings` – list all user recordings (returns metadata: ID, name, timestamp).
-   `list_file_search_stores` – list all file search / knowledge base stores and metadata.
-   `list_custom_functions` – list all custom functions (metadata only, not code).
-   `list_schedules` – list all scheduled workflow configurations. Works offline.
-   `list_triggers` – list all trigger workflow configurations. Works offline.
-   `user_function` – user-defined tools created in Cloud and executed in the extension sandbox.

### Tool name aliases

Use canonical snake\_case tool names. For direct HTTP, `action` can mirror `tool`:

-   `tool: "agent"` – unified alias that routes to `cloud_agent` (default) or `planner` (when extension/local session is requested).
-   `tool: "scrape"` – unified alias that routes to `cloud_scrape` (default) or extension scrape.
-   `tool: "planner"` is equivalent to `action: "planner"`.
-   `tool: "act_on_tab"` is equivalent to `action: "act_on_tab"`.
-   For MCP protocol calls, use tool names exactly as returned by `tools/list`.

### Parameter aliases

Prefer snake\_case parameters. Common compatibility aliases include:

-   `user_input` ⇔ `userInput`
-   `tab_urls` ⇔ `tabUrls`
-   `device_id` ⇔ `deviceId`
-   `max_steps` ⇔ `maxSteps`
-   `task_id` ⇔ `taskId`
-   `recording_id` ⇔ `recordingId`
-   `file_urls` ⇔ `fileUrls`
-   `image_urls` ⇔ `imageUrls`
-   `shared_workflow_url` ⇔ `sharedWorkflowUrl`
-   `store_id` ⇔ `storeId`
-   `conversation_context` ⇔ `conversationContext`
-   `tab_ids` ⇔ `tabIds`
-   `web_page_map` ⇔ `webPageMap`
-   `auth_token` ⇔ `authToken`

All credit tools support file and image inputs via publicly fetchable URLs. Files are automatically uploaded to Firebase Storage and passed to the agent as context.

Supported input parameters:

-   `file_urls` – array of publicly fetchable file URLs (CSV, PDF, etc.)
-   `image_urls` – array of publicly fetchable image URLs (JPG, PNG, etc.)
-   `recording_id` – ID of a recorded workflow to use as context

`file_urls`

CSV, PDF, text files to use as input data

`image_urls`

JPG, PNG images for visual context

`recording_id`

Recorded workflow to use as context

Example with file inputs

```
{
  "tool": "planner",
  "params": {
    "user_input": "Upload this CSV and submit the form",
    "tab_urls": ["https://example.com/upload"],
    "file_urls": ["https://example.com/data.csv"],
    "image_urls": ["https://example.com/screenshot.png"],
    "recording_id": "rec_abc123"
  }
}
```

The `replay_workflow` tool allows you to re-execute a previously completed workflow. You can replay your own workflows by task ID, or replay workflows shared by other users via a shared URL.

Two ways to replay:

-   `task_id` – replay your own workflow by execution ID
-   `shared_workflow_url` – replay a workflow shared by another user

At least one of these must be provided.

### Parameters

`task_id`string

The task ID from a previous workflow execution in your history.

`shared_workflow_url`string

A shared workflow URL (e.g., https://rtrvr.ai/shared/Tasks/userId/taskId/token). Use this to replay workflows shared by other users.

`tab_execution_mode`stringdefault: `new_tabs`

How to handle tabs during replay.

`"new_tabs"``"reuse_tabs"``"current_context"`

`recording_id`string

Optional recording ID to use as additional context.

`file_urls`string\[\]

Optional file URLs to include as input.

`image_urls`string\[\]

Optional image URLs to include as input.

Replay workflow examples

```
// By task ID (your own workflow)
{
  "tool": "replay_workflow",
  "params": {
    "task_id": "abc123xyz",
    "tab_execution_mode": "new_tabs"
  }
}

// By shared URL (another user's workflow)
{
  "tool": "replay_workflow", 
  "params": {
    "shared_workflow_url": "https://rtrvr.ai/shared/Tasks/userId/taskId/token"
  }
}
```

Below is a conceptual TypeScript view of each tool's parameters.

Agentic tool parameters

```
// Free tools
get_browser_tabs({
  filter?: "all" | "active" | "domain";
  domain?: string;
  device_id?: string;
});

get_page_data({
  tabIds: number[];
  device_id?: string;
});

take_page_action({
  actions: {
    tab_id?: number;
    tool_name: SystemToolName;
    args: Record<string, any>;
  }[];
  device_id?: string;
});

execute_javascript({
  code: string;
  timeout?: number;
  context?: Record<string, any>;
  device_id?: string;
});

// Credit tools
planner({
  user_input: string;
  context?: string;
  tab_urls?: string[];
  max_steps?: number;
  device_id?: string;
  recording_id?: string;      // Recording ID to use as workflow context
  file_urls?: string[];       // Publicly fetchable file URLs
  image_urls?: string[];      // Publicly fetchable image URLs
});

act_on_tab({
  user_input: string;
  tab_urls?: string[];
  schema?: {
    fields: {
      name: string;
      description: string;
      type: string;
      required?: boolean;
    }[];
  };
  tab_id?: number;
  device_id?: string;
  recording_id?: string;
  file_urls?: string[];
  image_urls?: string[];
});

extract_from_tab({
  user_input: string;
  tab_urls?: string[];
  schema?: {
    fields: {
      name: string;
      description: string;
      type: string;
      required?: boolean;
    }[];
  };
  output_destination?: {
    type: "json" | "google_sheet";
    new_sheet_title?: string;
    new_tab_title?: string;
    existing_sheet_id?: string;
    existing_tab_title?: string;
  };
  tab_id?: number;
  device_id?: string;
  recording_id?: string;
  file_urls?: string[];
  image_urls?: string[];
});

crawl_and_extract_from_tab({
  user_input: string;
  tab_urls?: string[];
  schema?: { fields: { name: string; description: string; type: string; required?: boolean; }[]; };
  max_pages?: number;
  follow_links?: boolean;
  link_pattern?: string;
  output_destination?: { type: "json" | "google_sheet"; new_sheet_title?: string; };
  tab_id?: number;
  device_id?: string;
  recording_id?: string;
  file_urls?: string[];
  image_urls?: string[];
});

replay_workflow({
  task_id?: string;              // Task ID from previous execution
  shared_workflow_url?: string;  // Shared workflow URL from another user
  tab_execution_mode?: "new_tabs" | "reuse_tabs" | "current_context";
  recording_id?: string;
  file_urls?: string[];
  image_urls?: string[];
});

// Cloud-only knowledge base tools (no device_id required)
knowledge_base_create_store({
  display_name: string;
});

knowledge_base_list_stores({});

knowledge_base_query({
  store_id: string;
  query: string;
  conversation_context?: string;
  model?: string;
});

knowledge_base_batch_index({
  store_id: string;
  tab_ids: number[];
  web_page_map: Record<number, {
    url: string;
    title: string;
    content_type: string;
    content?: string;
    tree?: string;
    element_link_record?: Record<number, string>;
    acc_tree_id?: string;
  }>;
  results?: { tab_id: number; success: boolean; url?: string; error?: string }[];
  auth_token?: string;
});

// User functions (defined in Cloud and executed in the browser sandbox)
user_function({
  functionName: string;
  // your custom parameters...
});
```

All tools return a consistent envelope with tool-specific `data` plus metadata:

BrowserAgentApiResponse (conceptual)

```
interface BrowserAgentApiResponse<TData = any> {
  success: boolean;
  data: TData | null;
  error: string | null;
  metadata: {
    requestId: string;
    executionTime: number;
    tool: string;
    deviceId?: string;
    creditsUsed?: number;
    creditsRemaining?: number;
    inlineOutputMaxBytes?: number;
    outputTooLarge?: boolean;
    responseRef?: StorageReference;
  };
  timestamp: string;
}
```

Example: get\_browser\_tabs response

```
{
  "success": true,
  "data": {
    "tabs": [{ "id": 1, "url": "https://example.com" }],
    "activeTab": { "id": 1, "url": "https://example.com" },
    "tabCount": 1
  },
  "error": null,
  "metadata": {
    "requestId": "req_abc123",
    "executionTime": 1234,
    "tool": "get_browser_tabs",
    "deviceId": "dj75mmaTWP0",
    "creditsUsed": 0,
    "creditsRemaining": 10000
  },
  "timestamp": "2025-01-01T12:00:00.000Z"
}
```

-   `metadata.deviceId` shows which device executed the request.
-   `metadata.requestId` is useful for joining logs with rtrvr.ai's internal state.
-   HTTP headers like `X-Credits-Used` and `X-Credits-Remaining` are also surfaced.
-   Large payloads keep inline previews and expose storage refs via `metadata.responseRef` and/or `outputRef`/`resultRef`.

Remote tool execution can be configured at a per-user and per-tool level via the extension settings and the Cloud dashboard.

-   Disable remote browser tools entirely for a given user.
-   Independently enable/disable free tools and credit tools.
-   Toggle individual tools inside each family.
-   All configuration is respected by the MCP handler; disabled tools yield an explanatory error.

**Important:** Remote tool execution is enabled by default. Review allowed tools in the Chrome extension MCP / Remote Tools settings to match your security posture.

If `/mcp` calls fail with `No online devices found` or other errors:

-   Open/close the rtrvr.ai Chrome extension popup to refresh its connection.
-   Rotate your API key from the extension dropdown to re-sync backend state.
-   Sign out and sign back into the extension to reset device registration.
-   Call `list_devices` to see which devices are online.
-   Ensure the deviceId you're targeting matches an online device.

Most device issues are resolved by refreshing the extension, rotating the API key, or resetting the sign-in.

1.  Install the [rtrvr.ai Chrome Extension](https://chromewebstore.google.com/detail/retriever-ai-web-agent/jldogdgepmcedfdhgnmclgemehfhpomg).
2.  Open the MCP / Remote Browser section in the extension.
3.  Copy the generated MCP URL — your **API key** and **deviceId** are already embedded. Alternatively, get your API key from [rtrvr.ai/cloud](/cloud?view=api-keys).
4.  For MCP clients: paste the URL directly into your MCP-enabled application (e.g. Claude).
5.  For direct HTTP: call `https://mcp.rtrvr.ai` with your API key in headers. Optionally pass `deviceId` to target a specific device.
6.  **Multi-device setup:** Install the extension on multiple Chrome profiles. Each gets its own deviceId. Target specific devices or leave blank for auto-selection.
7.  Configure which tools to enable/disable in extension settings.

The MCP URL generated by the extension includes your API key and deviceId. Treat this URL as a secret: store it safely, avoid logging it in plaintext, and rotate as needed.

These snippets show the recommended integration pattern: a thin server-side helper that wraps POST `/mcp`, keeps your API key off the frontend, and centralizes rate limiting + logging.

cURL

```
# 1) Your browser as an agentic API endpoint
curl -X POST "https://mcp.rtrvr.ai" \
  -H "Authorization: Bearer rtrvr_xxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "planner",
    "params": {
      "user_input": "Go to ChatGPT.com, ask for top Indian restaurants in SF, and extract back citations.",
      "tab_urls": ["https://chatgpt.com"],
      "options": {
        "ui": {
          "emitEvents": true
        }
      }
    },
    "deviceId": "dj75mmaTWP0"
  }'

# 2) Free tool: list all tabs on your most recent device (no deviceId = auto-select)
curl -X POST "https://mcp.rtrvr.ai" \
  -H "X-API-Key: rtrvr_xxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "get_browser_tabs",
    "params": { "filter": "all" }
  }'

# 3) Replay a workflow with file inputs
curl -X POST "https://mcp.rtrvr.ai" \
  -H "Authorization: Bearer rtrvr_xxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "replay_workflow",
    "params": {
      "task_id": "abc123xyz",
      "tab_execution_mode": "new_tabs",
      "file_urls": ["https://example.com/data.csv"]
    }
  }'

# 4) Planner with image context
curl -X POST "https://mcp.rtrvr.ai" \
  -H "Authorization: Bearer rtrvr_xxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "planner",
    "params": {
      "user_input": "Find similar products to the one in this image",
      "tab_urls": ["https://amazon.com"],
      "image_urls": ["https://example.com/product.jpg"]
    }
  }'

# 5) Cloud-only knowledge base create store (no deviceId required)
curl -X POST "https://mcp.rtrvr.ai" \
  -H "Authorization: Bearer rtrvr_xxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "knowledge_base_create_store",
    "params": {
      "display_name": "Research Docs"
    }
  }'

# 6) Cloud-only knowledge base query (no deviceId required)
curl -X POST "https://mcp.rtrvr.ai" \
  -H "Authorization: Bearer rtrvr_xxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "knowledge_base_query",
    "params": {
      "store_id": "hn-w0wa7ckmk7ch",
      "query": "Summarize what this knowledge base says about pricing."
    }
  }'
```

### Ready to automate?

Join teams using rtrvr.ai to build playful, powerful web automation workflows.

[](/cloud)[](/docs/agent)

---

# Shortcuts

Save time by replaying successful automations — rerun, replay, or save as one-command shortcuts.

# Workflows & Shortcuts

Every successful task in rtrvr.ai can be reused. The system offers three levels of reuse — from quick iteration to permanent one-command shortcuts — so you never have to re-explain a task the agent already knows how to do.

## Quick Rerun

Every user message in the chat history has a "Rerun" button. Clicking it opens a new chat thread and re-executes the prompt from scratch — the planner reconsiders the approach, which is useful when you want to iterate on phrasing or try a different strategy.

## Perfect Replay

When a workflow works perfectly, click "Replay" on the agent's response. This skips the planning phase and runs the exact same steps instantly. Replay is deterministic — the agent follows the recorded action sequence without re-planning. This is the foundation for Shortcuts, Schedules, and Triggers.

> **info**: Rerun = re-plan from scratch (AI may choose different steps). Replay = identical steps, no planning (deterministic).

## Creating Shortcuts

Turn any successful task into a shortcut you can trigger with a simple "/" command. Shortcuts bundle the exact steps, recordings, and tab configuration that worked.

- Find a successful task in your chat
- Click the "/" Create Shortcut button on the agent's response
- Give it a memorable name (e.g., "weekly-report", "leads", "price-check")
- Type "/weekly-report" anytime to run it instantly

## Smart Tab Handling

When creating shortcuts, choose how tabs are managed:

| Mode | Behavior | Best For |
| --- | --- | --- |
| Open in New Tabs | Recreates the exact URLs from the original run | Always scraping the same specific sites |
| Use Current Context | Applies the workflow logic to whatever tabs you have open | Repetitive tasks on different pages (e.g., extract data from any product page) |

## Combining Workflows

Build powerful automations by chaining multiple shortcuts or merging different successful workflows into a single new shortcut. You can also chain them inline:

```text
"First /login then /extract-data and /export-csv"
```

> **success**: Configure a single shortcut from multiple prior workflows for the ultimate one-command automation.

## Platform Availability

| Capability | Extension | Cloud | API |
| --- | --- | --- | --- |
| Quick Rerun | ✅ | — | — |
| Perfect Replay | ✅ | ✅ (replay_workflow) | ✅ (replay_workflow) |
| "/" Shortcuts | ✅ | — | — |
| Share workflow via URL | ✅ | ✅ | ✅ |
| Import shared workflows | ✅ | ✅ | ✅ |

---

# Triggers

Monitor tab notifications and fire workflows automatically — no API keys, no server setup, no site webhooks required.

# Browser Triggers

Most websites do not expose webhooks or APIs for real-time events. Browser Triggers solve this by monitoring your open tabs for web push notifications and executing workflows when matching notifications arrive.

[Video: Browser Triggers from Tab Notifications](https://www.youtube.com/watch?v=ZRyyI8qffUI)

> **success**: If a site can send browser notifications, you can trigger workflows from it.

## How Browser Triggers Work

The extension injects an interceptor into monitored tabs that patches Notification and ServiceWorkerRegistration.showNotification. Notification title/body metadata is matched against your trigger filters, and matching events run the linked workflow automatically.

- Intercept notifications from page and Service Worker paths
- Match by hostname plus optional title/body contains filters
- Run linked workflow in a new tab when a match is found
- Store execution history with matched payload and status

## Creating a Browser Trigger

- Run a workflow successfully in side panel chat
- Click the "Trigger" button on the workflow response
- Enter the URL to monitor (for example: x.com, app.slack.com)
- Add optional title/body filters
- Set cooldown to prevent rapid re-firing
- Save to activate monitoring

## Notification Matching

| Filter | Example | Matches |
| --- | --- | --- |
| Title contains | "new message" | Notifications with "new message" in title |
| Body contains | "price drop" | Notifications with "price drop" in body |
| Both | Title: "alert", Body: "urgent" | Only when both match |
| None | — | All notifications from monitored hostname |

> **info**: Matching is case-insensitive and uses simple contains logic.

## Cooldown Recommendations

| Site Type | Recommended Cooldown |
| --- | --- |
| Chat apps (Slack, Discord) | 5-15 minutes |
| Social apps (X, LinkedIn) | 15-30 minutes |
| Price alerts | 30-60 minutes |
| Daily digests | 1440 minutes (24 hours) |

## Monitored Tab Groups

- Matching tabs are grouped into a yellow "Monitored" tab group
- Closing a tab pauses monitoring for that tab
- Reopening the URL resumes monitoring automatically

## Manual Testing

Use the "Test" action in the Triggers dropdown to run the workflow immediately without waiting for a live notification.

## Limitations

- Monitored tabs must stay open
- Pure Service Worker push flows may not always be interceptable
- Chrome must be running; for 24/7 inbound signals use /docs/webhooks

## Best Practices

- Use specific filters to avoid false positives
- Set cooldowns for chatty sources to manage credit burn
- Validate every trigger with Test before production usage

---

# Webhooks

Trigger workflows from external systems and receive async completion callbacks with signatures and retries.

# Webhooks

Webhooks let external systems trigger rtrvr workflows via HTTP POST and let rtrvr send completion/failure callbacks back to your infrastructure. Use this for Zapier, Make, n8n, backend jobs, and server-to-server pipelines.

[Video: n8n + Extension Integration](https://www.youtube.com/watch?v=pcPRZsBs2aQ)

[Video: Cloud Browser API from n8n](https://www.youtube.com/watch?v=FOmwVh1tJw0)

> **info**: Need browser push-notification based automation instead? See /docs/triggers.

## Platform Compatibility

| Platform | HTTP Timeout | Compatibility | Recommended Approach |
| --- | --- | --- | --- |
| n8n | 100s (Cloud) | Excellent | Direct calls work for most tasks |
| Make.com | Up to 300s | Excellent | Set timeout to 120s+ |
| Zapier | 30s fixed | Use callbacks | Provide webhookUrl for async results |

## Inbound Webhooks (Zapier, Make, n8n → rtrvr)

Any service that can send an HTTP POST can trigger rtrvr.ai workflows. Use the MCP endpoint to control your logged-in browser, or the /agent endpoint for cloud browser execution.

### MCP Endpoint (Your Browser)

```text
POST https://mcp.rtrvr.ai

Headers:
  Authorization: Bearer rtrvr_your_api_key
  Content-Type: application/json

Body:
{
  "tool": "planner" | "extract" | "act" | "crawl" | "replay_workflow" | ...,
  "params": { ... tool-specific parameters ... },
  "deviceId": "optional_device_id",
  "webhookUrl": "https://your-server.com/callback"  // optional: receive results
}
```

### Agent Endpoint (Cloud Browser)

```bash
POST https://api.rtrvr.ai/agent

Headers:
  Authorization: Bearer YOUR_API_KEY
  Content-Type: application/json

Body:
{
  "input": "Extract company info and contact details",
  "urls": ["https://example.com"],
  "webhookUrl": "https://your-server.com/callback",
  "response": { "verbosity": "final" }
}
```

### n8n Integration

n8n Cloud has a 100-second timeout — comfortably above most rtrvr task durations. Use either endpoint depending on whether you need your logged-in browser or a cloud browser.

```json
// n8n → MCP (your logged-in browser)
{
  "method": "POST",
  "url": "https://mcp.rtrvr.ai",
  "body": {
    "tool": "planner",
    "params": {
      "user_input": "{{ $json.task_description }}",
      "tab_urls": ["{{ $json.target_url }}"]
    },
    "webhookUrl": "{{ $node.Webhook.url }}"
  }
}

// n8n → /agent (cloud browser)
{
  "method": "POST",
  "url": "https://api.rtrvr.ai/agent",
  "sendHeaders": true,
  "headerParameters": {
    "parameters": [
      { "name": "Authorization", "value": "Bearer {{ $credentials.rtrvrApiKey }}" },
      { "name": "Content-Type", "value": "application/json" }
    ]
  },
  "sendBody": true,
  "bodyParameters": {
    "parameters": [
      { "name": "input", "value": "={{ $json.taskDescription }}" },
      { "name": "urls", "value": "={{ [$json.targetUrl] }}" }
    ]
  }
}
```

### Make (Integromat) Integration

> **warning**: Make's default timeout is 30 seconds. Click "Show advanced settings" and set Timeout to 120 seconds for reliable results.

- Add an "HTTP > Make a request" module
- URL: https://mcp.rtrvr.ai  |  Method: POST
- Headers: Authorization (Bearer token), Content-Type (application/json)
- Body type: Raw, Content type: JSON
- Request content: Your tool + params JSON

### Zapier Integration

Zapier has a fixed 30-second HTTP timeout. Use the webhook pattern: Zap 1 triggers rtrvr with a webhookUrl pointing to a Catch Hook in Zap 2, which receives and processes the results.

- Add a "Webhooks by Zapier" action to your Zap
- Select "POST" as the method
- Set URL to: https://mcp.rtrvr.ai
- Add headers: Authorization = Bearer rtrvr_your_api_key, Content-Type = application/json
- Set Data to your JSON payload (tool + params)
- Set webhookUrl to a Catch Hook URL in a second Zap

```json
// Example: Extract data when a new row is added to Google Sheets
{
  "tool": "extract",
  "params": {
    "user_input": "Extract the company name, employee count, and funding info",
    "tab_urls": ["{{Google Sheets Row URL}}"]
  },
  "webhookUrl": "https://hooks.zapier.com/hooks/catch/123/abc/"
}
```

### Available Tools

| Tool | Use Case | Execution |
| --- | --- | --- |
| planner | Complex multi-step tasks from natural language | Local or cloud browser |
| extract | Structured data extraction with optional schema | Local or cloud browser |
| act | Page interactions (click, type, navigate) | Local or cloud browser |
| crawl | Multi-page crawling with extraction | Local or cloud browser |
| replay_workflow | Re-run a previous workflow by ID or URL | Local or cloud browser |
| get_browser_tabs | List open tabs | Local browser only |
| execute_javascript | Run JS in browser sandbox | Local browser only |

## Outbound Webhooks (rtrvr → Your Server)

Include a webhookUrl in any API request to receive results when the workflow completes. rtrvr.ai will POST the full response to your endpoint.

### Enabling Outbound Webhooks

```bash
curl -X POST "https://mcp.rtrvr.ai" \
  -H "Authorization: Bearer rtrvr_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "planner",
    "params": {
      "user_input": "Find pricing for iPhone 16 Pro on Apple.com",
      "tab_urls": ["https://apple.com"]
    },
    "webhookUrl": "https://your-server.com/rtrvr-callback",
    "webhookSecret": "your_hmac_secret"
  }'
```

## Webhook Payloads

### Success Payload

```json
{
  "event": "workflow.completed",
  "timestamp": "2025-01-15T12:00:00.000Z",
  "requestId": "req_abc123xyz",
  "success": true,
  "data": {
    "taskCompleted": true,
    "output": { ... },
    "extractedData": [ ... ],
    "creditsUsed": 5
  },
  "metadata": {
    "tool": "planner",
    "deviceId": "dj75mmaTWP0",
    "executionTime": 15234,
    "creditsRemaining": 9995
  },
  "originalRequest": {
    "tool": "planner",
    "params": { ... }
  }
}
```

### Error Payload

```json
{
  "event": "workflow.failed",
  "timestamp": "2025-01-15T12:00:00.000Z",
  "requestId": "req_abc123xyz",
  "success": false,
  "error": {
    "message": "Device offline: no available browser extensions",
    "code": "DEVICE_UNAVAILABLE",
    "details": { ... }
  },
  "metadata": { "tool": "planner", "executionTime": 1234 }
}
```

## Verifying Webhook Signatures

If you provide a webhookSecret, rtrvr.ai signs the payload with HMAC-SHA256. Verify it to ensure authenticity:

```typescript
// Express.js
import crypto from 'crypto';

app.post('/rtrvr-callback', express.raw({ type: 'application/json' }), (req, res) => {
  const signature = req.headers['x-rtrvr-signature'] as string;
  const timestamp = req.headers['x-rtrvr-timestamp'] as string;

  // Reject stale timestamps (> 5 minutes)
  if (Date.now() - parseInt(timestamp) > 300000) {
    return res.status(400).json({ error: 'Timestamp too old' });
  }

  const payload = timestamp + '.' + req.body.toString();
  const expected = crypto
    .createHmac('sha256', process.env.RTRVR_WEBHOOK_SECRET!)
    .update(payload)
    .digest('hex');

  if (!crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))) {
    return res.status(401).json({ error: 'Invalid signature' });
  }

  const data = JSON.parse(req.body.toString());
  // Process asynchronously — respond 200 immediately
  res.status(200).json({ received: true });
  processWebhook(data);
});
```

```python
# Flask
import hmac, hashlib, time, os
from flask import Flask, request, jsonify

WEBHOOK_SECRET = os.environ['RTRVR_WEBHOOK_SECRET']

@app.route('/rtrvr-callback', methods=['POST'])
def handle_webhook():
    signature = request.headers.get('X-Rtrvr-Signature')
    timestamp = request.headers.get('X-Rtrvr-Timestamp')

    if abs(time.time() * 1000 - int(timestamp)) > 300000:
        return jsonify({'error': 'Timestamp too old'}), 400

    payload = f"{timestamp}.{request.data.decode()}"
    expected = hmac.new(WEBHOOK_SECRET.encode(), payload.encode(), hashlib.sha256).hexdigest()

    if not hmac.compare_digest(signature, expected):
        return jsonify({'error': 'Invalid signature'}), 401

    data = request.json
    return jsonify({'received': True})  # Process async
```

## Retry Behavior

Failed deliveries are retried with exponential backoff:

- Attempt 1: Immediate
- Attempt 2: After 5 seconds
- Attempt 3: After 30 seconds
- Attempt 4: After 2 minutes
- Attempt 5: After 10 minutes (final)

> **warning**: Respond with 200 OK immediately. Process webhook payloads asynchronously to avoid timeouts.

## Common Patterns

### Lead Enrichment Pipeline

New lead in CRM → rtrvr extracts company data → webhook returns enriched info → update CRM record.

```json
// Trigger: New HubSpot contact
// Action: POST to https://api.rtrvr.ai/agent
{
  "input": "Visit this company website and extract: company size, industry, tech stack, and key contacts",
  "urls": ["{{contact.company_website}}"],
  "webhookUrl": "https://hooks.zapier.com/catch/123/enrich/",
  "response": { "verbosity": "final" }
}
// Webhook receives enriched data → Update CRM
```

### Browser Trigger → Sheet Log (Zero Server)

The simplest pattern: monitor a site for notifications and log events to Google Sheets. No server, no API, no webhook endpoint needed.

- Create a workflow that extracts data from the site and appends to a Google Sheet
- Set up a Browser Trigger on that site with appropriate filters
- Configure sheet output to "Append to same sheet on each run"
- Every matching notification adds a row — building a running log automatically

### Scheduled Price Monitoring

Cron schedule → rtrvr checks competitor prices → compare with previous data → alert if changed.

```json
// Schedule: Daily at 9am via n8n Cron node
// Action: POST to https://api.rtrvr.ai/scrape
{
  "urls": [
    "https://competitor1.com/pricing",
    "https://competitor2.com/pricing"
  ]
}
// Compare extracted prices with yesterday's data
// If changed → Send Slack/email notification
```

### Browser Trigger + Outbound Webhook (Hybrid)

```text
Flow:
  1. Browser Trigger monitors slack.com for "deployment failed" notifications
  2. Trigger fires → workflow extracts error details from the Slack thread
  3. Workflow calls your server via rtrvr.ai API with webhookUrl set
  4. Server receives error details → creates a Jira ticket automatically

Result: Slack notification → browser extraction → server-side ticket creation
        No Slack API required — the browser does the heavy lifting
```

### Authenticated Data Sync (MCP)

Your app triggers → rtrvr uses your logged-in browser via MCP → data synced to your database.

```json
// Trigger: Webhook from your application
// Action: POST to https://mcp.rtrvr.ai
{
  "tool": "extract",
  "params": {
    "user_input": "Export my order history from the last 30 days",
    "tab_urls": ["https://vendor-portal.com/orders"]
  },
  "webhookUrl": "https://your-app.com/api/orders/sync"
}
// Your browser navigates using your login session
// Results sent to webhook → stored in database
```

### Zapier → rtrvr → Zapier (Round-trip)

- Zap 1: New Google Form submission → POST to mcp.rtrvr.ai (include webhookUrl pointing to Zap 2)
- Zap 2: Catch Hook receives results → Add row to Google Sheets

### Slack Command → rtrvr → Slack Message

```typescript
app.post('/slack/commands', async (req, res) => {
  const { text, response_url } = req.body;
  res.status(200).json({ text: '🔄 Running extraction...' }); // Ack < 3s

  await fetch('https://mcp.rtrvr.ai', {
    method: 'POST',
    headers: { 'Authorization': 'Bearer rtrvr_xxx', 'Content-Type': 'application/json' },
    body: JSON.stringify({
      tool: 'extract',
      params: { user_input: text, tab_urls: [extractUrlFromText(text)] },
      webhookUrl: 'https://your-server.com/slack-callback',
      webhookMetadata: { response_url },
    }),
  });
});

app.post('/slack-callback', async (req, res) => {
  const { data, originalRequest } = req.body;
  const { response_url } = originalRequest.webhookMetadata;

  await fetch(response_url, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      text: `✅ Extracted: ${JSON.stringify(data.extractedData, null, 2)}`,
    }),
  });
  res.status(200).json({ received: true });
});
```

## Choosing the Right Approach

| Approach | Trigger Signal | Best For | Availability |
| --- | --- | --- | --- |
| Browser Triggers | Push notification in a tab | Sites without APIs — social media, chat, SaaS dashboards | While Chrome is open |
| Schedules | Cron / interval timer | Periodic collection, monitoring, recurring reports | Chrome or 24/7 cloud |
| Inbound Webhooks | HTTP POST from external service | Zapier / Make / n8n, CI/CD, server-to-server | 24/7 cloud |

> **success**: All three approaches can output to Google Sheets, call custom tools, chain multi-step workflows, and include recordings for grounding. Mix and match.

## Best Practices

- Verify signatures on every callback in production
- Implement idempotency to handle retries safely
- Use HTTPS-only endpoints
- Pass webhookMetadata for correlation IDs and routing context

---

# Schedules

Automated task scheduling — run workflows on a timer from your Chrome browser or 24/7 from the cloud.

# Schedules

[Video: Schedules Overview](https://www.youtube.com/watch?v=EWnxfLbTm1g)

Turn any successful workflow into a scheduled task that runs on autopilot. Schedule from the Chrome Extension for local execution using your logged-in sessions, or export to Cloud for 24/7 headless operation. Perfect for price tracking, inventory monitoring, job board scraping, news aggregation, and competitive analysis.

## Creating a Schedule

- Run a task successfully in the chat
- Click "Schedule" on the agent's response
- Set a title and choose frequency: once, hourly, daily, weekly, or custom cron
- Pick your start time and configure advanced options
- Save — your schedule is now active

> **info**: Click the ⓘ button in the Schedule Settings header for quick inline help anytime.

## Browser Execution (Chrome Extension)

Scheduled tasks run in your Chrome browser using your existing logged-in sessions — no re-authentication required. Tasks execute while your browser is open. If a scheduled run is missed (browser closed), it runs automatically when Chrome reopens.

```text
Schedule set for 9 AM → Browser closed overnight → Opens at 10 AM → Task runs immediately
```

### Smart Tab Reuse

Configure schedules to reuse matching open tabs instead of opening new ones. Ideal for authenticated sites where you're already logged in — avoids re-signin and works great with sites that have bot detection.

- In Schedule Config, find "Tab Execution Mode"
- Select "Reuse matching open tabs when possible"
- Keep your target site open in a tab
- The agent uses your existing logged-in session

> **success**: Tab reuse is especially powerful for walled gardens like LinkedIn, Zillow, and Discord where fresh tabs often trigger CAPTCHA or login walls.

### Auto-Append to Sheets

Build a master tracking sheet that grows over time. Configure your schedule to append new data as rows to the same Google Sheet instead of creating a new sheet each run.

- In Schedule Config, expand the workflow step
- Find "Sheet Output Configuration"
- Select "Append to same sheet on each run"
- Each scheduled run adds new rows to your master sheet

```text
Daily job scrape → New listings append as rows → One sheet tracks all jobs over time
```

### Multi-Step Workflows

Chain multiple workflows together in a single schedule. Each step can have its own tab handling and sheet configuration. Steps can pass context using "Reuse tabs from previous step."

```text
Step 1: Scrape job listings → Step 2: Analyze fit with resume → Step 3: Export to tracking sheet
```

## Cloud Scheduling (24/7)

Export workflows to rtrvr.ai Cloud for execution on headless browsers that run 24/7 — even when your laptop is closed. Cloud schedules support the same features (sheet append, multi-step, recordings) plus cron expressions and notification delivery.

### Cron Expressions

```json
{
  "schedule": {
    "type": "cron",
    "expression": "0 */6 * * *",
    "timezone": "America/New_York"
  }
}
```

| Pattern | Meaning |
| --- | --- |
| 0 9 * * 1-5 | Weekdays at 9 AM |
| 0 */2 * * * | Every 2 hours |
| 0 0 1 * * | First day of each month |
| 0 0 * * 0 | Every Sunday at midnight |

### API-Based Schedule Creation

```bash
POST /schedules

{
  "name": "Daily Price Check",
  "prompt": "Check product prices on competitor websites",
  "schedule": {
    "type": "recurring",
    "interval": "daily",
    "time": "09:00",
    "timezone": "UTC"
  },
  "webhook_url": "https://your-app.com/price-updates",
  "active": true
}
```

### Monitoring & Alerts

Get notifications when cloud schedules complete via email, Slack, Discord, or WhatsApp. Set up change-detection alerts for monitoring pipelines:

```json
{
  "monitoring": {
    "enabled": true,
    "alert_on_change": true,
    "alert_threshold": 0.1
  },
  "notifications": [
    { "type": "email", "address": "alerts@yourcompany.com" },
    { "type": "slack", "webhook": "https://hooks.slack.com/..." }
  ]
}
```

## Schedule Management

Manage schedules from the Schedules dropdown in the extension side panel, the Cloud dashboard, or programmatically via API:

```bash
GET    /schedules                          # List all
GET    /schedules/{id}                     # Details
PUT    /schedules/{id}                     # Update
POST   /schedules/{id}/pause               # Pause
POST   /schedules/{id}/resume              # Resume
DELETE /schedules/{id}                     # Delete
GET    /schedules/{id}/executions          # History
```

## Execution History

```json
{
  "executions": [
    {
      "id": "exec_123",
      "started_at": "2025-01-01T09:00:00Z",
      "completed_at": "2025-01-01T09:02:15Z",
      "status": "completed",
      "result": { ... },
      "duration": 135.2
    }
  ],
  "stats": {
    "total_executions": 30,
    "success_rate": 96.7,
    "average_duration": 142.5
  }
}
```

## Error Handling

- Automatic retries with exponential backoff for transient failures
- Alert notifications for persistent failures
- Automatic schedule pausing after repeated failures
- Full error details in execution history

## Best Practices

- Test schedules with a one-time execution before enabling recurring runs
- Use tab reuse for authenticated sites to avoid login walls
- Use sheet append mode to build historical datasets over time
- Set appropriate intervals — don't overwhelm target sites
- Use webhooks for real-time notifications on completion
- Monitor execution history to spot performance degradation early
- Consider timezone differences for global operations

## Platform Availability

| Capability | Extension | Cloud | API |
| --- | --- | --- | --- |
| Create schedules from chat | ✅ | — | — |
| Local browser execution | ✅ | — | — |
| Smart tab reuse | ✅ | — | — |
| 24/7 headless execution | — | ✅ | ✅ |
| Cron expressions | — | ✅ | ✅ |
| Auto-append to sheets | ✅ | ✅ | ✅ |
| Multi-step workflows | ✅ | ✅ | ✅ |
| Email / Slack / Discord alerts | — | ✅ | ✅ |
| API-based management | — | ✅ | ✅ |
| Missed-run catch-up | ✅ | — | — |

> **success**: Pro tip: Combine smart tab reuse + sheet append for the ultimate monitoring workflow on authenticated sites.

---

# Cookie Sync

Sync browser cookies to the cloud so headless browsers can use your logged-in sessions — per-profile isolation, auto-sync scheduling, and encrypted storage.

# Cookie Sync

Cookie Sync lets you securely upload cookies from your local Chrome to the cloud, so rtrvr.ai cloud browsers can use your logged-in sessions. Cloud automations can access authenticated sites (LinkedIn, Gmail, internal tools) without you sharing passwords or manually logging in to cloud browsers.

> **info**: Cookies are encrypted and stored per-device in your private Firebase path. No other users can access them.

## Profile-Based Isolation

Cookie Sync is scoped to the current Chrome profile. Each profile has its own cookie store, and sync only reads from the profile where the extension is running. Use separate profiles for separate accounts.

```text
Profile "Work"     → syncs work Gmail cookies     → deviceId: device-work
Profile "Personal"  → syncs personal Gmail cookies  → deviceId: device-personal
Profile "Client"    → syncs client portal cookies   → deviceId: device-client
```

## How It Works

- Grant the "cookies" permission when prompted (optional, revocable anytime)
- Browse your cookies using the search button — only the current profile's cookies appear
- Check the domains you want to sync (e.g., linkedin.com, github.com)
- Click "Sync All" or enable auto-sync on a schedule
- Cloud browsers automatically restore cookies before running your workflows

## Auto-Sync Scheduling

Cookies expire and rotate frequently. Enable auto-sync to keep them fresh. Choose intervals from 15 minutes to 24 hours based on how aggressively your target sites rotate session tokens.

| Site Type | Recommended Interval |
| --- | --- |
| Banking / sensitive sites | 15–30 minutes |
| Social media | 1–6 hours |
| Internal tools | 12–24 hours |

> **info**: Auto-sync runs via Chrome Alarms, so it works as long as your browser is open.

## Domain Selection

The cookie browser shows all domains with cookies in your current Chrome profile. Use the search bar to filter, then check the domains you want to sync. Selecting "google.com" includes all subdomains (mail.google.com, etc.).

## Cloud Storage

Cookies are stored at a Firebase path unique to your account and device:

```text
user_cookies/{userId}/{deviceId}/{domain}

Example: user_cookies/abc123/device-work/linkedin_com
```

## Security & Privacy

- Cookies are stored in your private Firebase path — no other users can access them
- The "cookies" permission is optional and can be revoked anytime
- You choose exactly which domains to sync — no automatic collection
- "Clear Cloud Data" instantly deletes all synced cookies
- "Revoke Access" removes the permission and clears all cloud data

> **warning**: For maximum security, use a dedicated Chrome profile for automation and only sync the domains you need.

## Troubleshooting

- Verify cookies are synced: check "last synced" timestamps in the domain list
- Re-sync manually: click the refresh icon to force an immediate upload
- Check the right profile: make sure you're syncing from the profile where you're logged in
- Session expired: log back into the site in your browser, then re-sync
- DeviceId mismatch: ensure your cloud workflow targets the correct deviceId

---

# Permissions & Privacy

How rtrvr.ai accesses your data, what permissions it needs, and what it can never touch — enforced at the API level.

# Permissions & Privacy

## Why Google Sheets?

rtrvr.ai uses Google Sheets as a "memory layer" for complex workflows. When you run multi-tab automations, each tab operates as an independent sub-agent that needs to read inputs and write results. Sheets provide a structured way to orchestrate these parallel operations.

## Google Drive: drive.file Scope

We use Google's most restrictive Drive scope: drive.file. This is the smallest permission available and gives us access ONLY to files we create or files you explicitly select through the Google Drive Picker.

| Allowed | Not Allowed |
| --- | --- |
| Create new Google Sheets for workflow outputs | Browse or search your Drive |
| Access Sheets you explicitly select via file picker | Access your existing documents |
| Write results to Sheets rtrvr.ai created | Read files you haven't explicitly shared |
| — | Access shared drives or team files |

> **success**: This restriction is enforced by Google at the API level — not just by our code. It's technically impossible for rtrvr.ai to access files you haven't explicitly granted.

## Local Actions

rtrvr.ai is built on a local-first architecture. The agent runs primarily in YOUR browser using Chrome Extension APIs. Your credentials never leave your machine — we use your existing logged-in sessions and never ask for passwords or store login tokens.

## Data Retention

- We don't store Sheet data — it lives in YOUR Google account
- Task artifacts are retained only for workflow replay purposes
- Execution logs are automatically deleted after one week
- Logs are retained only for debugging and customer support

## AI & Training Data

Your data is NOT used to train AI models. Our contract with Google explicitly forbids training on inference data, and rtrvr.ai does not perform any model training. Your workflows are processed and results returned — nothing more.

## Best Practice: Dedicated Chrome Profile

For maximum peace of mind, use a dedicated Chrome profile for rtrvr.ai. This provides complete isolation from your main browsing and work accounts.

- Click your profile icon in Chrome (top right)
- Click "Add" to create a new profile
- Name it "rtrvr.ai Automation"
- Install the extension in this profile only

> **info**: This eliminates any concerns about cross-contamination with work accounts. Questions? Email security@rtrvr.ai or join our Discord at rtrvr.ai/discord.