mirror/computer
1
0
Fork 0
mirror of https://github.com/trycua/computer.git synced 2025-12-19 03:20:09 -06:00
Code Issues Packages Projects Releases Wiki Activity

rewrite for clarity and link from jupyter nbs

Browse Source
This commit is contained in:
Adam
2025-12-17 20:39:14 -05:00
parent c784cb8a49
commit 4a6e5c99a2
13 changed files with 290 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files

333
Development.md
View File

@@ -1,81 +1,74 @@
# Getting Started
# Development Guide
This guide covers setting up and developing the CUA (Computer Use Agent) monorepo.
## Project Structure
The project is organized as a monorepo with these main packages:
- `libs/core/` - Base package with telemetry support
- `libs/computer/` - Computer-use interface (CUI) library
- `libs/agent/` - AI agent library with multi-provider support
- `libs/som/` - Set-of-Mark parser
- `libs/computer-server/` - Server component for VM
- `libs/lume/` - Lume CLI
**Python Packages** (located in `libs/python/`):
- `core/` - Base package with telemetry support
- `computer/` - Computer-use interface (CUI) library
- `agent/` - AI agent library with multi-provider support
- `som/` - Set-of-Mark parser
- `computer-server/` - Server component for VM
- `mcp-server/` - MCP server implementation
- `bench-ui/` - Benchmark UI utilities
These packages are part of a uv workspace which manages a shared virtual environment and dependencies.
**Other Packages**:
- `libs/lume/` - Lume CLI (Swift)
- `libs/typescript/` - TypeScript packages including `cua-cli`
## Local Development Setup
All Python packages are part of a [uv workspace](https://docs.astral.sh/uv/workspaces/) which manages a shared virtual environment and dependencies.
1. Install Lume CLI:
## Quick Start
1. **Install Lume CLI** (required for local VM management):
```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
```
2. Clone the repository:
2. **Clone the repository**:
```bash
git clone https://github.com/trycua/cua.git
cd cua
```
3. Create a `.env.local` file in the root directory with your API keys:
3. **Create `.env.local`** in the root directory with your API keys:
```bash
# Required for Anthropic provider
ANTHROPIC_API_KEY=your_anthropic_key_here
# Required for OpenAI provider
OPENAI_API_KEY=your_openai_key_here
```
4. Install Node.js dependencies for Prettier and other scripts:
4. **Install Node.js dependencies**:
```bash
# Install pnpm if you don't have it
npm install -g pnpm
# Install all JS/TS dependencies
npm install -g pnpm # if not already installed
pnpm install
```
5. Install Python dependencies and workspace packages:
5. **Install Python dependencies**:
```bash
# First install uv if you don't have it
pip install uv
# Then install all Python dependencies
pip install uv # if not already installed
uv sync
```
6. Open the workspace in VSCode or Cursor:
6. **Open workspace in VS Code/Cursor**:
```bash
# For Cua Python development
# For Python development
code .vscode/py.code-workspace
# For Lume (Swift) development
code .vscode/lume.code-workspace
```
7. Install Pre-commit hooks:
This ensures code formatting and validation run automatically on each commit.
7. **Install pre-commit hooks**:
```bash
uv run pre-commit install
```
<details>
<summary>Why use workspace files?</summary>
Using the workspace file is strongly recommended as it:
- Sets up correct Python environments for each package
@@ -83,159 +76,100 @@ Using the workspace file is strongly recommended as it:
- Enables debugging configurations
- Maintains consistent settings across packages
## Lume Development
Refer to the [Lume README](./libs/lume/Development.md) for instructions on how to develop the Lume CLI.
</details>
## Python Development
### Requirements
- **Python 3.12+** (see [`pyproject.toml`](./pyproject.toml) for exact requirements)
- **uv** - Python package manager
- **pnpm** - Node.js package manager
### Setup
Install all of workspace dependencies with a single command:
Install all workspace dependencies with a single command:
```bash
uv sync
```
This installs all dependencies in the virtual environment `.venv`.
This installs all dependencies in the virtual environment `.venv`. Each Cua package is installed in editable mode, so changes to source code are immediately reflected.
Each Cua package is installed in editable mode, which means changes to the source code are immediately reflected in the installed package.
The `.venv` environment is also configured as the default VS Code Python interpreter in `.vscode/settings.json`.
The `.venv` environment is configured as the default VS Code Python interpreter in [`.vscode/settings.json`](.vscode/settings.json).
### Running Python Scripts
To run Python scripts in the workspace, use the `uv run` command:
Use `uv run` to execute Python scripts:
```bash
uv run python examples/agent_examples.py
```
Or activate the virtual environment manually:
Alternatively, activate the virtual environment manually:
```bash
source .venv/bin/activate
source .venv/bin/activate # On Windows: .venv\Scripts\activate
python examples/agent_examples.py
```
## Running Examples
### Running Examples
The Python workspace includes launch configurations for all packages:
The Python workspace includes launch configurations for running examples:
- "Run Computer Examples" - Runs computer examples
- "Run Agent Examples" - Runs agent examples
- "SOM" configurations - Various settings for running SOM
- "Run Computer Examples + Server" - Runs both Computer Examples and Server simultaneously
To run examples from VSCode / Cursor:
To run examples from VS Code/Cursor:
1. Press F5 or use the Run/Debug view
2. Select the desired configuration
The workspace also includes compound launch configurations:
- "Run Computer Examples + Server" - Runs both the Computer Examples and Server simultaneously
## Code Formatting Standards
## Code Formatting
The Cua project follows strict code formatting standards to ensure consistency across all packages.
### Python Code Formatting
### Python Formatting
#### Tools
The project uses the following tools for code formatting and linting:
- **[Black](https://black.readthedocs.io/)** - Code formatter
- **[isort](https://pycqa.github.io/isort/)** - Import sorter
- **[Ruff](https://beta.ruff.rs/docs/)** - Fast linter and formatter
- **[MyPy](https://mypy.readthedocs.io/)** - Static type checker (configured but not enforced in pre-commit)
- **[Black](https://black.readthedocs.io/)**: Code formatter
- **[isort](https://pycqa.github.io/isort/)**: Import sorter
- **[Ruff](https://beta.ruff.rs/docs/)**: Fast linter and formatter
- **[MyPy](https://mypy.readthedocs.io/)**: Static type checker
These tools are automatically installed when you set up the development environment.
All tools are automatically installed when you run `uv sync`.
#### Configuration
The formatting configuration is defined in the root `pyproject.toml` file:
Formatting configuration is defined in [`pyproject.toml`](./pyproject.toml). See the `[tool.black]`, `[tool.ruff]`, `[tool.mypy]`, and `[tool.isort]` sections for all settings.
```toml
[tool.black]
line-length = 100
target-version = ["py311"]
[tool.ruff]
fix = true
line-length = 100
target-version = "py311"
[tool.ruff.lint]
select = ["E", "F", "B", "I"]
ignore = [
"E501", "E402", "I001", "I002", "B007", "B023", "B024", "B027", "B028",
"B904", "B905", "E711", "E712", "E722", "E731", "F401", "F403", "F405",
"F811", "F821", "F841"
]
fix = true
[tool.ruff.format]
docstring-code-format = true
[tool.mypy]
check_untyped_defs = true
disallow_untyped_defs = true
ignore_missing_imports = true
python_version = "3.11"
show_error_codes = true
strict = true
warn_return_any = true
warn_unused_ignores = false
[tool.isort]
profile = "black"
```
#### Key Formatting Rules
#### Key Rules
- **Line Length**: Maximum of 100 characters
- **Python Version**: Code should be compatible with Python 3.11+
- **Python Version**: Code must be compatible with Python 3.12+
- **Imports**: Automatically sorted (using Ruff's "I" rule)
- **Type Hints**: Required for all function definitions (strict mypy mode)
#### IDE Integration
The repository includes VSCode workspace configurations that enable automatic formatting. When you open the workspace files (as recommended in the setup instructions), the correct formatting settings are automatically applied.
<details>
<summary>IDE configuration details</summary>
##### Python-specific settings
These are configured in `.vscode/settings.json`:
Python-specific IDE settings are configured in [`.vscode/settings.json`](.vscode/settings.json), including:
- Python interpreter path
- Format on save
- Code actions on save
- Black formatter configuration
- Ruff and MyPy integration
```json
{
"python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit",
"source.fixAll": "explicit"
},
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter"
},
"python.formatting.provider": "black",
"ruff.configuration": "${workspaceFolder}/pyproject.toml",
"mypy-type-checker.args": ["--config-file", "${workspaceFolder}/pyproject.toml"],
"mypy-type-checker.path": ["${workspaceFolder}"]
}
```
##### JS/TS-specific settings
##### **JS/TS-specific settings**
JavaScript/TypeScript formatting settings are also in [`.vscode/settings.json`](.vscode/settings.json), ensuring Prettier is used for all JS/TS files.
```json
"[javascript][typescript][typescriptreact][javascriptreact]": {
"editor.defaultFormatter": "esbenp.prettier-vscode"
}
```
- Ensures Prettier is used for all JS/TS files for consistent formatting.
Recommended VS Code Extensions
##### Recommended VS Code Extensions
- **Black Formatter** – `ms-python.black-formatter`
- **Ruff** – `charliermarsh.ruff`
@@ -244,9 +178,12 @@ Recommended VS Code Extensions
- **Prettier** – `esbenp.prettier-vscode`
- **Mypy Type Checker** – `ms-python.mypy-type-checker`
> VSCode will automatically suggest installing the recommended extensions when you open the workspace.
> VS Code will automatically suggest installing the recommended extensions when you open the workspace.
#### Manual Formatting
</details>
<details>
<summary>Manual formatting commands</summary>
To manually format code:
@@ -264,20 +201,24 @@ uv run ruff check .
uv run mypy .
```
</details>
#### Pre-commit Validation
Before submitting a pull request, ensure your code passes all formatting checks:
Before submitting a pull request, ensure your code passes all formatting checks.
**Option 1: Run all hooks via pre-commit (all in a single command)**
**Recommended: Run all hooks via pre-commit**
```bash
# Run hooks on staged files (recommended for quick checks)
uv run pre-commit run
```
- Automatically runs Black, Ruff, isort, Mypy, Prettier, and any other configured hooks.
This automatically runs Black, Ruff, isort, Prettier, TypeScript type checking, and other configured hooks. See [`.pre-commit-config.yaml`](.pre-commit-config.yaml) for the complete list.
**Option 2: Run individual tools manually**
> **Note:** MyPy is currently disabled in pre-commit hooks due to untyped codebase, but it's still configured and can be run manually.
<details>
<summary>Run individual tools manually</summary>
```bash
# Python checks
@@ -287,57 +228,45 @@ uv run ruff check .
uv run mypy .
# JavaScript/TypeScript checks
uv run prettier --check "**/*.{ts,tsx,js,jsx,json,md,yaml,yml}"
# TypeScript typecheck
pnpm prettier:check
node ./scripts/typescript-typecheck.js
```
### JavaScript / TypeScript Formatting (Prettier)
</details>
### JavaScript / TypeScript Formatting
The project uses **Prettier** to ensure consistent formatting across all JS/TS/JSON/Markdown/YAML files.
#### Installation
All Node.js dependencies are managed via `pnpm`. Make sure you have run:
All Node.js dependencies are managed via `pnpm`:
```bash
# Install pnpm if you don't have it
npm install -g pnpm
# Install project dependencies
npm install -g pnpm # if not already installed
pnpm install
```
This installs Prettier and other JS/TS dependencies defined in `package.json`.
#### Usage
- **Check formatting** (without making changes):
```bash
pnpm prettier:check
```
```bash
pnpm prettier:check
```
- **Automatically format files**:
```bash
pnpm prettier:format
```
```bash
pnpm prettier:format
```
- **TypeScript type checking**:
```bash
node ./scripts/typescript-typecheck.js
```
#### Type Checking (TypeScript)
#### VS Code Integration
- Run the TypeScript type checker:
```bash
node ./scripts/typescript-typecheck.js
```
#### VSCode Integration
- The workspace config ensures Prettier is used automatically for JS/TS/JSON/Markdown/YAML files.
- Recommended extension: Prettier – Code Formatter
- Ensure `editor.formatOnSave` is enabled in VSCode for automatic formatting.
The workspace config ensures Prettier is used automatically for JS/TS/JSON/Markdown/YAML files. Ensure `editor.formatOnSave` is enabled in VS Code.
### Swift Code (Lume)
@@ -347,6 +276,8 @@ For Swift code in the `libs/lume` directory:
- Use SwiftFormat for consistent formatting
- Code will be automatically formatted on save when using the lume workspace
Refer to [`libs/lume/Development.md`](./libs/lume/Development.md) for detailed Lume development instructions.
## Releasing Packages
Cua uses an automated GitHub Actions workflow to bump package versions.
@@ -359,13 +290,13 @@ All packages are managed through a single consolidated workflow: [Bump Version](
**Supported packages:**
- cua-agent
- cua-computer
- cua-computer-server
- cua-core
- cua-mcp-server
- cua-som
- pylume
- `cua-agent`
- `cua-computer`
- `cua-computer-server`
- `cua-core`
- `cua-mcp-server`
- `cua-som`
- `pylume`
**How to use:**
@@ -376,9 +307,26 @@ All packages are managed through a single consolidated workflow: [Bump Version](
5. Click "Run workflow" to start the version bump
6. The workflow will automatically commit changes and push to main
<details>
<summary>Local Testing (Advanced)</summary>
The Makefile provides utility targets for local testing only:
```bash
# Test version bump locally (dry run)
make dry-run-patch-core
# View current versions
make show-versions
```
**Note:** For production releases, always use the GitHub Actions workflows above instead of running Makefile commands directly.
</details>
## Releasing a New CLI Version
To release a new version of the CUA CLI, follow these steps:
To release a new version of the CUA CLI (`@trycua/cli`):
### 1. Update the Version
@@ -395,12 +343,14 @@ To release a new version of the CUA CLI, follow these steps:
5. Click "Run workflow"
The workflow will:
- Build single-file executables for all supported platforms
- Publish the package to npm
- Create a GitHub release with the version tag (format: `cua-vX.Y.Z`)
- Attach all platform-specific binaries to the release
<details>
<summary>3-5. Verify, update docs, and announce</summary>
### 3. Verify the Release
1. Check the GitHub Releases page to ensure the new version is published
@@ -418,7 +368,6 @@ The workflow will:
### 4. Update Documentation
Update any relevant documentation with the new version number, including:
- Example code in documentation
- Any version-specific instructions
- Compatibility matrices
@@ -429,8 +378,13 @@ Update any relevant documentation with the new version number, including:
- Update the changelog if maintained separately
- Announce in relevant channels (Slack, Discord, etc.)
</details>
---
<details>
<summary>Rolling Back a Version Bump</summary>
### Rolling Back a Version Bump
If you need to revert a version bump, follow these steps:
@@ -474,7 +428,8 @@ git push origin :refs/tags/core-v0.1.9
git push origin main
```
**Per-package tag patterns:**
<details>
<summary>Per-package tag patterns</summary>
Each package uses its own tag format defined in `.bumpversion.cfg`:
@@ -486,16 +441,6 @@ Each package uses its own tag format defined in `.bumpversion.cfg`:
- **cua-computer-server**: `computer-server-v{version}` (e.g., `computer-server-v0.1.27`)
- **cua-mcp-server**: `mcp-server-v{version}` (e.g., `mcp-server-v0.1.14`)
### Local Testing (Advanced)
</details>
The Makefile targets are kept for local testing only:
```bash
# Test version bump locally (dry run)
make dry-run-patch-core
# View current versions
make show-versions
```
**Note:** For production releases, always use the GitHub Actions workflows above instead of running Makefile commands directly.
</details>

2
notebooks/README.md
View File

@@ -2,6 +2,8 @@
This folder contains Jupyter notebooks that demonstrate the core functionality of the CUA (Computer Use Automation) system. These notebooks serve as interactive examples and quickstart guides for different components of the CUA platform.
> **For development setup and contributing**, see the [Development Guide](../Development.md).
## Available Notebooks
### Core Components

24
notebooks/agent_nb.ipynb
View File

@@ -1,5 +1,12 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -78,7 +85,16 @@
{
"cell_type": "markdown",
"metadata": {},
"source": "### Prerequisites for Cloud Sandbox\n\nTo use Cua Cloud Sandbox, you need to:\n1. Sign up at https://cua.ai\n2. Create a Cloud Sandbox\n3. Generate an API Key\n\nOnce you have these, you can connect to your Cloud Sandbox and run agents on it."
"source": [
"### Prerequisites for Cloud Sandbox\n",
"\n",
"To use Cua Cloud Sandbox, you need to:\n",
"1. Sign up at https://cua.ai\n",
"2. Create a Cloud Sandbox\n",
"3. Generate an API Key\n",
"\n",
"Once you have these, you can connect to your Cloud Sandbox and run agents on it."
]
},
{
"cell_type": "markdown",
@@ -493,7 +509,9 @@
{
"cell_type": "markdown",
"metadata": {},
"source": "All agent runs save trajectories that can be viewed at https://cua.ai/trajectory-viewer"
"source": [
"All agent runs save trajectories that can be viewed at https://cua.ai/trajectory-viewer"
]
},
{
"cell_type": "code",
@@ -527,4 +545,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}

13
notebooks/blog/build-your-own-operator-on-macos-1.ipynb
View File

@@ -1,5 +1,12 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -10,7 +17,9 @@
{
"cell_type": "markdown",
"metadata": {},
"source": "Welcome to Part 1 of our tutorial series on building a Computer Use Automation (CUA) operator using OpenAI. For the complete guide, check out our [full blog post](https://cua.ai/blog/build-your-own-operator-on-macos-1)."
"source": [
"Welcome to Part 1 of our tutorial series on building a Computer Use Automation (CUA) operator using OpenAI. For the complete guide, check out our [full blog post](https://cua.ai/blog/build-your-own-operator-on-macos-1)."
]
},
{
"cell_type": "markdown",
@@ -309,4 +318,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}

13
notebooks/blog/build-your-own-operator-on-macos-2.ipynb
View File

@@ -1,5 +1,12 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -10,7 +17,9 @@
{
"cell_type": "markdown",
"metadata": {},
"source": "Welcome to Part 2 of our tutorial series on building a Computer Use Automation (CUA) operator, this time using the `cua-agent` package. For the complete guide, check out our [full blog post](https://cua.ai/blog/build-your-own-operator-on-macos-2)."
"source": [
"Welcome to Part 2 of our tutorial series on building a Computer Use Automation (CUA) operator, this time using the `cua-agent` package. For the complete guide, check out our [full blog post](https://cua.ai/blog/build-your-own-operator-on-macos-2)."
]
},
{
"cell_type": "markdown",
@@ -174,4 +183,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}

7
notebooks/composite_agents_docker_nb.ipynb
View File

@@ -1,5 +1,12 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},

20
notebooks/computer_nb.ipynb
View File

@@ -1,5 +1,12 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -38,7 +45,16 @@
{
"cell_type": "markdown",
"metadata": {},
"source": "### Prerequisites for Cloud Sandbox\n\nTo use Cua Cloud Sandbox, you need to:\n1. Sign up at https://cua.ai\n2. Create a Cloud Sandbox\n3. Generate an API Key\n\nOnce you have these, you can connect to your Cloud Sandbox using its name."
"source": [
"### Prerequisites for Cloud Sandbox\n",
"\n",
"To use Cua Cloud Sandbox, you need to:\n",
"1. Sign up at https://cua.ai\n",
"2. Create a Cloud Sandbox\n",
"3. Generate an API Key\n",
"\n",
"Once you have these, you can connect to your Cloud Sandbox using its name."
]
},
{
"cell_type": "code",
@@ -599,4 +615,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}

7
notebooks/computer_server_nb.ipynb
View File

@@ -1,5 +1,12 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},

10
notebooks/customizing_computeragent.ipynb
View File

@@ -1,5 +1,13 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "00bf368c",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -200,4 +208,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}

7
notebooks/eval_osworld.ipynb
View File

@@ -1,5 +1,12 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},

8
notebooks/ollama_nb.ipynb
View File

@@ -1,5 +1,13 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d703f36d",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"id": "title-intro",

18
notebooks/sota_hackathon.ipynb
View File

@@ -1,5 +1,13 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "ebcbc0ad",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"id": "a5d6b2ed",
@@ -257,7 +265,13 @@
"cell_type": "markdown",
"id": "5b89a103",
"metadata": {},
"source": "## 🦾 Improve your agent\n\nTo improve your agent for OSWorld-Verified, experiment with different models and add custom tools that fit your use case. You can also dive into the ComputerAgent source code to design an improved version or subclass tailored to your needs.\n\nLearn more about [Customizing Your ComputerAgent](https://cua.ai/docs/agent-sdk/customizing-computeragent) in the docs."
"source": [
"## 🦾 Improve your agent\n",
"\n",
"To improve your agent for OSWorld-Verified, experiment with different models and add custom tools that fit your use case. You can also dive into the ComputerAgent source code to design an improved version or subclass tailored to your needs.\n",
"\n",
"Learn more about [Customizing Your ComputerAgent](https://cua.ai/docs/agent-sdk/customizing-computeragent) in the docs."
]
}
],
"metadata": {
@@ -273,4 +287,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}

39
notebooks/sota_hackathon_cloud.ipynb
View File

@@ -1,5 +1,13 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "5a127c81",
"metadata": {},
"source": [
"> **Development Setup**: For setting up the development environment, see the [Development Guide](../Development.md).\n"
]
},
{
"cell_type": "markdown",
"id": "a5d6b2ed",
@@ -56,7 +64,12 @@
"cell_type": "markdown",
"id": "47171dc3",
"metadata": {},
"source": "1. Create a Cua account at https://cua.ai/\n2. Start a small Cua sandbox at https://cua.ai/dashboard/containers (If you need credits, ask us!)\n3. Create a HUD account at https://www.hud.so/\n4. Create a .env file:"
"source": [
"1. Create a Cua account at https://cua.ai/\n",
"2. Start a small Cua sandbox at https://cua.ai/dashboard/containers (If you need credits, ask us!)\n",
"3. Create a HUD account at https://www.hud.so/\n",
"4. Create a .env file:"
]
},
{
"cell_type": "code",
@@ -158,7 +171,11 @@
"cell_type": "markdown",
"id": "12b9c22c",
"metadata": {},
"source": "Connect to an existing Cloud Sandbox through the Cua SDK.\n\nYou can access the computer through VNC on the [Cua Dashboard](https://cua.ai/dashboard)."
"source": [
"Connect to an existing Cloud Sandbox through the Cua SDK.\n",
"\n",
"You can access the computer through VNC on the [Cua Dashboard](https://cua.ai/dashboard)."
]
},
{
"cell_type": "code",
@@ -185,7 +202,13 @@
"cell_type": "markdown",
"id": "87a307e3",
"metadata": {},
"source": "Try running the computer use agent on a simple task.\n\nTo view a replay of the agent's actions, upload the trajectory to the [trajectory viewer](https://cua.ai/trajectory-viewer).\n\nTrajectories are saved in the format: `trajectories/YYYY-MM-DD_computer-use-pre_XXX`."
"source": [
"Try running the computer use agent on a simple task.\n",
"\n",
"To view a replay of the agent's actions, upload the trajectory to the [trajectory viewer](https://cua.ai/trajectory-viewer).\n",
"\n",
"Trajectories are saved in the format: `trajectories/YYYY-MM-DD_computer-use-pre_XXX`."
]
},
{
"cell_type": "code",
@@ -252,7 +275,13 @@
"cell_type": "markdown",
"id": "5b89a103",
"metadata": {},
"source": "## 🦾 Improve your agent\n\nTo improve your agent for OSWorld-Verified, experiment with different models and add custom tools that fit your use case. You can also dive into the ComputerAgent source code to design an improved version or subclass tailored to your needs.\n\nLearn more about [Customizing Your ComputerAgent](https://cua.ai/docs/agent-sdk/customizing-computeragent) in the docs."
"source": [
"## 🦾 Improve your agent\n",
"\n",
"To improve your agent for OSWorld-Verified, experiment with different models and add custom tools that fit your use case. You can also dive into the ComputerAgent source code to design an improved version or subclass tailored to your needs.\n",
"\n",
"Learn more about [Customizing Your ComputerAgent](https://cua.ai/docs/agent-sdk/customizing-computeragent) in the docs."
]
}
],
"metadata": {
@@ -268,4 +297,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}