computer/Development.md

Getting Started

Project Structure

The project is organized as a monorepo with these main packages:

• libs/core/ - Base package with telemetry support
• libs/computer/ - Computer-use interface (CUI) library
• libs/agent/ - AI agent library with multi-provider support
• libs/som/ - Set-of-Mark parser
• libs/computer-server/ - Server component for VM
• libs/lume/ - Lume CLI

These packages are part of a uv workspace which manages a shared virtual environment and dependencies.

Local Development Setup

1. Install Lume CLI:

   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
   

2. Clone the repository:

   git clone https://github.com/trycua/cua.git
   cd cua
   

3. Create a .env.local file in the root directory with your API keys:

   # Required for Anthropic provider
   ANTHROPIC_API_KEY=your_anthropic_key_here
   
   # Required for OpenAI provider
   OPENAI_API_KEY=your_openai_key_here
   

4. Open the workspace in VSCode or Cursor:

   # For Cua Python development
   code .vscode/py.code-workspace
   
   # For Lume (Swift) development
   code .vscode/lume.code-workspace
   

5. Install Pre-commit hooks (see below):

Using the workspace file is strongly recommended as it:

• Sets up correct Python environments for each package
• Configures proper import paths
• Enables debugging configurations
• Maintains consistent settings across packages

Pre-commit Hook Installation

Pre-commit hooks are scripts that run automatically before each commit to ensure code quality and consistency. To install the hooks:

# First, ensure dependencies are installed (only needed once)
uv sync

# Then install the pre-commit hooks
uv run pre-commit install

Lume Development

Refer to the Lume README for instructions on how to develop the Lume CLI.

Python Development

Setup

Install all of workspace dependencies with a single command:

uv sync

This installs all dependencies in the virtual environment .venv.

Each Cua package is installed in editable mode, which means changes to the source code are immediately reflected in the installed package.

The .venv environment is also configured as the default VS Code Python interpreter in .vscode/settings.json.

Running Python Scripts

To run Python scripts in the workspace, use the uv run command:

uv run python examples/agent_examples.py

Or activate the virtual environment manually:

source .venv/bin/activate
python examples/agent_examples.py

Running Examples

The Python workspace includes launch configurations for all packages:

• "Run Computer Examples" - Runs computer examples
• "Run Agent Examples" - Runs agent examples
• "SOM" configurations - Various settings for running SOM

To run examples from VSCode / Cursor:

1. Press F5 or use the Run/Debug view
2. Select the desired configuration

The workspace also includes compound launch configurations:

• "Run Computer Examples + Server" - Runs both the Computer Examples and Server simultaneously

Code Formatting Standards

The Cua project follows strict code formatting standards to ensure consistency across all packages.

Python Code Formatting

Tools

The project uses the following tools for code formatting and linting:

• Black: Code formatter
• isort: Import sorter
• Ruff: Fast linter and formatter
• MyPy: Static type checker

These tools are automatically installed when you set up the development environment.

Configuration

The formatting configuration is defined in the root pyproject.toml file:

[tool.black]
line-length = 100
target-version = ["py311"]

[tool.ruff]
line-length = 100
target-version = "py311"
select = ["E", "F", "B", "I"]
fix = true

[tool.ruff.format]
docstring-code-format = true

[tool.mypy]
strict = true
python_version = "3.11"
ignore_missing_imports = true
disallow_untyped_defs = true
check_untyped_defs = true
warn_return_any = true
show_error_codes = true
warn_unused_ignores = false

Key Formatting Rules

• Line Length: Maximum of 100 characters
• Python Version: Code should be compatible with Python 3.11+
• Imports: Automatically sorted (using Ruff's "I" rule)
• Type Hints: Required for all function definitions (strict mypy mode)

IDE Integration

The repository includes VSCode workspace configurations that enable automatic formatting. When you open the workspace files (as recommended in the setup instructions), the correct formatting settings are automatically applied.

Python-specific settings

These are configured in .vscode/settings.json:

{
  "python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
  "editor.formatOnSave": true,
  "editor.codeActionsOnSave": {
    "source.organizeImports": "explicit",
    "source.fixAll": "explicit"
  },
  "[python]": {
    "editor.defaultFormatter": "ms-python.black-formatter"
  },
  "python.formatting.provider": "black",
  "ruff.configuration": "${workspaceFolder}/pyproject.toml",
  "mypy-type-checker.args": ["--config-file", "${workspaceFolder}/pyproject.toml"],
  "mypy-type-checker.path": ["${workspaceFolder}"]
}

JS/TS-specific settings

"[javascript][typescript][typescriptreact][javascriptreact]": {
  "editor.defaultFormatter": "esbenp.prettier-vscode"
}

• Ensures Prettier is used for all JS/TS files for consistent formatting.

Recommended VS Code Extensions

• Black Formatter – ms-python.black-formatter
• Ruff – charliermarsh.ruff
• Pylance – ms-python.vscode-pylance
• isort – ms-python.isort
• Prettier – esbenp.prettier-vscode
• Mypy Type Checker – ms-python.mypy-type-checker

VSCode will automatically suggest installing the recommended extensions when you open the workspace.

Manual Formatting

To manually format code:

# Format all Python files using Black
uv run black .

# Sort imports using isort
uv run isort .

# Run Ruff linter with auto-fix
uv run ruff check --ignore E501,E402,I001,I002 .

# Run type checking with MyPy
uv run mypy .

Pre-commit Validation

Before submitting a pull request, ensure your code passes all formatting checks:

Option 1: Run all hooks via pre-commit (all in a single command)

# Run hooks on staged files (recommended for quick checks)
uv run pre-commit run

• Automatically runs Black, Ruff, isort, Mypy, Prettier, and any other configured hooks.

Option 2: Run individual tools manually

# Python checks
uv run black --check .
uv run isort --check .
uv run ruff check --ignore E501,E402,I001,I002 .
uv run mypy .

# JavaScript/TypeScript checks
uv run prettier --check "**/*.{ts,tsx,js,jsx,json,md,yaml,yml}"

# TypeScript typecheck
node ./scripts/typescript-typecheck.js

Swift Code (Lume)

For Swift code in the libs/lume directory:

• Follow the Swift API Design Guidelines
• Use SwiftFormat for consistent formatting
• Code will be automatically formatted on save when using the lume workspace