diff --git a/docs/content/docs/home/libraries/computer/index.mdx b/docs/content/docs/home/libraries/computer/index.mdx new file mode 100644 index 00000000..56d337c8 --- /dev/null +++ b/docs/content/docs/home/libraries/computer/index.mdx @@ -0,0 +1,32 @@ +--- +title: Computer +--- + +**@cua/computer** is a Computer-Use Interface (CUI) framework powering Cua for interacting with local macOS and Linux sandboxes. It's PyAutoGUI-compatible and pluggable with any AI agent systems (Cua, Langchain, CrewAI, AutoGen). Computer relies on [Lume](https://github.com/trycua/lume) for creating and managing sandbox environments. + +
+ +
+ +## Available Implementations + +The Computer library is available in multiple programming languages: + +- [Python Implementation](/home/libraries/computer/python) - For Python developers +- [TypeScript Implementation](/home/libraries/computer/typescript) - For TypeScript/JavaScript developers + +Both implementations offer equivalent functionality with language-specific syntax and conventions. + +## Key Features + +- Create and manage virtual machine sandboxes +- Take screenshots of the virtual machine +- Control mouse movements and clicks +- Simulate keyboard input +- Manage clipboard content +- Interact with the operating system interface +- Support for macOS and Linux environments + +## Getting Started + +Select your preferred language implementation from the links above to get started with installation instructions and usage examples. diff --git a/docs/content/docs/home/libraries/computer.mdx b/docs/content/docs/home/libraries/computer/python.mdx similarity index 85% rename from docs/content/docs/home/libraries/computer.mdx rename to docs/content/docs/home/libraries/computer/python.mdx index d10cb255..5300c86e 100644 --- a/docs/content/docs/home/libraries/computer.mdx +++ b/docs/content/docs/home/libraries/computer/python.mdx @@ -1,5 +1,5 @@ --- -title: Computer +title: Computer (Python) ---
-**cua-computer** is a Computer-Use Interface (CUI) framework powering Cua for interacting with local macOS and Linux sandboxes, PyAutoGUI-compatible, and pluggable with any AI agent systems (Cua, Langchain, CrewAI, AutoGen). Computer relies on [Lume](https://github.com/trycua/lume) for creating and managing sandbox environments. - -### Get started with Computer - -
- -
+## Python API ```python from computer import Computer @@ -74,13 +68,13 @@ finally: ## Install -To install the Computer-Use Interface (CUI): +To install the Computer-Use Interface (CUI) for Python: ```bash -pip install "cua-computer[all]" +pip install "@cua/computer[all]" ``` -The `cua-computer` PyPi package pulls automatically the latest executable version of Lume through [pylume](https://github.com/trycua/pylume). +The `@cua/computer` PyPi package pulls automatically the latest executable version of Lume through [pylume](https://github.com/trycua/pylume). ## Run @@ -94,7 +88,7 @@ The computer module includes a Gradio UI for creating and sharing demonstration ```bash # Install with UI support -pip install "cua-computer[ui]" +pip install "@cua/computer[ui]" ``` > **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI. @@ -176,4 +170,4 @@ Upload your dataset to Huggingface by: #### Examples and Resources - Example Dataset: [ddupont/test-dataset](https://huggingface.co/datasets/ddupont/test-dataset) -- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua) +- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua) \ No newline at end of file diff --git a/docs/content/docs/home/libraries/computer/typescript.mdx b/docs/content/docs/home/libraries/computer/typescript.mdx new file mode 100644 index 00000000..10a6b7cb --- /dev/null +++ b/docs/content/docs/home/libraries/computer/typescript.mdx @@ -0,0 +1,188 @@ +--- +title: Computer (TypeScript) +--- + +
+ + TypeScript + + + macOS + + + Discord + + + npm + +
+ +## TypeScript API + +```typescript +import { Computer } from '@cua/computer'; + +const main = async () => { + const computer = new Computer({ + osType: "macos", + display: "1024x768", + memory: "8GB", + cpu: "4" + }); + + try { + await computer.run(); + + const screenshot = await computer.interface.screenshot(); + // Save screenshot using fs or other method + require('fs').writeFileSync('screenshot.png', screenshot); + + await computer.interface.moveCursor(100, 100); + await computer.interface.leftClick(); + await computer.interface.rightClick(300, 300); + await computer.interface.doubleClick(400, 400); + + await computer.interface.type("Hello, World!"); + await computer.interface.pressKey("enter"); + + await computer.interface.setClipboard("Test clipboard"); + const content = await computer.interface.copyToClipboard(); + console.log(`Clipboard content: ${content}`); + } finally { + await computer.stop(); + } +}; + +main().catch(console.error); +``` + +## Install + +To install the Computer-Use Interface (CUI) for TypeScript: + +```bash +npm install @cua/computer +# or +yarn add @cua/computer +``` + +The TypeScript package automatically handles the necessary dependencies for working with Lume. + +## Provider System + +The TypeScript implementation includes a flexible provider system: + +- **Lume Provider**: For local macOS and Linux sandboxes +- **Lumier Provider**: For Docker-based containers +- **Cloud Provider**: For remote cloud-based VMs + +```typescript +// Using specific provider options +import { Computer } from '@cua/computer'; + +const computer = new Computer({ + osType: "macos", + provider: "lume", + providerOptions: { + ephemeral: true, + sharedDirectories: [ + { hostPath: "/path/to/share", tag: "shared", readOnly: true } + ] + } +}); +``` + +## Interface System + +The TypeScript implementation provides a comprehensive interface system for interacting with virtual machines: + +```typescript +// Example of using the interface +import { Computer } from '@cua/computer'; + +const main = async () => { + const computer = new Computer({ osType: "macos" }); + await computer.run(); + + // Screenshot and image operations + const screenshot = await computer.interface.screenshot(); + + // Mouse operations + await computer.interface.moveCursor(100, 100); + await computer.interface.leftClick(); + + // Keyboard operations + await computer.interface.type("Hello from TypeScript!"); + await computer.interface.hotkey(["command", "a"]); + + // Clipboard operations + await computer.interface.setClipboard("Clipboard content"); + const content = await computer.interface.copyToClipboard(); + + await computer.stop(); +}; +``` + +## Advanced Usage + +### Cloud Provider + +For using the cloud provider with WebSocket communication: + +```typescript +import { Computer } from '@cua/computer'; + +const computer = new Computer({ + osType: "macos", + provider: "cloud", + providerOptions: { + apiKey: "your-api-key", + endpoint: "wss://your-cloud-endpoint.com" + } +}); + +await computer.run(); +// Use computer.interface methods as usual +await computer.stop(); +``` + +### Telemetry + +The TypeScript implementation includes built-in telemetry tracking: + +```typescript +import { TelemetryManager } from '@cua/computer'; + +// Disable telemetry if needed +TelemetryManager.getInstance().disable(); + +// Or track custom events +TelemetryManager.getInstance().trackEvent('custom_event', { + property: 'value' +}); +``` + +## Examples + +For more examples and detailed usage, refer to the [TypeScript examples](https://github.com/trycua/cua/tree/main/examples/typescript) in the repository. \ No newline at end of file