mirror of
https://github.com/trycua/computer.git
synced 2026-01-03 12:00:00 -06:00
More organization, create typescript nad python doc files
This commit is contained in:
32
docs/content/docs/home/libraries/computer/index.mdx
Normal file
32
docs/content/docs/home/libraries/computer/index.mdx
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
title: Computer
|
||||
---
|
||||
|
||||
**@cua/computer** is a Computer-Use Interface (CUI) framework powering Cua for interacting with local macOS and Linux sandboxes. It's PyAutoGUI-compatible and pluggable with any AI agent systems (Cua, Langchain, CrewAI, AutoGen). Computer relies on [Lume](https://github.com/trycua/lume) for creating and managing sandbox environments.
|
||||
|
||||
<div align="center">
|
||||
<img src="/img/computer.png" />
|
||||
</div>
|
||||
|
||||
## Available Implementations
|
||||
|
||||
The Computer library is available in multiple programming languages:
|
||||
|
||||
- [Python Implementation](/home/libraries/computer/python) - For Python developers
|
||||
- [TypeScript Implementation](/home/libraries/computer/typescript) - For TypeScript/JavaScript developers
|
||||
|
||||
Both implementations offer equivalent functionality with language-specific syntax and conventions.
|
||||
|
||||
## Key Features
|
||||
|
||||
- Create and manage virtual machine sandboxes
|
||||
- Take screenshots of the virtual machine
|
||||
- Control mouse movements and clicks
|
||||
- Simulate keyboard input
|
||||
- Manage clipboard content
|
||||
- Interact with the operating system interface
|
||||
- Support for macOS and Linux environments
|
||||
|
||||
## Getting Started
|
||||
|
||||
Select your preferred language implementation from the links above to get started with installation instructions and usage examples.
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: Computer
|
||||
title: Computer (Python)
|
||||
---
|
||||
|
||||
<div
|
||||
@@ -38,13 +38,7 @@ title: Computer
|
||||
</a>
|
||||
</div>
|
||||
|
||||
**cua-computer** is a Computer-Use Interface (CUI) framework powering Cua for interacting with local macOS and Linux sandboxes, PyAutoGUI-compatible, and pluggable with any AI agent systems (Cua, Langchain, CrewAI, AutoGen). Computer relies on [Lume](https://github.com/trycua/lume) for creating and managing sandbox environments.
|
||||
|
||||
### Get started with Computer
|
||||
|
||||
<div align="center">
|
||||
<img src="/img/computer.png" />
|
||||
</div>
|
||||
## Python API
|
||||
|
||||
```python
|
||||
from computer import Computer
|
||||
@@ -74,13 +68,13 @@ finally:
|
||||
|
||||
## Install
|
||||
|
||||
To install the Computer-Use Interface (CUI):
|
||||
To install the Computer-Use Interface (CUI) for Python:
|
||||
|
||||
```bash
|
||||
pip install "cua-computer[all]"
|
||||
pip install "@cua/computer[all]"
|
||||
```
|
||||
|
||||
The `cua-computer` PyPi package pulls automatically the latest executable version of Lume through [pylume](https://github.com/trycua/pylume).
|
||||
The `@cua/computer` PyPi package pulls automatically the latest executable version of Lume through [pylume](https://github.com/trycua/pylume).
|
||||
|
||||
## Run
|
||||
|
||||
@@ -94,7 +88,7 @@ The computer module includes a Gradio UI for creating and sharing demonstration
|
||||
|
||||
```bash
|
||||
# Install with UI support
|
||||
pip install "cua-computer[ui]"
|
||||
pip install "@cua/computer[ui]"
|
||||
```
|
||||
|
||||
> **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI.
|
||||
@@ -176,4 +170,4 @@ Upload your dataset to Huggingface by:
|
||||
#### Examples and Resources
|
||||
|
||||
- Example Dataset: [ddupont/test-dataset](https://huggingface.co/datasets/ddupont/test-dataset)
|
||||
- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua)
|
||||
- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua)
|
||||
188
docs/content/docs/home/libraries/computer/typescript.mdx
Normal file
188
docs/content/docs/home/libraries/computer/typescript.mdx
Normal file
@@ -0,0 +1,188 @@
|
||||
---
|
||||
title: Computer (TypeScript)
|
||||
---
|
||||
|
||||
<div
|
||||
align="center"
|
||||
style={{
|
||||
display: 'flex',
|
||||
gap: '10px',
|
||||
margin: '0 auto',
|
||||
width: '100%',
|
||||
justifyContent: 'center',
|
||||
}}
|
||||
>
|
||||
<a href="#">
|
||||
<img
|
||||
src="https://img.shields.io/badge/TypeScript-3178C6?logo=typescript&logoColor=white&labelColor=3178C6"
|
||||
alt="TypeScript"
|
||||
/>
|
||||
</a>
|
||||
<a href="#">
|
||||
<img
|
||||
src="https://img.shields.io/badge/macOS-000000?logo=apple&logoColor=F0F0F0"
|
||||
alt="macOS"
|
||||
/>
|
||||
</a>
|
||||
<a href="https://discord.com/invite/mVnXXpdE85">
|
||||
<img
|
||||
src="https://img.shields.io/badge/Discord-%235865F2.svg?&logo=discord&logoColor=white"
|
||||
alt="Discord"
|
||||
/>
|
||||
</a>
|
||||
<a href="https://www.npmjs.com/package/@cua/computer">
|
||||
<img
|
||||
src="https://img.shields.io/npm/v/@cua/computer?color=3178C6"
|
||||
alt="npm"
|
||||
/>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
## TypeScript API
|
||||
|
||||
```typescript
|
||||
import { Computer } from '@cua/computer';
|
||||
|
||||
const main = async () => {
|
||||
const computer = new Computer({
|
||||
osType: "macos",
|
||||
display: "1024x768",
|
||||
memory: "8GB",
|
||||
cpu: "4"
|
||||
});
|
||||
|
||||
try {
|
||||
await computer.run();
|
||||
|
||||
const screenshot = await computer.interface.screenshot();
|
||||
// Save screenshot using fs or other method
|
||||
require('fs').writeFileSync('screenshot.png', screenshot);
|
||||
|
||||
await computer.interface.moveCursor(100, 100);
|
||||
await computer.interface.leftClick();
|
||||
await computer.interface.rightClick(300, 300);
|
||||
await computer.interface.doubleClick(400, 400);
|
||||
|
||||
await computer.interface.type("Hello, World!");
|
||||
await computer.interface.pressKey("enter");
|
||||
|
||||
await computer.interface.setClipboard("Test clipboard");
|
||||
const content = await computer.interface.copyToClipboard();
|
||||
console.log(`Clipboard content: ${content}`);
|
||||
} finally {
|
||||
await computer.stop();
|
||||
}
|
||||
};
|
||||
|
||||
main().catch(console.error);
|
||||
```
|
||||
|
||||
## Install
|
||||
|
||||
To install the Computer-Use Interface (CUI) for TypeScript:
|
||||
|
||||
```bash
|
||||
npm install @cua/computer
|
||||
# or
|
||||
yarn add @cua/computer
|
||||
```
|
||||
|
||||
The TypeScript package automatically handles the necessary dependencies for working with Lume.
|
||||
|
||||
## Provider System
|
||||
|
||||
The TypeScript implementation includes a flexible provider system:
|
||||
|
||||
- **Lume Provider**: For local macOS and Linux sandboxes
|
||||
- **Lumier Provider**: For Docker-based containers
|
||||
- **Cloud Provider**: For remote cloud-based VMs
|
||||
|
||||
```typescript
|
||||
// Using specific provider options
|
||||
import { Computer } from '@cua/computer';
|
||||
|
||||
const computer = new Computer({
|
||||
osType: "macos",
|
||||
provider: "lume",
|
||||
providerOptions: {
|
||||
ephemeral: true,
|
||||
sharedDirectories: [
|
||||
{ hostPath: "/path/to/share", tag: "shared", readOnly: true }
|
||||
]
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Interface System
|
||||
|
||||
The TypeScript implementation provides a comprehensive interface system for interacting with virtual machines:
|
||||
|
||||
```typescript
|
||||
// Example of using the interface
|
||||
import { Computer } from '@cua/computer';
|
||||
|
||||
const main = async () => {
|
||||
const computer = new Computer({ osType: "macos" });
|
||||
await computer.run();
|
||||
|
||||
// Screenshot and image operations
|
||||
const screenshot = await computer.interface.screenshot();
|
||||
|
||||
// Mouse operations
|
||||
await computer.interface.moveCursor(100, 100);
|
||||
await computer.interface.leftClick();
|
||||
|
||||
// Keyboard operations
|
||||
await computer.interface.type("Hello from TypeScript!");
|
||||
await computer.interface.hotkey(["command", "a"]);
|
||||
|
||||
// Clipboard operations
|
||||
await computer.interface.setClipboard("Clipboard content");
|
||||
const content = await computer.interface.copyToClipboard();
|
||||
|
||||
await computer.stop();
|
||||
};
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Cloud Provider
|
||||
|
||||
For using the cloud provider with WebSocket communication:
|
||||
|
||||
```typescript
|
||||
import { Computer } from '@cua/computer';
|
||||
|
||||
const computer = new Computer({
|
||||
osType: "macos",
|
||||
provider: "cloud",
|
||||
providerOptions: {
|
||||
apiKey: "your-api-key",
|
||||
endpoint: "wss://your-cloud-endpoint.com"
|
||||
}
|
||||
});
|
||||
|
||||
await computer.run();
|
||||
// Use computer.interface methods as usual
|
||||
await computer.stop();
|
||||
```
|
||||
|
||||
### Telemetry
|
||||
|
||||
The TypeScript implementation includes built-in telemetry tracking:
|
||||
|
||||
```typescript
|
||||
import { TelemetryManager } from '@cua/computer';
|
||||
|
||||
// Disable telemetry if needed
|
||||
TelemetryManager.getInstance().disable();
|
||||
|
||||
// Or track custom events
|
||||
TelemetryManager.getInstance().trackEvent('custom_event', {
|
||||
property: 'value'
|
||||
});
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
For more examples and detailed usage, refer to the [TypeScript examples](https://github.com/trycua/cua/tree/main/examples/typescript) in the repository.
|
||||
Reference in New Issue
Block a user