diff --git a/docs/content/docs/computer-sdk/computer-ui.mdx b/docs/content/docs/computer-sdk/computer-ui.mdx new file mode 100644 index 00000000..e90e9f08 --- /dev/null +++ b/docs/content/docs/computer-sdk/computer-ui.mdx @@ -0,0 +1,78 @@ +--- +title: Computer UI +--- + +The computer module includes a Gradio UI for creating and sharing demonstration data. We make it easy for people to build community datasets for better computer use models with an upload to Huggingface feature. + +```bash +# Install with UI support +pip install "cua-computer[ui]" +``` + +> **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI. + +### Building and Sharing Demonstrations with Huggingface + +Follow these steps to contribute your own demonstrations: + +#### 1. Set up Huggingface Access + +Set your HF_TOKEN in a .env file or in your environment variables: + +```bash +# In .env file +HF_TOKEN=your_huggingface_token +``` + +#### 2. Launch the Computer UI + +```python +# launch_ui.py +from computer.ui.gradio.app import create_gradio_ui +from dotenv import load_dotenv +load_dotenv('.env') + +app = create_gradio_ui() +app.launch(share=False) +``` + +For examples, see [Computer UI Examples](https://github.com/trycua/cua/tree/main/examples/computer_ui_examples.py) + +#### 3. Record Your Tasks + +
+View demonstration video + +
+ +Record yourself performing various computer tasks using the UI. + +#### 4. Save Your Demonstrations + +
+View demonstration video + +
+ +Save each task by picking a descriptive name and adding relevant tags (e.g., "office", "web-browsing", "coding"). + +#### 5. Record Additional Demonstrations + +Repeat steps 3 and 4 until you have a good amount of demonstrations covering different tasks and scenarios. + +#### 6. Upload to Huggingface + +
+View demonstration video + +
+ +Upload your dataset to Huggingface by: +- Naming it as `{your_username}/{dataset_name}` +- Choosing public or private visibility +- Optionally selecting specific tags to upload only tasks with certain tags + +#### Examples and Resources + +- Example Dataset: [ddupont/test-dataset](https://huggingface.co/datasets/ddupont/test-dataset) +- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua) \ No newline at end of file diff --git a/docs/content/docs/computer-sdk/meta.json b/docs/content/docs/computer-sdk/meta.json index f632538b..92e14612 100644 --- a/docs/content/docs/computer-sdk/meta.json +++ b/docs/content/docs/computer-sdk/meta.json @@ -4,6 +4,7 @@ "pages": [ "computers", "commands", + "computer-ui", "sandboxed-python" ] } diff --git a/libs/python/computer/README.md b/libs/python/computer/README.md index a75c4fe3..78c66566 100644 --- a/libs/python/computer/README.md +++ b/libs/python/computer/README.md @@ -63,82 +63,4 @@ The `cua-computer` PyPi package pulls automatically the latest executable versio Refer to this notebook for a step-by-step guide on how to use the Computer-Use Interface (CUI): -- [Computer-Use Interface (CUI)](https://github.com/trycua/cua/blob/main/notebooks/computer_nb.ipynb) - -## Using the Gradio Computer UI - -The computer module includes a Gradio UI for creating and sharing demonstration data. We make it easy for people to build community datasets for better computer use models with an upload to Huggingface feature. - -```bash -# Install with UI support -pip install "cua-computer[ui]" -``` - -> **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI. - -### Building and Sharing Demonstrations with Huggingface - -Follow these steps to contribute your own demonstrations: - -#### 1. Set up Huggingface Access - -Set your HF_TOKEN in a .env file or in your environment variables: - -```bash -# In .env file -HF_TOKEN=your_huggingface_token -``` - -#### 2. Launch the Computer UI - -```python -# launch_ui.py -from computer.ui.gradio.app import create_gradio_ui -from dotenv import load_dotenv -load_dotenv('.env') - -app = create_gradio_ui() -app.launch(share=False) -``` - -For examples, see [Computer UI Examples](https://github.com/trycua/cua/tree/main/examples/computer_ui_examples.py) - -#### 3. Record Your Tasks - -
-View demonstration video - -
- -Record yourself performing various computer tasks using the UI. - -#### 4. Save Your Demonstrations - -
-View demonstration video - -
- -Save each task by picking a descriptive name and adding relevant tags (e.g., "office", "web-browsing", "coding"). - -#### 5. Record Additional Demonstrations - -Repeat steps 3 and 4 until you have a good amount of demonstrations covering different tasks and scenarios. - -#### 6. Upload to Huggingface - -
-View demonstration video - -
- -Upload your dataset to Huggingface by: -- Naming it as `{your_username}/{dataset_name}` -- Choosing public or private visibility -- Optionally selecting specific tags to upload only tasks with certain tags - -#### Examples and Resources - -- Example Dataset: [ddupont/test-dataset](https://huggingface.co/datasets/ddupont/test-dataset) -- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua) - +- [Computer-Use Interface (CUI)](https://github.com/trycua/cua/blob/main/notebooks/computer_nb.ipynb) \ No newline at end of file