From 34fdf3abd95ac080a89aa7bebfd80d729e60077d Mon Sep 17 00:00:00 2001 From: KernelDeimos Date: Mon, 24 Feb 2025 14:18:15 -0500 Subject: [PATCH] doc: improve docs organization - Move extension docs back into this repo - Add index README.md under `doc/` - Improve documentation of project structure --- README.md | 2 +- doc/README.md | 24 ++++++ doc/contributors/extensions.md | 39 +++++++++- doc/contributors/extensions/README.md | 89 ++++++++++++++++++++++ doc/contributors/extensions/definitions.md | 46 +++++++++++ doc/contributors/extensions/events.json.js | 27 +++++++ doc/contributors/extensions/events.md | 38 +++++++++ doc/contributors/extensions/gen.js | 30 ++++++++ doc/contributors/index.md | 25 ------ doc/contributors/structure.md | 65 ++++++++++++++++ doc/{ => self-hosters}/first-run-issues.md | 0 11 files changed, 357 insertions(+), 28 deletions(-) create mode 100644 doc/README.md create mode 100644 doc/contributors/extensions/README.md create mode 100644 doc/contributors/extensions/definitions.md create mode 100644 doc/contributors/extensions/events.json.js create mode 100644 doc/contributors/extensions/events.md create mode 100644 doc/contributors/extensions/gen.js delete mode 100644 doc/contributors/index.md create mode 100644 doc/contributors/structure.md rename doc/{ => self-hosters}/first-run-issues.md (100%) diff --git a/README.md b/README.md index 4760e803..78c11199 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ npm start This will launch Puter at http://puter.localhost:4100 (or the next available port). -If this does not work, see [First Run Issues](./doc/first-run-issues.md) for +If this does not work, see [First Run Issues](./doc/self-hosters/first-run-issues.md) for troubleshooting steps.
diff --git a/doc/README.md b/doc/README.md new file mode 100644 index 00000000..7596f7a6 --- /dev/null +++ b/doc/README.md @@ -0,0 +1,24 @@ +## User Documentation + +- [Hosting Instructions](./self-hosters/instructions.md) +- [Domain Setup](./self-hosters/domains.md) +- [Support Levels](./self-hosters/support.md) + +## Contributor Documentation + +### Where to Start + +Start with [Repo Structure and Tooling](./contributors/structure.md). + +### Index + +- **Conventions** + - [Repo Structure and Tooling](./contributors/structure.md) + - How directories and files are organized in our GitHub repo + - What tools are used to build parts of Puter + - [Comment Prefixes](./contributors/comment_prefixes.md) + - A convention we use for line comments in code + +- [Frontend Documentation](/src/gui/doc) +- [Backend Documentation](/src/backend/doc) +- [Extensions](./contributors/extensions/) diff --git a/doc/contributors/extensions.md b/doc/contributors/extensions.md index e95570e6..41750508 100644 --- a/doc/contributors/extensions.md +++ b/doc/contributors/extensions.md @@ -1,3 +1,38 @@ -## Puter Extensions +# Puter Extensions -See the [Wiki Page](https://github.com/HeyPuter/puter/wiki/ex_extensions) +## Quickstart + +Create and edit this file: `mods/mods_enabled/hello-puter.js` + +```javascript +const { UserActorType, AppUnderUserActorType } = use.core; + +extension.get('/hello-puter', (req, res) => { + const actor = req.actor; + let who = 'unknown'; + if ( actor.type instanceof UserActorType ) { + who = actor.type.user.username; + } + if ( actor.type instanceof AppUnderUserActorType ) { + who = actor.type.app.name + ' on behalf of ' + actor.type.user.username; + } + res.send(`Hello, ${who}!`); +}); +``` + +## Events + +// + +This is subject to change as we make efforts to simplify the process. + +### Step 1: Configure a Mod Directory + +Add this to your config: +```json +"mod_directories": [ + "{source}/../mods/mods_available" +] +``` + +This adds the `mods/mods_available` directory to this diff --git a/doc/contributors/extensions/README.md b/doc/contributors/extensions/README.md new file mode 100644 index 00000000..be00793f --- /dev/null +++ b/doc/contributors/extensions/README.md @@ -0,0 +1,89 @@ +# Puter Extensions + +## Quickstart + +Create and edit this file: `mods/mods_enabled/hello-puter.js` + +```javascript +// You can get definitions exposed by Puter via `use` +const { UserActorType, AppUnderUserActorType } = use.core; + +// Endpoints can be registered directly on an extension +extension.get('/hello-puter', (req, res) => { + const actor = req.actor; + + + // Make a string "who" which says: + // "", or: + // " acting on behalf of " + let who = 'unknown'; + if ( actor.type instanceof UserActorType ) { + who = actor.type.user.username; + } + if ( actor.type instanceof AppUnderUserActorType ) { + who = actor.type.app.name + + ' on behalf of ' + + actor.type.user.username; + } + + res.send(`Hello, ${who}!`); +}); + +// Extensions can listen to events and manipulate Puter's behavior +extension.on('core.email.validate', event => { + if ( event.email.includes('evil') ) { + event.allow = false; + } +}); +``` + +### Scope of `extension` and `use` + +It is important to know that the `extension` global is temporary and does not +exist after your extension is loaded. If you wish to access the extension +object within a callback you will need to first bind it to a variable in +your extension's scope. + +```javascript +const ext = extension; +extension.on('some-event', () => { + // This would throw an error + // extension.something(); + + // This works + ext.example(); +}) +``` + +The same is true for `use`. Calls to `use` should happen at the top of +the file, just like imports in ES6. + +## Database Access + +A database access object is provided to the extension via `extension.db`. +You **must** scope `extension` to another variable (`ext` in this example) +in order to access `db` from callbacks. + +```javascript +const ext = extension; + +extension.get('/user-count', { noauth: true }, (req, res) => { + const [count] = await ext.db.read( + 'SELECT COUNT(*) as c FROM `user`' + ); +}); +``` + +The database access object has the following methods: +- `read(query, params)` - read from the database using a prepared statement. If read-replicas are enabled, this will use a replica. +- `write(query, params)` - write to the database using a prepared statement. If read-replicas are enabled, this will write to the primary. +- `pread(query, params)` - read from the database using a prepared statement. If read-replicas are enabled, this will read from the primary. +- `requireRead(query, params)` - read from the database using a prepared statement. If read-replicas are enabled, this will try reading from the replica first. If there are no results, a second attempt will be made on the primary. + +## Events + +See [events.md](./events.md) + +## Definitions + +See [definitions.md](./definitions.md) diff --git a/doc/contributors/extensions/definitions.md b/doc/contributors/extensions/definitions.md new file mode 100644 index 00000000..4e2dbc80 --- /dev/null +++ b/doc/contributors/extensions/definitions.md @@ -0,0 +1,46 @@ +## Definitions + +### `core.config` - Configuration + +Puter's configuration object. This includes values from `config.json` or their +defaults, and computed values like `origin` and `api_origin`. + +```javascript +const config = use('core.config'); + +extension.get('/get-origin', { noauth: true }, (req, res) => { + res.send(config.origin); +}) +``` + +### `core.util.*` - Utility Functions + +These utilities come from `src/backend/src/util` in Puter's repo. +Each file in this directory has its exports auto-loaded into this +namespace. For example, `src/backend/src/util/langutil.js` is available +via `use('core.util.langutil')` or `use.core.util.langutil`. + +#### `core.util.helpers` - Helper Functions + +Common utility functions used throughout Puter's backend. Use with caution as +some of these functions may be deprecated. + +> **note:** the following documentation is incomplete + +#### `core.util.langutil` - Language Helpers + +##### `whatis(thing :any)` + +- Returns `"array"` if `thing` is an array. +- Returns `"null"` if `thing` is `null`. +- Returns `typeof thing` for any other case. + +##### `nou(value :any)` + +Simply a "null or undefined" check. + +##### `can(value :any, capabilities :Array)` + +Checks if something has the specified capabilities. At the time of +writing the only one supported is `iterate`, which will check if +`value[Symbol.iterator]` is truthy diff --git a/doc/contributors/extensions/events.json.js b/doc/contributors/extensions/events.json.js new file mode 100644 index 00000000..55cf9197 --- /dev/null +++ b/doc/contributors/extensions/events.json.js @@ -0,0 +1,27 @@ +module.exports = [ + { + id: 'core.email.validate', + description: ` + This event is emitted when an email is being validated. + The event can be used to block certain emails from being validated. + `, + properties: { + email: { + type: 'string', + mutability: 'no-effect', + summary: 'the email being validated', + notes: [ + 'The email may have already been cleaned.', + ] + }, + allow: { + type: 'boolean', + mutability: 'mutable', + summary: 'whether the email is allowed', + notes: [ + 'If set to false, the email will be considered invalid.', + ] + }, + }, + } +]; \ No newline at end of file diff --git a/doc/contributors/extensions/events.md b/doc/contributors/extensions/events.md new file mode 100644 index 00000000..0b68a2d6 --- /dev/null +++ b/doc/contributors/extensions/events.md @@ -0,0 +1,38 @@ +### `core.email.validate` + +This event is emitted when an email is being validated. +The event can be used to block certain emails from being validated. + +#### Property `email` + +the email being validated +- **Type**: string +- **Mutability**: no-effect +- **Notes**: undefined + - The email may have already been cleaned. +#### Property `allow` + +whether the email is allowed +- **Type**: boolean +- **Mutability**: mutable +- **Notes**: undefined + - If set to false, the email will be considered invalid. + +### `core.request.measured` + +This event is emitted when a requests incoming and outgoing bytes +have been measured. + +```javascript +extension.on('core.request.measured', data => { + const measurements = data.measurements; + // measurements = { sz_incoming: integer, sz_outgoing: integer } + + const actor = data.actor; // instance of Actor + + console.log('\x1B[36;1m === MEASUREMENT ===\x1B[0m\n', { + actor: data.actor.uid, + measurements: data.measurements + }); +}); +``` diff --git a/doc/contributors/extensions/gen.js b/doc/contributors/extensions/gen.js new file mode 100644 index 00000000..d0963794 --- /dev/null +++ b/doc/contributors/extensions/gen.js @@ -0,0 +1,30 @@ +const dedent = require('dedent'); +const events = require('./events.json.js'); + +const mdlib = {}; +mdlib.h = (out, n, str) => { + out(`${'#'.repeat(n)} ${str}\n\n`); +} + +const N_START = 3; + +const out = str => process.stdout.write(str); +for ( const event of events ) { + mdlib.h(out, N_START, `\`${event.id}\``); + out(dedent(event.description) + '\n\n'); + + for ( const k in event.properties ) { + const prop = event.properties[k]; + mdlib.h(out, N_START + 1, `Property \`${k}\``); + out(prop.summary + '\n'); + out(`- **Type**: ${prop.type}\n`); + out(`- **Mutability**: ${prop.mutability}\n`); + if ( prop.notes ) { + out(`- **Notes**: ${prop.n}\n`); + for ( const note of prop.notes ) { + out(` - ${note}\n`); + } + } + } + +} diff --git a/doc/contributors/index.md b/doc/contributors/index.md deleted file mode 100644 index 34d976f0..00000000 --- a/doc/contributors/index.md +++ /dev/null @@ -1,25 +0,0 @@ -# Contributing to Puter - -## Essential / General Knowledge - -### Repository Dichotomy - -- Puter's GUI is at the root; `/src` is the GUI -- Puter's backend is a workspace npm package; - it resides in `packages/backend(/src)` - -The above may seem counter-intuitive; backend and frontend are siblings, right? -Consider this: by a different intuition, the backend is at a "deeper" level -of function; this directory structure better adheres to soon-to-be contributors -sifting around through the files to discover "what's what". - -The directory `volatile` exists _for your convenience_ to simplify running -Puter for development. When Puter is run -run with the backend with this repository as its working directory, it -will use `volatile/config` and `volatile/runtime` instead of -`/etc/puter` and `/var/puter`. - -## See Next - -- [Backend Documentation](../../packages/backend/doc/contributors/index.md) - diff --git a/doc/contributors/structure.md b/doc/contributors/structure.md new file mode 100644 index 00000000..6b005c28 --- /dev/null +++ b/doc/contributors/structure.md @@ -0,0 +1,65 @@ +# Repository Structure and Tooling + +Puter has many of its parts in a single [monorepo](https://en.wikipedia.org/wiki/Monorepo), +rather than a single repository for each cohesive part. +We feel this makes it easier for new contributors to develop Puter since you don't +need to figure out how to tie the parts together or how to work with Git submodules. +It also makes it easier for us to maintain project-wide conventions and tooling. + +Some tools, like [puter-cli](https://github.com/HeyPuter/puter-cli), exist in separate +repositories. The `puter-cli` tool is used externally and can communicate with Puter's +API on our production (puter.com) instance or your own instance of Puter, so there's +not really any advantage to putting it in the monorepo. + +## Top-Level directories + +### The `doc` directory + +The top-level `doc` directory contains the file you're reading right now. +Its scope is documentation for using and contributing to Puter in general, +and linking to more specific documentation in other places. + +All `doc` directories will have a `README.md` which should be considered as +the index file for the documentation. All documentation under a `doc` +directory should be accessible via a path of links starting from `README.md`. + +### The `src` directory + +Every directory under `/tools` is [an npm "workspaces" module](https://docs.npmjs.com/cli/v8/using-npm/workspaces). Every direct child of this directory (generally) has a `package.json` and a `src` directory. + +Some of these modules are core pieces of Puter: +- **Puter's backend** is [`/src/backend`](/src/backend) +- **Puter's GUI** is [`/src/gui`](/src/gui) + +Some of these modules are apps: +- **Puter's Terminal**: [`/src/terminal`](/src/terminal) +- **Puter's Shell**: [`/src/phoenix`](/src/phoenix) +- **Experimental v86 Integration**: [`/src/emulator`](/src/emulator) + - **Note:** development is focused on Puter PDE files instead (docs pending) + +Some of these modules are libraries: +- **common javascript**: [`/src/putility`](/src/putility) +- **runtime import mechanism**: [`/src/useapi`](/src/useapi) +- **Puter's "puter.js" browser SDK**: [`/src/puter-js`](/src/puter-js) + +### The `volatile` directory + +When you're running Puter with development instructions (i.e. `npm start`), +Puter's configuration directory will be `volatile/config` and Puter's +runtime directory will be `volatile/runtime`, instead of the standard +`/etc/puter` and `/var/puter` directories in production installations. + +We should probably rename this directory, actually, but it would inconvenience +a lot of people right now if we did. + +### The `tools` directory + +Every directory under `/tools` is [an npm "workspaces" module](https://docs.npmjs.com/cli/v8/using-npm/workspaces). + +This is where `run-selfhosted.js` is. That's the entrypoint for `npm start`. + +These tools are underdocumented and may not behave well if they're not executed +from the correct working directory (which is different for different tools). +Consider this a work-in-progress. If you want to use or contribute to anything +under this directory, for now you should +[tag @KernelDeimos on the community Discord](https://discord.gg/PQcx7Teh8u). diff --git a/doc/first-run-issues.md b/doc/self-hosters/first-run-issues.md similarity index 100% rename from doc/first-run-issues.md rename to doc/self-hosters/first-run-issues.md