mirror/opencloud
1
0
Fork 0
mirror of https://github.com/opencloud-eu/opencloud.git synced 2026-01-27 23:18:49 -06:00
Code Issues Packages Projects Releases Wiki Activity

Revert "run remark linter locally"

Browse Source 
This reverts commit 97c348f1ae.
This commit is contained in:
A.Unger
2020-09-23 11:48:59 +02:00
parent 97c348f1ae
commit 8b9e02ec0b
447 changed files with 6088 additions and 6178 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
ocis/docs/_index.md
View File

@@ -1,16 +1,15 @@
* * *
---
title: "Infinite Scale"
date: 2020-02-27T20:35:00+01:00
weight: -10
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
## geekdocFilePath: \_index.md
geekdocFilePath: _index.md
---
This tool provides a single entrypoint for the whole ownCloud Infinite Scale stack.
{{&lt; mermaid class="text-center">}}
{{< mermaid class="text-center">}}
graph TD
ocis-proxy -->
ocis-konnectd & ocis-phoenix & ocis-thumbnails & ocis-ocs & ocis-webdav
@@ -19,12 +18,13 @@ ocis-phoenix --> ocis-reva-fronted
ocis-reva-fronted --> ocis-reva-gateway
ocis-konnectd --> ocis-glauth
ocis-reva-gateway --> ocis-reva-users
ocis-reva-gateway --> ocis-reva-authbasic
ocis-reva-gateway --> ocis-reva-auth-bearer
ocis-reva-gateway --> ocis-reva-sharing
ocis-reva-gateway --> ocis-reva-storage-home-_
ocis-reva-storage-home-_ --> ocis-reva-storage-home-\*-data
ocis-reva-gateway --> ocis-reva-storage-home-*
ocis-reva-storage-home-* --> ocis-reva-storage-home-*-data
ocis-reva-sharing --> redis
{{&lt; /mermaid >}}
{{< /mermaid >}}

25
ocis/docs/basic-remote-setup.md
View File

@@ -1,14 +1,13 @@
* * *
---
title: "Basic Remote Setup"
date: 2020-02-27T20:35:00+01:00
weight: 16
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: basic-remote-setup.md
---
## geekdocFilePath: basic-remote-setup.md
{{&lt; toc >}}
{{< toc >}}
Out of the box the ocis single binary and the `owncloud/ocis` docker image are configured to run on localhost for quick testing and development.
@@ -18,16 +17,16 @@ If you need to access ocis on a VM or a remote machine e.g. when testing a mobil
If you start the ocis fullstack for the first time with `./bin/ocis server` it will generate a file `identifier-registration.yml` in the config folder relative to its location. This file is used to configure the clients for the built-in Identity Provider.
{{&lt; hint warning >}}
**Outdated version**\\
{{< hint warning >}}
**Outdated version**\
The `identifier-registration.yml` file will only be generated if there is no such file in place. You could miss updates on this file. Run `make clean` to delete the file and keep the development environment tidy otherwise as well.
{{&lt; /hint >}}
{{< /hint >}}
### Add your hostname to the idp config
Let us assume `your-host` is your remote domain name or IP adress. Add your host to the `identifier-registration.yml` like this:
```yaml {linenos=table,hl_lines=\["15-17",21]}
```yaml {linenos=table,hl_lines=["15-17",21]}
# OpenID Connect client registry.
clients:
- id: phoenix
@@ -74,10 +73,10 @@ KONNECTD_TLS=0 \
For more configuration options check the configuration secion in [ocis](https://owncloud.github.io/ocis/configuration/) and every ocis extension.
{{&lt; hint info >}}
**TLS Certificate**\\
{{< hint info >}}
**TLS Certificate**\
In this example, we are replacing the default self signed cert with a CA signed one to avoid the certificate warning when accessing the login page.
{{&lt; /hint >}}
{{< /hint >}}
## Use Docker Compose

112
ocis/docs/bridge.md
View File

@@ -1,14 +1,13 @@
* * *
---
title: "Bridge"
date: 2020-02-27T20:35:00+01:00
weight: 30
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: bridge.md
---
## geekdocFilePath: bridge.md
{{&lt; toc >}}
{{< toc >}}
We are planning to build a bridge from ownCloud 10 to ocis. The idea is to have a reverse proxy infront of ownCloud 10 that will forward requests to ownCloud 10 or ocis-reva, depending on the migration status of the logged in user.
@@ -23,17 +22,19 @@ Using ocis and the ownCloud 10 openidconnect and graphapi plugins it is possible
### Install the owncloud 10 graphapi app
In an owncloud 10 apps folder
$ git clone git@github.com:owncloud/graphapi.git
$ cd graphapi
$ composer install
```
$ git clone git@github.com:owncloud/graphapi.git
$ cd graphapi
$ composer install
```
### Enable the graphapi app
occ a:e graphapi
```
occ a:e graphapi
```
No configuration necessary. You can test with `curl`:
```console
$ curl https://cloud.example.com/index.php/apps/graphapi/v1.0/users -u admin | jq
Enter host password for user 'admin':
@@ -68,17 +69,17 @@ We are going to use the above ownCloud 10 and graphapi app to turn it into the d
#### Grab it!
In an `ocis` folder
$ git clone git@github.com:owncloud/ocis-glauth.git
$ cd ocis-glauth
$ make
```
$ git clone git@github.com:owncloud/ocis-glauth.git
$ cd ocis-glauth
$ make
```
This should give you a `bin/ocis-glauth` binary. Try listing the help with `bin/ocis-glauth --help`.
#### Run it!
You need to point `ocis-glauth` to your owncloud domain:
```console
$ bin/ocis-glauth --log-level debug server --backend-datastore owncloud --backend-server https://cloud.example.com --backend-basedn dc=example,dc=com
```
@@ -91,13 +92,11 @@ $ bin/ocis-glauth --log-level debug server --backend-datastore owncloud --backen
#### Check it is up and running
You should now be able to list accounts from your ownCloud 10 oc_accounts table using:
```console
$ ldapsearch -x -H ldap://localhost:9125 -b dc=example,dc=com -D "cn=admin,dc=example,dc=com" -W '(objectclass=posixaccount)'
```
Groups should work as well:
```console
$ ldapsearch -x -H ldap://localhost:9125 -b dc=example,dc=com -D "cn=admin,dc=example,dc=com" -W '(objectclass=posixgroup)'
```
@@ -109,44 +108,41 @@ $ ldapsearch -x -H ldap://localhost:9125 -b dc=example,dc=com -D "cn=admin,dc=ex
#### Get it!
In an `ocis` folder
$ git clone git@github.com:owncloud/ocis-phoenix.git
$ cd ocis-phoenix
$ make
```
$ git clone git@github.com:owncloud/ocis-phoenix.git
$ cd ocis-phoenix
$ make
```
This should give you a `bin/ocis-phoenix` binary. Try listing the help with `bin/ocis-phoenix --help`.
#### Run it!
Point `ocis-phoenix` to your owncloud domain and tell it where to find the openid connect issuing authority:
```console
$ bin/ocis-phoenix server --web-config-server https://cloud.example.com --oidc-authority https://192.168.1.100:9130 --oidc-metadata-url https://192.168.1.100:9130/.well-known/openid-configuration --oidc-client-id ocis
```
`ocis-phoenix` needs to know
- `--web-config-server https://cloud.example.com` is ownCloud url with webdav and ocs endpoints (oc10 or ocis)
- `--oidc-authority https://192.168.1.100:9130` the openid connect issuing authority, in our case `oidc-konnectd`, running on port 9130
- `--oidc-metadata-url https://192.168.1.100:9130/.well-known/openid-configuration` the openid connect configuration endpoint, typically the issuer host with `.well-known/openid-configuration`, but there are cases when another endpoint is used, eg. ping identity provides multiple endpoints to separate domains
- `--oidc-client-id ocis` the client id we will register later with `ocis-konnectd` in the `identifier-registration.yaml`
- `--web-config-server https://cloud.example.com` is ownCloud url with webdav and ocs endpoints (oc10 or ocis)
- `--oidc-authority https://192.168.1.100:9130` the openid connect issuing authority, in our case `oidc-konnectd`, running on port 9130
- `--oidc-metadata-url https://192.168.1.100:9130/.well-known/openid-configuration` the openid connect configuration endpoint, typically the issuer host with `.well-known/openid-configuration`, but there are cases when another endpoint is used, eg. ping identity provides multiple endpoints to separate domains
- `--oidc-client-id ocis` the client id we will register later with `ocis-konnectd` in the `identifier-registration.yaml`
### Start ocis-konnectd
#### Get it!
In an `ocis` folder
$ git clone git@github.com:owncloud/ocis-konnectd.git
$ cd ocis-konnectd
$ make
```
$ git clone git@github.com:owncloud/ocis-konnectd.git
$ cd ocis-konnectd
$ make
```
This should give you a `bin/ocis-konnectd` binary. Try listing the help with `bin/ocis-konnectd --help`.
#### Set environment variables
Konnectd needs environment variables to configure the LDAP server:
```console
export LDAP_URI=ldap://192.168.1.100:9125
export LDAP_BINDDN="cn=admin,dc=example,dc=com"
@@ -160,13 +156,11 @@ export LDAP_UUID_ATTRIBUTE=uid
export LDAP_UUID_ATTRIBUTE_TYPE=text
export LDAP_FILTER="(objectClass=posixaccount)"
```
Don't forget to use an existing user and the correct password.
### Configure clients
Now we need to configure a client we can later use to configure the ownCloud 10 openidconnect app. In the `assets/identifier-registration.yaml` have:
```yaml
---
@@ -182,7 +176,6 @@ clients:
- http://localhost:9100
- http://localhost:9100/
```
You will need the `insecure: yes` if you are using self signed certificates.
Replace `cloud.example.com` in the redirect URI with your ownCloud 10 host and port.
@@ -191,44 +184,43 @@ Replace `localhost:9100` in the redirect URIs with your the `ocis-phoenix` host
#### Run it!
You can now bring up `ocis-connectd` with:
```console
$ bin/ocis-konnectd server --iss https://192.168.1.100:9130 --identifier-registration-conf assets/identifier-registration.yaml --signing-kid gen1-2020-02-27
```
`ocis-konnectd` needs to know
- `--iss https://192.168.1.100:9130` the issuer, which must be a reachable https endpoint. For testing an ip works. HTTPS is NOT optional. This url is exposed in the `https://192.168.1.100:9130/.well-known/openid-configuration` endpoint and clients need to be able to connect to it
- `--identifier-registration-conf assets/identifier-registration.yaml` the identifier-registration.yaml you created
- `--signing-kid gen1-2020-02-27` a signature key id, otherwise the jwks key has no name, which might cause problems with clients. a random key is ok, but it should change when the actual signing key changes.
- `--iss https://192.168.1.100:9130` the issuer, which must be a reachable https endpoint. For testing an ip works. HTTPS is NOT optional. This url is exposed in the `https://192.168.1.100:9130/.well-known/openid-configuration` endpoint and clients need to be able to connect to it
- `--identifier-registration-conf assets/identifier-registration.yaml` the identifier-registration.yaml you created
- `--signing-kid gen1-2020-02-27` a signature key id, otherwise the jwks key has no name, which might cause problems with clients. a random key is ok, but it should change when the actual signing key changes.
#### Check it is up and running
1. Try getting the configuration:
1. Try getting the configuration:
```console
$ curl https://192.168.1.100:9130/.well-known/openid-configuration
```
2. Check if the login works at <https://192.168.1.100:9130/signin/v1/identifier>
2. Check if the login works at https://192.168.1.100:9130/signin/v1/identifier
> Note: If you later get a `Unable to find a key for (algorithm, kid):PS256, )` Error make sure you did set a `--signing-kid` when starting `ocis-konnectd` by checking it is present in <https://192.168.1.100:9130/konnect/v1/jwks.json>
> Note: If you later get a `Unable to find a key for (algorithm, kid):PS256, )` Error make sure you did set a `--signing-kid` when starting `ocis-konnectd` by checking it is present in https://192.168.1.100:9130/konnect/v1/jwks.json
### Patch owncloud
While the UserSession in ownCloud 10 is currently used to test all available IAuthModule implementations, it immediately logs out the user when an exception occurs. However, existing owncloud 10 instances use the oauth2 app to create Bearer tokens for mobile and desktop clients.
To give the openidconnect app a chance to verify the tokens we need to change the code a bit. See <https://github.com/owncloud/core/pull/37043> for a possible solution.
To give the openidconnect app a chance to verify the tokens we need to change the code a bit. See https://github.com/owncloud/core/pull/37043 for a possible solution.
> Note: The PR is hot ... as in _younger than this list of steps_. And it messes with authentication. Use with caution.
> Note: The PR is hot ... as in *younger than this list of steps*. And it messes with authentication. Use with caution.
### Install the owncloud 10 openidconnect app
In an owncloud 10 apps folder
$ git clone git@github.com:owncloud/openidconnect.git
$ cd openidconnect
$ composer install
```
$ git clone git@github.com:owncloud/openidconnect.git
$ cd openidconnect
$ composer install
```
After enabling the app configure it in `config/oidc.config.php`
@@ -249,16 +241,14 @@ $CONFIG = [
```
In the above configuration replace
- `provider-url` with the URL to your `ocis-konnectd` issuer
- `https://cloud.example.com` with the URL to your ownCloud 10 instance
- `http://localhost:9100` with the URL to your phoenix instance
- `provider-url` with the URL to your `ocis-konnectd` issuer
- `https://cloud.example.com` with the URL to your ownCloud 10 instance
- `http://localhost:9100` with the URL to your phoenix instance
> Note: By default the openidconnect app will use the email of the user to match the user from the oidc userinfo endpoint with the ownCloud account. So make sure your users have a unique primary email.
## Next steps
Aside from the above todos these are the next stepo
- tie it all together behind `ocis-proxy`
- create an `ocis bridge` command that runs all the ocis services in one step with a properly preconfigured `ocis-konnectd` `identifier-registration.yaml` file for `phoenix` and the owncloud 10 `openidconnect` app, as well as a randomized `--signing-kid`.
- tie it all together behind `ocis-proxy`
- create an `ocis bridge` command that runs all the ocis services in one step with a properly preconfigured `ocis-konnectd` `identifier-registration.yaml` file for `phoenix` and the owncloud 10 `openidconnect` app, as well as a randomized `--signing-kid`.

21
ocis/docs/building-docs.md
View File

@@ -1,14 +1,13 @@
* * *
---
title: "Building the documentation"
date: 2020-07-27T08:39:38+00:00
weight: 99
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: building-docs.md
---
## geekdocFilePath: building-docs.md
{{&lt; toc >}}
{{< toc >}}
## Buildling the documentation
@@ -16,22 +15,22 @@ Following steps can be applied for every oCIS extension repository.
### Setting up
- Install [hugo](https://gohugo.io/getting-started/installing/)
- Run `make docs`
- Install [hugo](https://gohugo.io/getting-started/installing/)
- Run `make docs`
### Viewing the documentation
To view the rendered docs in the browser run:
```bash
cd hugo
hugo -D server
```
Then open "http&#x3A;//localhost:1313/"
Then open "http://localhost:1313/"
When making changes to the docs, run `make docs` again and the server will pick up the changes and reload the page automatically
### Deploying the documentation
The documentation is automatically deployed from the master branch to <https://owncloud.github.io/phoenix/>
The documentation is automatically deployed from the master branch to https://owncloud.github.io/phoenix/

36
ocis/docs/building.md
View File

@@ -1,36 +1,36 @@
* * *
---
title: "Building"
date: 2020-02-27T20:35:00+01:00
weight: 50
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
## geekdocFilePath: building.md
geekdocFilePath: building.md
---
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.13. After the installation of the required tools you need to get the sources:
{{&lt; highlight txt >}}
git clone <https://github.com/owncloud/ocis.git>
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis.git
cd ocis
{{&lt; / highlight >}}
{{< / highlight >}}
All required tools besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`. To build the `ocis` binary run:
{{&lt; highlight txt >}}
{{< highlight txt >}}
make generate
make build
{{&lt; / highlight >}}
{{< / highlight >}}
Finally, you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis -h` to see all available options.
## Simple Ocis fo extonsions example
Currently, we are using a go build tag to allow building a more simple set of the binary. It was intended to let extension developers focus on only the necessary services.
{{&lt; hint info >}}
{{< hint info >}}
While it the tag based simple build demonstrates how to use ocis as a framework for a micro service architecture, we may change to an approach that uses an explicit command to run only a subset of the services.
{{&lt; / hint >}}
{{< / hint >}}
```console
TAGS=simple make build
@@ -40,8 +40,10 @@ The artifact lives in `/bin/ocis`
The generated simple ocis binary is a subset of the ocis command with a restricted set of services meant for ease up development. The services included are
ocis-hello
ocis-phoenix
ocis-konnectd
ocis-glauth
micro's own services
```
ocis-hello
ocis-phoenix
ocis-konnectd
ocis-glauth
micro's own services
```

50
ocis/docs/configuration.md
View File

@@ -1,20 +1,19 @@
* * *
---
title: "Configuration"
date: "2020-09-21T13:14:56+0200"
weight: 20
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: configuration.md
---
## geekdocFilePath: configuration.md
{{&lt; toc >}}
{{< toc >}}
## Configuration
oCIS Single Binary is not responsible for configuring extensions. Instead, each extension could either be configured by environment variables, cli flags or config files.
Each extension has its dedicated documentation page (e.g. <https://owncloud.github.io/extensions/ocis_proxy/configuration>) which lists all possible configurations. Config files and environment variables are picked up if you use the `./bin/ocis server` command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
Each extension has its dedicated documentation page (e.g. https://owncloud.github.io/extensions/ocis_proxy/configuration) which lists all possible configurations. Config files and environment variables are picked up if you use the `./bin/ocis server` command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
### Configuration using config files
@@ -26,7 +25,7 @@ $HOME/.ocis
./config
```
For this configuration to be picked up, have a look at your extension `root` command and look for which default config name it has assigned. _i.e: ocis-proxy reads `proxy.json | yaml | toml ...`_.
For this configuration to be picked up, have a look at your extension `root` command and look for which default config name it has assigned. *i.e: ocis-proxy reads `proxy.json | yaml | toml ...`*.
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/ocis.yml`, `${HOME}/.ocis/ocis.yml` or `$(pwd)/config/ocis.yml`.
@@ -44,16 +43,16 @@ ownCloud Infinite Scale Stack
Usage: `ocis [global options] command [command options] [arguments...]`
\--config-file | $OCIS_CONFIG_FILE
--config-file | $OCIS_CONFIG_FILE
: Path to config file.
\--log-level | $OCIS_LOG_LEVEL
--log-level | $OCIS_LOG_LEVEL
: Set logging level. Default: `info`.
\--log-pretty | $OCIS_LOG_PRETTY
--log-pretty | $OCIS_LOG_PRETTY
: Enable pretty logging. Default: `true`.
\--log-color | $OCIS_LOG_COLOR
--log-color | $OCIS_LOG_COLOR
: Enable colored logging. Default: `true`.
## Sub Commands
@@ -64,7 +63,7 @@ Check health status
Usage: `ocis health [command options] [arguments...]`
\--debug-addr | $OCIS_DEBUG_ADDR
--debug-addr | $OCIS_DEBUG_ADDR
: Address to debug endpoint. Default: `0.0.0.0:9010`.
### ocis server
@@ -73,40 +72,40 @@ Start fullstack server
Usage: `ocis server [command options] [arguments...]`
\--tracing-enabled | $OCIS_TRACING_ENABLED
--tracing-enabled | $OCIS_TRACING_ENABLED
: Enable sending traces.
\--tracing-type | $OCIS_TRACING_TYPE
--tracing-type | $OCIS_TRACING_TYPE
: Tracing backend type. Default: `jaeger`.
\--tracing-endpoint | $OCIS_TRACING_ENDPOINT
--tracing-endpoint | $OCIS_TRACING_ENDPOINT
: Endpoint for the agent. Default: `localhost:6831`.
\--tracing-collector | $OCIS_TRACING_COLLECTOR
--tracing-collector | $OCIS_TRACING_COLLECTOR
: Endpoint for the collector. Default: `http://localhost:14268/api/traces`.
\--tracing-service | $OCIS_TRACING_SERVICE
--tracing-service | $OCIS_TRACING_SERVICE
: Service name for tracing. Default: `ocis`.
\--debug-addr | $OCIS_DEBUG_ADDR
--debug-addr | $OCIS_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9010`.
\--debug-token | $OCIS_DEBUG_TOKEN
--debug-token | $OCIS_DEBUG_TOKEN
: Token to grant metrics access.
\--debug-pprof | $OCIS_DEBUG_PPROF
--debug-pprof | $OCIS_DEBUG_PPROF
: Enable pprof debugging.
\--debug-zpages | $OCIS_DEBUG_ZPAGES
--debug-zpages | $OCIS_DEBUG_ZPAGES
: Enable zpages debugging.
\--http-addr | $OCIS_HTTP_ADDR
--http-addr | $OCIS_HTTP_ADDR
: Address to bind http server. Default: `0.0.0.0:9000`.
\--http-root | $OCIS_HTTP_ROOT
--http-root | $OCIS_HTTP_ROOT
: Root path of http server. Default: `/`.
\--grpc-addr | $OCIS_GRPC_ADDR
--grpc-addr | $OCIS_GRPC_ADDR
: Address to bind grpc server. Default: `0.0.0.0:9001`.
### List of available Extension subcommands
@@ -220,3 +219,4 @@ Start thumbnails server
#### ocis webdav
Start webdav server

60
ocis/docs/debugging.md
View File

@@ -1,12 +1,11 @@
* * *
---
title: "Debugging"
date: 2020-03-19T08:21:00+01:00
weight: 50
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
## geekdocFilePath: debugging.md
geekdocFilePath: debugging.md
---
## Debugging
@@ -14,7 +13,7 @@ As a single binary for easy deployment running `ocis server` just forks itself t
Ultimately, we want to be able to stop a single service using eg. `ocis kill phoenix` so that you can start the service you want to debug in debug mode. We need to [change the way we fork processes](https://github.com/owncloud/ocis/issues/77) though, otherwise the runtime will automatically restart a service if killed.
### Start ocis
### Start ocis
For debugging there are two workflows that work well, depending on your preferences.
@@ -68,8 +67,7 @@ Then you can set a breakpoint in the service you need and attach to the process
#### Start all services independently to replace one of them with a debug process
1. You can use this `./ocis.sh` script to start all services independently, so they don't get restrarted by the runtime when you kill them:
1. You can use this `./ocis.sh` script to start all services independently, so they don't get restrarted by the runtime when you kill them:
```bash
#/bin/sh
LOG_LEVEL="debug"
@@ -99,7 +97,7 @@ bin/ocis --log-level=$LOG_LEVEL reva-users &
bin/ocis --log-level=$LOG_LEVEL proxy &
```
2. Get the list of running processes:
2. Get the list of running processes:
```console
# ps ax | grep ocis
@@ -128,13 +126,14 @@ bin/ocis --log-level=$LOG_LEVEL proxy &
13015 pts/1 Sl 0:00 bin/ocis-debug reva-auth-basic
```
3. Kill the service you want to start in debug mode:
3. Kill the service you want to start in debug mode:
```console
# kill 17628
```
4. Start the service you are interested in in debug mode. When using make to build the binary there is already a `bin/ocis-debug` binary for you. When running an IDE tell it which service to start by providing the corresponding sub command, eg. `bin\ocis-debug reva-frontend`.
4. Start the service you are interested in in debug mode. When using make to build the binary there is already a `bin/ocis-debug` binary for you. When running an IDE tell it which service to start by providing the corresponding sub command, eg. `bin\ocis-debug reva-frontend`.
### Gather error messages
@@ -158,22 +157,24 @@ This popped up when I tried to add `marie` as a collaborator in phoenix. That tr
</ocs>
```
{{&lt; hint info >}}
{{< hint info >}}
The username and password only work when basic auth is available. Otherwise you have to obtain a bearer token, eg. by grabbing it from the browser.
{{&lt; /hint >}}
{{&lt; hint danger >}}
{{< /hint >}}
{{< hint danger >}}
TODO add ocis cli tool to obtain a bearer token.
{{&lt; /hint >}}
{{< /hint >}}
We also have a few interesting log entries:
0:43PM INF home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/grpc/interceptors/log/log.go:69 > unary code=OK end="18/Mar/2020:22:43:40 +0100" from=tcp://[::1]:44078 pid=17836 pkg=rgrpc start="18/Mar/2020:22:43:40 +0100" time_ns=95841 traceid=b4eb9a9f45921f7d3632523ca32a42b0 uri=/cs3.storage.registry.v1beta1.RegistryAPI/GetStorageProvider user-agent=grpc-go/1.26.0
10:43PM ERR home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/grpc/interceptors/log/log.go:69 > unary code=Unknown end="18/Mar/2020:22:43:40 +0100" from=tcp://[::1]:43910 pid=17836 pkg=rgrpc start="18/Mar/2020:22:43:40 +0100" time_ns=586115 traceid=b4eb9a9f45921f7d3632523ca32a42b0 uri=/cs3.gateway.v1beta1.GatewayAPI/Stat user-agent=grpc-go/1.26.0
10:43PM ERR home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/http/services/owncloud/ocs/reqres.go:94 > error sending a grpc stat request error="rpc error: code = Unknown desc = gateway: error calling Stat: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp [::1]:9152: connect: connection refused\"" pid=17832 pkg=rhttp traceid=b4eb9a9f45921f7d3632523ca32a42b0
```
0:43PM INF home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/grpc/interceptors/log/log.go:69 > unary code=OK end="18/Mar/2020:22:43:40 +0100" from=tcp://[::1]:44078 pid=17836 pkg=rgrpc start="18/Mar/2020:22:43:40 +0100" time_ns=95841 traceid=b4eb9a9f45921f7d3632523ca32a42b0 uri=/cs3.storage.registry.v1beta1.RegistryAPI/GetStorageProvider user-agent=grpc-go/1.26.0
10:43PM ERR home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/grpc/interceptors/log/log.go:69 > unary code=Unknown end="18/Mar/2020:22:43:40 +0100" from=tcp://[::1]:43910 pid=17836 pkg=rgrpc start="18/Mar/2020:22:43:40 +0100" time_ns=586115 traceid=b4eb9a9f45921f7d3632523ca32a42b0 uri=/cs3.gateway.v1beta1.GatewayAPI/Stat user-agent=grpc-go/1.26.0
10:43PM ERR home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/http/services/owncloud/ocs/reqres.go:94 > error sending a grpc stat request error="rpc error: code = Unknown desc = gateway: error calling Stat: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp [::1]:9152: connect: connection refused\"" pid=17832 pkg=rhttp traceid=b4eb9a9f45921f7d3632523ca32a42b0
```
{{&lt; hint danger >}}
TODO return the trace id in the response so we can correlate easier. For reva tracked in <https://github.com/cs3org/reva/issues/587>
{{&lt; /hint >}}
{{< hint danger >}}
TODO return the trace id in the response so we can correlate easier. For reva tracked in https://github.com/cs3org/reva/issues/587
{{< /hint >}}
The last line gives us a hint where the log message originated: `.../github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/http/services/owncloud/ocs/reqres.go:94`. Which looks like this:
@@ -202,14 +203,15 @@ Debug wherever the call trace leads you to ... good luck!
You can either run and manage the services independently, or you can update the `go.mod` file and replace dependencies with your local version.
To debug the reva frontend we need to add two replacements:
// use the local ocis-reva repo
replace github.com/owncloud/ocis-reva => ../ocis-reva
// also use the local reva repo
replace github.com/cs3org/reva => ../reva
{{&lt; hint info >}}
```
// use the local ocis-reva repo
replace github.com/owncloud/ocis-reva => ../ocis-reva
// also use the local reva repo
replace github.com/cs3org/reva => ../reva
```
{{< hint info >}}
The username and password only work when basic auth is available. Otherwise you have to obtain a bearer token, eg. by grabbing it from the browser.
{{&lt; /hint >}}
{{< /hint >}}
Rebuild ocis to make sure the dependency is used. It should be sufficient to just restart the service you want to debug.

94
ocis/docs/development.md
View File

@@ -1,14 +1,13 @@
* * *
---
title: "Getting Started with Development"
date: 2020-07-07T20:35:00+01:00
weight: 15
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: development.md
---
## geekdocFilePath: development.md
{{&lt; toc >}}
{{< toc >}}
## Docker dev environment
@@ -16,59 +15,66 @@ geekdocEditPath: edit/master/docs
To build and run your local ocis code with default storage driver
docker run --rm -ti --name ocis -v $PWD:/ocis -p 9200:9200 owncloud/eos-ocis-dev
```
docker run --rm -ti --name ocis -v $PWD:/ocis -p 9200:9200 owncloud/eos-ocis-dev
```
The eos-ocis-dev container will build and run ocis using the owncloud storage driver and store files in the container at `/var/tmp/reva/data/<username>/files`
To check the uploaded files start digging with: `docker exec -it ocis ls -l /var/tmp/reva/`
{{&lt; hint info >}}
{{< hint info >}}
On MacOS do not mount a local folder to the `/var/tmp/reva/` path. The fuse driver used by docker [does not support extended attributes](https://docs.docker.com/v18.09/docker-for-mac/osxfs/). See [#182](https://github.com/owncloud/ocis/issues/182) for more details.
{{&lt; /hint >}}
{{< /hint >}}
### Option 2: Docker compose
With the `docker-compose.yml` file in ocis repo you can also start ocis via compose:
docker-compose up -d ocis
```
docker-compose up -d ocis
```
{{&lt; hint info >}}
{{< hint info >}}
We are only starting the `ocis` container here.
{{&lt; /hint >}}
{{< /hint >}}
## Verification
Check the services are running
$ docker-compose exec ocis ./bin/ocis list
+--------------------------+-----+
| EXTENSION | PID |
+--------------------------+-----+
| accounts | 172 |
| api | 204 |
| glauth | 187 |
| graph | 41 |
| graph-explorer | 55 |
| konnectd | 196 |
| ocs | 59 |
| phoenix | 29 |
| proxy | 22 |
| registry | 226 |
| reva-auth-basic | 96 |
| reva-auth-bearer | 104 |
| reva-frontend | 485 |
| reva-gateway | 78 |
| reva-sharing | 286 |
| reva-storage-eos | 129 |
| reva-storage-eos-data | 134 |
| reva-storage-home | 442 |
| reva-storage-home-data | 464 |
| reva-storage-oc | 149 |
| reva-storage-oc-data | 155 |
| reva-storage-public-link | 168 |
| reva-users | 420 |
| settings | 23 |
| thumbnails | 201 |
| web | 218 |
| webdav | 63 |
+--------------------------+-----+
```
$ docker-compose exec ocis ./bin/ocis list
+--------------------------+-----+
| EXTENSION | PID |
+--------------------------+-----+
| accounts | 172 |
| api | 204 |
| glauth | 187 |
| graph | 41 |
| graph-explorer | 55 |
| konnectd | 196 |
| ocs | 59 |
| phoenix | 29 |
| proxy | 22 |
| registry | 226 |
| reva-auth-basic | 96 |
| reva-auth-bearer | 104 |
| reva-frontend | 485 |
| reva-gateway | 78 |
| reva-sharing | 286 |
| reva-storage-eos | 129 |
| reva-storage-eos-data | 134 |
| reva-storage-home | 442 |
| reva-storage-home-data | 464 |
| reva-storage-oc | 149 |
| reva-storage-oc-data | 155 |
| reva-storage-public-link | 168 |
| reva-users | 420 |
| settings | 23 |
| thumbnails | 201 |
| web | 218 |
| webdav | 63 |
+--------------------------+-----+
```

239
ocis/docs/eos.md
View File

@@ -1,14 +1,13 @@
* * *
---
title: "EOS"
date: 2020-02-27T20:35:00+01:00
weight: 30
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: eos.md
---
## geekdocFilePath: eos.md
{{&lt; toc >}}
{{< toc >}}
OCIS can be configured to run on top of [eos](https://eos.web.cern.ch/). While the [eos documentation](http://eos-docs.web.cern.ch/) does cover a lot of topics it leaves out some details that you may have to either pull from various [docker containers](https://gitlab.cern.ch/eos/eos-docker), the [forums](https://eos-community.web.cern.ch/) or even the [source](https://github.com/cern-eos/eos) itself.
@@ -16,151 +15,164 @@ This document is a work in progress of the current setup.
## Docker dev environment for eos storage
We begin with the docker-compose.yml found in <https://github.com/owncloud/ocis/> and
We begin with the docker-compose.yml found in https://github.com/owncloud/ocis/ and
switch it to eos-storage.
### 1. Start eos & ocis containers
Start the eos cluster and ocis via the compose stack.
```
docker-compose up -d
```
docker-compose up -d
{{&lt; hint info >}}
{{< hint info >}}
The first time the **ocis** container starts up, it will compile ocis from scratch which can take a while.
To follow progress, run `docker-compose logs -f --tail=10 ocis`
{{&lt; /hint >}}
{{< /hint >}}
### 2. LDAP support
Configure the OS to resolve users and groups using ldap
docker-compose exec -d ocis /start-ldap
```
docker-compose exec -d ocis /start-ldap
```
Check that the OS in the ocis container can now resolve einstein or the other demo users
$ docker-compose exec ocis id einstein
uid=20000(einstein) gid=30000(users) groups=30000(users),30001(sailing-lovers),30002(violin-haters),30007(physics-lovers)
```
$ docker-compose exec ocis id einstein
uid=20000(einstein) gid=30000(users) groups=30000(users),30001(sailing-lovers),30002(violin-haters),30007(physics-lovers)
```
{{&lt; hint info >}}
{{< hint info >}}
If the user is not found at first you might need to wait a few more minutes in case the ocis container is still compiling.
{{&lt; /hint >}}
{{< /hint >}}
We also need to restart the reva-users service so it picks up the changed environment. Without a restart it is not able to resolve users from LDAP.
docker-compose exec ocis ./bin/ocis kill reva-users
docker-compose exec ocis ./bin/ocis run reva-users
```
docker-compose exec ocis ./bin/ocis kill reva-users
docker-compose exec ocis ./bin/ocis run reva-users
```
### 3. Home storage
Kill the home storage. By default it uses the `owncloud` storage driver. We need to switch it to the `eoshome` driver and make it use the storage id of the eos storage provider:
docker-compose exec ocis ./bin/ocis kill reva-storage-home
docker-compose exec -e REVA_STORAGE_HOME_DRIVER=eoshome -e REVA_STORAGE_HOME_MOUNT_ID=1284d238-aa92-42ce-bdc4-0b0000009158 ocis ./bin/ocis run reva-storage-home
```
docker-compose exec ocis ./bin/ocis kill reva-storage-home
docker-compose exec -e REVA_STORAGE_HOME_DRIVER=eoshome -e REVA_STORAGE_HOME_MOUNT_ID=1284d238-aa92-42ce-bdc4-0b0000009158 ocis ./bin/ocis run reva-storage-home
```
### 4. Home data provider
Kill the home data provider. By default it uses the `owncloud` storage driver. We need to switch it to the `eoshome` driver and make it use the storage id of the eos storage provider:
docker-compose exec ocis ./bin/ocis kill reva-storage-home-data
docker-compose exec -e REVA_STORAGE_HOME_DATA_DRIVER=eoshome ocis ./bin/ocis run reva-storage-home-data
```
docker-compose exec ocis ./bin/ocis kill reva-storage-home-data
docker-compose exec -e REVA_STORAGE_HOME_DATA_DRIVER=eoshome ocis ./bin/ocis run reva-storage-home-data
```
{{&lt; hint info >}}
The difference between the _home storage_ and the _home data provider_ are that the former is responsible for metadata changes while the latter is responsible for actual data transfer. The _home storage_ uses the cs3 api to manage a folder hierarchy, while the _home data provider_ is responsible for moving bytes to and from the storage.
{{&lt; /hint >}}
{{< hint info >}}
The difference between the *home storage* and the *home data provider* are that the former is responsible for metadata changes while the latter is responsible for actual data transfer. The *home storage* uses the cs3 api to manage a folder hierarchy, while the *home data provider* is responsible for moving bytes to and from the storage.
{{< /hint >}}
## Verification
Login with `einstein / relativity`, upload a file to einsteins home and verify the file is there using
docker-compose exec ocis eos ls -l /eos/dockertest/reva/users/4/4c510ada-c86b-4815-8820-42cdf82c3d51/
-rw-r--r-- 1 einstein users 10 Jul 1 15:24 newfile.txt
```
docker-compose exec ocis eos ls -l /eos/dockertest/reva/users/4/4c510ada-c86b-4815-8820-42cdf82c3d51/
-rw-r--r-- 1 einstein users 10 Jul 1 15:24 newfile.txt
```
If the problem persists, please check the [troubleshooting section about uploads](#creation-and-upload-of-files-does-not-work).
## Further exploration
EOS has a built in shell that you can enter using
$ docker-compose exec mgm-master eos
# ---------------------------------------------------------------------------
# EOS Copyright (C) 2011-2019 CERN/Switzerland
# This program comes with ABSOLUTELY NO WARRANTY; for details type `license'.
# This is free software, and you are welcome to redistribute it
# under certain conditions; type `license' for details.
# ---------------------------------------------------------------------------
EOS_INSTANCE=eostest
EOS_SERVER_VERSION=4.6.5 EOS_SERVER_RELEASE=1
EOS_CLIENT_VERSION=4.6.5 EOS_CLIENT_RELEASE=1
EOS Console [root://localhost] |/> help
access Access Interface
accounting Accounting Interface
acl Acl Interface
archive Archive Interface
attr Attribute Interface
backup Backup Interface
clear Clear the terminal
cd Change directory
chmod Mode Interface
chown Chown Interface
config Configuration System
console Run Error Console
cp Cp command
debug Set debug level
exit Exit from EOS console
file File Handling
fileinfo File Information
find Find files/directories
newfind Find files/directories (new implementation)
fs File System configuration
fsck File System Consistency Checking
fuse Fuse Mounting
fusex Fuse(x) Administration
geosched Geoscheduler Interface
group Group configuration
health Health information about system
help Display this text
info Retrieve file or directory information
inspector Interact with File Inspector
io IO Interface
json Toggle JSON output flag for stdout
license Display Software License
ls List a directory
ln Create a symbolic link
map Path mapping interface
member Check Egroup membership
mkdir Create a directory
motd Message of the day
mv Rename file or directory
node Node configuration
ns Namespace Interface
pwd Print working directory
quit Exit from EOS console
quota Quota System configuration
reconnect Forces a re-authentication of the shell
recycle Recycle Bin Functionality
rmdir Remove a directory
rm Remove a file
role Set the client role
route Routing interface
rtlog Get realtime log output from mgm & fst servers
silent Toggle silent flag for stdout
space Space configuration
stagerrm Remove disk replicas of a file if it has tape replicas
stat Run 'stat' on a file or directory
squash Run 'squashfs' utility function
test Run performance test
timing Toggle timing flag for execution time measurement
touch Touch a file
token Token interface
tracker Interact with File Tracker
transfer Transfer Interface
version Verbose client/server version
vid Virtual ID System Configuration
whoami Determine how we are mapped on server side
who Statistics about connected users
? Synonym for 'help'
.q Exit from EOS console
EOS Console [root://localhost] |/>
```
$ docker-compose exec mgm-master eos
# ---------------------------------------------------------------------------
# EOS Copyright (C) 2011-2019 CERN/Switzerland
# This program comes with ABSOLUTELY NO WARRANTY; for details type `license'.
# This is free software, and you are welcome to redistribute it
# under certain conditions; type `license' for details.
# ---------------------------------------------------------------------------
EOS_INSTANCE=eostest
EOS_SERVER_VERSION=4.6.5 EOS_SERVER_RELEASE=1
EOS_CLIENT_VERSION=4.6.5 EOS_CLIENT_RELEASE=1
EOS Console [root://localhost] |/> help
access Access Interface
accounting Accounting Interface
acl Acl Interface
archive Archive Interface
attr Attribute Interface
backup Backup Interface
clear Clear the terminal
cd Change directory
chmod Mode Interface
chown Chown Interface
config Configuration System
console Run Error Console
cp Cp command
debug Set debug level
exit Exit from EOS console
file File Handling
fileinfo File Information
find Find files/directories
newfind Find files/directories (new implementation)
fs File System configuration
fsck File System Consistency Checking
fuse Fuse Mounting
fusex Fuse(x) Administration
geosched Geoscheduler Interface
group Group configuration
health Health information about system
help Display this text
info Retrieve file or directory information
inspector Interact with File Inspector
io IO Interface
json Toggle JSON output flag for stdout
license Display Software License
ls List a directory
ln Create a symbolic link
map Path mapping interface
member Check Egroup membership
mkdir Create a directory
motd Message of the day
mv Rename file or directory
node Node configuration
ns Namespace Interface
pwd Print working directory
quit Exit from EOS console
quota Quota System configuration
reconnect Forces a re-authentication of the shell
recycle Recycle Bin Functionality
rmdir Remove a directory
rm Remove a file
role Set the client role
route Routing interface
rtlog Get realtime log output from mgm & fst servers
silent Toggle silent flag for stdout
space Space configuration
stagerrm Remove disk replicas of a file if it has tape replicas
stat Run 'stat' on a file or directory
squash Run 'squashfs' utility function
test Run performance test
timing Toggle timing flag for execution time measurement
touch Touch a file
token Token interface
tracker Interact with File Tracker
transfer Transfer Interface
version Verbose client/server version
vid Virtual ID System Configuration
whoami Determine how we are mapped on server side
who Statistics about connected users
? Synonym for 'help'
.q Exit from EOS console
EOS Console [root://localhost] |/>
```
But this is a different adventure. See the links at the top of this page for other sources of information on eos.
@@ -184,10 +196,10 @@ The ocis logs can be accessed using `docker-compose logs ocis`. Add `-f` for fol
### How do I update a service in the ocis container?
1. `docker-compose exec ocis make clean build` to update the binary
2. `docker-compose exec ocis ./bin/ocis kill <service>` to kill the service
3. `docker-compose exec ocis ./bin/ocis run <service>` to start the service. Do not forget to set any env vars, eg.
`docker-compose exec -e REVA_STORAGE_EOS_LAYOUT="{{substr 0 1 .Id.OpaqueId}}/{{.Id.OpaqueId}}" -e REVA_STORAGE_HOME_DRIVER=eoshome ocis ./bin/ocis run reva-storage-home`
1. `docker-compose exec ocis make clean build` to update the binary
2. `docker-compose exec ocis ./bin/ocis kill <service>` to kill the service
3. `docker-compose exec ocis ./bin/ocis run <service>` to start the service. Do not forget to set any env vars, eg.
`docker-compose exec -e REVA_STORAGE_EOS_LAYOUT="{{substr 0 1 .Id.OpaqueId}}/{{.Id.OpaqueId}}" -e REVA_STORAGE_HOME_DRIVER=eoshome ocis ./bin/ocis run reva-storage-home`
### Creation and upload of files does not work
@@ -205,3 +217,4 @@ The EOS dockers are configured with replication, so every file uploaded there wi
so make sure there is enough physical space on disk when testing.
Also please note that older failed uploads might still be present in the "/tmp" directory of the "ocis" container.

127
ocis/docs/extensions.md
View File

@@ -1,14 +1,13 @@
* * *
---
title: "Extension"
date: 2020-02-27T20:35:00+01:00
weight: 40
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: extensions.md
---
## geekdocFilePath: extensions.md
{{&lt; toc >}}
{{< toc >}}
## How to build and run ocis-simple
@@ -18,7 +17,7 @@ ocis uses build tags to build different flavors of the binary. In order to work
mkdir ocis-extension-workshop && ocis-extension-workshop
```
Following <https://github.com/owncloud/ocis>
Following https://github.com/owncloud/ocis
```console
git clone https://github.com/owncloud/ocis.git
@@ -27,20 +26,20 @@ cd ocis
TAGS=simple make generate build
```
_Q: Can you specify which version of phoenix to use?_
_A: No, the phoenix that is used is compiled into the [assets of ocis-phoenix](https://github.com/owncloud/ocis-phoenix/blob/master/pkg/assets/embed.go) which is currently not automatically updated. We'll see how to use a custom phoenix later._
*Q: Can you specify which version of phoenix to use?*
*A: No, the phoenix that is used is compiled into the [assets of ocis-phoenix](https://github.com/owncloud/ocis-phoenix/blob/master/pkg/assets/embed.go) which is currently not automatically updated. We'll see how to use a custom phoenix later.*
`bin/ocis server`
Open the browser at http&#x3A;//localhost:9100
Open the browser at http://localhost:9100
1. You land on the login screen. click login
2. You are redirected to an idp at http&#x3A;//localhost:9140/oauth2/auth with a login mask. Use `einstein:relativity`to login (one of the three demo users)
3. You are redirected to http&#x3A;//localhost:9100/#/hello the ocis-hello app
4. Replace `World` with something else and submit. You should see `Hello %something else%`
1. You land on the login screen. click login
2. You are redirected to an idp at http://localhost:9140/oauth2/auth with a login mask. Use `einstein:relativity`to login (one of the three demo users)
3. You are redirected to http://localhost:9100/#/hello the ocis-hello app
4. Replace `World` with something else and submit. You should see `Hello %something else%`
_Q: One of the required ports is already in use. Ocis seems to be trying to restart the service over and over. What gives?_
_A: Using the ocis binary to start the server will case ocis to keep track of the different services and restart them in case they crash._
*Q: One of the required ports is already in use. Ocis seems to be trying to restart the service over and over. What gives?*
*A: Using the ocis binary to start the server will case ocis to keep track of the different services and restart them in case they crash.*
## Hacking ocis-hello
@@ -50,25 +49,25 @@ go back to the ocis-extension-workshop folder
cd ..
```
Following <https://github.com/owncloud/ocis-hello>
Following https://github.com/owncloud/ocis-hello
git clone https://github.com/owncloud/ocis-hello.git
cd ocis-hello
```
git clone https://github.com/owncloud/ocis-hello.git
cd ocis-hello
yarn install
# this actually creates the assets
yarn build
yarn install
# this actually creates the assets
yarn build
# this will compile the assets into the binary
make generate build
# this will compile the assets into the binary
make generate build
```
Two options:
1. run only the necessery services from ocis and ocis-hello independently
2. compile ocis with the updated ocis-hello
1. run only the necessery services from ocis and ocis-hello independently
2. compile ocis with the updated ocis-hello
### Option 1:
get a list of ocis services:
```console
@@ -83,10 +82,12 @@ In order to be able to manage the processes ourselves we need to start them inde
`bin/ocis server` starts the same services as:
bin/ocis micro &
bin/ocis phoenix &
bin/ocis hello &
bin/ocis reva &
```
bin/ocis micro &
bin/ocis phoenix &
bin/ocis hello &
bin/ocis reva &
```
Now we can kill the `ocis hello` and use our custom built ocis-hello binary:
@@ -97,13 +98,15 @@ bin/ocis-hello server
## Hacking phoenix (and ocis-phoenix)
Following <https://github.com/owncloud/phoenix> we are going to build the current phoenix
Following https://github.com/owncloud/phoenix we are going to build the current phoenix
git clone https://github.com/owncloud/phoenix.git
cd phoenix
```
git clone https://github.com/owncloud/phoenix.git
cd phoenix
yarn install
yarn dist
yarn install
yarn dist
```
We can tell ocis to use the compiled assets:
@@ -118,7 +121,7 @@ PHOENIX_ASSET_PATH="`pwd`/../phoenix/dist" bin/ocis phoenix
The owncloud design system contains a set of ownCloud vue components for phoenix or your own ocis extensions. Use it for a consistent look and feel.
Point your browser to <https://owncloud.github.io/owncloud-design-system> and check the available components. Live editing the examples in the browser is supported.
Point your browser to https://owncloud.github.io/owncloud-design-system and check the available components. Live editing the examples in the browser is supported.
note: There is a bug with navigation sub items: either click a nav item twice or refresh the page
@@ -126,10 +129,10 @@ note: There is a bug with navigation sub items: either click a nav item twice or
This is what hello is: copy and extend!
1. Phoenix is configured using the config.json which is served by the phoenix service (either `bin/ocis phoenix` or `bin/ocis-phoenix server`)
1. Phoenix is configured using the config.json which is served by the phoenix service (either `bin/ocis phoenix` or `bin/ocis-phoenix server`)
2. point ocis phoenix to the web config which you extended with an external app:
`PHOENIX_WEB_CONFIG="`pwd`/../phoenix/config.json" PHOENIX_ASSET_PATH="`pwd`/../phoenix/dist" bin/ocis phoenix`
2. point ocis phoenix to the web config which you extended with an external app:
`PHOENIX_WEB_CONFIG="`pwd`/../phoenix/config.json" PHOENIX_ASSET_PATH="`pwd`/../phoenix/dist" bin/ocis phoenix`
```json
{
@@ -166,26 +169,23 @@ This is what hello is: copy and extend!
## Phoenix extension points
{{&lt; hint info >}}
{{< hint info >}}
For an up to date list check out [the phoenix documentation](https://github.com/owncloud/phoenix/issues/2423).
{{&lt; /hint >}}
{{< /hint >}}
Several ones available:
### Phoenix core
- App switcher (defined in config.json)
- App container (loads UI of your extension)
- App switcher (defined in config.json)
- App container (loads UI of your extension)
### Files app
- File action
- Create new file action
- Sidebar
- Quick access for sidebar inside of file actions (in the file row)
- File action
- Create new file action
- Sidebar
- Quick access for sidebar inside of file actions (in the file row)
Example of a file action in the `app.js`:
```js
const appInfo = {
name: 'MarkdownEditor',
@@ -221,20 +221,17 @@ Short answer: any way you like
Long answer: micro and ocis-hello follow a protocol driven development:
- specify the API using protobuf
- specify the API using protobuf
- generate client and server code
- evolve based on the protocol
- generate client and server code
- CS3 api uses protobuf as well and uses GRPC
- evolve based on the protocol
- ocis uses go-micro, which provides http and grpc gateways
- the gateways and protocols are optional
- CS3 api uses protobuf as well and uses GRPC
- owncloud and kopano are looking into a [MS graph](https://developer.microsoft.com/de-de/graph) like api to handle phoenix requests.
- they might be about user, contacrs, calendars ... which is covered by the graph api
- we want to integrate with eg. kopano and provide a commen api (file sync and share is covered as well)
- ocis uses go-micro, which provides http and grpc gateways
- the gateways and protocols are optional
- owncloud and kopano are looking into a [MS graph](https://developer.microsoft.com/de-de/graph) like api to handle phoenix requests.
- they might be about user, contacrs, calendars ... which is covered by the graph api
- we want to integrate with eg. kopano and provide a commen api (file sync and share is covered as well)
- as an example for protobuf take a look at [ocis-hello](https://github.com/owncloud/ocis-hello/tree/master/pkg/proto/v0)
- as an example for protobuf take a look at [ocis-hello](https://github.com/owncloud/ocis-hello/tree/master/pkg/proto/v0)

37
ocis/docs/getting-started.md
View File

@@ -1,14 +1,13 @@
* * *
---
title: "Getting Started"
date: 2020-02-27T20:35:00+01:00
weight: 15
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
## geekdocFilePath: getting-started.md
{{&lt; toc >}}
{{< toc >}}
## Installation
@@ -16,7 +15,7 @@ So far we are offering two different variants for the installation. You can choo
### Docker
Docker images for ocis are hosted on <https://hub.docker.com/r/owncloud/ocis>.
Docker images for ocis are hosted on https://hub.docker.com/r/owncloud/ocis.
The `latest` tag always reflects the current master branch.
@@ -26,8 +25,8 @@ docker pull owncloud/ocis
#### Dependencies
- Running ocis currently needs a working Redis caching server
- The default storage location in the container is `/var/tmp/reva/data`. You may want to create a volume to persist the files in the primary storage
- Running ocis currently needs a working Redis caching server
- The default storage location in the container is `/var/tmp/reva/data`. You may want to create a volume to persist the files in the primary storage
#### Docker compose
@@ -41,7 +40,7 @@ docker-compose -f ocis.yml -f ../cache/redis-ocis.yml up
### Binaries
The pre-built binaries for different platforms are downloadable at <https://download.owncloud.com/ocis/ocis/> . Specific releases are organized in separate folders. They are in sync which every release tag on GitHub. The binaries from the current master branch can be found in <https://download.owncloud.com/ocis/ocis/testing/>
The pre-built binaries for different platforms are downloadable at https://download.owncloud.com/ocis/ocis/ . Specific releases are organized in separate folders. They are in sync which every release tag on GitHub. The binaries from the current master branch can be found in https://download.owncloud.com/ocis/ocis/testing/
```console
curl https://download.owncloud.com/ocis/ocis/1.0.0-beta1/ocis-1.0.0-beta1-darwin-amd64 --output ocis
@@ -51,8 +50,8 @@ chmod +x ocis
#### Dependencies
- Running ocis currently needs a working Redis caching server
- The default promary storage location is `/var/tmp/reva/data`. You can change that value by configuration.
- Running ocis currently needs a working Redis caching server
- The default promary storage location is `/var/tmp/reva/data`. You can change that value by configuration.
## Usage
@@ -62,21 +61,21 @@ The program provides a few sub-commands on execution. The available configuratio
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{&lt; highlight txt >}}
{{< highlight txt >}}
ocis server --help
{{&lt; / highlight >}}
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{&lt; highlight txt >}}
{{< highlight txt >}}
ocis health --help
{{&lt; / highlight >}}
{{< / highlight >}}
## Quickstart for Developers
Following <https://github.com/owncloud/ocis#development>
Following https://github.com/owncloud/ocis#development
```console
git clone https://github.com/owncloud/ocis.git
@@ -84,7 +83,7 @@ cd ocis
make generate build
```
Open https&#x3A;//localhost:9200 and login using one of the demo accounts:
Open https://localhost:9200 and login using one of the demo accounts:
```console
einstein:relativity
@@ -226,4 +225,4 @@ promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code
: Total number of scrapes by HTTP status code

9
ocis/docs/license.md
View File

@@ -1,11 +1,10 @@
* * *
---
title: "License"
date: 2020-02-27T20:35:00+01:00
weight: 100
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
## geekdocFilePath: license.md
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/blob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

19
ocis/docs/login-flow.md
View File

@@ -1,23 +1,23 @@
* * *
---
title: "Login Flow"
date: 2020-05-04T20:47:00+01:00
weight: 43
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: login-flow.md
---
## geekdocFilePath: login-flow.md
## Login Flow
The following sequence diagram describes the [openid connect auth code flow](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth). The eight numbered steps and notes correspond to the [openid connect auth code flow steps](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowSteps). Example requests are based on the spec as well.:
{{&lt; mermaid class="text-center">}}
{{< mermaid class="text-center">}}
sequenceDiagram
%% we have comments!! \\o/
%% we have comments!! \o/
%% this documents the login workflow
%% examples taken from the oidc spec <https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth>
%% TODO add PKCE, see <https://developer.okta.com/blog/2019/08/22/okta-authjs-pkce#use-pkce-to-make-your-apps-more-secure>
%% examples taken from the oidc spec https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth
%% TODO add PKCE, see https://developer.okta.com/blog/2019/08/22/okta-authjs-pkce#use-pkce-to-make-your-apps-more-secure
participant user as User
participant client as Client
participant proxy as ocis-proxy
@@ -82,5 +82,4 @@ sequenceDiagram
client->>+proxy: PROPFIND <br> With access token
proxy-->>-client: 207 Multi-Status
client-->>-user: List of Files X, Y, Z ...
{{&lt; /mermaid >}}
{{< /mermaid >}}

10
ocis/docs/public-upload-flow.md
View File

@@ -1,15 +1,15 @@
* * *
---
title: "Public upload Flow"
date: 2020-07-27T14:16:00+01:00
weight: 47
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: public-upload-flow.md
---
## geekdocFilePath: public-upload-flow.md
## Public Upload flow
The following diagram describes the flow of requests:
{{&lt; svg src="static/ocis/tus-public-upload.svg" >}}
{{< svg src="static/ocis/tus-public-upload.svg" >}}

15
ocis/docs/request-flow.md
View File

@@ -1,20 +1,20 @@
* * *
---
title: "Request Flow"
date: 2020-04-27T16:07:00+01:00
weight: 45
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: request-flow.md
---
## geekdocFilePath: request-flow.md
## Request Flow
The following sequence diagram describes the general request flow. It shows where account provisioning and token minting are happening:
{{&lt; mermaid class="text-center">}}
{{< mermaid class="text-center">}}
sequenceDiagram
%% we have comments!! \\o/
%% we have comments!! \o/
participant user as User
participant client as Client
participant proxy as ocis-proxy
@@ -93,5 +93,4 @@ sequenceDiagram
proxy-->>-client: Multistatus response
client-->>-user: List of Files X, Y, Z ...
{{&lt; /mermaid >}}
{{< /mermaid >}}

62
ocis/docs/testing.md
View File

@@ -1,12 +1,12 @@
* * *
---
title: "Testing"
date: 2018-05-02T00:00:00+00:00
weight: 37
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: testing.md
---
## geekdocFilePath: testing.md
## Acceptance tests
@@ -15,8 +15,9 @@ We are using the ownCloud 10 acceptance testsuite against ocis. To set this up y
### Getting the tests
All you need to do to get the acceptance tests is check out the core repo:
git clone https://github.com/owncloud/core.git
```
git clone https://github.com/owncloud/core.git
```
### Run a redis server in a docker container
@@ -27,24 +28,27 @@ File versions need a redis server. Start one with docker by using:
### Run ocis
To start ocis:
bin/ocis server
```
bin/ocis server
```
### Run the acceptance tests
First we will need to clone the testing app in owncloud which contains the skeleton files required for running the tests.
In the ownCloud 10 core clone the testing app with the following command:
git clone https://github.com/owncloud/testing apps/testing
```
git clone https://github.com/owncloud/testing apps/testing
```
Then run the api acceptance tests with the following command:
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
SKELETON_DIR=apps/testing/data/apiSkeleton \
BEHAT_FILTER_TAGS='~@notToImplementOnOCIS&&~@toImplementOnOCIS'
```
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
SKELETON_DIR=apps/testing/data/apiSkeleton \
BEHAT_FILTER_TAGS='~@notToImplementOnOCIS&&~@toImplementOnOCIS'
```
Make sure to adjust the settings `TEST_SERVER_URL` and `OCIS_REVA_DATA_ROOT` according to your environment.
@@ -80,30 +84,24 @@ If you want to work on a specific issue
2. locally run each of the tests marked with that issue in the expected failures file
E.g.:
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
BEHAT_FEATURE='tests/acceptance/features/apiComments/comments.feature:123'
```
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
BEHAT_FEATURE='tests/acceptance/features/apiComments/comments.feature:123'
```
3. the tests will fail, try to understand how and why they are failing
4. fix the code
5. go back to 2. and repeat till the tests are passing.
6. remove those tests from the expected failures file
7. run each of the local tests that were demonstrating the **buggy** behavior. They should fail.
8. delete each of the local tests that were demonstrating the **buggy** behavior.
9. make a PR that has the fixed code, relevant lines removed from the expected failures file and bug demonstration tests deleted.
If the changes also affect the `ocis-reva` repository make sure the changes get ported over there.
### Notes
- in a normal case the test-code cleans up users after the test-run, but if a test-run is interrupted (e.g. by CTRL+C) users might have been left on the LDAP server. In that case rerunning the tests requires wiping the users in the ldap server, otherwise the tests will fail when trying to populate the users.
- the tests usually create users in the OU `TestUsers` with usernames specified in the feature file. If not defined in the feature file, most users have the password `123456`, defined by `regularUserPassword` in `behat.yml`, but other passwords are also used, see [`\FeatureContext::getPasswordForUser()`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L386) for mapping and [`\FeatureContext::__construct`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L1668) for the password definitions.
- in a normal case the test-code cleans up users after the test-run, but if a test-run is interrupted (e.g. by CTRL+C) users might have been left on the LDAP server. In that case rerunning the tests requires wiping the users in the ldap server, otherwise the tests will fail when trying to populate the users.
- the tests usually create users in the OU `TestUsers` with usernames specified in the feature file. If not defined in the feature file, most users have the password `123456`, defined by `regularUserPassword` in `behat.yml`, but other passwords are also used, see [`\FeatureContext::getPasswordForUser()`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L386) for mapping and [`\FeatureContext::__construct`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L1668) for the password definitions.

25
ocis/docs/tracing.md
View File

@@ -1,17 +1,16 @@
* * *
---
title: "Tracing"
date: 2020-05-13T12:09:00+01:00
weight: 55
geekdocRepo: <https://github.com/owncloud/ocis>
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
## geekdocFilePath: tracing.md
geekdocFilePath: tracing.md
---
By default, we use [Jaeger](https://www.jaegertracing.io) for request tracing within oCIS. You can follow these steps
to get started:
1. Start Jaeger by using the all-in-one docker image:
1. Start Jaeger by using the all-in-one docker image:
```console
docker run -d --name jaeger \
-e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
@@ -25,8 +24,8 @@ to get started:
-p 9411:9411 \
jaegertracing/all-in-one:1.17
```
2. Every single oCIS service has its own environment variables for enabling and configuring tracing.
1. You can enable and configure tracing on each service individually. For example, enable tracing
2. Every single oCIS service has its own environment variables for enabling and configuring tracing.
1. You can enable and configure tracing on each service individually. For example, enable tracing
in Reva when starting the oCIS single binary like this:
```console
REVA_TRACING_ENABLED=true \
@@ -34,16 +33,16 @@ to get started:
REVA_TRACING_COLLECTOR=http://localhost:14268/api/traces \
./bin/ocis server
```
2. Enabling and configuring tracing on oCIS itself will forward the configuration to all services:
2. Enabling and configuring tracing on oCIS itself will forward the configuration to all services:
```console
OCIS_TRACING_ENABLED=true \
OCIS_TRACING_ENDPOINT=localhost:6831 \
OCIS_TRACING_COLLECTOR=http://localhost:14268/api/traces \
./bin/ocis server
```
If you want to set individual tracing configuration for each service, make sure to set
`OCIS_TRACING_ENABLED=false`.
3. Make the actual request that you want to trace.
4. Open up the [Jaeger UI](http://localhost:16686) to analyze request traces.
If you want to set individual tracing configuration for each service, make sure to set
`OCIS_TRACING_ENABLED=false`.
3. Make the actual request that you want to trace.
4. Open up the [Jaeger UI](http://localhost:16686) to analyze request traces.
For more information on Jaeger, please refer to their [Documentation](https://www.jaegertracing.io/docs/1.17/).