mirror/opencloud
1
0
Fork 0
mirror of https://github.com/opencloud-eu/opencloud.git synced 2026-01-06 20:29:54 -06:00
Code Issues Packages Projects Releases Wiki Activity

cleanup docs for ocis and remaining extensions

Browse Source
This commit is contained in:
Michael Barz
2020-09-30 19:55:05 +02:00
parent 1ebe0f14a1
commit 456ba52d05
68 changed files with 28 additions and 5621 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
ocis-phoenix/Makefile
View File

@@ -3,7 +3,6 @@ NAME := ocis-phoenix
IMPORT := github.com/owncloud/ocis/$(NAME)
BIN := bin
DIST := dist
HUGO := hugo
PHOENIX_ASSETS_VERSION = v0.17.0
ifeq ($(OS), Windows_NT)
@@ -59,7 +58,7 @@ sync:
.PHONY: clean
clean:
go clean -i ./...
rm -rf $(BIN) $(DIST) $(HUGO) assets
rm -rf $(BIN) $(DIST) assets
.PHONY: fmt
fmt:
@@ -132,24 +131,9 @@ release-check:
.PHONY: release-finish
release-finish: release-copy release-check
.PHONY: docs-copy
docs-copy:
mkdir -p $(HUGO); \
mkdir -p $(HUGO)/content/extensions; \
cd $(HUGO); \
git init; \
git remote rm origin; \
git remote add origin https://github.com/owncloud/owncloud.github.io; \
git fetch; \
git checkout origin/source -f; \
rsync --delete -ax ../docs/ content/extensions/$(NAME)
.PHONY: docs-build
docs-build:
cd $(HUGO); hugo
.PHONY: docs
docs: docs-copy docs-build
.PHONY: config-docs-generate
config-docs-generate:
# go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md
.PHONY: watch
watch:

10
ocis-phoenix/docs/_index.md
View File

@@ -1,10 +0,0 @@
---
title: "Phoenix"
date: 2018-05-02T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-phoenix
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
This service embeds [Phoenix](https://github.com/owncloud/phoenix) to provide a UI for ownCloud Infinite Scale.

26
ocis-phoenix/docs/building.md
View File

@@ -1,26 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis-phoenix
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.12. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis-phoenix.git
cd ocis-phoenix
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
## Backend
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
The above commands will download a [Phoenix](https://github.com/owncloud/phoenix) release and embed it into the binary. Finally you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis-phoenix -h` to see all available options.

339
ocis-phoenix/docs/getting-started.md
View File

@@ -1,339 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 20
geekdocRepo: https://github.com/owncloud/ocis-phoenix
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
TBD
### Binaries
TBD
## Configuration
We provide overall three different variants of configuration. The variant based on environment variables and commandline flags are split up into global values and command-specific values.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
#### Global
PHOENIX_CONFIG_FILE
: Path to config file, empty default value
PHOENIX_LOG_LEVEL
: Set logging level, defaults to `info`
PHOENIX_LOG_COLOR
: Enable colored logging, defaults to `true`
PHOENIX_LOG_PRETTY
: Enable pretty logging, defaults to `true`
#### Server
PHOENIX_TRACING_ENABLED
: Enable sending traces, defaults to `false`
PHOENIX_TRACING_TYPE
: Tracing backend type, defaults to `jaeger`
PHOENIX_TRACING_ENDPOINT
: Endpoint for the agent, empty default value
PHOENIX_TRACING_COLLECTOR
: Endpoint for the collector, empty default value
PHOENIX_TRACING_SERVICE
: Service name for tracing, defaults to `phoenix`
PHOENIX_DEBUG_ADDR
: Address to bind debug server, defaults to `0.0.0.0:9104`
PHOENIX_DEBUG_TOKEN
: Token to grant metrics access, empty default value
PHOENIX_DEBUG_PPROF
: Enable pprof debugging, defaults to `false`
PHOENIX_DEBUG_ZPAGES
: Enable zpages debugging, defaults to `false`
PHOENIX_HTTP_ADDR
: Address to bind http server, defaults to `0.0.0.0:9100`
PHOENIX_HTTP_ROOT
: Root path of http server, defaults to `/`
PHOENIX_ASSET_PATH
: Path to custom assets, empty default value
PHOENIX_WEB_CONFIG
: Path to phoenix config, empty default value
PHOENIX_WEB_CONFIG_SERVER
: Server URL, defaults to `http://localhost:9135`
PHOENIX_WEB_CONFIG_THEME
: Theme, defaults to `owncloud`
PHOENIX_WEB_CONFIG_VERSION
: Version, defaults to `0.1.0`
PHOENIX_APPS
: Use multiple times to provide multiple apps
PHOENIX_EXTERNAL_APPS
: Not supported yet, specify a config.json file via `PHOENIX_WEB_CONFIG`
PHOENIX_OIDC_METADATA_URL
: OpenID Connect metadata URL, defaults to `http://localhost:9130/.well-known/openid-configuration`
PHOENIX_OIDC_AUTHORITY
: OpenID Connect authority, defaults to `http://localhost:9130`
PHOENIX_OIDC_CLIENT_ID
: OpenID Connect client ID, defaults to `phoenix`
PHOENIX_OIDC_RESPONSE_TYPE
: OpenID Connect response type, defaults to `code`
PHOENIX_OIDC_SCOPE
: OpenID Connect scope, defaults to `openid profile email`
In case you want to render any additional properties in the config.json you can provide a custom config.json using eg. `PHOENIX_WEB_CONFIG=/path/to/config.json ocis-phoenix server`
#### Health
PHOENIX_DEBUG_ADDR
: Address to debug endpoint, defaults to `0.0.0.0:9104`
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below.
#### Global
--config-file
: Path to config file, empty default value
--log-level
: Set logging level, defaults to `info`
--log-color
: Enable colored logging, defaults to `true`
--log-pretty
: Enable pretty logging, defaults to `true`
#### Server
--tracing-enabled
: Enable sending traces, defaults to `false`
--tracing-type
: Tracing backend type, defaults to `jaeger`
--tracing-endpoint
: Endpoint for the agent, empty default value
--tracing-collector
: Endpoint for the collector, empty default value
--tracing-service
: Service name for tracing, defaults to `phoenix`
--debug-addr
: Address to bind debug server, defaults to `0.0.0.0:9104`
--debug-token
: Token to grant metrics access, empty default value
--debug-pprof
: Enable pprof debugging, defaults to `false`
--debug-zpages
: Enable zpages debugging, defaults to `false`
--http-addr
: Address to bind http server, defaults to `0.0.0.0:9100`
--http-root
: Root path of http server, defaults to `/`
--asset-path
: Path to custom assets, empty default value
--web-config
: Path to phoenix config, empty default value
--web-config-server
: Server URL, defaults to `http://localhost:9135`
--web-config-theme
: Theme, defaults to `owncloud`
--web-config-version
: Version, defaults to `0.1.0`
--web-config-app
: Provide multiple apps, defaults to `""`. Usage: `phoenix --web-config-app files --web-config-app pdf-viewer`
--oidc-metadata-url
: OpenID Connect metadata URL, defaults to `http://localhost:9130/.well-known/openid-configuration`
--oidc-authority
: OpenID Connect authority, defaults to `http://localhost:9130`
--oidc-client-id
: OpenID Connect client ID, defaults to `phoenix`
--oidc-response-type
: OpenID Connect response type, defaults to `code`
--oidc-scope
: OpenID Connect scope, defaults to `openid profile email`
In case you want to render any additional properties in the config.json you can provide a custom config.json using eg. `ocis-phoenix server --web-config=/path/to/config.json`
#### Health
--debug-addr
: Address to debug endpoint, defaults to `0.0.0.0:9104`
### Configuration file
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/ocis-phoenix/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/phoenix.yml`, `${HOME}/.ocis/phoenix.yml` or `$(pwd)/config/phoenix.yml`.
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis-phoenix --help`.
### Server
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{< highlight txt >}}
ocis-phoenix server --help
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
ocis-phoenix health --help
{{< / highlight >}}
## Metrics
This service provides some [Prometheus](https://prometheus.io/) metrics through the debug endpoint, you can optionally secure the metrics endpoint by some random token, which got to be configured through one of the flag `--debug-token` or the environment variable `PHOENIX_DEBUG_TOKEN` mentioned above. By default the metrics endpoint is bound to `http://0.0.0.0:9104/metrics`.
go_gc_duration_seconds
: A summary of the GC invocation durations
go_gc_duration_seconds_sum
: A summary of the GC invocation durations
go_gc_duration_seconds_count
: A summary of the GC invocation durations
go_goroutines
: Number of goroutines that currently exist
go_info
: Information about the Go environment
go_memstats_alloc_bytes
: Number of bytes allocated and still in use
go_memstats_alloc_bytes_total
: Total number of bytes allocated, even if freed
go_memstats_buck_hash_sys_bytes
: Number of bytes used by the profiling bucket hash table
go_memstats_frees_total
: Total number of frees
go_memstats_gc_cpu_fraction
: The fraction of this program's available CPU time used by the GC since the program started
go_memstats_gc_sys_bytes
: Number of bytes used for garbage collection system metadata
go_memstats_heap_alloc_bytes
: Number of heap bytes allocated and still in use
go_memstats_heap_idle_bytes
: Number of heap bytes waiting to be used
go_memstats_heap_inuse_bytes
: Number of heap bytes that are in use
go_memstats_heap_objects
: Number of allocated objects
go_memstats_heap_released_bytes
: Number of heap bytes released to OS
go_memstats_heap_sys_bytes
: Number of heap bytes obtained from system
go_memstats_last_gc_time_seconds
: Number of seconds since 1970 of last garbage collection
go_memstats_lookups_total
: Total number of pointer lookups
go_memstats_mallocs_total
: Total number of mallocs
go_memstats_mcache_inuse_bytes
: Number of bytes in use by mcache structures
go_memstats_mcache_sys_bytes
: Number of bytes used for mcache structures obtained from system
go_memstats_mspan_inuse_bytes
: Number of bytes in use by mspan structures
go_memstats_mspan_sys_bytes
: Number of bytes used for mspan structures obtained from system
go_memstats_next_gc_bytes
: Number of heap bytes when next garbage collection will take place
go_memstats_other_sys_bytes
: Number of bytes used for other system allocations
go_memstats_stack_inuse_bytes
: Number of bytes in use by the stack allocator
go_memstats_stack_sys_bytes
: Number of bytes obtained from system for stack allocator
go_memstats_sys_bytes
: Number of bytes obtained from system
go_threads
: Number of OS threads created
promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code

10
ocis-phoenix/docs/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2018-05-02T00:00:00+00:00
weight: 50
geekdocRepo: https://github.com/owncloud/ocis-phoenix
geekdocEditPath: edit/master/docs
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/ocis-phoenix/blob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

47
ocis-phoenix/docs/releasing.md
View File

@@ -1,47 +0,0 @@
---
title: "Releasing"
weight: 40
geekdocRepo: https://github.com/owncloud/ocis-phoenix
geekdocEditPath: edit/master/docs
geekdocFilePath: releasing.md
---
{{< toc >}}
## Releasing
The next generation Web Frontend is shipped as an ocis Extension. The `ocis-phoenix` extension is also embedded in the single binary and part of the `ocis server` command.
To update this package within all the deliveries, we need to update the package in the following chain from the bottom to the top.
### Package Hierarchy
- [ocis](https://githug.com/owncloud/ocis)
- [ocis-phoenix](https://github.com/owncloud/ocis-phoenix)
- [ocis-pkg](https://github.com/owncloud/ocis-pkg)
- [phoenix](https://github.com/owncloud/phoenix)
#### Prerequisites
Before updating the assets, make sure that [Phoenix](https://github.com/owncloud/phoenix) has been released first
and take note of its release tag name.
#### Updating ocis-phoenix
1. Create a branch `release-$version`. in <https://github.com/owncloud/ocis-phoenix>
2. Create a Folder in `changelog` for the release version and date `mkdir $major.$minor.$patchVersion_YYYY-MM-DD`.
3. Move all changelog items from the `changelog/unreleased/` folder to the `$major.$minor.$patchVersion_YYYY-MM-DD` folder.
4. Update the go module `ocis-pkg` to the latest version <https://blog.golang.org/using-go-modules> .
5. Update the phoenix asset by adjusting the value of `PHOENIX_ASSETS_VERSION` at the top of the Makefile and specify the tag name of the latest [Phoenix release](https://github.com/owncloud/phoenix/tags).
6. Run `make clean generate`.
7. Create a changelog item for the update in the `changelog/$major.$minor.$patchVersion_YYYY-MM-DD` folder.
8. Commit your changes.
9. After merging, wait for the CI to run on the merge commit.
10. Go to "Releases" in GH click "Draft a new Release".
11. Use `v$major.$minor.$patch` as a tag (the `v` prefix is important) and publish it.
12. The tag and the Release artifacts will be created automatically.
#### Next steps
Next steps is usually updating the ocis-phoenix version in the [ocis](https://githug.com/owncloud/ocis) repository.

4
ocis-pkg/Makefile
View File

@@ -68,3 +68,7 @@ test:
.PHONY: build
build:
$(GOBUILD) -v -tags '$(TAGS)' -ldflags '$(LDFLAGS)' ./...
.PHONY: config-docs-generate
config-docs-generate:
# go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md

24
ocis-reva/Makefile
View File

@@ -3,7 +3,6 @@ NAME := ocis-reva
IMPORT := github.com/owncloud/ocis/$(NAME)
BIN := bin
DIST := dist
HUGO := hugo
ifeq ($(OS), Windows_NT)
EXECUTABLE := $(NAME).exe
@@ -60,7 +59,7 @@ sync:
.PHONY: clean
clean:
go clean -i ./...
rm -rf $(BIN) $(DIST) $(HUGO)
rm -rf $(BIN) $(DIST)
.PHONY: fmt
fmt:
@@ -133,28 +132,9 @@ release-check:
.PHONY: release-finish
release-finish: release-copy release-check
.PHONY: docs-copy
docs-copy:
mkdir -p $(HUGO); \
mkdir -p $(HUGO)/content/extensions; \
cd $(HUGO); \
git init; \
git remote rm origin; \
git remote add origin https://github.com/owncloud/owncloud.github.io; \
git fetch; \
git checkout origin/source -f; \
rsync --delete -ax ../docs/ content/extensions/$(NAME)
.PHONY: config-docs-generate
config-docs-generate:
go run github.com/owncloud/flaex >| docs/configuration.md
.PHONY: docs-build
docs-build:
cd $(HUGO); hugo
.PHONY: docs
docs: config-docs-generate docs-copy docs-build
go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md
.PHONY: watch
watch:

45
ocis-reva/docs/_index.md
View File

@@ -1,45 +0,0 @@
---
title: "Reva"
date: 2018-05-02T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
This service provides an ocis extension that wraps [reva](https://github.com/cs3org/reva/) and adds an opinionated configuration to it.
It uses the port range 9140-9179 to preconfigure several services.
| port | service |
|------|---------|
| 9109 | health? |
| 9140 | frontend |
| 9141 | frontend debug |
| 9142 | gateway |
| 9143 | gateway debug |
| 9144 | users |
| 9145 | users debug |
| 9146 | authbasic |
| 9147 | authbasic debug |
| 9148 | authbearer |
| 9149 | authbearer debug |
| 9150 | sharing |
| 9151 | sharing debug |
| 9152 | storage root |
| 9153 | storage root debug |
| 9154 | storage home |
| 9155 | storage home debug |
| 9156 | storage home data |
| 9157 | storage home data debug |
| 9158 | storage eos |
| 9159 | storage eos debug |
| 9160 | storage eos data |
| 9161 | storage eos data debug |
| 9162 | storage oc |
| 9163 | storage oc debug |
| 9164 | storage oc data |
| 9165 | storage oc data debug |
| 9166-9177 | reserved for s3, wnd, custom + data providers |
| 9178 | storage public link |
| 9179 | storage public link data |

26
ocis-reva/docs/building.md
View File

@@ -1,26 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install).To build this project you have to install Go >= v1.13. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis-reva.git
cd ocis-reva
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
### Backend
{{< highlight txt >}}
make build
{{< / highlight >}}
The above command should produce the binary within the `bin/` folder now, give it a try with `./bin/ocis-reva -h` to see all available options.

656
ocis-reva/docs/configuration.md
View File

@@ -1,656 +0,0 @@
---
title: "Configuration"
date: "2020-09-21T13:14:49+0200"
weight: 20
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: configuration.md
---
{{< toc >}}
## Configuration
oCIS Single Binary is not responsible for configuring extensions. Instead, each extension could either be configured by environment variables, cli flags or config files.
Each extension has its dedicated documentation page (e.g. https://owncloud.github.io/extensions/ocis_proxy/configuration) which lists all possible configurations. Config files and environment variables are picked up if you use the `./bin/ocis server` command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
### Configuration using config files
Out of the box extensions will attempt to read configuration details from:
```console
/etc/ocis
$HOME/.ocis
./config
```
For this configuration to be picked up, have a look at your extension `root` command and look for which default config name it has assigned. *i.e: ocis-proxy reads `proxy.json | yaml | toml ...`*.
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/ocis.yml`, `${HOME}/.ocis/ocis.yml` or `$(pwd)/config/ocis.yml`.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below. Command line flags are only working when calling the subcommand directly.
## Root Command
Example service for Reva/oCIS
Usage: `ocis-reva [global options] command [command options] [arguments...]`
--config-file | $REVA_CONFIG_FILE
: Path to config file.
--log-level | $REVA_LOG_LEVEL
: Set logging level. Default: `info`.
--log-pretty | $REVA_LOG_PRETTY
: Enable pretty logging.
--log-color | $REVA_LOG_COLOR
: Enable colored logging.
## Sub Commands
### ocis-reva auth-basic
Start reva authprovider for basic auth
Usage: `ocis-reva auth-basic [command options] [arguments...]`
--debug-addr | $REVA_AUTH_BASIC_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9147`.
--auth-driver | $REVA_AUTH_DRIVER
: auth driver: 'demo', 'json' or 'ldap'. Default: `ldap`.
--auth-json | $REVA_AUTH_JSON
: Path to users.json file.
--network | $REVA_AUTH_BASIC_NETWORK
: Network to use for the reva auth-basic service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_AUTH_BASIC_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_AUTH_BASIC_ADDR
: Address to bind reva service. Default: `0.0.0.0:9146`.
--url | $REVA_AUTH_BASIC_URL
: URL to use for the reva service. Default: `localhost:9146`.
### ocis-reva frontend
Start reva frontend service
Usage: `ocis-reva frontend [command options] [arguments...]`
--debug-addr | $REVA_FRONTEND_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9141`.
--transfer-secret | $REVA_TRANSFER_SECRET
: Transfer secret for datagateway. Default: `replace-me-with-a-transfer-secret`.
--webdav-namespace | $WEBDAV_NAMESPACE
: Namespace prefix for the /webdav endpoint. Default: `/home/`.
--dav-files-namespace | $DAV_FILES_NAMESPACE
: Namespace prefix for the webdav /dav/files endpoint. Default: `/oc/`.
--network | $REVA_FRONTEND_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_FRONTEND_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
--addr | $REVA_FRONTEND_ADDR
: Address to bind reva service. Default: `0.0.0.0:9140`.
--url | $REVA_FRONTEND_URL
: URL to use for the reva service. Default: `https://localhost:9200`.
--datagateway-prefix | $REVA_FRONTEND_DATAGATEWAY_PREFIX
: datagateway prefix. Default: `data`.
--ocdav-prefix | $REVA_FRONTEND_OCDAV_PREFIX
: owncloud webdav endpoint prefix.
--ocs-prefix | $REVA_FRONTEND_OCS_PREFIX
: open collaboration services endpoint prefix. Default: `ocs`.
--gateway-url | $REVA_GATEWAY_URL
: URL to use for the reva gateway service. Default: `localhost:9142`.
--upload-disable-tus | $REVA_FRONTEND_UPLOAD_DISABLE_TUS
: Disables TUS upload mechanism. Default: `false`.
--upload-http-method-override | $REVA_FRONTEND_UPLOAD_HTTP_METHOD_OVERRIDE
: Specify an HTTP method (ex: POST) that clients should to use when uploading instead of PATCH.
### ocis-reva gateway
Start reva gateway
Usage: `ocis-reva gateway [command options] [arguments...]`
--debug-addr | $REVA_GATEWAY_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9143`.
--transfer-secret | $REVA_TRANSFER_SECRET
: Transfer secret for datagateway. Default: `replace-me-with-a-transfer-secret`.
--network | $REVA_GATEWAY_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_GATEWAY_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_GATEWAY_ADDR
: Address to bind reva service. Default: `0.0.0.0:9142`.
--url | $REVA_GATEWAY_URL
: URL to use for the reva service. Default: `localhost:9142`.
--commit-share-to-storage-grant | $REVA_GATEWAY_COMMIT_SHARE_TO_STORAGE_GRANT
: Commit shares to the share manager. Default: `true`.
--commit-share-to-storage-ref | $REVA_GATEWAY_COMMIT_SHARE_TO_STORAGE_REF
: Commit shares to the storage. Default: `true`.
--share-folder | $REVA_GATEWAY_SHARE_FOLDER
: mount shares in this folder of the home storage provider. Default: `Shares`.
--disable-home-creation-on-login | $REVA_GATEWAY_DISABLE_HOME_CREATION_ON_LOGIN
: Disable creation of home folder on login.
--storage-registry-driver | $REVA_STORAGE_REGISTRY_DRIVER
: driver of the storage registry. Default: `static`.
--storage-home-provider | $REVA_STORAGE_HOME_PROVIDER
: mount point of the storage provider for user homes in the global namespace. Default: `/home`.
--frontend-url | $REVA_FRONTEND_URL
: URL to use for the reva service. Default: `https://localhost:9200`.
--datagateway-url | $REVA_DATAGATEWAY_URL
: URL to use for the reva datagateway. Default: `https://localhost:9200/data`.
--users-url | $REVA_USERS_URL
: URL to use for the reva service. Default: `localhost:9144`.
--auth-basic-url | $REVA_AUTH_BASIC_URL
: URL to use for the reva service. Default: `localhost:9146`.
--auth-bearer-url | $REVA_AUTH_BEARER_URL
: URL to use for the reva service. Default: `localhost:9148`.
--sharing-url | $REVA_SHARING_URL
: URL to use for the reva service. Default: `localhost:9150`.
--storage-root-url | $REVA_STORAGE_ROOT_URL
: URL to use for the reva service. Default: `localhost:9152`.
--storage-root-mount-path | $REVA_STORAGE_ROOT_MOUNT_PATH
: mount path. Default: `/`.
--storage-root-mount-id | $REVA_STORAGE_ROOT_MOUNT_ID
: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009152`.
--storage-home-url | $REVA_STORAGE_HOME_URL
: URL to use for the reva service. Default: `localhost:9154`.
--storage-home-mount-path | $REVA_STORAGE_HOME_MOUNT_PATH
: mount path. Default: `/home`.
--storage-home-mount-id | $REVA_STORAGE_HOME_MOUNT_ID
: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009154`.
--storage-eos-url | $REVA_STORAGE_EOS_URL
: URL to use for the reva service. Default: `localhost:9158`.
--storage-eos-mount-path | $REVA_STORAGE_EOS_MOUNT_PATH
: mount path. Default: `/eos`.
--storage-eos-mount-id | $REVA_STORAGE_EOS_MOUNT_ID
: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009158`.
--storage-oc-url | $REVA_STORAGE_OC_URL
: URL to use for the reva service. Default: `localhost:9162`.
--storage-oc-mount-path | $REVA_STORAGE_OC_MOUNT_PATH
: mount path. Default: `/oc`.
--storage-oc-mount-id | $REVA_STORAGE_OC_MOUNT_ID
: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
--public-link-url | $REVA_STORAGE_PUBLIC_LINK_URL
: URL to use for the public links service. Default: `localhost:9178`.
--storage-public-link-mount-path | $REVA_STORAGE_PUBLIC_LINK_MOUNT_PATH
: mount path. Default: `/public/`.
### ocis-reva sharing
Start reva sharing service
Usage: `ocis-reva sharing [command options] [arguments...]`
--debug-addr | $REVA_SHARING_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9151`.
--network | $REVA_SHARING_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_SHARING_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_SHARING_ADDR
: Address to bind reva service. Default: `0.0.0.0:9150`.
--url | $REVA_SHARING_URL
: URL to use for the reva service. Default: `localhost:9150`.
--user-driver | $REVA_SHARING_USER_DRIVER
: driver to use for the UserShareProvider. Default: `json`.
--user-json-file | $REVA_SHARING_USER_JSON_FILE
: file used to persist shares for the UserShareProvider. Default: `/var/tmp/reva/shares.json`.
--public-driver | $REVA_SHARING_PUBLIC_DRIVER
: driver to use for the PublicShareProvider. Default: `json`.
### ocis-reva storage-home-data
Start reva storage-home-data service
Usage: `ocis-reva storage-home-data [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_HOME_DATA_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9157`.
--network | $REVA_STORAGE_HOME_DATA_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_HOME_DATA_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
--addr | $REVA_STORAGE_HOME_DATA_ADDR
: Address to bind reva service. Default: `0.0.0.0:9156`.
--url | $REVA_STORAGE_HOME_DATA_URL
: URL to use for the reva service. Default: `localhost:9156`.
--driver | $REVA_STORAGE_HOME_DATA_DRIVER
: storage driver for home data mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
--prefix | $REVA_STORAGE_HOME_DATA_PREFIX
: prefix for the http endpoint, without leading slash. Default: `data`.
--temp-folder | $REVA_STORAGE_HOME_DATA_TEMP_FOLDER
: temp folder. Default: `/var/tmp/`.
--enable-home | $REVA_STORAGE_HOME_ENABLE_HOME
: enable the creation of home directories. Default: `true`.
--gateway-url | $REVA_GATEWAY_URL
: URL to use for the reva gateway service. Default: `localhost:9142`.
--users-url | $REVA_USERS_URL
: URL to use for the reva service. Default: `localhost:9144`.
### ocis-reva health
Check health status
Usage: `ocis-reva health [command options] [arguments...]`
--debug-addr | $REVA_DEBUG_ADDR
: Address to debug endpoint. Default: `0.0.0.0:9109`.
### ocis-reva storage-home
Start reva storage-home service
Usage: `ocis-reva storage-home [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_HOME_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9155`.
--network | $REVA_STORAGE_HOME_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_HOME_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_STORAGE_HOME_ADDR
: Address to bind reva service. Default: `0.0.0.0:9154`.
--url | $REVA_STORAGE_HOME_URL
: URL to use for the reva service. Default: `localhost:9154`.
--driver | $REVA_STORAGE_HOME_DRIVER
: storage driver for home mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
--mount-path | $REVA_STORAGE_HOME_MOUNT_PATH
: mount path. Default: `/home`.
--mount-id | $REVA_STORAGE_HOME_MOUNT_ID
: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
--expose-data-server | $REVA_STORAGE_HOME_EXPOSE_DATA_SERVER
: exposes a dedicated data server. Default: `false`.
--data-server-url | $REVA_STORAGE_HOME_DATA_SERVER_URL
: data server url. Default: `http://localhost:9156/data`.
--enable-home | $REVA_STORAGE_HOME_ENABLE_HOME
: enable the creation of home directories. Default: `true`.
--users-url | $REVA_USERS_URL
: URL to use for the reva service. Default: `localhost:9144`.
### ocis-reva storage-eos-data
Start reva storage-eos-data service
Usage: `ocis-reva storage-eos-data [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_OC_DATA_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9161`.
--network | $REVA_STORAGE_EOS_DATA_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_EOS_DATA_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
--addr | $REVA_STORAGE_EOS_DATA_ADDR
: Address to bind reva service. Default: `0.0.0.0:9160`.
--url | $REVA_STORAGE_EOS_DATA_URL
: URL to use for the reva service. Default: `localhost:9160`.
--driver | $REVA_STORAGE_EOS_DATA_DRIVER
: storage driver for eos data mount: eg. local, eos, owncloud, ocis or s3. Default: `eos`.
--prefix | $REVA_STORAGE_EOS_DATA_PREFIX
: prefix for the http endpoint, without leading slash. Default: `data`.
--temp-folder | $REVA_STORAGE_EOS_DATA_TEMP_FOLDER
: temp folder. Default: `/var/tmp/`.
--gateway-url | $REVA_GATEWAY_URL
: URL to use for the reva gateway service. Default: `localhost:9142`.
--users-url | $REVA_USERS_URL
: URL to use for the reva service. Default: `localhost:9144`.
### ocis-reva reva-storage-public-link
Start reva storage-public-link service
Usage: `ocis-reva reva-storage-public-link [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_PUBLIC_LINK_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9179`.
--network | $REVA_STORAGE_PUBLIC_LINK_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_PUBLIC_LINK_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_STORAGE_PUBLIC_LINK_ADDR
: Address to bind reva service. Default: `0.0.0.0:9178`.
--url | $REVA_STORAGE_PUBLIC_LINK_URL
: Address to bind reva service. Default: `localhost:9178`.
--mount-path | $REVA_STORAGE_PUBLIC_LINK_MOUNT_PATH
: mount path. Default: `/public/`.
--gateway-url | $REVA_GATEWAY_URL
: URL to use for the reva gateway service. Default: `localhost:9142`.
### ocis-reva users
Start reva users service
Usage: `ocis-reva users [command options] [arguments...]`
--debug-addr | $REVA_SHARING_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9145`.
--network | $REVA_USERS_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_USERS_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_USERS_ADDR
: Address to bind reva service. Default: `0.0.0.0:9144`.
--url | $REVA_USERS_URL
: URL to use for the reva service. Default: `localhost:9144`.
--driver | $REVA_USERS_DRIVER
: user driver: 'demo', 'json', 'ldap', or 'rest'. Default: `ldap`.
--json-config | $REVA_USERS_JSON
: Path to users.json file.
--rest-client-id | $REVA_REST_CLIENT_ID
: User rest driver Client ID.
--rest-client-secret | $REVA_REST_CLIENT_SECRET
: User rest driver Client Secret.
--rest-redis-address | $REVA_REST_REDIS_ADDRESS
: Address for redis server. Default: `localhost:6379`.
--rest-redis-username | $REVA_REST_REDIS_USERNAME
: Username for redis server.
--rest-redis-password | $REVA_REST_REDIS_PASSWORD
: Password for redis server.
--rest-id-provider | $REVA_REST_ID_PROVIDER
: The OIDC Provider.
--rest-api-base-url | $REVA_REST_API_BASE_URL
: Base API Endpoint.
--rest-oidc-token-endpoint | $REVA_REST_OIDC_TOKEN_ENDPOINT
: Endpoint to generate token to access the API.
--rest-target-api | $REVA_REST_TARGET_API
: The target application.
### ocis-reva auth-bearer
Start reva authprovider for bearer auth
Usage: `ocis-reva auth-bearer [command options] [arguments...]`
--debug-addr | $REVA_AUTH_BEARER_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9149`.
--oidc-issuer | $REVA_OIDC_ISSUER
: OIDC issuer. Default: `https://localhost:9200`.
--oidc-insecure | $REVA_OIDC_INSECURE
: OIDC allow insecure communication. Default: `true`.
--oidc-id-claim | $REVA_OIDC_ID_CLAIM
: OIDC id claim. Default: `preferred_username`.
--oidc-uid-claim | $REVA_OIDC_UID_CLAIM
: OIDC uid claim.
--oidc-gid-claim | $REVA_OIDC_GID_CLAIM
: OIDC gid claim.
--network | $REVA_AUTH_BEARER_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_AUTH_BEARER_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_AUTH_BEARER_ADDR
: Address to bind reva service. Default: `0.0.0.0:9148`.
--url | $REVA_AUTH_BEARER_URL
: URL to use for the reva service. Default: `localhost:9148`.
### ocis-reva storage-eos
Start reva storage-eos service
Usage: `ocis-reva storage-eos [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_EOS_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9159`.
--network | $REVA_STORAGE_EOS_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_EOS_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_STORAGE_EOS_ADDR
: Address to bind reva service. Default: `0.0.0.0:9158`.
--url | $REVA_STORAGE_EOS_URL
: URL to use for the reva service. Default: `localhost:9158`.
--driver | $REVA_STORAGE_EOS_DRIVER
: storage driver for eos mount: eg. local, eos, owncloud, ocis or s3. Default: `eos`.
--mount-path | $REVA_STORAGE_EOS_MOUNT_PATH
: mount path. Default: `/eos`.
--mount-id | $REVA_STORAGE_EOS_MOUNT_ID
: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009158`.
--expose-data-server | $REVA_STORAGE_EOS_EXPOSE_DATA_SERVER
: exposes a dedicated data server. Default: `false`.
--data-server-url | $REVA_STORAGE_EOS_DATA_SERVER_URL
: data server url. Default: `http://localhost:9160/data`.
### ocis-reva storage-oc
Start reva storage-oc service
Usage: `ocis-reva storage-oc [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_OC_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9163`.
--network | $REVA_STORAGE_OC_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_OC_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_STORAGE_OC_ADDR
: Address to bind reva service. Default: `0.0.0.0:9162`.
--url | $REVA_STORAGE_OC_URL
: URL to use for the reva service. Default: `localhost:9162`.
--driver | $REVA_STORAGE_OC_DRIVER
: storage driver for oc mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
--mount-path | $REVA_STORAGE_OC_MOUNT_PATH
: mount path. Default: `/oc`.
--mount-id | $REVA_STORAGE_OC_MOUNT_ID
: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
--expose-data-server | $REVA_STORAGE_OC_EXPOSE_DATA_SERVER
: exposes a dedicated data server. Default: `false`.
--data-server-url | $REVA_STORAGE_OC_DATA_SERVER_URL
: data server url. Default: `http://localhost:9164/data`.
--users-url | $REVA_USERS_URL
: URL to use for the reva service. Default: `localhost:9144`.
### ocis-reva storage-oc-data
Start reva storage-oc-data service
Usage: `ocis-reva storage-oc-data [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_OC_DATA_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9165`.
--network | $REVA_STORAGE_OC_DATA_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_OC_DATA_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
--addr | $REVA_STORAGE_OC_DATA_ADDR
: Address to bind reva service. Default: `0.0.0.0:9164`.
--url | $REVA_STORAGE_OC_DATA_URL
: URL to use for the reva service. Default: `localhost:9164`.
--driver | $REVA_STORAGE_OC_DATA_DRIVER
: storage driver for oc data mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
--prefix | $REVA_STORAGE_OC_DATA_PREFIX
: prefix for the http endpoint, without leading slash. Default: `data`.
--temp-folder | $REVA_STORAGE_OC_DATA_TEMP_FOLDER
: temp folder. Default: `/var/tmp/`.
--gateway-url | $REVA_GATEWAY_URL
: URL to use for the reva gateway service. Default: `localhost:9142`.
--users-url | $REVA_USERS_URL
: URL to use for the reva service. Default: `localhost:9144`.
### ocis-reva storage-root
Start reva storage-root service
Usage: `ocis-reva storage-root [command options] [arguments...]`
--debug-addr | $REVA_STORAGE_ROOT_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9153`.
--network | $REVA_STORAGE_ROOT_NETWORK
: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
--protocol | $REVA_STORAGE_ROOT_PROTOCOL
: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
--addr | $REVA_STORAGE_ROOT_ADDR
: Address to bind reva service. Default: `0.0.0.0:9152`.
--url | $REVA_STORAGE_ROOT_URL
: URL to use for the reva service. Default: `localhost:9152`.
--driver | $REVA_STORAGE_ROOT_DRIVER
: storage driver for root mount: eg. local, eos, owncloud, ocis or s3. Default: `local`.
--mount-path | $REVA_STORAGE_ROOT_MOUNT_PATH
: mount path. Default: `/`.
--mount-id | $REVA_STORAGE_ROOT_MOUNT_ID
: mount id. Default: `123e4567-e89b-12d3-a456-426655440001`.
--expose-data-server | $REVA_STORAGE_ROOT_EXPOSE_DATA_SERVER
: exposes a dedicated data server.
--data-server-url | $REVA_STORAGE_ROOT_DATA_SERVER_URL
: data server url.

56
ocis-reva/docs/getting-started.md
View File

@@ -1,56 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 15
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
Docker images for ocis-reva are hosted on https://hub.docker.com/r/owncloud/ocis-reva.
The `latest` tag always reflects the current master branch.
```console
docker pull owncloud/ocis-reva
```
#### Dependencies
- Running ocis-reva currently needs a working Redis caching server
- The default storage location in the container is `/var/tmp/reva/data`. You may want to create a volume to persist the files in the primary storage
### Binaries
The pre-built binaries for different platforms are downloadable at https://download.owncloud.com/ocis/ocis-reva/ . Specific releases are organized in separate folders. They are in sync which every release tag on GitHub. The binaries from the current master branch can be found in https://download.owncloud.com/ocis/ocis-reva/testing/
```console
curl https://download.owncloud.com/ocis/ocis/1.0.0-beta1/ocis-reva-1.0.0-beta1-darwin-amd64 --output ocis-reva
chmod +x ocis
./ocis-reva sharing
```
#### Dependencies
- Running ocis currently needs a working Redis caching server
- The default promary storage location is `/var/tmp/reva/data`. You can change that value by configuration.
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis-reva --help`.
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
ocis-reva health --help
{{< / highlight >}}

10
ocis-reva/docs/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2018-05-02T00:00:00+00:00
weight: 40
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/ocis-revablob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

47
ocis-reva/docs/releasing.md
View File

@@ -1,47 +0,0 @@
---
title: "Releasing"
date: 2020-05-22T00:00:00+00:00
weight: 60
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: releasing.md
---
{{< toc >}}
To release a new version of ocis-reva, you have to follow a few simple steps.
## Preparation
1. Before releasing, make sure that reva has been [updated to the desired version]({{< ref "updating.md" >}})
2. Create a new branch e.g. `release-x.x.x` where `x.x.x` is the version you want to release.
3. Checkout the preparation branch.
4. Create a new changelog folder and move the unreleased snippets there.
{{< highlight txt >}}
mkdir changelog/x.x.x_yyyy-MM-dd/ # yyyy-MM-dd is the current date
mv changelog/unreleased/* changelog/x.x.x_yyyy-MM-dd/
{{< / highlight >}}
5. Commit and push the changes
{{< highlight txt >}}
git add --all
git commit -m "prepare release x.x.x"
git push origin release-x.x.x
{{< / highlight >}}
6. Create a pull request to the master branch.
## Release
1. After the preparation branch has been merged update your local master.
2. [Wait for CI](https://cloud.drone.io/owncloud/ocis-reva) to generate a commit for the changelog update
3. Check out master (or make sure to check out the generated changelog commit in case of subsequent merges)
{{< highlight txt >}}
git checkout master
git pull origin master
{{< / highlight >}}
4. Create a new tag (preferably signed) and replace the version number accordingly.
{{< highlight txt >}}
git tag -s vx.x.x -m "release vx.x.x"
git push origin vx.x.x
{{< / highlight >}}
5. Wait for CI and check that the GitHub release was published.
Congratulations, you just released ocis-reva!

194
ocis-reva/docs/storages.md
View File

@@ -1,194 +0,0 @@
---
title: "Storages"
date: 2020-04-27T18:46:00+01:00
weight: 37
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: storages.md
---
## Storage commands
`ocis-reva` has multiple storage provider commands to preconfigure different default configurations for the reva *storage provider* service. While you could rerun `ocis-reva storage-oc` multiple times with different flags to get multiple instances we are giving the different commands the necessary default configuration to allow the `ocis` binary to simply start them and not deal with configuration.
## Storage providers
To manage the file tree ocis uses reva *storage providers* that are accessing the underlying storage using a *storage driver*. The driver can be used to change the implementation of a storage aspect to better reflect the actual underlying storage capabilities. As an example a move operation on a POSIX filesystem ([theoretically](https://danluu.com/deconstruct-files/)) is an atomic operation. When trying to implement a file tree on top of S3 there is no native move operation that can be used. A naive implementation might fall back on a COPY and DELETE. Some S3 implementations provide a COPY operation that uses an existing key as the source, so the file at least does not need to be reuploaded. In the worst case scenario, which is renaming a folder with hundreds of thousands of objects, a reupload for every file has to be made. Instead of hiding this complexity a better choice might be to disable renaming of files or at least folders on S3. There are however implementations of filesystems on top of S3 that store the tree metadata in dedicated objects or use a completely different persistence mechanism like a distributed key value store to implement the file tree aspect of a storage.
{{< hint info >}}
While the *storage provider* is responsible for managing the tree, file up and download is delegated to a dedicated *data provider*. See below.
{{< /hint >}}
## Storage aspects
A lot of different storage technologies exist, ranging from general purpose file systems with POSIX semantics to software defined storage with multiple APIs. Choosing any of them is making a tradeoff decision. Or, if a storage technology is already in place it automatically predetermines the capabilities that can be made available. *Not all storage systems are created equal.*
Unfortunately, no POSIX filesystem natively supports all storage aspects that ownCloud 10 requires:
### A hierarchical file tree
An important aspect of a filesystem is organizing files and directories in a file hierarchy, or tree. It allows you to create, move and delete nodes. Beside the name a node also has well known metadata like size and mtime that are persisted in the tree as well.
{{< hint info >}}
**Folders are not directories**
There is a difference between *folder* and *directory*: a *directory* is a file system concept. A *folder* is a metaphor for the concept of a physical file folder. There are also *virtual folders* or *smart folders* like the recent files folder which are no file system *directories*. So, every *directory* and every *virtual folder* is a *folder*, but not every *folder* is a *directory*. See [the folder metaphor in wikipedia](https://en.wikipedia.org/wiki/Directory_(computing)#Folder_metaphor). Also see the activity history below.
{{< /hint >}}
#### Id based lookup
While traditionally nodes in the tree are reached by traversing the path the tree persistence should be prepared to look up a node by an id. Think of an inode in a POSIX filesystem. If this operation needs to be cached for performance reasons keep in mind that cache invalidation is hard and crawling all files to update the inode to path mapping takes O(n), not O(1).
#### ETag propagation
For the state based sync a client can discover changes by recursively descending the tree and comparing the ETag for every node. If the storage technology supports propagating ETag changes up the tree, only the root node of a tree needs to be checked to determine if a discovery needs to be started and which nodes need to be traversed. This allows using the storage technology itself to persist all metadata that is necessary for sync, without additional services or caches.
#### Subtree size accounting
The tree can keep track of how many bytes are stored in a folder. Similar to ETag propagation a change in file size is propagated up the hierarchy.
{{< hint info >}}
**ETag and Size propagation**
When propagating the ETag (mtime) and size changes up the tree the question is where to stop. If all changes need to be propagated to the root of a storage then the root or busy folders will become a hotspot. There are two things to keep in mind: 1. propagation only happens up to the root of a single space (a user private drive or a single group drive), 2. no cross storage propagation. The latter was used in oc10 to let clients detect when a file in a received shared folder changed. This functionality is moving to the storage registry which caches the ETag for every root so clients can discover if and which storage changed.
{{< /hint >}}
#### Rename
Depending on the underlying storage technology some operations may either be slow, up to a point where it makes more sense to disable them entirely. One example is a folder rename: on S3 a *simple* folder rename translates to a copy and delete operation for every child of the renamed folder. There is an exception though: this restriction only applies if the S3 storage is treated like a filesystem, where the keys are the path and the value is the file content. There are smarter ways to implement file systems on top of S3, but again: there is always a tradeoff.
{{< hint info >}}
**S3 has no rename**
Technically, [S3 has no rename operation at all](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-s3-objects.html#copy-object). By design, the location of the value is determined by the key, so it always has to do a copy and delete. Another example is the [redis RENAME operation](https://redis.io/commands/rename): while being specified as O(1) it *executes an implicit DEL operation, so if the deleted key contains a very big value it may cause high latency...*
{{< /hint >}}
#### Arbitrary metadata persistence
In addition to well known metadata like name size and mtime, users might be able to add arbitrary metadata like tags, comments or [dublin core](https://en.wikipedia.org/wiki/Dublin_Core). In POSIX filesystems this maps to extended attributes.
### Grant persistence
The CS3 API uses grants to describe access permissions. Storage systems have a wide range of permissions granularity and not all grants may be supported by every storage driver. POSIX ACLs for example have no expiry. If the storage system does not support certain grant properties, e.g. expiry, then the storage driver may choose to implement them in a different way. Expiries could be persisted in a different way and checked periodically to remove the grants. Again: every decision is a tradeoff.
### Trash persistence
After deleting a node the storage allows listing the deleted nodes and has an undo mechanism for them.
### Versions persistence
A user can restore a previous version of a file.
{{< hint info >}}
**Snapshots are not versions**
Modern POSIX filesystems support snapshotting of volumes. This is different from keeping track of versions to a file or folder, but might be another implementation strategy for a storage driver to allow users to restore content.
{{< /hint >}}
### Activity History
The storage keeps an activity history, tracking the different actions that have been performed. This does not only include file changes but also metadata changes like renames and permission changes.
## Storage drivers
Reva currently has four storage driver implementations that can be used for *storage providers* an well as *data providers*.
### Local Storage Driver
The *minimal* storage driver for a POSIX based filesystem. It literally supports none of the storage aspect other than basic file tree management. Sharing can - to a degree - be implemented using POSIX ACLs.
- tree provided by a POSIX filesystem
- inefficient path by id lookup, currently uses the file path as id, so ids are not stable
- can store a uuid in extended attributes and use a cache to look them up, similar to the ownCloud driver
- no native ETag propagation, five options are available:
- built in propagation (changes bypassing ocis are not picked up until a rescan)
- built in inotify (requires 48 bytes of RAM per file, needs to keep track of every file and folder)
- external inotify (same RAM requirement, but could be triggered by external tools, e.g. a workflow engine)
- kernel audit log (use the linux kernel audit to capture file events on the storage and offload them to a queue)
- fuse filesystem overlay
- no subtree accounting, same options as for ETag propagation
- efficient rename
- arbitrary metadata using extended attributes
- grant persistence
- using POSIX ACLs
- requires an LDAP server to make guest accounts available in the OS
- OCIS has glauth which contains all users
- an existing LDAP could be used if guests ar provisioned in another way
- using extended attributes to implement expiry or sharing that does not require OS level integration
- fuse filesystem overlay
- no native trash
- could use the [The FreeDesktop.org Trash specification](https://specifications.freedesktop.org/trash-spec/trashspec-latest.html)
- fuse filesystem overlay
- no native versions, multiple options possible
- git for folders
- rcs for single files
- rsnapshot for hourly / daily / weekly / monthly backups ... but this is not versioning as known from oc10
- design new freedesktop spec, basically what is done in oc10 without the limitations or borrow ideas from the freedesktop trash spec
- fuse filesystem overlay
To provide the other storage aspects we plan to implement a FUSE overlay filesystem which will add the different aspects on top of local filesystems like ext4, btrfs or xfs. It should work on NFSv45 as well, although NFSv4 supports RichACLs and we will explore how to leverage them to implement sharing at a future date. The idea is to use the storages native capabilities to deliver the best user experience. But again: that means making the right tradeoffs.
### OwnCloud Storage Driver
This is the current default storage driver. While it implements the file tree (using redis, including id based lookup), ETag propagation, trash, versions and sharing (including expiry) using the data directory layout of ownCloud 10 it has [known limitations](https://github.com/owncloud/core/issues/28095) that cannot be fixed without changing the actual layout on disk.
To setup it up properly in a distributed fashion, the storage-home and the storage-oc need to share the same underlying FS. Their "data" counterparts also need access to the same shared FS.
For a simple docker-compose setup, you can create a volume which will be used by the "ocis-reva-storage-home", "ocis-reva-storage-home-data", "ocis-reva-storage-oc" and "ocis-reva-storage-oc-data" containers. Using the `owncloud/ocis-reva` docker image, the volume would need to be hooked in the `/var/tmp/reva` folder insde the containers.
- tree provided by a POSIX filesystem
- file layout is mapped to the old ownCloud 10 layout
- the root of tree for a user on disk is prefixed with `/path/to/data/<username>/files/`
- efficient path by id lookup
- all files and folders get assigned a uuid in the extended attributes
- when starting the storage provider it will walk all files to populate a redis kv store for uuid to path lookup
- slow to boot trees with lots of nodes
- build in ETag propagation
- ETags are calculated based on mtime
- mtime is propagated by the storage driver
- changes bypassing ocis are not picked up until a restart of the storage provider
- no subtree accounting, same options as for local storage
- efficient rename
- TODO [update the kv store for path lookup](https://github.com/cs3org/reva/issues/985), this is an O(n) operation
- arbitrary metadata using extended attributes
- grant persistence
- using custom ACLs that are stored as extended attributes
- a grant corresponds to one extended attribute of 40-100 bytes, effectively limiting the number of shares to ~100-40
- extended attributes have varying limitations, based on the underlying filesystem
- the linux kernel imposes a limit of 255bytes per name and 64KiB per value
- ext2/3/4: total bytes for all attributes of a file is limited to 4KiB (a filesystem block)
- xfs: limit of 64KiB per value
- btrfs: total bytes used for the name, value, and implementation overhead bytes 16KiB (the default filesystem nodesize value)
- does not require OS level integration
- built in trash
- trashed files are moved to `/path/to/data/<username>/files_trashbin/`
- trashed files are appended a timestamp `.d<unixtime>`, which [breaks trashing of files that reach the filesystems specific name limit](https://github.com/owncloud/core/issues/28095)
- built in versions
- file versions are stored in `/path/to/data/<username>/files_versions/`
- file versions are appended a timestamp `.d<unixtime>`, which [breaks versioning of files that reach the filesystems specific name limit](https://github.com/owncloud/core/issues/28095)
### EOS Storage Driver
The CERN eos storage has evolved with ownCloud and natively supports id based lookup, ETag propagation, subtree size accounting, sharing, trash and versions. To use it you need to change the default configuration of the `ocis-reva storage-home` command (or have a look at the Makefile ̀ eos-start` target):
```
export REVA_STORAGE_HOME_DRIVER=eos
export REVA_STORAGE_EOS_NAMESPACE=/eos
export REVA_STORAGE_EOS_MASTER_URL="root://eos-mgm1.eoscluster.cern.ch:1094"
export REVA_STORAGE_EOS_ENABLE_HOME=true
export REVA_STORAGE_EOS_LAYOUT="dockertest/{{.Username}}"
```
Running it locally also requires the `eos` and `xrootd` binaries. Running it using `make eos-start` will use CentOS based containers that already have the necessary packages installed.
{{< hint info >}}
Pull requests to add explicit `reva storage-(s3|custom|...)` commands with working defaults are welcome.
{{< /hint >}}
### S3 Storage Driver
A naive driver that treats the keys in an S3 capable storage as `/` delimited path names. While it does not support MOVE or ETag propagation it can be used to read and write files. Better integration with native capabilities like versioning is possible but depends on the Use Case. Several storage solutions that provide an S3 interface also support some form of notifications that can be used to implement ETag propagation.
## Data Providers
Clients using the CS3 API use an [InitiateFileDownload](https://cs3org.github.io/cs3apis/#cs3.storage.provider.v1beta1.InitiateFileDownloadRequest) and ]InitiateUpload](https://cs3org.github.io/cs3apis/#cs3.storage.provider.v1beta1.InitiateFileUploadRequest) request at the [reva gateway](https://cs3org.github.io/cs3apis/#cs3.gateway.v1beta1.GatewayAPI) to obtain a URL endpoint that can be used to either GET the file content or upload content using the resumable [tus.io](https://tus.io) protocol.
The *data provider* uses the same *storage driver* as the *storage provider* but can be scaled independently.
The dataprovider allows uploading the file to a quarantine area where further data analysis may happen before making the file accessible again. One use case for this is anti virus scanning for files coming from untrusted sources.
## Future work
### FUSE overlay filesystem
We are planning to further separate the concerns and use a local storage provider with a FUSE filesystem overlaying the actual POSIX storage that can be used to capture deletes and writes that might happen outside of ocis/reva.
It would allow us to extend the local storage driver with missing storage aspects while keeping a tree like filesystem that end users are used to see when sshing into the machine.
### Upload to Quarantine area
Antivirus scanning of random files uploaded from untrusted sources and executing metadata extraction or thumbnail generation should happen in a sandboxed system to prevent malicious users from gaining any information about the system. By spawning a new container with access to only the uploaded data we can further limit the attack surface.

148
ocis-reva/docs/testing.md
View File

@@ -1,148 +0,0 @@
---
title: "Testing"
date: 2018-05-02T00:00:00+00:00
weight: 37
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: testing.md
---
## API Acceptance tests
We are using the ownCloud 10 API acceptance testsuite against ocis. To set this up you need the owncloud 10 core repo, a ldap server that the acceptance tests can use to manage users, a redis server for file-versions and the ocis-reva code.
### Getting the tests
All you need to do to get the acceptance tests is check out the core repo:
```
git clone https://github.com/owncloud/core.git
```
### Run a ldap server in a docker container
The ownCloud 10 acceptance tests will need write permission. You can start a suitable ldap server in a docker container with:
```
docker run --hostname ldap.my-company.com \
-e LDAP_TLS_VERIFY_CLIENT=never \
-e LDAP_DOMAIN=owncloud.com \
-e LDAP_ORGANISATION=ownCloud \
-e LDAP_ADMIN_PASSWORD=admin \
--name docker-slapd \
-p 127.0.0.1:389:389 \
-p 636:636 -d osixia/openldap
```
### Run a redis server in a docker container
File versions need a redis server. Start one with docker by using:
`docker run -e REDIS_DATABASES=1 -p 6379:6379 -d webhippie/redis:latest`
### Run ocis-reva with that ldap server
`ocis-reva` provides multiple subcommands. To configure them all via env vars you can export these environment variables.
```
export REVA_USERS_DRIVER=ldap
export REVA_LDAP_HOSTNAME=localhost
export REVA_LDAP_PORT=636
export REVA_LDAP_BASE_DN='dc=owncloud,dc=com'
export REVA_LDAP_USERFILTER='(&(objectclass=posixAccount)(cn=%s))'
export REVA_LDAP_GROUPFILTER='(&(objectclass=posixGroup)(cn=%s))'
export REVA_LDAP_BIND_DN='cn=admin,dc=owncloud,dc=com'
export REVA_LDAP_BIND_PASSWORD=admin
export REVA_LDAP_SCHEMA_UID=uid
export REVA_LDAP_SCHEMA_MAIL=mail
export REVA_LDAP_SCHEMA_DISPLAYNAME=displayName
export REVA_LDAP_SCHEMA_CN=cn
export REVA_FRONTEND_URL=http://localhost:9140 # needed because the proxy is not started
export REVA_DATAGATEWAY_URL=http://localhost:9140/data # needed because the proxy is not started
```
Then you need to start the ocis-reva services
```
bin/ocis-reva frontend & \
bin/ocis-reva gateway & \
bin/ocis-reva auth-basic & \
bin/ocis-reva auth-bearer & \
bin/ocis-reva sharing & \
bin/ocis-reva storage-home & \
bin/ocis-reva storage-home-data & \
bin/ocis-reva storage-oc & \
bin/ocis-reva storage-oc-data & \
bin/ocis-reva users &
```
### Run the API acceptance tests
In the ownCloud 10 core repo run
```
make test-acceptance-api \
TEST_SERVER_URL=http://localhost:9140 \
TEST_EXTERNAL_USER_BACKENDS=true \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
BEHAT_FILTER_TAGS='~@notToImplementOnOCIS&&~@toImplementOnOCIS&&~@preview-extension-required' \
SKELETON_DIR=apps/testing/data/apiSkeleton
```
Make sure to adjust the settings `TEST_SERVER_URL`,`OCIS_REVA_DATA_ROOT` and `SKELETON_DIR` according to your environment.
This will run all tests that are relevant to OCIS.
To run a single test add `BEHAT_FEATURE=<feature file>` and specify the path to the feature file and an optional line number. For example: `BEHAT_FEATURE='tests/acceptance/features/apiWebdavUpload1/uploadFile.feature:12'`
### use existing tests for BDD
As a lot of scenarios are written for oC10, we can use those tests for Behaviour driven development in ocis.
Every scenario that does not work in OCIS with OC storage, is listed in `tests/acceptance/expected-failures-on-OC-storage.txt` with a link to the related issue.
Similarly, scenarios that do not work in OCIS with EOS storage are listed in `tests/acceptance/expected-failures-on-EOS-storage.txt`.
Scenarios from the oC10 API acceptance tests are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the
expected failures. If there are any differences then the CI pipeline fails.
Additionally, some issues have scenarios that demonstrate the current buggy behaviour in ocis(reva).
Those scenarios are in this ocis-reva repository in `tests/acceptance/features/apiOcisSpecific`.
Have a look into the [documentation](https://doc.owncloud.com/server/developer_manual/testing/acceptance-tests.html#writing-scenarios-for-bugs) to understand why we are writing those tests.
Also, ocis behaves partly differently with EOS-Storage and OC-Storage. There are scenarios that do not work in OCIS when run on EOS-storage, but works when on OC-Storage, and vice-versa. For those kind of scenarios, ` @skipOnOcis-EOS-Storage` and `@skipOnOcis-OC-Storage` tags are used. For instance, for a scenario that fails on EOS-Storage but passes on OC-Storage, we use `@skipOnOcis-EOS-Storage` tag to let it run on OC-Storage, where it works as expected, instead of skipping the test completely.
If you want to work on a specific issue
1. adjust the core commit id to the latest commit in core so that CI will run the latest test code and scenarios from core.
For that change `coreCommit` in the `config` section:
config = {
'apiTests': {
'coreBranch': 'master',
'coreCommit': 'a06b1bd5ba8e5244bfaf7fa04f441961e6fb0daa',
'numberOfParts': 2
}
}
2. locally run each of the tests marked with that issue in the expected failures file:
E.g.:
```
make test-acceptance-api \
TEST_SERVER_URL=http://localhost:9140 \
TEST_EXTERNAL_USER_BACKENDS=true \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
BEHAT_FEATURE='tests/acceptance/features/apiComments/comments.feature:123'
```
3. the tests will fail, try to understand how and why they are failing
4. fix the code
5. go back to 2. and repeat till the tests are passing.
6. remove those tests from the expected failures file.
7. run each of the local tests that were demonstrating the **buggy** behavior. They should fail.
8. delete each of the local tests that were demonstrating the **buggy** behavior.
9. make a PR that has the fixed code, relevant lines removed from the expected failures file and bug demonstration tests deleted.
If the changes also affect the `ocis` repository make sure the changes get ported over there.
That will need the fixed code in `ocis-reva` to be applied to `ocis` along with the test-related changes.
### Notes
- in a normal case the test-code cleans up users after the test-run, but if a test-run is interrupted (e.g. by CTRL+C) users might have been left on the LDAP server. In that case rerunning the tests requires wiping the users in the ldap server, otherwise the tests will fail when trying to populate the users. This can be done by simply running `docker stop docker-slapd && docker rm docker-slapd` and [restarting the LDAP server container](#run-a-ldap-server-in-a-docker-container)
- the tests usually create users in the OU `TestUsers` with usernames specified in the feature file. If not defined in the feature file, most users have the password `123456`, defined by `regularUserPassword` in `behat.yml`, but other passwords are also used, see [`\FeatureContext::getPasswordForUser()`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L386) for mapping and [`\FeatureContext::__construct`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L1668) for the password definitions.

19
ocis-reva/docs/updating.md
View File

@@ -1,19 +0,0 @@
---
title: "Updating reva"
date: 2020-05-22T00:00:00+00:00
weight: 50
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: updating.md
---
{{< toc >}}
## Updating reva
1. Run `go get github.com/cs3org/reva@master`
2. Create a changelog entry containing changes that were done in [reva](https://github.com/cs3org/reva/commits/master)
3. Create a Pull Request to ocis-reva master with those changes
4. If test issues appear, you might need to [adjust the tests]({{< ref "testing.md" >}})
5. After the PR is merged, consider doing a [release of ocis-reva]({{< ref "releasing.md" >}})

52
ocis-reva/docs/users.md
View File

@@ -1,52 +0,0 @@
---
title: "Users"
date: 2020-01-16T00:00:00+00:00
weight: 35
geekdocRepo: https://github.com/owncloud/ocis-reva
geekdocEditPath: edit/master/docs
geekdocFilePath: users.md
---
### Demo driver
This is a simple user driver for testing. It contains three users:
```
einstein:relativity
marie:radioactivty
richard:superfluidity
```
In order to use the `demo` driver you need to export the relevant environment variable:
```
export REVA_USERS_DRIVER=demo
```
### JSON driver
In order to switch from the `ldap` driver to JSON based users you need to export the relevant environment variables:
```
export REVA_USERS_DRIVER=json
export REVA_USERS_JSON=/path/to/users.json
```
For the format of the users.json have a look at the [reva examples](https://github.com/cs3org/reva/blob/master/examples/separate/users.demo.json)
### LDAP driver
This is the default user driver.
If the below defaults don't match your environment change them accordingly:
```
export REVA_LDAP_HOSTNAME=localhost
export REVA_LDAP_PORT=9126
export REVA_LDAP_BASE_DN='dc=example,dc=org'
export REVA_LDAP_USERFILTER='(&(objectclass=posixAccount)(cn=%s))'
export REVA_LDAP_GROUPFILTER='(&(objectclass=posixGroup)(cn=%s))'
export REVA_LDAP_BIND_DN='cn=reva,ou=sysusers,dc=example,dc=org'
export REVA_LDAP_BIND_PASSWORD=reva
export REVA_LDAP_SCHEMA_UID=uid
export REVA_LDAP_SCHEMA_MAIL=mail
export REVA_LDAP_SCHEMA_DISPLAYNAME=sn
export REVA_LDAP_SCHEMA_CN=cn
```
Then restart the `bin/ocis-reva users` and `bin/ocis-reva auth-basic` services for the changes to take effect.

30
ocis/docs/_index.md
View File

@@ -1,30 +0,0 @@
---
title: "Infinite Scale"
date: 2020-02-27T20:35:00+01:00
weight: -10
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
This tool provides a single entrypoint for the whole ownCloud Infinite Scale stack.
{{< mermaid class="text-center">}}
graph TD
ocis-proxy -->
ocis-konnectd & ocis-phoenix & ocis-thumbnails & ocis-ocs & ocis-webdav
ocis-phoenix --> ocis-reva-fronted
ocis-reva-fronted --> ocis-reva-gateway
ocis-konnectd --> ocis-glauth
ocis-reva-gateway --> ocis-reva-users
ocis-reva-gateway --> ocis-reva-authbasic
ocis-reva-gateway --> ocis-reva-auth-bearer
ocis-reva-gateway --> ocis-reva-sharing
ocis-reva-gateway --> ocis-reva-storage-home-*
ocis-reva-storage-home-* --> ocis-reva-storage-home-*-data
ocis-reva-sharing --> redis
{{< /mermaid >}}

104
ocis/docs/basic-remote-setup.md
View File

@@ -1,104 +0,0 @@
---
title: "Basic Remote Setup"
date: 2020-02-27T20:35:00+01:00
weight: 16
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: basic-remote-setup.md
---
{{< toc >}}
Out of the box the ocis single binary and the `owncloud/ocis` docker image are configured to run on localhost for quick testing and development.
If you need to access ocis on a VM or a remote machine e.g. when testing a mobile client you need to configure ocis to run on a different host.
## Use the binary
If you start the ocis fullstack for the first time with `./bin/ocis server` it will generate a file `identifier-registration.yml` in the config folder relative to its location. This file is used to configure the clients for the built-in Identity Provider.
{{< hint warning >}}
**Outdated version**\
The `identifier-registration.yml` file will only be generated if there is no such file in place. You could miss updates on this file. Run `make clean` to delete the file and keep the development environment tidy otherwise as well.
{{< /hint >}}
### Add your hostname to the idp config
Let us assume `your-host` is your remote domain name or IP adress. Add your host to the `identifier-registration.yml` like this:
```yaml {linenos=table,hl_lines=["15-17",21]}
# OpenID Connect client registry.
clients:
- id: phoenix
name: ownCloud web app
application_type: web
insecure: yes
trusted: yes
redirect_uris:
- http://localhost:9100/
- http://localhost:9100/oidc-callback.html
- http://localhost:9100/oidc-silent-redirect.html
- https://localhost:9200/
- https://localhost:9200/oidc-callback.html
- https://localhost:9200/oidc-silent-redirect.html
- https://your-server:9200/
- https://your-server:9200/oidc-callback.html
- https://your-server:9200/oidc-silent-redirect.html
origins:
- http://localhost:9100
- https://localhost:9200
- https://your-server:9200
```
In this example we do not change the default port (`9200`). But this could be changed to another port.
### Start the ocis fullstack server
You need to configure `your-host` in some services to provide the needed public resources. When using the owncloud storage driver (which is the default) oCIS currently needs a running Redis Server reachable locally on the machine at the default port (`localhost:6379`). You can change this using the following option `REVA_STORAGE_OWNCLOUD_REDIS_ADDR=some-host:6379`.
```bash
PROXY_HTTP_ADDR=0.0.0.0:9200 \
KONNECTD_ISS=https://your-server:9200 \
REVA_OIDC_ISSUER=https://your-server:9200 \
PHOENIX_OIDC_AUTHORITY=https://your-server:9200 \
PHOENIX_WEB_CONFIG_SERVER=https://your-server:9200 \
PHOENIX_OIDC_METADATA_URL=https://your-server:9200/.well-known/openid-configuration \
REVA_DATAGATEWAY_URL=https://your-server:9200/data \
REVA_FRONTEND_URL=https://your-server:9200 \
PROXY_TRANSPORT_TLS_KEY=./certs/your-host.key \
PROXY_TRANSPORT_TLS_CERT=./certs/your-host.crt \
KONNECTD_TLS=0 \
./bin/ocis server
```
For more configuration options check the configuration secion in [ocis](https://owncloud.github.io/ocis/configuration/) and every ocis extension.
{{< hint info >}}
**TLS Certificate**\
In this example, we are replacing the default self signed cert with a CA signed one to avoid the certificate warning when accessing the login page.
{{< /hint >}}
## Use Docker Compose
We are using our [docker compose playground](https://github.com/owncloud-docker/compose-playground) as a repository to share snippets that make our test setups easier and more aligned.
You can start oCIS with docker very easily on a different host using this snippet.
Let us assume your local IP is `192.168.103.195`
```bash
git clone https://github.com/owncloud-docker/compose-playground.git
cd compose-playground/compose/ocis
sed -i -e 's/your-url/192.168.103.195/g' config/identifier-registration.yml
cat << EOF > .env
OCIS_BASE_URL=192.168.103.195
OCIS_HTTP_PORT=9200
OCIS_DOCKER_TAG=latest
EOF
docker-compose -f ocis.yml -f ../cache/redis-ocis.yml up -d
curl -k https://192.168.103.195:9200/status.php
```

254
ocis/docs/bridge.md
View File

@@ -1,254 +0,0 @@
---
title: "Bridge"
date: 2020-02-27T20:35:00+01:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: bridge.md
---
{{< toc >}}
We are planning to build a bridge from ownCloud 10 to ocis. The idea is to have a reverse proxy infront of ownCloud 10 that will forward requests to ownCloud 10 or ocis-reva, depending on the migration status of the logged in user.
This document is a work in progress of the current setup.
## Current status
Using ocis and the ownCloud 10 openidconnect and graphapi plugins it is possible today to introduce openid connect based authentication to existing instances. That is a prerequisite for migrating to ocis.
## How to do it
### Install the owncloud 10 graphapi app
In an owncloud 10 apps folder
```
$ git clone git@github.com:owncloud/graphapi.git
$ cd graphapi
$ composer install
```
### Enable the graphapi app
```
occ a:e graphapi
```
No configuration necessary. You can test with `curl`:
```console
$ curl https://cloud.example.com/index.php/apps/graphapi/v1.0/users -u admin | jq
Enter host password for user 'admin':
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 694 100 694 0 0 4283 0 --:--:-- --:--:-- --:--:-- 4283
{
"value": [
{
"id": "admin",
"displayName": "admin",
"mail": null
},
{
"id": "demo",
"displayName": "Demo",
"mail": null
},
...
],
"@odata.nextLink": "https://oc.butonic.de/apps/graphapi/v1.0/users?$top=10&$skip=10"
}
```
> Note: The MS graph api actually asks for `Bearer` auth, but in order to check users passwords during an LDAP bind we are exploiting ownClouds authentication implementation that will grant access when `Basic` auth is used. An LDAP Bind you may ask? Read on!
### Start ocis-glauth
We are going to use the above ownCloud 10 and graphapi app to turn it into the datastore for an LDAP proxy.
#### Grab it!
In an `ocis` folder
```
$ git clone git@github.com:owncloud/ocis-glauth.git
$ cd ocis-glauth
$ make
```
This should give you a `bin/ocis-glauth` binary. Try listing the help with `bin/ocis-glauth --help`.
#### Run it!
You need to point `ocis-glauth` to your owncloud domain:
```console
$ bin/ocis-glauth --log-level debug server --backend-datastore owncloud --backend-server https://cloud.example.com --backend-basedn dc=example,dc=com
```
`--log-level debug` is only used to generate more verbose output
`--backend-datastore owncloud` switches to tho owncloud datastore
`--backend-server https://cloud.example.com` is the url to an ownCloud instance with an enabled graphapi app
`--backend-basedn dc=example,dc=com` is used to construct the LDAP dn. The user `admin` will become `cn=admin,dc=example,dc=com`.
#### Check it is up and running
You should now be able to list accounts from your ownCloud 10 oc_accounts table using:
```console
$ ldapsearch -x -H ldap://localhost:9125 -b dc=example,dc=com -D "cn=admin,dc=example,dc=com" -W '(objectclass=posixaccount)'
```
Groups should work as well:
```console
$ ldapsearch -x -H ldap://localhost:9125 -b dc=example,dc=com -D "cn=admin,dc=example,dc=com" -W '(objectclass=posixgroup)'
```
> Note: This is currently a readonly implementation and minimal to the usecase of authenticating users with konnectd.
### Start ocis-phoenix
#### Get it!
In an `ocis` folder
```
$ git clone git@github.com:owncloud/ocis-phoenix.git
$ cd ocis-phoenix
$ make
```
This should give you a `bin/ocis-phoenix` binary. Try listing the help with `bin/ocis-phoenix --help`.
#### Run it!
Point `ocis-phoenix` to your owncloud domain and tell it where to find the openid connect issuing authority:
```console
$ bin/ocis-phoenix server --web-config-server https://cloud.example.com --oidc-authority https://192.168.1.100:9130 --oidc-metadata-url https://192.168.1.100:9130/.well-known/openid-configuration --oidc-client-id ocis
```
`ocis-phoenix` needs to know
- `--web-config-server https://cloud.example.com` is ownCloud url with webdav and ocs endpoints (oc10 or ocis)
- `--oidc-authority https://192.168.1.100:9130` the openid connect issuing authority, in our case `oidc-konnectd`, running on port 9130
- `--oidc-metadata-url https://192.168.1.100:9130/.well-known/openid-configuration` the openid connect configuration endpoint, typically the issuer host with `.well-known/openid-configuration`, but there are cases when another endpoint is used, eg. ping identity provides multiple endpoints to separate domains
- `--oidc-client-id ocis` the client id we will register later with `ocis-konnectd` in the `identifier-registration.yaml`
### Start ocis-konnectd
#### Get it!
In an `ocis` folder
```
$ git clone git@github.com:owncloud/ocis-konnectd.git
$ cd ocis-konnectd
$ make
```
This should give you a `bin/ocis-konnectd` binary. Try listing the help with `bin/ocis-konnectd --help`.
#### Set environment variables
Konnectd needs environment variables to configure the LDAP server:
```console
export LDAP_URI=ldap://192.168.1.100:9125
export LDAP_BINDDN="cn=admin,dc=example,dc=com"
export LDAP_BINDPW="its-a-secret"
export LDAP_BASEDN="dc=example,dc=com"
export LDAP_SCOPE=sub
export LDAP_LOGIN_ATTRIBUTE=uid
export LDAP_EMAIL_ATTRIBUTE=mail
export LDAP_NAME_ATTRIBUTE=givenName
export LDAP_UUID_ATTRIBUTE=uid
export LDAP_UUID_ATTRIBUTE_TYPE=text
export LDAP_FILTER="(objectClass=posixaccount)"
```
Don't forget to use an existing user and the correct password.
### Configure clients
Now we need to configure a client we can later use to configure the ownCloud 10 openidconnect app. In the `assets/identifier-registration.yaml` have:
```yaml
---
# OpenID Connect client registry.
clients:
- id: ocis
name: ownCloud Infinite Scale
insecure: yes
application_type: web
redirect_uris:
- https://cloud.example.com/apps/openidconnect/redirect
- http://localhost:9100/oidc-callback.html
- http://localhost:9100
- http://localhost:9100/
```
You will need the `insecure: yes` if you are using self signed certificates.
Replace `cloud.example.com` in the redirect URI with your ownCloud 10 host and port.
Replace `localhost:9100` in the redirect URIs with your the `ocis-phoenix` host and port.
#### Run it!
You can now bring up `ocis-connectd` with:
```console
$ bin/ocis-konnectd server --iss https://192.168.1.100:9130 --identifier-registration-conf assets/identifier-registration.yaml --signing-kid gen1-2020-02-27
```
`ocis-konnectd` needs to know
- `--iss https://192.168.1.100:9130` the issuer, which must be a reachable https endpoint. For testing an ip works. HTTPS is NOT optional. This url is exposed in the `https://192.168.1.100:9130/.well-known/openid-configuration` endpoint and clients need to be able to connect to it
- `--identifier-registration-conf assets/identifier-registration.yaml` the identifier-registration.yaml you created
- `--signing-kid gen1-2020-02-27` a signature key id, otherwise the jwks key has no name, which might cause problems with clients. a random key is ok, but it should change when the actual signing key changes.
#### Check it is up and running
1. Try getting the configuration:
```console
$ curl https://192.168.1.100:9130/.well-known/openid-configuration
```
2. Check if the login works at https://192.168.1.100:9130/signin/v1/identifier
> Note: If you later get a `Unable to find a key for (algorithm, kid):PS256, )` Error make sure you did set a `--signing-kid` when starting `ocis-konnectd` by checking it is present in https://192.168.1.100:9130/konnect/v1/jwks.json
### Patch owncloud
While the UserSession in ownCloud 10 is currently used to test all available IAuthModule implementations, it immediately logs out the user when an exception occurs. However, existing owncloud 10 instances use the oauth2 app to create Bearer tokens for mobile and desktop clients.
To give the openidconnect app a chance to verify the tokens we need to change the code a bit. See https://github.com/owncloud/core/pull/37043 for a possible solution.
> Note: The PR is hot ... as in *younger than this list of steps*. And it messes with authentication. Use with caution.
### Install the owncloud 10 openidconnect app
In an owncloud 10 apps folder
```
$ git clone git@github.com:owncloud/openidconnect.git
$ cd openidconnect
$ composer install
```
After enabling the app configure it in `config/oidc.config.php`
```php
$CONFIG = [
'openid-connect' => [
'provider-url' => 'https://192.168.1.100:9130',
'client-id' => 'ocis',
'loginButtonName' => 'OpenId Connect @ Konnectd',
],
'debug' => true, // if using self signed certificates
// allow the different domains access to the ocs and wabdav endpoints:
'cors.allowed-domains' => [
'https://cloud.example.com',
'http://localhost:9100',
],
];
```
In the above configuration replace
- `provider-url` with the URL to your `ocis-konnectd` issuer
- `https://cloud.example.com` with the URL to your ownCloud 10 instance
- `http://localhost:9100` with the URL to your phoenix instance
> Note: By default the openidconnect app will use the email of the user to match the user from the oidc userinfo endpoint with the ownCloud account. So make sure your users have a unique primary email.
## Next steps
Aside from the above todos these are the next stepo
- tie it all together behind `ocis-proxy`
- create an `ocis bridge` command that runs all the ocis services in one step with a properly preconfigured `ocis-konnectd` `identifier-registration.yaml` file for `phoenix` and the owncloud 10 `openidconnect` app, as well as a randomized `--signing-kid`.

36
ocis/docs/building-docs.md
View File

@@ -1,36 +0,0 @@
---
title: "Building the documentation"
date: 2020-07-27T08:39:38+00:00
weight: 99
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: building-docs.md
---
{{< toc >}}
## Buildling the documentation
Following steps can be applied for every oCIS extension repository.
### Setting up
- Install [hugo](https://gohugo.io/getting-started/installing/)
- Run `make docs`
### Viewing the documentation
To view the rendered docs in the browser run:
```bash
cd hugo
hugo -D server
```
Then open "http://localhost:1313/"
When making changes to the docs, run `make docs` again and the server will pick up the changes and reload the page automatically
### Deploying the documentation
The documentation is automatically deployed from the master branch to https://owncloud.github.io/phoenix/

49
ocis/docs/building.md
View File

@@ -1,49 +0,0 @@
---
title: "Building"
date: 2020-02-27T20:35:00+01:00
weight: 50
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.13. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis.git
cd ocis
{{< / highlight >}}
All required tools besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`. To build the `ocis` binary run:
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
Finally, you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis -h` to see all available options.
## Simple Ocis fo extonsions example
Currently, we are using a go build tag to allow building a more simple set of the binary. It was intended to let extension developers focus on only the necessary services.
{{< hint info >}}
While it the tag based simple build demonstrates how to use ocis as a framework for a micro service architecture, we may change to an approach that uses an explicit command to run only a subset of the services.
{{< / hint >}}
```console
TAGS=simple make build
```
The artifact lives in `/bin/ocis`
The generated simple ocis binary is a subset of the ocis command with a restricted set of services meant for ease up development. The services included are
```
ocis-hello
ocis-phoenix
ocis-konnectd
ocis-glauth
micro's own services
```

222
ocis/docs/configuration.md
View File

@@ -1,222 +0,0 @@
---
title: "Configuration"
date: "2020-09-21T13:14:56+0200"
weight: 20
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: configuration.md
---
{{< toc >}}
## Configuration
oCIS Single Binary is not responsible for configuring extensions. Instead, each extension could either be configured by environment variables, cli flags or config files.
Each extension has its dedicated documentation page (e.g. https://owncloud.github.io/extensions/ocis_proxy/configuration) which lists all possible configurations. Config files and environment variables are picked up if you use the `./bin/ocis server` command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
### Configuration using config files
Out of the box extensions will attempt to read configuration details from:
```console
/etc/ocis
$HOME/.ocis
./config
```
For this configuration to be picked up, have a look at your extension `root` command and look for which default config name it has assigned. *i.e: ocis-proxy reads `proxy.json | yaml | toml ...`*.
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/ocis.yml`, `${HOME}/.ocis/ocis.yml` or `$(pwd)/config/ocis.yml`.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below. Command line flags are only working when calling the subcommand directly.
## Root Command
ownCloud Infinite Scale Stack
Usage: `ocis [global options] command [command options] [arguments...]`
--config-file | $OCIS_CONFIG_FILE
: Path to config file.
--log-level | $OCIS_LOG_LEVEL
: Set logging level. Default: `info`.
--log-pretty | $OCIS_LOG_PRETTY
: Enable pretty logging. Default: `true`.
--log-color | $OCIS_LOG_COLOR
: Enable colored logging. Default: `true`.
## Sub Commands
### ocis health
Check health status
Usage: `ocis health [command options] [arguments...]`
--debug-addr | $OCIS_DEBUG_ADDR
: Address to debug endpoint. Default: `0.0.0.0:9010`.
### ocis server
Start fullstack server
Usage: `ocis server [command options] [arguments...]`
--tracing-enabled | $OCIS_TRACING_ENABLED
: Enable sending traces.
--tracing-type | $OCIS_TRACING_TYPE
: Tracing backend type. Default: `jaeger`.
--tracing-endpoint | $OCIS_TRACING_ENDPOINT
: Endpoint for the agent. Default: `localhost:6831`.
--tracing-collector | $OCIS_TRACING_COLLECTOR
: Endpoint for the collector. Default: `http://localhost:14268/api/traces`.
--tracing-service | $OCIS_TRACING_SERVICE
: Service name for tracing. Default: `ocis`.
--debug-addr | $OCIS_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9010`.
--debug-token | $OCIS_DEBUG_TOKEN
: Token to grant metrics access.
--debug-pprof | $OCIS_DEBUG_PPROF
: Enable pprof debugging.
--debug-zpages | $OCIS_DEBUG_ZPAGES
: Enable zpages debugging.
--http-addr | $OCIS_HTTP_ADDR
: Address to bind http server. Default: `0.0.0.0:9000`.
--http-root | $OCIS_HTTP_ROOT
: Root path of http server. Default: `/`.
--grpc-addr | $OCIS_GRPC_ADDR
: Address to bind grpc server. Default: `0.0.0.0:9001`.
### List of available Extension subcommands
There are more subcommands to start the individual extensions. Please check the documentation about their usage and options in the dedicated section of the documentation.
#### ocis konnectd
Start konnectd server
#### ocis run
Runs an extension
#### ocis store
Start a go-micro store
#### ocis glauth
Start glauth server
#### ocis ocs
Start ocs server
#### ocis reva-storage-eos-data
Start reva storage data provider for eos mount
#### ocis reva-storage-home-data
Start reva storage data provider for home mount
#### ocis kill
Kill an extension by name
#### ocis proxy
Start proxy server
#### ocis reva-auth-bearer
Start reva auth-bearer service
#### ocis reva-storage-oc-data
Start reva storage data provider for oc mount
#### ocis settings
Start settings server
#### ocis accounts
Start accounts server
#### ocis phoenix
Start phoenix server
#### ocis reva-storage-eos
Start reva storage service for eos mount
#### ocis reva-storage-home
Start reva storage service for home mount
#### ocis reva-storage-oc
Start reva storage service for oc mount
#### ocis reva-storage-root
Start reva root storage
#### ocis reva-gateway
Start reva gateway
#### ocis reva-sharing
Start reva sharing service
#### ocis reva-users
Start reva users service
#### ocis list
Lists running ocis extensions
#### ocis reva-auth-basic
Start reva auth-basic service
#### ocis reva-frontend
Start reva frontend
#### ocis reva-storage-public-link
Start reva public link storage
#### ocis thumbnails
Start thumbnails server
#### ocis webdav
Start webdav server

217
ocis/docs/debugging.md
View File

@@ -1,217 +0,0 @@
---
title: "Debugging"
date: 2020-03-19T08:21:00+01:00
weight: 50
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: debugging.md
---
## Debugging
As a single binary for easy deployment running `ocis server` just forks itself to start all the services, which makes debugging those processes a little harder.
Ultimately, we want to be able to stop a single service using eg. `ocis kill phoenix` so that you can start the service you want to debug in debug mode. We need to [change the way we fork processes](https://github.com/owncloud/ocis/issues/77) though, otherwise the runtime will automatically restart a service if killed.
### Start ocis
For debugging there are two workflows that work well, depending on your preferences.
#### Use the debug binary and attach to the process as needed
Run the debug binary with `OCIS_LOG_LEVEL=debug bin/ocis-debug server` and then find the service you want to debug using:
```console
# ps ax | grep ocis
12837 pts/1 Sl+ 0:00 bin/ocis-debug server
12845 pts/1 Sl 0:00 bin/ocis-debug graph
12847 pts/1 Sl 0:00 bin/ocis-debug reva-auth-bearer
12848 pts/1 Sl 0:00 bin/ocis-debug graph-explorer
12849 pts/1 Sl 0:00 bin/ocis-debug ocs
12850 pts/1 Sl 0:00 bin/ocis-debug reva-storage-oc-data
12863 pts/1 Sl 0:00 bin/ocis-debug webdav
12874 pts/1 Sl 0:00 bin/ocis-debug reva-frontend
12897 pts/1 Sl 0:00 bin/ocis-debug reva-sharing
12905 pts/1 Sl 0:00 bin/ocis-debug reva-gateway
12912 pts/1 Sl 0:00 bin/ocis-debug reva-storage-home
12920 pts/1 Sl 0:00 bin/ocis-debug reva-users
12929 pts/1 Sl 0:00 bin/ocis-debug glauth
12940 pts/1 Sl 0:00 bin/ocis-debug reva-storage-home-data
12948 pts/1 Sl 0:00 bin/ocis-debug konnectd
12952 pts/1 Sl 0:00 bin/ocis-debug proxy
12961 pts/1 Sl 0:00 bin/ocis-debug thumbnails
12971 pts/1 Sl 0:00 bin/ocis-debug reva-storage-oc
12981 pts/1 Sl 0:00 bin/ocis-debug web
12993 pts/1 Sl 0:00 bin/ocis-debug api
12998 pts/1 Sl 0:00 bin/ocis-debug registry
13004 pts/1 Sl 0:00 bin/ocis-debug phoenix
13015 pts/1 Sl 0:00 bin/ocis-debug reva-auth-basic
```
Then you can set a breakpoint in the service you need and attach to the process via processid. To debug the `reva-sharing` service the VS Code `launch.json` would look like this:
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "ocis attach",
"type": "go",
"request": "attach",
"mode": "local",
"processId": 12897,
}
]
}
```
#### Start all services independently to replace one of them with a debug process
1. You can use this `./ocis.sh` script to start all services independently, so they don't get restrarted by the runtime when you kill them:
```bash
#/bin/sh
LOG_LEVEL="debug"
bin/ocis --log-level=$LOG_LEVEL micro &
bin/ocis --log-level=$LOG_LEVEL glauth &
bin/ocis --log-level=$LOG_LEVEL graph-explorer &
bin/ocis --log-level=$LOG_LEVEL graph &
#bin/ocis --log-level=$LOG_LEVEL hello &
bin/ocis --log-level=$LOG_LEVEL konnectd &
#bin/ocis --log-level=$LOG_LEVEL ocs &
bin/ocis --log-level=$LOG_LEVEL phoenix &
bin/ocis --log-level=$LOG_LEVEL reva-auth-basic &
bin/ocis --log-level=$LOG_LEVEL reva-auth-bearer &
bin/ocis --log-level=$LOG_LEVEL reva-frontend &
bin/ocis --log-level=$LOG_LEVEL reva-gateway &
bin/ocis --log-level=$LOG_LEVEL reva-sharing &
bin/ocis --log-level=$LOG_LEVEL reva-storage-home &
bin/ocis --log-level=$LOG_LEVEL reva-storage-home-data &
bin/ocis --log-level=$LOG_LEVEL reva-storage-oc &
bin/ocis --log-level=$LOG_LEVEL reva-storage-oc-data &
bin/ocis --log-level=$LOG_LEVEL reva-storage-root &
bin/ocis --log-level=$LOG_LEVEL reva-users &
#bin/ocis --log-level=$LOG_LEVEL webdav
bin/ocis --log-level=$LOG_LEVEL proxy &
```
2. Get the list of running processes:
```console
# ps ax | grep ocis
12837 pts/1 Sl+ 0:00 bin/ocis-debug server
12845 pts/1 Sl 0:00 bin/ocis-debug graph
12847 pts/1 Sl 0:00 bin/ocis-debug reva-auth-bearer
12848 pts/1 Sl 0:00 bin/ocis-debug graph-explorer
12849 pts/1 Sl 0:00 bin/ocis-debug ocs
12850 pts/1 Sl 0:00 bin/ocis-debug reva-storage-oc-data
12863 pts/1 Sl 0:00 bin/ocis-debug webdav
12874 pts/1 Sl 0:00 bin/ocis-debug reva-frontend
12897 pts/1 Sl 0:00 bin/ocis-debug reva-sharing
12905 pts/1 Sl 0:00 bin/ocis-debug reva-gateway
12912 pts/1 Sl 0:00 bin/ocis-debug reva-storage-home
12920 pts/1 Sl 0:00 bin/ocis-debug reva-users
12929 pts/1 Sl 0:00 bin/ocis-debug glauth
12940 pts/1 Sl 0:00 bin/ocis-debug reva-storage-home-data
12948 pts/1 Sl 0:00 bin/ocis-debug konnectd
12952 pts/1 Sl 0:00 bin/ocis-debug proxy
12961 pts/1 Sl 0:00 bin/ocis-debug thumbnails
12971 pts/1 Sl 0:00 bin/ocis-debug reva-storage-oc
12981 pts/1 Sl 0:00 bin/ocis-debug web
12993 pts/1 Sl 0:00 bin/ocis-debug api
12998 pts/1 Sl 0:00 bin/ocis-debug registry
13004 pts/1 Sl 0:00 bin/ocis-debug phoenix
13015 pts/1 Sl 0:00 bin/ocis-debug reva-auth-basic
```
3. Kill the service you want to start in debug mode:
```console
# kill 17628
```
4. Start the service you are interested in in debug mode. When using make to build the binary there is already a `bin/ocis-debug` binary for you. When running an IDE tell it which service to start by providing the corresponding sub command, eg. `bin\ocis-debug reva-frontend`.
### Gather error messages
We recommend you collect all related information in a single file or in a github issue. Let us start with an error that pops up in the Web UI:
> Error while sharing.
> error sending a grpc stat request
This popped up when I tried to add `marie` as a collaborator in phoenix. That triggers a request to the server which I copied as curl. We can strip a lot of headers and the gist of it is:
```console
# curl 'https://localhost:9200/ocs/v1.php/apps/files_sharing/api/v1/shares' -d 'shareType=0&shareWith=marie&path=%2FNeuer+Ordner&permissions=1' -u einstein:relativity -k -v | xmllint -format -
[... headers ...]
<?xml version="1.0" encoding="UTF-8"?>
<ocs>
<meta>
<status>error</status>
<statuscode>998</statuscode>
<message>error sending a grpc stat request</message>
</meta>
</ocs>
```
{{< hint info >}}
The username and password only work when basic auth is available. Otherwise you have to obtain a bearer token, eg. by grabbing it from the browser.
{{< /hint >}}
{{< hint danger >}}
TODO add ocis cli tool to obtain a bearer token.
{{< /hint >}}
We also have a few interesting log entries:
```
0:43PM INF home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/grpc/interceptors/log/log.go:69 > unary code=OK end="18/Mar/2020:22:43:40 +0100" from=tcp://[::1]:44078 pid=17836 pkg=rgrpc start="18/Mar/2020:22:43:40 +0100" time_ns=95841 traceid=b4eb9a9f45921f7d3632523ca32a42b0 uri=/cs3.storage.registry.v1beta1.RegistryAPI/GetStorageProvider user-agent=grpc-go/1.26.0
10:43PM ERR home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/grpc/interceptors/log/log.go:69 > unary code=Unknown end="18/Mar/2020:22:43:40 +0100" from=tcp://[::1]:43910 pid=17836 pkg=rgrpc start="18/Mar/2020:22:43:40 +0100" time_ns=586115 traceid=b4eb9a9f45921f7d3632523ca32a42b0 uri=/cs3.gateway.v1beta1.GatewayAPI/Stat user-agent=grpc-go/1.26.0
10:43PM ERR home/jfd/go/pkg/mod/github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/http/services/owncloud/ocs/reqres.go:94 > error sending a grpc stat request error="rpc error: code = Unknown desc = gateway: error calling Stat: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp [::1]:9152: connect: connection refused\"" pid=17832 pkg=rhttp traceid=b4eb9a9f45921f7d3632523ca32a42b0
```
{{< hint danger >}}
TODO return the trace id in the response so we can correlate easier. For reva tracked in https://github.com/cs3org/reva/issues/587
{{< /hint >}}
The last line gives us a hint where the log message originated: `.../github.com/cs3org/reva@v0.0.2-0.20200318111623-a2f97d4aa741/internal/http/services/owncloud/ocs/reqres.go:94`. Which looks like this:
```go
89: // WriteOCSResponse handles writing ocs responses in json and xml
90: func WriteOCSResponse(w http.ResponseWriter, r *http.Request, res *Response, err error) {
91: var encoded []byte
92:
93: if err != nil {
94: appctx.GetLogger(r.Context()).Error().Err(err).Msg(res.OCS.Meta.Message)
95: }
```
Ok, so this seems to be a convenience method that is called from multiple places an also handles errors. Unfortunately, this hides the actual source of the error. We could set a breakpoint in line 94 and reproduce the problem, which can be a lot harder than just clicking the share button or sending a curl request again. So let us see what else the log tells us.
The previous line tells us that a Stat request failed: `uri=/cs3.gateway.v1beta1.GatewayAPI/Stat`. This time the line is written by the grpc log interceptor. What else is there?
The first line tells us that looking up the responsible storage provider seems to have succeeded: `uri=/cs3.storage.registry.v1beta1.RegistryAPI/GetStorageProvider`.
At this point it your familiarity with the codebase starts to become a factor. If you are new you should probably go back to setting a break point on the log line and check the stack trace.
Debug wherever the call trace leads you to ... good luck!
### Managing dependencies and testing changes
You can either run and manage the services independently, or you can update the `go.mod` file and replace dependencies with your local version.
To debug the reva frontend we need to add two replacements:
```
// use the local ocis-reva repo
replace github.com/owncloud/ocis-reva => ../ocis-reva
// also use the local reva repo
replace github.com/cs3org/reva => ../reva
```
{{< hint info >}}
The username and password only work when basic auth is available. Otherwise you have to obtain a bearer token, eg. by grabbing it from the browser.
{{< /hint >}}
Rebuild ocis to make sure the dependency is used. It should be sufficient to just restart the service you want to debug.

80
ocis/docs/development.md
View File

@@ -1,80 +0,0 @@
---
title: "Getting Started with Development"
date: 2020-07-07T20:35:00+01:00
weight: 15
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: development.md
---
{{< toc >}}
## Docker dev environment
### Option 1: Plain docker
To build and run your local ocis code with default storage driver
```
docker run --rm -ti --name ocis -v $PWD:/ocis -p 9200:9200 owncloud/eos-ocis-dev
```
The eos-ocis-dev container will build and run ocis using the owncloud storage driver and store files in the container at `/var/tmp/reva/data/<username>/files`
To check the uploaded files start digging with: `docker exec -it ocis ls -l /var/tmp/reva/`
{{< hint info >}}
On MacOS do not mount a local folder to the `/var/tmp/reva/` path. The fuse driver used by docker [does not support extended attributes](https://docs.docker.com/v18.09/docker-for-mac/osxfs/). See [#182](https://github.com/owncloud/ocis/issues/182) for more details.
{{< /hint >}}
### Option 2: Docker compose
With the `docker-compose.yml` file in ocis repo you can also start ocis via compose:
```
docker-compose up -d ocis
```
{{< hint info >}}
We are only starting the `ocis` container here.
{{< /hint >}}
## Verification
Check the services are running
```
$ docker-compose exec ocis ./bin/ocis list
+--------------------------+-----+
| EXTENSION | PID |
+--------------------------+-----+
| accounts | 172 |
| api | 204 |
| glauth | 187 |
| graph | 41 |
| graph-explorer | 55 |
| konnectd | 196 |
| ocs | 59 |
| phoenix | 29 |
| proxy | 22 |
| registry | 226 |
| reva-auth-basic | 96 |
| reva-auth-bearer | 104 |
| reva-frontend | 485 |
| reva-gateway | 78 |
| reva-sharing | 286 |
| reva-storage-eos | 129 |
| reva-storage-eos-data | 134 |
| reva-storage-home | 442 |
| reva-storage-home-data | 464 |
| reva-storage-oc | 149 |
| reva-storage-oc-data | 155 |
| reva-storage-public-link | 168 |
| reva-users | 420 |
| settings | 23 |
| thumbnails | 201 |
| web | 218 |
| webdav | 63 |
+--------------------------+-----+
```

220
ocis/docs/eos.md
View File

@@ -1,220 +0,0 @@
---
title: "EOS"
date: 2020-02-27T20:35:00+01:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: eos.md
---
{{< toc >}}
OCIS can be configured to run on top of [eos](https://eos.web.cern.ch/). While the [eos documentation](http://eos-docs.web.cern.ch/) does cover a lot of topics it leaves out some details that you may have to either pull from various [docker containers](https://gitlab.cern.ch/eos/eos-docker), the [forums](https://eos-community.web.cern.ch/) or even the [source](https://github.com/cern-eos/eos) itself.
This document is a work in progress of the current setup.
## Docker dev environment for eos storage
We begin with the docker-compose.yml found in https://github.com/owncloud/ocis/ and
switch it to eos-storage.
### 1. Start eos & ocis containers
Start the eos cluster and ocis via the compose stack.
```
docker-compose up -d
```
{{< hint info >}}
The first time the **ocis** container starts up, it will compile ocis from scratch which can take a while.
To follow progress, run `docker-compose logs -f --tail=10 ocis`
{{< /hint >}}
### 2. LDAP support
Configure the OS to resolve users and groups using ldap
```
docker-compose exec -d ocis /start-ldap
```
Check that the OS in the ocis container can now resolve einstein or the other demo users
```
$ docker-compose exec ocis id einstein
uid=20000(einstein) gid=30000(users) groups=30000(users),30001(sailing-lovers),30002(violin-haters),30007(physics-lovers)
```
{{< hint info >}}
If the user is not found at first you might need to wait a few more minutes in case the ocis container is still compiling.
{{< /hint >}}
We also need to restart the reva-users service so it picks up the changed environment. Without a restart it is not able to resolve users from LDAP.
```
docker-compose exec ocis ./bin/ocis kill reva-users
docker-compose exec ocis ./bin/ocis run reva-users
```
### 3. Home storage
Kill the home storage. By default it uses the `owncloud` storage driver. We need to switch it to the `eoshome` driver and make it use the storage id of the eos storage provider:
```
docker-compose exec ocis ./bin/ocis kill reva-storage-home
docker-compose exec -e REVA_STORAGE_HOME_DRIVER=eoshome -e REVA_STORAGE_HOME_MOUNT_ID=1284d238-aa92-42ce-bdc4-0b0000009158 ocis ./bin/ocis run reva-storage-home
```
### 4. Home data provider
Kill the home data provider. By default it uses the `owncloud` storage driver. We need to switch it to the `eoshome` driver and make it use the storage id of the eos storage provider:
```
docker-compose exec ocis ./bin/ocis kill reva-storage-home-data
docker-compose exec -e REVA_STORAGE_HOME_DATA_DRIVER=eoshome ocis ./bin/ocis run reva-storage-home-data
```
{{< hint info >}}
The difference between the *home storage* and the *home data provider* are that the former is responsible for metadata changes while the latter is responsible for actual data transfer. The *home storage* uses the cs3 api to manage a folder hierarchy, while the *home data provider* is responsible for moving bytes to and from the storage.
{{< /hint >}}
## Verification
Login with `einstein / relativity`, upload a file to einsteins home and verify the file is there using
```
docker-compose exec ocis eos ls -l /eos/dockertest/reva/users/4/4c510ada-c86b-4815-8820-42cdf82c3d51/
-rw-r--r-- 1 einstein users 10 Jul 1 15:24 newfile.txt
```
If the problem persists, please check the [troubleshooting section about uploads](#creation-and-upload-of-files-does-not-work).
## Further exploration
EOS has a built in shell that you can enter using
```
$ docker-compose exec mgm-master eos
# ---------------------------------------------------------------------------
# EOS Copyright (C) 2011-2019 CERN/Switzerland
# This program comes with ABSOLUTELY NO WARRANTY; for details type `license'.
# This is free software, and you are welcome to redistribute it
# under certain conditions; type `license' for details.
# ---------------------------------------------------------------------------
EOS_INSTANCE=eostest
EOS_SERVER_VERSION=4.6.5 EOS_SERVER_RELEASE=1
EOS_CLIENT_VERSION=4.6.5 EOS_CLIENT_RELEASE=1
EOS Console [root://localhost] |/> help
access Access Interface
accounting Accounting Interface
acl Acl Interface
archive Archive Interface
attr Attribute Interface
backup Backup Interface
clear Clear the terminal
cd Change directory
chmod Mode Interface
chown Chown Interface
config Configuration System
console Run Error Console
cp Cp command
debug Set debug level
exit Exit from EOS console
file File Handling
fileinfo File Information
find Find files/directories
newfind Find files/directories (new implementation)
fs File System configuration
fsck File System Consistency Checking
fuse Fuse Mounting
fusex Fuse(x) Administration
geosched Geoscheduler Interface
group Group configuration
health Health information about system
help Display this text
info Retrieve file or directory information
inspector Interact with File Inspector
io IO Interface
json Toggle JSON output flag for stdout
license Display Software License
ls List a directory
ln Create a symbolic link
map Path mapping interface
member Check Egroup membership
mkdir Create a directory
motd Message of the day
mv Rename file or directory
node Node configuration
ns Namespace Interface
pwd Print working directory
quit Exit from EOS console
quota Quota System configuration
reconnect Forces a re-authentication of the shell
recycle Recycle Bin Functionality
rmdir Remove a directory
rm Remove a file
role Set the client role
route Routing interface
rtlog Get realtime log output from mgm & fst servers
silent Toggle silent flag for stdout
space Space configuration
stagerrm Remove disk replicas of a file if it has tape replicas
stat Run 'stat' on a file or directory
squash Run 'squashfs' utility function
test Run performance test
timing Toggle timing flag for execution time measurement
touch Touch a file
token Token interface
tracker Interact with File Tracker
transfer Transfer Interface
version Verbose client/server version
vid Virtual ID System Configuration
whoami Determine how we are mapped on server side
who Statistics about connected users
? Synonym for 'help'
.q Exit from EOS console
EOS Console [root://localhost] |/>
```
But this is a different adventure. See the links at the top of this page for other sources of information on eos.
## Cleaning up
To clean up and start completely from scratch, run `docker-compose down -v`.
Then delete the local "bin" folder as root which contains the ocis binaries compiled by the "ocis" docker.
## Troubleshooting
### Docker-compose exits right away
When running `docker-compose up -d` ocis exits right away.
You can check the error code using `docker-compose ps` and investigate further by running only ocis again using `docker-compose up ocis` (without `-d` so you can see what is going on in the foreground).
One reason might be that the binary was already built but does not match the container env. Try running `make clean` before running `docker-compose up ocis` so it gets built inside the container.
### Where are the logs ?
The ocis logs can be accessed using `docker-compose logs ocis`. Add `-f` for following.
### How do I update a service in the ocis container?
1. `docker-compose exec ocis make clean build` to update the binary
2. `docker-compose exec ocis ./bin/ocis kill <service>` to kill the service
3. `docker-compose exec ocis ./bin/ocis run <service>` to start the service. Do not forget to set any env vars, eg.
`docker-compose exec -e REVA_STORAGE_EOS_LAYOUT="{{substr 0 1 .Id.OpaqueId}}/{{.Id.OpaqueId}}" -e REVA_STORAGE_HOME_DRIVER=eoshome ocis ./bin/ocis run reva-storage-home`
### Creation and upload of files does not work
If the upload did not work, please check the status of the eos space using the command `docker-compose exec mgm-master eos fs ls`.
In case the default space appears as offline, run `docker-compose exec mgm-master eos space set default on`.
### Uploading big files appears to hang
Please note that the uploads first go into the "ocis" docker and land in its "/tmp" folder, then gets copied over to the EOS docker using `xrdcopy`.
This is why uploading first transfers all bytes and then seem to hang for a while during the final copy.
### Running out of space quickly
The EOS dockers are configured with replication, so every file uploaded there will be replicated 4 times,
so make sure there is enough physical space on disk when testing.
Also please note that older failed uploads might still be present in the "/tmp" directory of the "ocis" container.

237
ocis/docs/extensions.md
View File

@@ -1,237 +0,0 @@
---
title: "Extension"
date: 2020-02-27T20:35:00+01:00
weight: 40
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: extensions.md
---
{{< toc >}}
## How to build and run ocis-simple
ocis uses build tags to build different flavors of the binary. In order to work on a new extension we are going to reduce the scope a little and use the `simple` tag. Let us begin by creating a dedicated folder:
```console
mkdir ocis-extension-workshop && ocis-extension-workshop
```
Following https://github.com/owncloud/ocis
```console
git clone https://github.com/owncloud/ocis.git
cd ocis
TAGS=simple make generate build
```
*Q: Can you specify which version of phoenix to use?*
*A: No, the phoenix that is used is compiled into the [assets of ocis-phoenix](https://github.com/owncloud/ocis-phoenix/blob/master/pkg/assets/embed.go) which is currently not automatically updated. We'll see how to use a custom phoenix later.*
`bin/ocis server`
Open the browser at http://localhost:9100
1. You land on the login screen. click login
2. You are redirected to an idp at http://localhost:9140/oauth2/auth with a login mask. Use `einstein:relativity`to login (one of the three demo users)
3. You are redirected to http://localhost:9100/#/hello the ocis-hello app
4. Replace `World` with something else and submit. You should see `Hello %something else%`
*Q: One of the required ports is already in use. Ocis seems to be trying to restart the service over and over. What gives?*
*A: Using the ocis binary to start the server will case ocis to keep track of the different services and restart them in case they crash.*
## Hacking ocis-hello
go back to the ocis-extension-workshop folder
```console
cd ..
```
Following https://github.com/owncloud/ocis-hello
```
git clone https://github.com/owncloud/ocis-hello.git
cd ocis-hello
yarn install
# this actually creates the assets
yarn build
# this will compile the assets into the binary
make generate build
```
Two options:
1. run only the necessery services from ocis and ocis-hello independently
2. compile ocis with the updated ocis-hello
### Option 1:
get a list of ocis services:
```console
ps ax | grep ocis
```
Try to kill `ocis hello`
Remember: for now, killing a service will cause ocis to restart it. This is subject to change.
In order to be able to manage the processes ourselves we need to start them independently:
`bin/ocis server` starts the same services as:
```
bin/ocis micro &
bin/ocis phoenix &
bin/ocis hello &
bin/ocis reva &
```
Now we can kill the `ocis hello` and use our custom built ocis-hello binary:
```console
cd ../ocis-hello
bin/ocis-hello server
```
## Hacking phoenix (and ocis-phoenix)
Following https://github.com/owncloud/phoenix we are going to build the current phoenix
```
git clone https://github.com/owncloud/phoenix.git
cd phoenix
yarn install
yarn dist
```
We can tell ocis to use the compiled assets:
Kill `ocis phoenix`, then use the compiled assets when starting phoenix.
```console
cd ../ocis
PHOENIX_ASSET_PATH="`pwd`/../phoenix/dist" bin/ocis phoenix
```
## The ownCloud design system
The owncloud design system contains a set of ownCloud vue components for phoenix or your own ocis extensions. Use it for a consistent look and feel.
Point your browser to https://owncloud.github.io/owncloud-design-system and check the available components. Live editing the examples in the browser is supported.
note: There is a bug with navigation sub items: either click a nav item twice or refresh the page
## External phoenix apps
This is what hello is: copy and extend!
1. Phoenix is configured using the config.json which is served by the phoenix service (either `bin/ocis phoenix` or `bin/ocis-phoenix server`)
2. point ocis phoenix to the web config which you extended with an external app:
`PHOENIX_WEB_CONFIG="`pwd`/../phoenix/config.json" PHOENIX_ASSET_PATH="`pwd`/../phoenix/dist" bin/ocis phoenix`
```json
{
"server": "http://localhost:9140",
"theme": "owncloud",
"version": "0.1.0",
"openIdConnect": {
"metadata_url": "http://localhost:9140/.well-known/openid-configuration",
"authority": "http://localhost:9140",
"client_id": "phoenix",
"response_type": "code",
"scope": "openid profile email"
},
"apps": [],
"external_apps": [
{
"id": "hello",
"path": "http://localhost:9105/hello.js",
"config": {
"url": "http://localhost:9105"
}
},
{
"id": "myapp",
"path": "http://localhost:6789/superapp.js",
"config": {
"backend": "http://someserver:1234",
"myconfig": "is awesome"
}
}
]
}
```
## Phoenix extension points
{{< hint info >}}
For an up to date list check out [the phoenix documentation](https://github.com/owncloud/phoenix/issues/2423).
{{< /hint >}}
Several ones available:
### Phoenix core
- App switcher (defined in config.json)
- App container (loads UI of your extension)
### Files app
- File action
- Create new file action
- Sidebar
- Quick access for sidebar inside of file actions (in the file row)
Example of a file action in the `app.js`:
```js
const appInfo = {
name: 'MarkdownEditor',
id: 'markdown-editor',
icon: 'text',
isFileEditor: true,
extensions: [{
extension: 'txt',
newFileMenu: {
menuTitle ($gettext) {
return $gettext('Create new plain text file…')
}
}
},
{
extension: 'md',
newFileMenu: {
menuTitle ($gettext) {
return $gettext('Create new mark-down file…')
}
}
}]
}
```
For the side bar have a look at the files app, `defaults.js` & `fileSideBars`
## API driven development
Until now we only had a look at the ui and how the extensions are managed on the cli. But how do apps actually talk to the server?
Short answer: any way you like
Long answer: micro and ocis-hello follow a protocol driven development:
- specify the API using protobuf
- generate client and server code
- evolve based on the protocol
- CS3 api uses protobuf as well and uses GRPC
- ocis uses go-micro, which provides http and grpc gateways
- the gateways and protocols are optional
- owncloud and kopano are looking into a [MS graph](https://developer.microsoft.com/de-de/graph) like api to handle phoenix requests.
- they might be about user, contacrs, calendars ... which is covered by the graph api
- we want to integrate with eg. kopano and provide a commen api (file sync and share is covered as well)
- as an example for protobuf take a look at [ocis-hello](https://github.com/owncloud/ocis-hello/tree/master/pkg/proto/v0)

228
ocis/docs/getting-started.md
View File

@@ -1,228 +0,0 @@
---
title: "Getting Started"
date: 2020-02-27T20:35:00+01:00
weight: 15
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
Docker images for ocis are hosted on https://hub.docker.com/r/owncloud/ocis.
The `latest` tag always reflects the current master branch.
```console
docker pull owncloud/ocis
```
#### Dependencies
- Running ocis currently needs a working Redis caching server
- The default storage location in the container is `/var/tmp/reva/data`. You may want to create a volume to persist the files in the primary storage
#### Docker compose
You can use our docker-compose [playground example](https://github.com/owncloud-docker/compose-playground/tree/master/ocis) to run ocis with dependencies with a single command in a docker network.
```console
git clone git@github.com:owncloud-docker/compose-playground.git
cd compose-playground/ocis
docker-compose -f ocis.yml -f ../cache/redis-ocis.yml up
```
### Binaries
The pre-built binaries for different platforms are downloadable at https://download.owncloud.com/ocis/ocis/ . Specific releases are organized in separate folders. They are in sync which every release tag on GitHub. The binaries from the current master branch can be found in https://download.owncloud.com/ocis/ocis/testing/
```console
curl https://download.owncloud.com/ocis/ocis/1.0.0-beta1/ocis-1.0.0-beta1-darwin-amd64 --output ocis
chmod +x ocis
./ocis server
```
#### Dependencies
- Running ocis currently needs a working Redis caching server
- The default promary storage location is `/var/tmp/reva/data`. You can change that value by configuration.
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis --help`.
### Server
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{< highlight txt >}}
ocis server --help
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
ocis health --help
{{< / highlight >}}
## Quickstart for Developers
Following https://github.com/owncloud/ocis#development
```console
git clone https://github.com/owncloud/ocis.git
cd ocis
make generate build
```
Open https://localhost:9200 and login using one of the demo accounts:
```console
einstein:relativity
marie:radioactivity
feynman:superfluidity
```
## Runtime
Included with the ocis binary is embedded a go-micro runtime that is in charge of starting services as a fork of the master process. This provides complete control over the services. Ocis extensions can be added as part of this runtime.
```console
./bin/ocis micro
```
This will currently boot:
```console
com.owncloud.api
com.owncloud.http.broker
com.owncloud.proxy
com.owncloud.registry
com.owncloud.router
com.owncloud.runtime
com.owncloud.web
go.micro.http.broker
```
Further ocis extensions can be added to the runtime via the ocis command like:
```console
./bin/ocis hello
```
Which will register:
```console
com.owncloud.web.hello
com.owncloud.api.hello
```
To the list of available services.
## Metrics
This service provides some [Prometheus](https://prometheus.io/) metrics through the debug endpoint, you can optionally secure the metrics endpoint by some random token, which got to be configured through one of the flag `--debug-token` or the environment variable `OCIS_DEBUG_TOKEN` mentioned above. By default the metrics endpoint is bound to `http://0.0.0.0:8001/metrics`.
go_gc_duration_seconds
: A summary of the GC invocation durations
go_gc_duration_seconds_sum
: A summary of the GC invocation durations
go_gc_duration_seconds_count
: A summary of the GC invocation durations
go_goroutines
: Number of goroutines that currently exist
go_info
: Information about the Go environment
go_memstats_alloc_bytes
: Number of bytes allocated and still in use
go_memstats_alloc_bytes_total
: Total number of bytes allocated, even if freed
go_memstats_buck_hash_sys_bytes
: Number of bytes used by the profiling bucket hash table
go_memstats_frees_total
: Total number of frees
go_memstats_gc_cpu_fraction
: The fraction of this program's available CPU time used by the GC since the program started
go_memstats_gc_sys_bytes
: Number of bytes used for garbage collection system metadata
go_memstats_heap_alloc_bytes
: Number of heap bytes allocated and still in use
go_memstats_heap_idle_bytes
: Number of heap bytes waiting to be used
go_memstats_heap_inuse_bytes
: Number of heap bytes that are in use
go_memstats_heap_objects
: Number of allocated objects
go_memstats_heap_released_bytes
: Number of heap bytes released to OS
go_memstats_heap_sys_bytes
: Number of heap bytes obtained from system
go_memstats_last_gc_time_seconds
: Number of seconds since 1970 of last garbage collection
go_memstats_lookups_total
: Total number of pointer lookups
go_memstats_mallocs_total
: Total number of mallocs
go_memstats_mcache_inuse_bytes
: Number of bytes in use by mcache structures
go_memstats_mcache_sys_bytes
: Number of bytes used for mcache structures obtained from system
go_memstats_mspan_inuse_bytes
: Number of bytes in use by mspan structures
go_memstats_mspan_sys_bytes
: Number of bytes used for mspan structures obtained from system
go_memstats_next_gc_bytes
: Number of heap bytes when next garbage collection will take place
go_memstats_other_sys_bytes
: Number of bytes used for other system allocations
go_memstats_stack_inuse_bytes
: Number of bytes in use by the stack allocator
go_memstats_stack_sys_bytes
: Number of bytes obtained from system for stack allocator
go_memstats_sys_bytes
: Number of bytes obtained from system
go_threads
: Number of OS threads created
promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code

10
ocis/docs/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2020-02-27T20:35:00+01:00
weight: 100
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/blob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

85
ocis/docs/login-flow.md
View File

@@ -1,85 +0,0 @@
---
title: "Login Flow"
date: 2020-05-04T20:47:00+01:00
weight: 43
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: login-flow.md
---
## Login Flow
The following sequence diagram describes the [openid connect auth code flow](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth). The eight numbered steps and notes correspond to the [openid connect auth code flow steps](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowSteps). Example requests are based on the spec as well.:
{{< mermaid class="text-center">}}
sequenceDiagram
%% we have comments!! \o/
%% this documents the login workflow
%% examples taken from the oidc spec https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth
%% TODO add PKCE, see https://developer.okta.com/blog/2019/08/22/okta-authjs-pkce#use-pkce-to-make-your-apps-more-secure
participant user as User
participant client as Client
participant proxy as ocis-proxy
participant idp as IdP
participant glauth as ocis-glauth
participant graph as ocis-graph
participant accounts as ocis-accounts
participant ldap as external LDAP server
user->>+client: What is the content of my home?
client->>+proxy: PROPFIND <br> no (or expired) auth
Note over client,proxy: ocis needs to know the IdP that is<br>used to authenticate users. The<br>proxy will redirect unauthenticated<br>requests to that IdP.
proxy-->>-client: 302 Found
Note over client, idp: HTTP/1.1 302 Found<br>Location: https://server.example.com/authorize?<br>response_type=code&<br>scope=openid%20profile%20email<br>&client_id=s6BhdRkqt3<br>&state=af0ifjsldkj<br>&redirect_uri=https%3A%2F%2Fclient.example.org%2Fcb
Note over client, idp: We should follow the OpenID Connect Discovery protocol
Note over client, idp: Clients might fall back to the ocis server if the discovery failed.<br>We can provide a webfinger endpoint there to let guests use an idp<br>that is backed by the accounts service.
Note over client, idp: For now, clients can only handle one IdP, which is configured in ocis.
client-->>client: 1. Client prepares an Authentication Request<br>containing the desired request parameters.
client->>+idp: 2. Client sends the request to the Authorization Server.
Note over client, idp: GET /authorize?<br>response_type=code<br>&scope=openid%20profile%20email<br>&client_id=s6BhdRkqt3<br>&state=af0ifjsldkj<br>&redirect_uri=https%3A%2F%2Fclient.example.org%2Fcb HTTP/1.1<br>Host: server.example.com
Note over user, idp: 3. Authorization Server Authenticates the End-User.
Note over idp,ldap: Either an IdP already exists or a new one is introduced. Since we are not yet using oidc discovery we can only use one IdP.
alt all users managed by konnectd/ocis
idp->>+glauth: LDAP query/bind
glauth->>+graph: GET user with Basic Auth<br>GraphAPI
graph->>+accounts: internal GRPC
accounts-->>-graph: response
graph-->>-glauth: OData response
glauth-->>-idp: LDAP result
Note over accounts,ldap: In case internal users are managed<br>in an external ldap they have to be<br>synced to the accounts service to<br>show up as recipients during sharing.
else all users authenticated by an external idp
idp->>+ldap: LDAP query/bind
ldap-->>-idp: LDAP result
alt guest accounts managed in ocis / lookup using glauth proxy:
Note over idp,glauth: Idp is configured to use glauth as a<br>second ldap server.
idp->>+glauth: LDAP query/bind
glauth->>+graph: GET user with Basic Auth<br>GraphAPI
graph->>+accounts: internal GRPC
accounts-->>-graph: response
graph-->>-glauth: OData response
glauth-->>-idp: LDAP result
else guest account provisioned by other means
Note over accounts, ldap: In case guest accounts are managed<br>in an existing ldap they need to be<br>synced to the accounts service to<br>be able to login and show up as<br>recipients during sharing.
end
end
Note over user, idp: 4. Authorization Server obtains End-User Consent/Authorization.
idp-->>-client: 5. Authorization Server sends the End-User back<br>to the Client with an Authorization Code.
Note over client, idp: HTTP/1.1 302 Found<br>Location: https://client.example.org/cb?<br>code=SplxlOBeZQQYbYS6WxSbIA&state=af0ifjsldkj
client->>+idp: 6. Client requests a response using the<br>Authorization Code at the Token Endpoint.
Note over client, idp: POST /token HTTP/1.1<br>Host: server.example.com<br>Content-Type: application/x-www-form-urlencoded<br>grant_type=authorization_code&code=SplxlOBeZQQYbYS6WxSbIA<br>&redirect_uri=https%3A%2F%2Fclient.example.org%2Fcb
idp-->>-client: 7. Client receives a response that contains an<br>ID Token and Access Token in the response body.
Note over client, idp: HTTP/1.1 200 OK<br>Content-Type: application/json<br>Cache-Control: no-store<br>Pragma: no-cache<br>{<br>"access_token": "SlAV32hkKG",<br>"token_type": "Bearer",<br>"refresh_token": "8xLOxBtZp8",<br>"expires_in": 3600,<br>"id_token": "a ... b.c ... d.e ... f" // must be a JWT<br>}
client-->>client: 8. Client validates the ID token and<br>retrieves the End-User's Subject Identifier.
client->>+proxy: PROPFIND <br> With access token
proxy-->>-client: 207 Multi-Status
client-->>-user: List of Files X, Y, Z ...
{{< /mermaid >}}

15
ocis/docs/public-upload-flow.md
View File

@@ -1,15 +0,0 @@
---
title: "Public upload Flow"
date: 2020-07-27T14:16:00+01:00
weight: 47
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: public-upload-flow.md
---
## Public Upload flow
The following diagram describes the flow of requests:
{{< svg src="static/ocis/tus-public-upload.svg" >}}

96
ocis/docs/request-flow.md
View File

@@ -1,96 +0,0 @@
---
title: "Request Flow"
date: 2020-04-27T16:07:00+01:00
weight: 45
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: request-flow.md
---
## Request Flow
The following sequence diagram describes the general request flow. It shows where account provisioning and token minting are happening:
{{< mermaid class="text-center">}}
sequenceDiagram
%% we have comments!! \o/
participant user as User
participant client as Client
participant proxy as ocis-proxy
participant idp as IdP
participant accounts as ocis-accounts
participant ldap as corporate LDAP server
user->>+client: What is the content of my home?
client->>+proxy: PROPFIND <br> Bearer auth using oidc auth token
Note over client,proxy: What is in a bearer token? <br> The spec recommends opaque tokens. <br> Treat it as random byte noise.
Note over client,proxy: the proxy MUST authenticate users <br> using ocis-accounts because it needs <br> to decide where to send the request
%% Mention introspection endpoint for opaque tokens
%% konnectd uses jwt, so we can save a request
%% either way the token can be used to look up the sub and iss of the user
%% or is token check enough?
proxy->>+idp: GET /userinfo
alt userinfo succeeds
idp-->>proxy: 200 OK
Note over proxy,accounts: Content-Type: application/json<br>{<br>"sub": "248289761001",<br>"name": "Jane Doe",<br>"given_name": "Jane",<br>"family_name": "Doe",<br>"preferred_username": "j.doe",<br>"email": "janedoe@example.com",<br>"picture": "http://example.com/janedoe/me.jpg"<br>}
%% see: https://openid.net/specs/openid-connect-core-1_0.html#UserInfoResponse
else userinfo fails
idp-->>-proxy: 401 Unauthorized
Note over proxy,accounts: WWW-Authenticate: error="invalid_token",<br>error_description="The Access Token expired"
proxy-->>client: 401 Unauthorized or <br>302 Found with redirect to idp
Note over client: start at login flow<br> or refresh the token
end
proxy->>+accounts: TODO API call to exchange sub@iss with account UUID
Note over proxy,accounts: does not autoprovision users. They are explicitly provsioned later.
alt account exists or has been migrated
accounts-->>proxy: existing account UUID
else account does not exist
opt oc10 endpoint is configured
Note over proxy,oc10: Check if user exists in oc10
proxy->>+oc10: GET /apps/graphapi/v1.0/users/&lt;uuid&gt;
opt user exists in oc10
oc10-->>-proxy: 200
%% TODO auth using internal token
proxy->>+oc10: PROPFIND
Note over proxy,oc10: forward existing bearer auth
oc10-->>-proxy: Multistatus response
proxy-->>client: Multistatus response
client-->>user: List of Files X, Y, Z ...
end
end
Note over proxy,accounts: provision a new account including displayname, email and sub@iss <br> TODO only if the user is allowed to login, based on group <br> membership in the ldap server
proxy->>proxy: generate new uuid
proxy->>+accounts: TODO create account with new generated uuid
accounts-->>-proxy: OK / error
else account has been disabled
accounts-->>-proxy: account is disabled
proxy-->>client: 401 Unauthorized or <br>302 Found with redirect to idp
Note over client: start at login flow<br> or refresh the token
end
proxy->>proxy: store uuid in context
%% what if oc10 does not support a certain request / API
proxy->>proxy: mint an internal jwt that includes the UUID and username using revas `x-access-token` header
proxy->>+reva: PROPFIND <br>Token auth using internal JWT
reva-->>-proxy: Multistatus response
proxy-->>-client: Multistatus response
client-->>-user: List of Files X, Y, Z ...
{{< /mermaid >}}

1
ocis/docs/static/src/tus-public-upload.drawio vendored
View File

File diff suppressed because one or more lines are too long

3
ocis/docs/static/tus-public-upload.svg vendored
View File

File diff suppressed because one or more lines are too long
Side by Side

Before

Width:  |  Height:  |  Size: 66 KiB

107
ocis/docs/testing.md
View File

@@ -1,107 +0,0 @@
---
title: "Testing"
date: 2018-05-02T00:00:00+00:00
weight: 37
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: testing.md
---
## Acceptance tests
We are using the ownCloud 10 acceptance testsuite against ocis. To set this up you need the owncloud 10 core repo, a ldap server that the acceptance tests can use to manage users, a redis server for file-versions and the ocis code.
### Getting the tests
All you need to do to get the acceptance tests is check out the core repo:
```
git clone https://github.com/owncloud/core.git
```
### Run a redis server in a docker container
File versions need a redis server. Start one with docker by using:
`docker run -e REDIS_DATABASES=1 -p 6379:6379 -d webhippie/redis:latest`
### Run ocis
To start ocis:
```
bin/ocis server
```
### Run the acceptance tests
First we will need to clone the testing app in owncloud which contains the skeleton files required for running the tests.
In the ownCloud 10 core clone the testing app with the following command:
```
git clone https://github.com/owncloud/testing apps/testing
```
Then run the api acceptance tests with the following command:
```
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
SKELETON_DIR=apps/testing/data/apiSkeleton \
BEHAT_FILTER_TAGS='~@notToImplementOnOCIS&&~@toImplementOnOCIS'
```
Make sure to adjust the settings `TEST_SERVER_URL` and `OCIS_REVA_DATA_ROOT` according to your environment.
This will run all tests that are relevant to OCIS.
To run a single test add `BEHAT_FEATURE=<feature file>`
### use existing tests for BDD
As a lot of scenarios are written for oC10, we can use those tests for Behaviour driven development in ocis.
Every scenario that does not work in OCIS with OC storage, is listed in `tests/acceptance/expected-failures-on-OC-storage.txt` with a link to the related issue.
Those scenarios are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the
expected failures. If there are any differences then the CI pipeline fails.
Similarly, scenarios that do not work in OCIS with EOS storage are listed in `tests/acceptance/expected-failures-on-EOS-storage.txt`.
Additionally, some issues have scenarios that demonstrate the current buggy behaviour in ocis(reva).
Those scenarios are in this ocis repository in `tests/acceptance/features/apiOcisSpecific`.
Have a look into the [documentation](https://doc.owncloud.com/server/developer_manual/testing/acceptance-tests.html#writing-scenarios-for-bugs) to understand why we are writing those tests.
If you want to work on a specific issue
1. adjust the core commit id to the latest commit in core so that CI will run the latest test code and scenarios from core.
For that change `coreCommit` in the `config` section:
config = {
'apiTests': {
'coreBranch': 'master',
'coreCommit': 'a06b1bd5ba8e5244bfaf7fa04f441961e6fb0daa',
'numberOfParts': 2
}
}
2. locally run each of the tests marked with that issue in the expected failures file
E.g.:
```
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
BEHAT_FEATURE='tests/acceptance/features/apiComments/comments.feature:123'
```
3. the tests will fail, try to understand how and why they are failing
4. fix the code
5. go back to 2. and repeat till the tests are passing.
6. remove those tests from the expected failures file
7. run each of the local tests that were demonstrating the **buggy** behavior. They should fail.
8. delete each of the local tests that were demonstrating the **buggy** behavior.
9. make a PR that has the fixed code, relevant lines removed from the expected failures file and bug demonstration tests deleted.
If the changes also affect the `ocis-reva` repository make sure the changes get ported over there.
### Notes
- in a normal case the test-code cleans up users after the test-run, but if a test-run is interrupted (e.g. by CTRL+C) users might have been left on the LDAP server. In that case rerunning the tests requires wiping the users in the ldap server, otherwise the tests will fail when trying to populate the users.
- the tests usually create users in the OU `TestUsers` with usernames specified in the feature file. If not defined in the feature file, most users have the password `123456`, defined by `regularUserPassword` in `behat.yml`, but other passwords are also used, see [`\FeatureContext::getPasswordForUser()`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L386) for mapping and [`\FeatureContext::__construct`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L1668) for the password definitions.

48
ocis/docs/tracing.md
View File

@@ -1,48 +0,0 @@
---
title: "Tracing"
date: 2020-05-13T12:09:00+01:00
weight: 55
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs
geekdocFilePath: tracing.md
---
By default, we use [Jaeger](https://www.jaegertracing.io) for request tracing within oCIS. You can follow these steps
to get started:
1. Start Jaeger by using the all-in-one docker image:
```console
docker run -d --name jaeger \
-e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
-p 5775:5775/udp \
-p 6831:6831/udp \
-p 6832:6832/udp \
-p 5778:5778 \
-p 16686:16686 \
-p 14268:14268 \
-p 14250:14250 \
-p 9411:9411 \
jaegertracing/all-in-one:1.17
```
2. Every single oCIS service has its own environment variables for enabling and configuring tracing.
1. You can enable and configure tracing on each service individually. For example, enable tracing
in Reva when starting the oCIS single binary like this:
```console
REVA_TRACING_ENABLED=true \
REVA_TRACING_ENDPOINT=localhost:6831 \
REVA_TRACING_COLLECTOR=http://localhost:14268/api/traces \
./bin/ocis server
```
2. Enabling and configuring tracing on oCIS itself will forward the configuration to all services:
```console
OCIS_TRACING_ENABLED=true \
OCIS_TRACING_ENDPOINT=localhost:6831 \
OCIS_TRACING_COLLECTOR=http://localhost:14268/api/traces \
./bin/ocis server
```
If you want to set individual tracing configuration for each service, make sure to set
`OCIS_TRACING_ENABLED=false`.
3. Make the actual request that you want to trace.
4. Open up the [Jaeger UI](http://localhost:16686) to analyze request traces.
For more information on Jaeger, please refer to their [Documentation](https://www.jaegertracing.io/docs/1.17/).

24
ocs/Makefile
View File

@@ -3,7 +3,6 @@ NAME := ocs
IMPORT := github.com/owncloud/ocis/$(NAME)
BIN := bin
DIST := dist
HUGO := hugo
ifeq ($(OS), Windows_NT)
EXECUTABLE := $(NAME).exe
@@ -59,7 +58,7 @@ sync:
.PHONY: clean
clean:
go clean -i ./...
rm -rf $(BIN) $(DIST) $(HUGO)
rm -rf $(BIN) $(DIST)
.PHONY: fmt
fmt:
@@ -132,24 +131,9 @@ release-check:
.PHONY: release-finish
release-finish: release-copy release-check
.PHONY: docs-copy
docs-copy:
mkdir -p $(HUGO); \
mkdir -p $(HUGO)/content/extensions; \
cd $(HUGO); \
git init; \
git remote rm origin; \
git remote add origin https://github.com/owncloud/owncloud.github.io; \
git fetch; \
git checkout origin/source -f; \
rsync --delete -ax ../docs/ content/extensions/$(NAME)
.PHONY: docs-build
docs-build:
cd $(HUGO); hugo
.PHONY: docs
docs: docs-copy docs-build
.PHONY: config-docs-generate
config-docs-generate:
# go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md
.PHONY: watch
watch:

10
ocs/docs/_index.md
View File

@@ -1,10 +0,0 @@
---
title: "Ocs"
date: 2018-05-02T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-ocs
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
This service provides the OCS API which is required by some ownCloud clients.

28
ocs/docs/building.md
View File

@@ -1,28 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis-ocs
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
{{< toc >}}
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.12. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis-ocs.git
cd ocis-ocs
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
## Backend
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
Finally you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis-ocs -h` to see all available options.

154
ocs/docs/getting-started.md
View File

@@ -1,154 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-ocs
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
Docker images for ocis-ocs are hosted on https://hub.docker.com/r/owncloud/ocis-ocs.
The `latest` tag always reflects the current master branch.
```console
docker pull owncloud/ocis-ocs
```
### Binaries
The pre-built binaries for different platforms are downloadable at https://download.owncloud.com/ocis/ocs/ . Specific releases are organized in separate folders. They are in sync which every release tag on GitHub. The binaries from the current master branch can be found in https://download.owncloud.com/ocis/ocs/testing/
```console
curl https://download.owncloud.com/ocis/ocis-ocs/1.0.0-beta1/ocis-ocs-1.0.0-beta1-darwin-amd64 --output ocis-ocs
chmod +x ocis-ocs
./ocis-ocs server
```
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis-ocs --help`.
### Server
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{< highlight txt >}}
ocis-ocs server --help
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
ocis-ocs health --help
{{< / highlight >}}
## Metrics
This service provides some [Prometheus](https://prometheus.io/) metrics through the debug endpoint, you can optionally secure the metrics endpoint by some random token, which got to be configured through one of the flag `--debug-token` or the environment variable `OCS_DEBUG_TOKEN` mentioned above. By default the metrics endpoint is bound to `http://0.0.0.0:9114/metrics`.
go_gc_duration_seconds
: A summary of the GC invocation durations
go_gc_duration_seconds_sum
: A summary of the GC invocation durations
go_gc_duration_seconds_count
: A summary of the GC invocation durations
go_goroutines
: Number of goroutines that currently exist
go_info
: Information about the Go environment
go_memstats_alloc_bytes
: Number of bytes allocated and still in use
go_memstats_alloc_bytes_total
: Total number of bytes allocated, even if freed
go_memstats_buck_hash_sys_bytes
: Number of bytes used by the profiling bucket hash table
go_memstats_frees_total
: Total number of frees
go_memstats_gc_cpu_fraction
: The fraction of this program's available CPU time used by the GC since the program started
go_memstats_gc_sys_bytes
: Number of bytes used for garbage collection system metadata
go_memstats_heap_alloc_bytes
: Number of heap bytes allocated and still in use
go_memstats_heap_idle_bytes
: Number of heap bytes waiting to be used
go_memstats_heap_inuse_bytes
: Number of heap bytes that are in use
go_memstats_heap_objects
: Number of allocated objects
go_memstats_heap_released_bytes
: Number of heap bytes released to OS
go_memstats_heap_sys_bytes
: Number of heap bytes obtained from system
go_memstats_last_gc_time_seconds
: Number of seconds since 1970 of last garbage collection
go_memstats_lookups_total
: Total number of pointer lookups
go_memstats_mallocs_total
: Total number of mallocs
go_memstats_mcache_inuse_bytes
: Number of bytes in use by mcache structures
go_memstats_mcache_sys_bytes
: Number of bytes used for mcache structures obtained from system
go_memstats_mspan_inuse_bytes
: Number of bytes in use by mspan structures
go_memstats_mspan_sys_bytes
: Number of bytes used for mspan structures obtained from system
go_memstats_next_gc_bytes
: Number of heap bytes when next garbage collection will take place
go_memstats_other_sys_bytes
: Number of bytes used for other system allocations
go_memstats_stack_inuse_bytes
: Number of bytes in use by the stack allocator
go_memstats_stack_sys_bytes
: Number of bytes obtained from system for stack allocator
go_memstats_sys_bytes
: Number of bytes obtained from system
go_threads
: Number of OS threads created
promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code

10
ocs/docs/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2018-05-02T00:00:00+00:00
weight: 40
geekdocRepo: https://github.com/owncloud/ocis-ocs
geekdocEditPath: edit/master/docs
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/ocs/blob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

24
proxy/Makefile
View File

@@ -3,7 +3,6 @@ NAME := proxy
IMPORT := github.com/owncloud/ocis/$(NAME)
BIN := bin
DIST := dist
HUGO := hugo
ifeq ($(OS), Windows_NT)
EXECUTABLE := $(NAME).exe
@@ -59,7 +58,7 @@ sync:
.PHONY: clean
clean:
go clean -i ./...
rm -rf $(BIN) $(DIST) $(HUGO)
rm -rf $(BIN) $(DIST)
.PHONY: fmt
fmt:
@@ -132,28 +131,9 @@ release-check:
.PHONY: release-finish
release-finish: release-copy release-check
.PHONY: docs-copy
docs-copy:
mkdir -p $(HUGO); \
mkdir -p $(HUGO)/content/extensions; \
cd $(HUGO); \
git init; \
git remote rm origin; \
git remote add origin https://github.com/owncloud/owncloud.github.io; \
git fetch; \
git checkout origin/source -f; \
rsync --delete -ax ../docs/ content/extensions/$(NAME)
.PHONY: config-docs-generate
config-docs-generate:
go run github.com/owncloud/flaex >| docs/configuration.md
.PHONY: docs-build
docs-build:
cd $(HUGO); hugo
.PHONY: docs
docs: config-docs-generate docs-copy docs-build
go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md
.PHONY: watch
watch:

8
proxy/docs/_index.md
View File

@@ -1,8 +0,0 @@
---
title: Proxy
geekdocRepo: https://github.com/owncloud/ocis-proxy
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
This service provides a basic proxy in front of the public ocis services.

10
proxy/docs/about.md
View File

@@ -1,10 +0,0 @@
---
title: "About"
date: 2020-02-07T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-proxy
geekdocEditPath: edit/master/docs
geekdocFilePath: about.md
---
This service provides an proxy service that routes requests to the correct services.

28
proxy/docs/building.md
View File

@@ -1,28 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis-proxy
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
{{< toc >}}
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.13. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis-proxy.git
cd ocis-proxy
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
## Backend
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
Finally you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis-proxy -h` to see all available options and subcommands.

134
proxy/docs/configuration.md
View File

@@ -1,134 +0,0 @@
---
title: "Configuration"
date: "2020-09-21T13:14:51+0200"
weight: 20
geekdocRepo: https://github.com/owncloud/ocis-proxy
geekdocEditPath: edit/master/docs
geekdocFilePath: configuration.md
---
{{< toc >}}
## Configuration
oCIS Single Binary is not responsible for configuring extensions. Instead, each extension could either be configured by environment variables, cli flags or config files.
Each extension has its dedicated documentation page (e.g. https://owncloud.github.io/extensions/ocis_proxy/configuration) which lists all possible configurations. Config files and environment variables are picked up if you use the `./bin/ocis server` command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
### Configuration using config files
Out of the box extensions will attempt to read configuration details from:
```console
/etc/ocis
$HOME/.ocis
./config
```
For this configuration to be picked up, have a look at your extension `root` command and look for which default config name it has assigned. *i.e: ocis-proxy reads `proxy.json | yaml | toml ...`*.
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/ocis.yml`, `${HOME}/.ocis/ocis.yml` or `$(pwd)/config/ocis.yml`.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below. Command line flags are only working when calling the subcommand directly.
## Root Command
proxy for Reva/oCIS
Usage: `ocis-proxy [global options] command [command options] [arguments...]`
--config-file | $PROXY_CONFIG_FILE
: Path to config file.
--log-level | $PROXY_LOG_LEVEL
: Set logging level. Default: `info`.
--log-pretty | $PROXY_LOG_PRETTY
: Enable pretty logging. Default: `true`.
--log-color | $PROXY_LOG_COLOR
: Enable colored logging. Default: `true`.
## Sub Commands
### ocis-proxy health
Check health status
Usage: `ocis-proxy health [command options] [arguments...]`
--debug-addr | $PROXY_DEBUG_ADDR
: Address to debug endpoint. Default: `0.0.0.0:9109`.
### ocis-proxy server
Start integrated server
Usage: `ocis-proxy server [command options] [arguments...]`
--tracing-enabled | $PROXY_TRACING_ENABLED
: Enable sending traces.
--tracing-type | $PROXY_TRACING_TYPE
: Tracing backend type. Default: `jaeger`.
--tracing-endpoint | $PROXY_TRACING_ENDPOINT
: Endpoint for the agent.
--tracing-collector | $PROXY_TRACING_COLLECTOR
: Endpoint for the collector. Default: `http://localhost:14268/api/traces`.
--tracing-service | $PROXY_TRACING_SERVICE
: Service name for tracing. Default: `proxy`.
--debug-addr | $PROXY_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9205`.
--debug-token | $PROXY_DEBUG_TOKEN
: Token to grant metrics access.
--debug-pprof | $PROXY_DEBUG_PPROF
: Enable pprof debugging.
--debug-zpages | $PROXY_DEBUG_ZPAGES
: Enable zpages debugging.
--http-addr | $PROXY_HTTP_ADDR
: Address to bind http server. Default: `0.0.0.0:9200`.
--http-root | $PROXY_HTTP_ROOT
: Root path of http server. Default: `/`.
--asset-path | $PROXY_ASSET_PATH
: Path to custom assets.
--http-namespace | $PROXY_HTTP_NAMESPACE
: Set the base namespace for the http namespace. Default: `com.owncloud`.
--transport-tls-cert | $PROXY_TRANSPORT_TLS_CERT
: Certificate file for transport encryption.
--transport-tls-key | $PROXY_TRANSPORT_TLS_KEY
: Secret file for transport encryption.
--tls | $PROXY_TLS
: Use TLS (disable only if proxy is behind a TLS-terminating reverse-proxy).. Default: `true`.
--jwt-secret | $PROXY_JWT_SECRET
: Used to create JWT to talk to reva, should equal reva's jwt-secret. Default: `Pive-Fumkiu4`.
--reva-gateway-addr | $PROXY_REVA_GATEWAY_ADDR
: REVA Gateway Endpoint. Default: `127.0.0.1:9142`.
--oidc-issuer | $PROXY_OIDC_ISSUER
: OIDC issuer. Default: `https://localhost:9200`.
--oidc-insecure | $PROXY_OIDC_INSECURE
: OIDC allow insecure communication. Default: `true`.

46
proxy/docs/getting-started.md
View File

@@ -1,46 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 15
geekdocRepo: https://github.com/owncloud/ocis-proxy
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
Docker images for ocis-reva are hosted on https://hub.docker.com/r/owncloud/ocis-proxy.
The `latest` tag always reflects the current master branch.
```console
docker pull owncloud/ocis-proxy
```
### Binaries
The pre-built binaries for different platforms are downloadable at https://download.owncloud.com/ocis/ocis-proxy/ . Specific releases are organized in separate folders. They are in sync which every release tag on GitHub. The binaries from the current master branch can be found in https://download.owncloud.com/ocis/ocis-proxy/testing/
```console
curl https://download.owncloud.com/ocis/ocis-proxy/1.0.0-beta1/ocis-proxy-1.0.0-beta1-darwin-amd64 --output ocis-proxy
chmod +x ocis-proxy
./ocis-proxy server
```
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis-proxy --help`.
### Server
The server command is used to start the http server. For further help please execute:
{{< highlight txt >}}
ocis-proxy server --help
{{< / highlight >}}

24
settings/Makefile
View File

@@ -3,7 +3,6 @@ NAME := settings
IMPORT := github.com/owncloud/ocis/$(NAME)
BIN := bin
DIST := dist
HUGO := hugo
PROTO_VERSION := v0
PROTO_SRC := pkg/proto/$(PROTO_VERSION)
@@ -62,7 +61,7 @@ sync:
.PHONY: clean
clean:
go clean -i ./...
rm -rf $(BIN) $(DIST) $(HUGO)
rm -rf $(BIN) $(DIST)
.PHONY: fmt
fmt:
@@ -139,24 +138,9 @@ release-finish: release-copy release-check
test-acceptance-webui:
./ui/tests/run-acceptance-test.sh $(FEATURE_PATH)
.PHONY: docs-copy
docs-copy:
mkdir -p $(HUGO); \
mkdir -p $(HUGO)/content/extensions; \
cd $(HUGO); \
git init; \
git remote rm origin; \
git remote add origin https://github.com/owncloud/owncloud.github.io; \
git fetch; \
git checkout origin/source -f; \
rsync --delete -ax ../docs/ content/extensions/$(NAME)
.PHONY: docs-build
docs-build:
cd $(HUGO); hugo
.PHONY: docs
docs: docs-copy docs-build
.PHONY: config-docs-generate
config-docs-generate:
# go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md
.PHONY: watch
watch:

46
settings/docs/_index.md
View File

@@ -1,46 +0,0 @@
---
title: "Settings"
date: 2018-05-02T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-settings
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
## Abstract
When using oCIS, the requirement to store settings arises. This extension provides functionality
for other extensions to register new settings within oCIS. It is responsible for storing the respective
settings values as well.
For ease of use, this extension provides an ocis-web extension which allows users to change their settings values.
Please refer to the [ocis-web extension docs](https://owncloud.github.io/ocis/extensions/#external-phoenix-apps)
for running ocis-web extensions.
{{< mermaid class="text-center">}}
graph TD
subgraph ow[ocis-web]
ows[ocis-web-settings]
owc[ocis-web-core]
end
ows ---|"listSettingsBundles(),<br>saveSettingsValue(value)"| os[ocis-settings]
owc ---|"listSettingsValues()"| sdk[oC SDK]
sdk --- sdks{ocis-settings<br>available?}
sdks ---|"yes"| os
sdks ---|"no"| defaults[Use set of<br>default values]
oa[oCIS extensions<br>e.g. ocis-accounts] ---|"saveSettingsBundle(bundle)"| os
{{< /mermaid >}}
The diagram shows how the settings service integrates into oCIS:
**Settings management:**
- oCIS extensions can register *settings bundles* with the ocis-settings service.
- The settings frontend can be plugged into ocis-web, showing forms for changing *settings values* as a user.
The forms are generated from the registered *settings bundles*.
**Settings usage:**
- Extensions can query ocis-settings for *settings values* of a user.
- The ownCloud SDK, used as a data abstraction layer for ocis-web, will query ocis-settings for *settings values* of a user,
if it's available. The SDK uses sensible defaults when ocis-settings is not part of the setup.
For compatibility with ownCloud 10, a migration of ownCloud 10 settings into the storage of ocis-settings will be available.

28
settings/docs/building.md
View File

@@ -1,28 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis-settings
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
{{< toc >}}
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.12. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis-settings.git
cd ocis-settings
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
## Backend
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
Finally you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis-settings -h` to see all available options.

75
settings/docs/bundles.md
View File

@@ -1,75 +0,0 @@
---
title: "Settings Bundles"
date: 2020-05-04T00:00:00+00:00
weight: 50
geekdocRepo: https://github.com/owncloud/ocis-settings
geekdocEditPath: edit/master/docs
geekdocFilePath: bundles.md
---
A **Settings Bundle** is a collection of settings, uniquely identified by the key of the
extension registering the bundle and the key of the bundle itself. It's purpose is to let
oCIS extensions define settings and make them available to users. They are dynamically
rendered into forms, available in the frontend.
As of now we support five different types of settings:
- boolean
- integer
- string
- single choice list of integers or strings
- multiple choice list of integers or strings
Each **Setting** is uniquely identified by a key within the bundle. Some attributes
depend on the chosen type of setting. Through the information provided with the
attributes of the setting, the settings frontend dynamically renders form elements,
allowing users to change their settings individually.
## Example
```json
{
"identifier": {
"extension": "ocis-accounts",
"bundleKey": "profile"
},
"displayName": "Profile",
"settings": [
{
"settingKey": "lastname",
"displayName": "Lastname",
"description": "Input for lastname",
"stringValue": {
"placeholder": "Set lastname"
}
},
{
"settingKey": "age",
"displayName": "Age",
"description": "Input for age",
"intValue": {
"min": "16",
"max": "200",
"step": "2",
"placeholder": "Set age"
}
},
{
"settingKey": "timezone",
"displayName": "Timezone",
"description": "User timezone",
"singleChoiceValue": {
"options": [
{
"stringValue": "Europe/Berlin",
"displayValue": "Europe/Berlin"
},
{
"stringValue": "Asia/Kathmandu",
"displayValue": "Asia/Kathmandu"
}
]
}
}
]
}
```

272
settings/docs/getting-started.md
View File

@@ -1,272 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 25
geekdocRepo: https://github.com/owncloud/ocis-settings
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
TBD
### Binaries
TBD
## Configuration
We provide overall three different variants of configuration. The variant based on environment variables and commandline flags are split up into global values and command-specific values.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
#### Global
SETTINGS_CONFIG_FILE
: Path to config file, empty default value
SETTINGS_LOG_LEVEL
: Set logging level, defaults to `info`
SETTINGS_LOG_COLOR
: Enable colored logging, defaults to `true`
SETTINGS_LOG_PRETTY
: Enable pretty logging, defaults to `true`
#### Server
SETTINGS_TRACING_ENABLED
: Enable sending traces, defaults to `false`
SETTINGS_TRACING_TYPE
: Tracing backend type, defaults to `jaeger`
SETTINGS_TRACING_ENDPOINT
: Endpoint for the agent, empty default value
SETTINGS_TRACING_COLLECTOR
: Endpoint for the collector, empty default value
SETTINGS_TRACING_SERVICE
: Service name for tracing, defaults to `settings`
SETTINGS_DEBUG_ADDR
: Address to bind debug server, defaults to `0.0.0.0:9194`
SETTINGS_DEBUG_TOKEN
: Token to grant metrics access, empty default value
SETTINGS_DEBUG_PPROF
: Enable pprof debugging, defaults to `false`
SETTINGS_DEBUG_ZPAGES
: Enable zpages debugging, defaults to `false`
SETTINGS_HTTP_ADDR
: Address to bind http server, defaults to `0.0.0.0:9190`
SETTINGS_HTTP_NAMESPACE
: The http namespace
SETTINGS_HTTP_ROOT
: Root path of http server, defaults to `/`
#### Health
SETTINGS_DEBUG_ADDR
: Address to debug endpoint, defaults to `0.0.0.0:9194`
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below.
#### Global
--config-file
: Path to config file, empty default value
--log-level
: Set logging level, defaults to `info`
--log-color
: Enable colored logging, defaults to `true`
--log-pretty
: Enable pretty logging, defaults to `true`
#### Server
--tracing-enabled
: Enable sending traces, defaults to `false`
--tracing-type
: Tracing backend type, defaults to `jaeger`
--tracing-endpoint
: Endpoint for the agent, empty default value
--tracing-collector
: Endpoint for the collector, empty default value
--tracing-service
: Service name for tracing, defaults to `settings`
--debug-addr
: Address to bind debug server, defaults to `0.0.0.0:9194`
--debug-token
: Token to grant metrics access, empty default value
--debug-pprof
: Enable pprof debugging, defaults to `false`
--debug-zpages
: Enable zpages debugging, defaults to `false`
--http-addr
: Address to bind http server, defaults to `0.0.0.0:9190`
--http-namespace
: Namespace for internal services communication, defaults to `com.owncloud.web`
--http-root
: Root path of http server, defaults to `/`
#### Health
--debug-addr
: Address to debug endpoint, defaults to `0.0.0.0:9194`
### Configuration file
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/settings/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/settings.yml`, `${HOME}/.ocis/settings.yml` or `$(pwd)/config/settings.yml`.
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis-settings --help`.
### Server
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{< highlight txt >}}
ocis-settings server --help
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
ocis-settings health --help
{{< / highlight >}}
## Metrics
This service provides some [Prometheus](https://prometheus.io/) metrics through the debug endpoint, you can optionally secure the metrics endpoint by some random token, which got to be configured through one of the flag `--debug-token` or the environment variable `SETTINGS_DEBUG_TOKEN` mentioned above. By default the metrics endpoint is bound to `http://0.0.0.0:9194/metrics`.
go_gc_duration_seconds
: A summary of the GC invocation durations
go_gc_duration_seconds_sum
: A summary of the GC invocation durations
go_gc_duration_seconds_count
: A summary of the GC invocation durations
go_goroutines
: Number of goroutines that currently exist
go_info
: Information about the Go environment
go_memstats_alloc_bytes
: Number of bytes allocated and still in use
go_memstats_alloc_bytes_total
: Total number of bytes allocated, even if freed
go_memstats_buck_hash_sys_bytes
: Number of bytes used by the profiling bucket hash table
go_memstats_frees_total
: Total number of frees
go_memstats_gc_cpu_fraction
: The fraction of this program's available CPU time used by the GC since the program started
go_memstats_gc_sys_bytes
: Number of bytes used for garbage collection system metadata
go_memstats_heap_alloc_bytes
: Number of heap bytes allocated and still in use
go_memstats_heap_idle_bytes
: Number of heap bytes waiting to be used
go_memstats_heap_inuse_bytes
: Number of heap bytes that are in use
go_memstats_heap_objects
: Number of allocated objects
go_memstats_heap_released_bytes
: Number of heap bytes released to OS
go_memstats_heap_sys_bytes
: Number of heap bytes obtained from system
go_memstats_last_gc_time_seconds
: Number of seconds since 1970 of last garbage collection
go_memstats_lookups_total
: Total number of pointer lookups
go_memstats_mallocs_total
: Total number of mallocs
go_memstats_mcache_inuse_bytes
: Number of bytes in use by mcache structures
go_memstats_mcache_sys_bytes
: Number of bytes used for mcache structures obtained from system
go_memstats_mspan_inuse_bytes
: Number of bytes in use by mspan structures
go_memstats_mspan_sys_bytes
: Number of bytes used for mspan structures obtained from system
go_memstats_next_gc_bytes
: Number of heap bytes when next garbage collection will take place
go_memstats_other_sys_bytes
: Number of bytes used for other system allocations
go_memstats_stack_inuse_bytes
: Number of bytes in use by the stack allocator
go_memstats_stack_sys_bytes
: Number of bytes obtained from system for stack allocator
go_memstats_sys_bytes
: Number of bytes obtained from system
go_threads
: Number of OS threads created
promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code

42
settings/docs/glossary.md
View File

@@ -1,42 +0,0 @@
---
title: "Glossary"
date: 2020-05-04T12:35:00+01:00
weight: 80
geekdocRepo: https://github.com/owncloud/ocis-settings
geekdocEditPath: edit/master/docs
geekdocFilePath: glossary.md
---
In the context of this extension and oCIS in general, we are using the following terminology.
### Configuration
- System configuration
- e.g. service host names and ports
- Changes need to be propagated to other services
- Typically modified on the CLI
### Settings
- Application level settings
- e.g. default language
- Can be modified at runtime without restarting the service
- Typically modified in the UI
### Preferences
- User settings
- Subset of "Settings"
- e.g. preferred language of a user
### Settings Bundle
- Collection of related settings
- Registered by an ocis extension
### Settings Value
- Manifestation of a setting for a specific user
- E.g. used for customization (at runtime) in `ocis-web`
- `ocis-web-settings` extension for modifying settings values is provided by this service
- Can be queried and modified by other ocis extensions

10
settings/docs/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2018-05-02T00:00:00+00:00
weight: 90
geekdocRepo: https://github.com/owncloud/ocis-settings
geekdocEditPath: edit/master/docs
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/settings/blob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

75
settings/docs/values.md
View File

@@ -1,75 +0,0 @@
---
title: "Settings Values"
date: 2020-05-04T00:00:00+00:00
weight: 51
geekdocRepo: https://github.com/owncloud/ocis-settings
geekdocEditPath: edit/master/docs
geekdocFilePath: values.md
---
A **Settings Value** is the value an authenticated user has chosen for a specific setting, defined in a
*settings bundle*. For choosing settings values as a user the sole entry point is the ocis-web extension
provided by this service.
## Identifying settings values
A *settings value* is uniquely identified by four attributes. Three of them are coming from the definition of
the setting within it's settings bundle (see [Settings Bundles](https://owncloud.github.io/extensions/ocis_settings/bundles/)
for an example). The fourth identifies the user.
- extension: Key of the extension that registered the settings bundle,
- bundleKey: Key of the settings bundle,
- settingKey: Key of the setting as defined within the bundle,
- accountUuid: The UUID of the authenticated user who has saved the setting.
{{< hint info >}}
When requests are going through `ocis-proxy`, the accountUuid attribute can be set to the static keyword `me`
instead of using a real UUID. `ocis-proxy` will take care of minting the UUID of the authenticated user into
a JWT, providing it in the HTTP header as `x-access-token`. That UUID is then used in this service, to replace
`me` with the actual UUID of the authenticated user.
{{< /hint >}}
## Example of stored settings values
```json
{
"values": {
"language": {
"identifier": {
"extension": "ocis-accounts",
"bundleKey": "profile",
"settingKey": "language",
"accountUuid": "5681371f-4a6e-43bc-8bb5-9c9237fa9c58"
},
"listValue": {
"values": [
{
"stringValue": "de"
}
]
}
},
"timezone": {
"identifier": {
"extension": "ocis-accounts",
"bundleKey": "profile",
"settingKey": "timezone",
"accountUuid": "5681371f-4a6e-43bc-8bb5-9c9237fa9c58"
},
"listValue": {
"values": [
{
"stringValue": "Europe/Berlin"
}
]
}
}
}
}
```
## gRPC endpoints
The obvious way of modifying settings is the ocis-web extension, as described earlier. However, services can
use the respective gRPC endpoints of the `ValueService` to query and modify *settings values* as well.
The gRPC endpoints require the same identifier attributes as described above, so for making a request to
the `ValueService` you will have to make sure that the accountUuid of the authenticated user is available in
your service at the time of the request.

24
store/Makefile
View File

@@ -3,7 +3,6 @@ NAME := store
IMPORT := github.com/owncloud/ocis/$(NAME)
BIN := bin
DIST := dist
HUGO := hugo
ifeq ($(OS), Windows_NT)
EXECUTABLE := $(NAME).exe
@@ -59,7 +58,7 @@ sync:
.PHONY: clean
clean:
go clean -i ./...
rm -rf $(BIN) $(DIST) $(HUGO)
rm -rf $(BIN) $(DIST)
.PHONY: fmt
fmt:
@@ -132,24 +131,9 @@ release-check:
.PHONY: release-finish
release-finish: release-copy release-check
.PHONY: docs-copy
docs-copy:
mkdir -p $(HUGO); \
mkdir -p $(HUGO)/content/extensions; \
cd $(HUGO); \
git init; \
git remote rm origin; \
git remote add origin https://github.com/owncloud/owncloud.github.io; \
git fetch; \
git checkout origin/source -f; \
rsync --delete -ax ../docs/ content/extensions/$(NAME)
.PHONY: docs-build
docs-build:
cd $(HUGO); hugo
.PHONY: docs
docs: docs-copy docs-build
.PHONY: config-docs-generate
config-docs-generate:
# go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md
.PHONY: watch
watch:

10
store/docs/_index.md
View File

@@ -1,10 +0,0 @@
---
title: "Store"
date: 2018-05-02T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-store
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
This service provides ...

28
store/docs/building.md
View File

@@ -1,28 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis-store
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
{{< toc >}}
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.12. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis-store.git
cd ocis-store
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
## Backend
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
Finally you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis-store -h` to see all available options.

272
store/docs/getting-started.md
View File

@@ -1,272 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 20
geekdocRepo: https://github.com/owncloud/ocis-store
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
TBD
### Binaries
TBD
## Configuration
We provide overall three different variants of configuration. The variant based on environment variables and commandline flags are split up into global values and command-specific values.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
#### Global
STORE_CONFIG_FILE
: Path to config file, empty default value
STORE_LOG_LEVEL
: Set logging level, defaults to `info`
STORE_LOG_COLOR
: Enable colored logging, defaults to `true`
STORE_LOG_PRETTY
: Enable pretty logging, defaults to `true`
#### Server
STORE_TRACING_ENABLED
: Enable sending traces, defaults to `false`
STORE_TRACING_TYPE
: Tracing backend type, defaults to `jaeger`
STORE_TRACING_ENDPOINT
: Endpoint for the agent, empty default value
STORE_TRACING_COLLECTOR
: Endpoint for the collector, empty default value
STORE_TRACING_SERVICE
: Service name for tracing, defaults to `store`
STORE_DEBUG_ADDR
: Address to bind debug server
STORE_DEBUG_TOKEN
: Token to grant metrics access, empty default value
STORE_DEBUG_PPROF
: Enable pprof debugging, defaults to `false`
STORE_DEBUG_ZPAGES
: Enable zpages debugging, defaults to `false`
STORE_HTTP_ADDR
: Address to bind http server, defaults to `0.0.0.0:9461`
STORE_HTTP_NAMESPACE
: The http namespace
STORE_HTTP_ROOT
: Root path of http server, defaults to `/`
#### Health
STORE_DEBUG_ADDR
: Address to debug endpoint
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below.
#### Global
--config-file
: Path to config file, empty default value
--log-level
: Set logging level, defaults to `info`
--log-color
: Enable colored logging, defaults to `true`
--log-pretty
: Enable pretty logging, defaults to `true`
#### Server
--tracing-enabled
: Enable sending traces, defaults to `false`
--tracing-type
: Tracing backend type, defaults to `jaeger`
--tracing-endpoint
: Endpoint for the agent, empty default value
--tracing-collector
: Endpoint for the collector, empty default value
--tracing-service
: Service name for tracing, defaults to `store`
--debug-addr
: Address to bind debug server
--debug-token
: Token to grant metrics access, empty default value
--debug-pprof
: Enable pprof debugging, defaults to `false`
--debug-zpages
: Enable zpages debugging, defaults to `false`
--http-addr
: Address to bind http server, defaults to `0.0.0.0:9461`
--http-namespace
: Namespace for internal services communication, defaults to `com.owncloud.web`
--http-root
: Root path of http server, defaults to `/`
#### Health
--debug-addr
: Address to debug endpoint
### Configuration file
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/store/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/store.yml`, `${HOME}/.ocis/store.yml` or `$(pwd)/config/store.yml`.
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis-store --help`.
### Server
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{< highlight txt >}}
ocis-store server --help
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
ocis-store health --help
{{< / highlight >}}
## Metrics
This service provides some [Prometheus](https://prometheus.io/) metrics through the debug endpoint, you can optionally secure the metrics endpoint by some random token, which got to be configured through one of the flag `--debug-token` or the environment variable `STORE_DEBUG_TOKEN` mentioned above. By default the metrics endpoint is bound to `http://0.0.0.0:9460/metrics`.
go_gc_duration_seconds
: A summary of the GC invocation durations
go_gc_duration_seconds_sum
: A summary of the GC invocation durations
go_gc_duration_seconds_count
: A summary of the GC invocation durations
go_goroutines
: Number of goroutines that currently exist
go_info
: Information about the Go environment
go_memstats_alloc_bytes
: Number of bytes allocated and still in use
go_memstats_alloc_bytes_total
: Total number of bytes allocated, even if freed
go_memstats_buck_hash_sys_bytes
: Number of bytes used by the profiling bucket hash table
go_memstats_frees_total
: Total number of frees
go_memstats_gc_cpu_fraction
: The fraction of this program's available CPU time used by the GC since the program started
go_memstats_gc_sys_bytes
: Number of bytes used for garbage collection system metadata
go_memstats_heap_alloc_bytes
: Number of heap bytes allocated and still in use
go_memstats_heap_idle_bytes
: Number of heap bytes waiting to be used
go_memstats_heap_inuse_bytes
: Number of heap bytes that are in use
go_memstats_heap_objects
: Number of allocated objects
go_memstats_heap_released_bytes
: Number of heap bytes released to OS
go_memstats_heap_sys_bytes
: Number of heap bytes obtained from system
go_memstats_last_gc_time_seconds
: Number of seconds since 1970 of last garbage collection
go_memstats_lookups_total
: Total number of pointer lookups
go_memstats_mallocs_total
: Total number of mallocs
go_memstats_mcache_inuse_bytes
: Number of bytes in use by mcache structures
go_memstats_mcache_sys_bytes
: Number of bytes used for mcache structures obtained from system
go_memstats_mspan_inuse_bytes
: Number of bytes in use by mspan structures
go_memstats_mspan_sys_bytes
: Number of bytes used for mspan structures obtained from system
go_memstats_next_gc_bytes
: Number of heap bytes when next garbage collection will take place
go_memstats_other_sys_bytes
: Number of bytes used for other system allocations
go_memstats_stack_inuse_bytes
: Number of bytes in use by the stack allocator
go_memstats_stack_sys_bytes
: Number of bytes obtained from system for stack allocator
go_memstats_sys_bytes
: Number of bytes obtained from system
go_threads
: Number of OS threads created
promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code

10
store/docs/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2018-05-02T00:00:00+00:00
weight: 40
geekdocRepo: https://github.com/owncloud/ocis-store
geekdocEditPath: edit/master/docs
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/store/blob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

24
thumbnails/Makefile
View File

@@ -3,7 +3,6 @@ NAME := thumbnails
IMPORT := github.com/owncloud/ocis/$(NAME)
BIN := bin
DIST := dist
HUGO := hugo
PROTO_VERSION := v0
PROTO_SRC := pkg/proto/$(PROTO_VERSION)/*.proto
@@ -60,7 +59,7 @@ sync:
.PHONY: clean
clean:
go clean -i ./...
rm -rf $(BIN) $(DIST) $(HUGO)
rm -rf $(BIN) $(DIST)
.PHONY: fmt
fmt:
@@ -138,24 +137,9 @@ proto-docs-build:
go get -u github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc; \
protoc --doc_out=./docs --doc_opt=./templates/grpc.tmpl,grpc.md $(PROTO_SRC)
.PHONY: docs-copy
docs-copy:
mkdir -p $(HUGO); \
mkdir -p $(HUGO)/content/extensions; \
cd $(HUGO); \
git init; \
git remote rm origin; \
git remote add origin https://github.com/owncloud/owncloud.github.io; \
git fetch; \
git checkout origin/source -f; \
rsync --delete -ax ../docs/ content/extensions/$(NAME)
.PHONY: docs-build
docs-build:
cd $(HUGO); hugo
.PHONY: docs
docs: proto-docs-build docs-copy docs-build
.PHONY: config-docs-generate
config-docs-generate:
# go run github.com/owncloud/flaex >| ../docs/extensions/$(NAME)/configuration.md
.PHONY: watch
watch:

10
thumbnails/docs/_index.md
View File

@@ -1,10 +0,0 @@
---
title: "Thumbnails"
date: 2018-05-02T00:00:00+00:00
weight: 10
geekdocRepo: https://github.com/owncloud/ocis-thumbnails
geekdocEditPath: edit/master/docs
geekdocFilePath: _index.md
---
This service provides an ocis extensions which generates thumbnails for image files.

28
thumbnails/docs/building.md
View File

@@ -1,28 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis-thumbnails
geekdocEditPath: edit/master/docs
geekdocFilePath: building.md
---
{{< toc >}}
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.12. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis-thubnails.git
cd {{ Name }}
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `Gopath`. All commands to build this project are part of our `Makefile`.
## Backend
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
Finally you should have the binary within the `bin/` folder now, give it a try with `./bin/ocis-thumbnails -h` to see all available options.

290
thumbnails/docs/getting-started.md
View File

@@ -1,290 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 20
geekdocRepo: https://github.com/owncloud/ocis-thumbnails
geekdocEditPath: edit/master/docs
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
TBD
### Binaries
TBD
## Configuration
We provide overall three different variants of configuration. The variant based on environment variables and commandline flags are split up into global values and command-specific values.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
#### Global
THUMBNAILS_CONFIG_FILE
: Path to config file, empty default value
THUMBNAILS_LOG_LEVEL
: Set logging level, defaults to `info`
THUMBNAILS_LOG_COLOR
: Enable colored logging, defaults to `true`
THUMBNAILS_LOG_PRETTY
: Enable pretty logging, defaults to `true`
#### Server
THUMBNAILS_TRACING_ENABLED
: Enable sending traces, defaults to `false`
THUMBNAILS_TRACING_TYPE
: Tracing backend type, defaults to `jaeger`
THUMBNAILS_TRACING_ENDPOINT
: Endpoint for the agent, empty default value
THUMBNAILS_TRACING_COLLECTOR
: Endpoint for the collector, empty default value
THUMBNAILS_TRACING_SERVICE
: Service name for tracing, defaults to `ocis-thumbnails`
THUMBNAILS_DEBUG_ADDR
: Address to bind debug server, defaults to `0.0.0.0:9114`
THUMBNAILS_DEBUG_TOKEN
: Token to grant metrics access, empty default value
THUMBNAILS_DEBUG_PPROF
: Enable pprof debugging, defaults to `false`
THUMBNAILS_DEBUG_ZPAGES
: Enable zpages debugging, defaults to `false`
THUMBNAILS_GRPC_NAME
: Name of the service, defaults to `thumbnails`
THUMBNAILS_GRPC_ADDR
: Address to bind grpc server, defaults to `0.0.0.0:9185`
THUMBNAILS_GRPC_NAMESPACE
: Set the base namespace for the grpc namespace", defaults to `com.owncloud.api`
THUMBNAILS_FILESYSTEMSTORAGE_ROOT
: Root path of the filesystem storage directory, defaults to `<os tempdir>/ocis-thumbnails/`
THUMBNAILS_WEBDAVSOURCE_BASEURL
: Base url for a webdav api, defaults to `https://localhost:9200/remote.php/webdav/`
THUMBNAILS_RESOLUTIONS
: List of resolutions supported by the service, defaults to `["16x16", "32x32", "64x64", "128x128"]
#### Health
THUMBNAILS_DEBUG_ADDR
: Address to debug endpoint, defaults to `0.0.0.0:9189`
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below.
#### Global
--config-file
: Path to config file, empty default value
--log-level
: Set logging level, defaults to `info`
--log-color
: Enable colored logging, defaults to `true`
--log-pretty
: Enable pretty logging, defaults to `true`
#### Server
--tracing-enabled
: Enable sending traces, defaults to `false`
--tracing-type
: Tracing backend type, defaults to `jaeger`
--tracing-endpoint
: Endpoint for the agent, empty default value
--tracing-collector
: Endpoint for the collector, empty default value
--tracing-service
: Service name for tracing, defaults to `thumbnails`
--debug-addr
: Address to bind debug server, defaults to `0.0.0.0:9189`
--debug-token
: Token to grant metrics access, empty default value
--debug-pprof
: Enable pprof debugging, defaults to `false`
--debug-zpages
: Enable zpages debugging, defaults to `false`
--grpc-name
: Name of the service, defaults to `thumbnails`
--grpc-addr
: Address to bind grpc server, defaults to `0.0.0.0:9185`
--grpc-namespace
: Set the base namespace for the grpc namespace", defaults to `com.owncloud.api`
--filesystemstorage-root
: Root path of the filesystem storage directory, defaults to `<os tempdir>/ocis-thumbnails/`
--webdavsource-baseurl
: Base url for a webdav api, defaults to `https://localhost:9200/remote.php/webdav/`
--thumbnail-resolution
: List of resolutions supported by the service, defaults to `["16x16", "32x32", "64x64", "128x128"]
#### Health
--debug-addr
: Address to debug endpoint, defaults to `0.0.0.0:9189`
### Configuration file
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/thumbnails/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/thumbnails.yml`, `${HOME}/.ocis/thumbnails.yml` or `$(pwd)/config/thumbnails.yml`.
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formated help output if you execute the binary via `ocis-thumbnails --help`.
### Server
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{< highlight txt >}}
{{ Name }} server --help
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
{{ Name }} health --help
{{< / highlight >}}
## Metrics
This service provides some [Prometheus](https://prometheus.io/) metrics through the debug endpoint, you can optionally secure the metrics endpoint by some random token, which got to be configured through one of the flag `--debug-token` or the environment variable `THUMBNAILS_DEBUG_TOKEN` mentioned above. By default the metrics endpoint is bound to `http://0.0.0.0:9114/metrics`.
go_gc_duration_seconds
: A summary of the GC invocation durations
go_gc_duration_seconds_sum
: A summary of the GC invocation durations
go_gc_duration_seconds_count
: A summary of the GC invocation durations
go_goroutines
: Number of goroutines that currently exist
go_info
: Information about the Go environment
go_memstats_alloc_bytes
: Number of bytes allocated and still in use
go_memstats_alloc_bytes_total
: Total number of bytes allocated, even if freed
go_memstats_buck_hash_sys_bytes
: Number of bytes used by the profiling bucket hash table
go_memstats_frees_total
: Total number of frees
go_memstats_gc_cpu_fraction
: The fraction of this program's available CPU time used by the GC since the program started
go_memstats_gc_sys_bytes
: Number of bytes used for garbage collection system metadata
go_memstats_heap_alloc_bytes
: Number of heap bytes allocated and still in use
go_memstats_heap_idle_bytes
: Number of heap bytes waiting to be used
go_memstats_heap_inuse_bytes
: Number of heap bytes that are in use
go_memstats_heap_objects
: Number of allocated objects
go_memstats_heap_released_bytes
: Number of heap bytes released to OS
go_memstats_heap_sys_bytes
: Number of heap bytes obtained from system
go_memstats_last_gc_time_seconds
: Number of seconds since 1970 of last garbage collection
go_memstats_lookups_total
: Total number of pointer lookups
go_memstats_mallocs_total
: Total number of mallocs
go_memstats_mcache_inuse_bytes
: Number of bytes in use by mcache structures
go_memstats_mcache_sys_bytes
: Number of bytes used for mcache structures obtained from system
go_memstats_mspan_inuse_bytes
: Number of bytes in use by mspan structures
go_memstats_mspan_sys_bytes
: Number of bytes used for mspan structures obtained from system
go_memstats_next_gc_bytes
: Number of heap bytes when next garbage collection will take place
go_memstats_other_sys_bytes
: Number of bytes used for other system allocations
go_memstats_stack_inuse_bytes
: Number of bytes in use by the stack allocator
go_memstats_stack_sys_bytes
: Number of bytes obtained from system for stack allocator
go_memstats_sys_bytes
: Number of bytes obtained from system
go_threads
: Number of OS threads created
promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code

71
thumbnails/docs/grpc.md
View File

@@ -1,71 +0,0 @@
---
title: "GRPC API"
date: 2018-05-02T00:00:00+00:00
weight: 50
geekdocRepo: https://github.com/owncloud/ocis-thumbnails
geekdocEditPath: edit/master/docs
geekdocFilePath: grpc.md
---
{{< toc >}}
## pkg/proto/v0/thumbnails.proto
### GetRequest
A request to retrieve a thumbnail
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| filepath | [string](#string) | | The path to the source image |
| filetype | [GetRequest.FileType](#getrequestfiletype) | | The type to which the thumbnail should get encoded to. |
| etag | [string](#string) | | The etag of the source image |
| width | [int32](#int32) | | The width of the thumbnail |
| height | [int32](#int32) | | The height of the thumbnail |
| authorization | [string](#string) | | The authorization token |
### GetResponse
The service response
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| thumbnail | [bytes](#bytes) | | The thumbnail as a binary |
| mimetype | [string](#string) | | The mimetype of the thumbnail |
### GetRequest.FileType
The file types to which the thumbnail cna get encoded to.
| Name | Number | Description |
| ---- | ------ | ----------- |
| PNG | 0 | Represents PNG type |
| JPG | 1 | Represents JPG type |
### ThumbnailService
A Service for handling thumbnail generation
| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
| GetThumbnail | [GetRequest](#getrequest) | [GetResponse](#getresponse) | Generates the thumbnail and returns it. |
## Scalar Value Types
| .proto Type | Notes | C++ | Java |
| ----------- | ----- | --- | ---- |
| {{< div id="double" content="double" >}} | | double | double |
| {{< div id="float" content="float" >}} | | float | float |
| {{< div id="int32" content="int32" >}} | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int |
| {{< div id="int64" content="int64" >}} | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long |
| {{< div id="uint32" content="uint32" >}} | Uses variable-length encoding. | uint32 | int |
| {{< div id="uint64" content="uint64" >}} | Uses variable-length encoding. | uint64 | long |
| {{< div id="sint32" content="sint32" >}} | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int |
| {{< div id="sint64" content="sint64" >}} | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long |
| {{< div id="fixed32" content="fixed32" >}} | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int |
| {{< div id="fixed64" content="fixed64" >}} | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long |
| {{< div id="sfixed32" content="sfixed32" >}} | Always four bytes. | int32 | int |
| {{< div id="sfixed64" content="sfixed64" >}} | Always eight bytes. | int64 | long |
| {{< div id="bool" content="bool" >}} | | bool | boolean |
| {{< div id="string" content="string" >}} | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String |
| {{< div id="bytes" content="bytes" >}} | May contain any arbitrary sequence of bytes. | string | ByteString |

10
thumbnails/docs/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2018-05-02T00:00:00+00:00
weight: 60
geekdocRepo: https://github.com/owncloud/ocis-thumbnails
geekdocEditPath: edit/master/docs
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/thumbnails/blob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.

45
thumbnails/docs/releasing.md
View File

@@ -1,45 +0,0 @@
---
title: "Releasing"
date: 2018-05-02T00:00:00+00:00
weight: 40
geekdocRepo: https://github.com/owncloud/ocis-thumbnails
geekdocEditPath: edit/master/docs
geekdocFilePath: releasing.md
---
{{< toc >}}
To release a new version of ocis-thumbnails, you have to follow a few simple steps.
## Preperation
1. Create a new branch e.g. `release-x.x.x` where `x.x.x` is the version you want to release.
2. Checkout the preparation branch.
3. Create a new changelog folder and move the unreleased snippets there.
{{< highlight txt >}}
mkdir changelog/x.x.x_yyyy-MM-dd/ # yyyy-MM-dd is the current date
mv changelog/unreleased/* changelog/x.x.x_yyyy-MM-dd/
{{< / highlight >}}
4. Commit and push the changes
{{< highlight txt >}}
git add --all
git commit -m "prepare release x.x.x"
git push
{{< / highlight >}}
5. Create a pull request to the master branch.
## Release
1. After the preparation branch has been merged update your local master.
{{< highlight txt >}}
git checkout master
git pull
{{< / highlight >}}
2. Create a new tag (preferably signed).
{{< highlight txt >}}
git tag -s vx.x.x -m "release vx.x.x"
git push --tags
{{< / highlight >}}
3. Wait for CI and check that the GitHub release was published.
Congratulations, you just released ocis-thumbnails!