mirror/opencloud
1
0
Fork 0
mirror of https://github.com/opencloud-eu/opencloud.git synced 2026-02-28 17:38:59 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into update-bridge-docs

Browse Source
This commit is contained in:
A.Unger
2021-09-29 12:09:29 +02:00
parent 5554c72d9a f36017f188
commit 6166bb5a4c
511 changed files with 12687 additions and 17566 deletions
Download Patch File Download Diff File Expand all files Collapse all files

257
docs/extensions/graph/spaces.md Normal file
View File

@@ -0,0 +1,257 @@
---
title: Spaces
weight: 20
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/extensions/graph
geekdocFilePath: spaces.md
---
{{< toc >}}
## Graph Service
The Graph service is a reference implementation of the MS Graph API. There are no libraries doing any work only a set of routes and handlers.
## Spaces API
The Spaces API makes use of the [MS Graph API Drive resource](https://docs.microsoft.com/en-us/graph/api/resources/drive?view=graph-rest-1.0) to represent the concept of a Storage Space. Natively the MS Graph Specification [does not provide a way for creating Drives](https://docs.microsoft.com/en-us/graph/api/resources/drive?view=graph-rest-1.0#methods), as a Drive is a read only resource.
We circumvented this limitation by adding a `POST /drive/{drive-name}` to the Graph router. A major drawback of this solution is that this endpoint does not have support from the official MS Graph SDK, however it is reachable by any HTTP clients.
### Methods
```
POST /drive/{drive-name}
```
Calls to the following endpoint will create a Space with all the default parameters since we do not parse the request body just yet.
## Examples
We can now create a `Marketing` space and retrieve its WebDAV endpoint. Let's see how to do this.
### Starting conditions
This is the status of a DecomposedFS `users` tree. As we can see it is empty because we have not yet logged in with any users. It is a fresh new installation.
```
❯ tree -a /var/tmp/ocis/storage/users
/var/tmp/ocis/storage/users
├── blobs
├── nodes
│ └── root
├── spaces
│ ├── personal
│ └── share
├── trash
└── uploads
```
Let's start with creating a space:
`curl -k -X POST 'https://localhost:9200/graph/v1.0/drives/marketing' -u einstein:relativity -v`
```
❯ tree -a /var/tmp/ocis/storage/users
/var/tmp/ocis/storage/users
├── blobs
├── nodes
│ ├── 02dc1ec5-28b5-41c5-a48a-fabd4fa0562e
│ │ └── .space -> ../e85d185f-cdaa-4618-a312-e33ea435acfe
│ ├── 52efe3c2-c95a-47a1-8f3d-924aa473c711
│ ├── e85d185f-cdaa-4618-a312-e33ea435acfe
│ └── root
│ ├── 4c510ada-c86b-4815-8820-42cdf82c3d51 -> ../52efe3c2-c95a-47a1-8f3d-924aa473c711
│ └── c42debb8-926e-4a46-83b0-39dba56e59a4 -> ../02dc1ec5-28b5-41c5-a48a-fabd4fa0562e
├── spaces
│ ├── personal
│ │ └── 52efe3c2-c95a-47a1-8f3d-924aa473c711 -> ../../nodes/52efe3c2-c95a-47a1-8f3d-924aa473c711
│ ├── project
│ │ └── 02dc1ec5-28b5-41c5-a48a-fabd4fa0562e -> ../../nodes/02dc1ec5-28b5-41c5-a48a-fabd4fa0562e
│ └── share
├── trash
└── uploads
```
we can see that the `project` folder was added to the spaces as well as the `.space` folder to the space node `02dc1ec5-28b5-41c5-a48a-fabd4fa0562e`. For demonstration purposes, let's list the extended attributes of the new node:
```
xattr -l /var/tmp/ocis/storage/users/nodes/root/c42debb8-926e-4a46-83b0-39dba56e59a4
user.ocis.blobid:
user.ocis.blobsize: 0
user.ocis.name: c42debb8-926e-4a46-83b0-39dba56e59a4
user.ocis.owner.id: 4c510ada-c86b-4815-8820-42cdf82c3d51
user.ocis.owner.idp: https://localhost:9200
user.ocis.owner.type: primary
user.ocis.parentid: root
user.ocis.quota: 65536
user.ocis.space.name: marketing
```
As seen here it contains the metadata from the default list of requirements for this ticket.
Let's list the drive we just created using the graph API:
```
curl -k 'https://localhost:9200/graph/v1.0/me/drives' -u einstein:relativity -v | jq .value
[
{
"driveType": "personal",
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!52efe3c2-c95a-47a1-8f3d-924aa473c711",
"lastModifiedDateTime": "2021-09-07T14:42:39.025050471+02:00",
"name": "root",
"owner": {
"user": {
"id": "4c510ada-c86b-4815-8820-42cdf82c3d51"
}
},
"root": {
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!52efe3c2-c95a-47a1-8f3d-924aa473c711",
"webDavUrl": "https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157!52efe3c2-c95a-47a1-8f3d-924aa473c711"
}
},
{
"driveType": "project",
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e",
"lastModifiedDateTime": "2021-09-07T14:42:39.030705579+02:00",
"name": "root",
"owner": {
"user": {
"id": "4c510ada-c86b-4815-8820-42cdf82c3d51"
}
},
"quota": {
"total": 65536
},
"root": {
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e",
"webDavUrl": "https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e"
}
}
]
As we can see the response already contains a space-aware dav endpoint, which we can use to upload files to the space:
```
curl -k https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e/test.txt -X PUT -d "beep-sboop" -v -u einstein:relativity
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* upload completely sent off: 10 out of 10 bytes
< HTTP/1.1 201 Created
< Access-Control-Allow-Origin: *
< Content-Length: 0
< Content-Security-Policy: default-src 'none';
< Content-Type: text/plain
< Date: Tue, 07 Sep 2021 12:45:54 GMT
< Etag: "e2942565a4eb52e8754c2806f215fe93"
< Last-Modified: Tue, 07 Sep 2021 12:45:54 +0000
< Oc-Etag: "e2942565a4eb52e8754c2806f215fe93"
< Oc-Fileid: MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OmU4ZDIwNDA5LTE1OWItNGI2Ny1iODZkLTlkM2U3ZjYyYmM0ZQ==
< Vary: Origin
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Frame-Options: SAMEORIGIN
< X-Permitted-Cross-Domain-Policies: none
< X-Robots-Tag: none
< X-Xss-Protection: 1; mode=block
<
* Connection #0 to host localhost left intact
* Closing connection 0
```
This is the state after every transformation:
```
tree -a /var/tmp/ocis/storage/users
/var/tmp/ocis/storage/users
├── blobs
│ └── 83842d56-91de-41d5-8800-b2fb7b2d31cf
├── nodes
│ ├── 02dc1ec5-28b5-41c5-a48a-fabd4fa0562e
│ │ ├── .space -> ../e85d185f-cdaa-4618-a312-e33ea435acfe
│ │ └── test.txt -> ../e8d20409-159b-4b67-b86d-9d3e7f62bc4e
│ ├── 52efe3c2-c95a-47a1-8f3d-924aa473c711
│ ├── e85d185f-cdaa-4618-a312-e33ea435acfe
│ ├── e8d20409-159b-4b67-b86d-9d3e7f62bc4e
│ └── root
│ ├── 4c510ada-c86b-4815-8820-42cdf82c3d51 -> ../52efe3c2-c95a-47a1-8f3d-924aa473c711
│ └── c42debb8-926e-4a46-83b0-39dba56e59a4 -> ../02dc1ec5-28b5-41c5-a48a-fabd4fa0562e
├── spaces
│ ├── personal
│ │ └── 52efe3c2-c95a-47a1-8f3d-924aa473c711 -> ../../nodes/52efe3c2-c95a-47a1-8f3d-924aa473c711
│ ├── project
│ │ └── 02dc1ec5-28b5-41c5-a48a-fabd4fa0562e -> ../../nodes/02dc1ec5-28b5-41c5-a48a-fabd4fa0562e
│ └── share
├── trash
└── uploads
```
Observe the `test.txt` in the `02dc1ec5-28b5-41c5-a48a-fabd4fa0562e` node.
To finalize, verify the new created file is webdav-listable:
```xml
curl -k https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e -X PROPFIND -v -u einstein:relativity | xmllint --format -
<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
<d:response>
<d:href>/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e/</d:href>
<d:propstat>
<d:prop>
<oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjAyZGMxZWM1LTI4YjUtNDFjNS1hNDhhLWZhYmQ0ZmEwNTYyZQ==</oc:id>
<oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjAyZGMxZWM1LTI4YjUtNDFjNS1hNDhhLWZhYmQ0ZmEwNTYyZQ==</oc:fileid>
<d:getetag>"35a2ce5f56592d79d1b7233eff033347"</d:getetag>
<oc:permissions>RDNVCK</oc:permissions>
<d:resourcetype>
<d:collection/>
</d:resourcetype>
<oc:size>0</oc:size>
<d:getlastmodified>Tue, 07 Sep 2021 12:45:54 GMT</d:getlastmodified>
<oc:favorite>0</oc:favorite>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e/.space/</d:href>
<d:propstat>
<d:prop>
<oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OmU4NWQxODVmLWNkYWEtNDYxOC1hMzEyLWUzM2VhNDM1YWNmZQ==</oc:id>
<oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OmU4NWQxODVmLWNkYWEtNDYxOC1hMzEyLWUzM2VhNDM1YWNmZQ==</oc:fileid>
<d:getetag>"2e9a84bffce8b648ba626185800ee8fa"</d:getetag>
<oc:permissions>SRDNVCK</oc:permissions>
<d:resourcetype>
<d:collection/>
</d:resourcetype>
<oc:size>0</oc:size>
<d:getlastmodified>Tue, 07 Sep 2021 12:42:39 GMT</d:getlastmodified>
<oc:favorite>0</oc:favorite>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157!02dc1ec5-28b5-41c5-a48a-fabd4fa0562e/test.txt</d:href>
<d:propstat>
<d:prop>
<oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OmU4ZDIwNDA5LTE1OWItNGI2Ny1iODZkLTlkM2U3ZjYyYmM0ZQ==</oc:id>
<oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OmU4ZDIwNDA5LTE1OWItNGI2Ny1iODZkLTlkM2U3ZjYyYmM0ZQ==</oc:fileid>
<d:getetag>"e2942565a4eb52e8754c2806f215fe93"</d:getetag>
<oc:permissions>RDNVW</oc:permissions>
<d:resourcetype/>
<d:getcontentlength>10</d:getcontentlength>
<d:getcontenttype>text/plain</d:getcontenttype>
<d:getlastmodified>Tue, 07 Sep 2021 12:45:54 GMT</d:getlastmodified>
<oc:checksums>
<oc:checksum>SHA1:8f4b4c83c565fc5ec54b78c30c94a6b65e411de5 MD5:6a3a4eca9a6726eef8f7be5b03ea9011 ADLER32:151303ed</oc:checksum>
</oc:checksums>
<oc:favorite>0</oc:favorite>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
</d:multistatus>
```

2
docs/extensions/onlyoffice/.gitignore vendored
View File

@@ -1,2 +0,0 @@
configuration.md
grpc.md

11
docs/extensions/onlyoffice/_index.md
View File

@@ -1,11 +0,0 @@
---
title: "OnlyOffice"
date: 2018-05-02T00:00:00+00:00
weight: 20
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/extensions/onlyoffice
geekdocFilePath: _index.md
geekdocCollapseSection: true
---
This service enables users open documents, spreadsheets and presentations in the OnlyOffice app installed in ownCloud 10.

28
docs/extensions/onlyoffice/building.md
View File

@@ -1,28 +0,0 @@
---
title: "Building"
date: 2018-05-02T00:00:00+00:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/extensions/onlyoffice
geekdocFilePath: building.md
---
{{< toc >}}
As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install), to build this project you have to install Go >= v1.12. After the installation of the required tools you need to get the sources:
{{< highlight txt >}}
git clone https://github.com/owncloud/ocis/onlyoffice.git
cd ocis-onlyoffice
{{< / highlight >}}
All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
## Backend
{{< highlight txt >}}
make generate
make build
{{< / highlight >}}
Finally you should have the binary within the `bin/` folder now, give it a try with `./bin/onlyoffice -h` to see all available options.

291
docs/extensions/onlyoffice/getting-started.md
View File

@@ -1,291 +0,0 @@
---
title: "Getting Started"
date: 2018-05-02T00:00:00+00:00
weight: 20
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/extensions/onlyoffice
geekdocFilePath: getting-started.md
---
{{< toc >}}
## Installation
So far we are offering two different variants for the installation. You can choose between [Docker](https://www.docker.com/) or pre-built binaries which are stored on our download mirrors and GitHub releases. Maybe we will also provide system packages for the major distributions later if we see the need for it.
### Docker
TBD
### Binaries
TBD
## Configuration
We provide overall three different variants of configuration. The variant based on environment variables and commandline flags are split up into global values and command-specific values.
### ownCloud Web configuration
When loading the extension in the ownCloud Web, it is necessary to specify to which ownCloud 10 server the extension is supposed to connect to. This can be done via `config` object when registering the extension in `config.json`. For more details, you can take a look at the following example:
```json
"external_apps": [
{
"id": "onlyoffice",
"path": "https://localhost:9200/onlyoffice.js",
"config": {
"server": "https://oc10.example.org"
}
}
]
```
### Environment variables
If you prefer to configure the service with environment variables you can see the available variables below.
#### Global
ONLYOFFICE_CONFIG_FILE
: Path to config file, empty default value
ONLYOFFICE_LOG_LEVEL
: Set logging level, defaults to `info`
ONLYOFFICE_LOG_COLOR
: Enable colored logging, defaults to `true`
ONLYOFFICE_LOG_PRETTY
: Enable pretty logging, defaults to `true`
#### Server
ONLYOFFICE_TRACING_ENABLED
: Enable sending traces, defaults to `false`
ONLYOFFICE_TRACING_TYPE
: Tracing backend type, defaults to `jaeger`
ONLYOFFICE_TRACING_ENDPOINT
: Endpoint for the agent, empty default value
ONLYOFFICE_TRACING_COLLECTOR
: Endpoint for the collector, empty default value
ONLYOFFICE_TRACING_SERVICE
: Service name for tracing, defaults to `onlyoffice`
ONLYOFFICE_DEBUG_ADDR
: Address to bind debug server, defaults to `0.0.0.0:9224`
ONLYOFFICE_DEBUG_TOKEN
: Token to grant metrics access, empty default value
ONLYOFFICE_DEBUG_PPROF
: Enable pprof debugging, defaults to `false`
ONLYOFFICE_DEBUG_ZPAGES
: Enable zpages debugging, defaults to `false`
ONLYOFFICE_HTTP_ADDR
: Address to bind http server, defaults to `0.0.0.0:9220`
ONLYOFFICE_HTTP_NAMESPACE
: The http namespace
ONLYOFFICE_HTTP_ROOT
: Root path of http server, defaults to `/`
#### Health
ONLYOFFICE_DEBUG_ADDR
: Address to debug endpoint, defaults to `0.0.0.0:9224`
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below.
#### Global
--config-file | $ONLYOFFICE_CONFIG_FILE
: Path to config file.
--log-level | $ONLYOFFICE_LOG_LEVEL
: Set logging level. Default: `info`.
--log-pretty | $ONLYOFFICE_LOG_PRETTY
: Enable pretty logging. Default: `true`.
--log-color | $ONLYOFFICE_LOG_COLOR
: Enable colored logging. Default: `true`.
#### Server
--tracing-enabled | $ONLYOFFICE_TRACING_ENABLED
: Enable sending traces.
--tracing-type | $ONLYOFFICE_TRACING_TYPE
: Tracing backend type. Default: `jaeger`.
--tracing-endpoint | $ONLYOFFICE_TRACING_ENDPOINT
: Endpoint for the agent.
--tracing-collector | $ONLYOFFICE_TRACING_COLLECTOR
: Endpoint for the collector.
--tracing-service | $ONLYOFFICE_TRACING_SERVICE
: Service name for tracing. Default: `onlyoffice`.
--debug-addr | $ONLYOFFICE_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9224`.
--debug-token | $ONLYOFFICE_DEBUG_TOKEN
: Token to grant metrics access.
--debug-pprof | $ONLYOFFICE_DEBUG_PPROF
: Enable pprof debugging.
--debug-zpages | $ONLYOFFICE_DEBUG_ZPAGES
: Enable zpages debugging.
--http-addr | $ONLYOFFICE_HTTP_ADDR
: Address to bind http server. Default: `0.0.0.0:9220`.
--http-namespace | $ONLYOFFICE_HTTP_NAMESPACE
: Set the base namespace for the http namespace. Default: `com.owncloud.web`.
--http-root | $ONLYOFFICE_HTTP_ROOT
: Root path of http server. Default: `/`.
--asset-path | $ONLYOFFICE_ASSET_PATH
: Path to custom assets.
#### Health
--debug-addr | $ONLYOFFICE_DEBUG_ADDR
: Address to debug endpoint. Default: `0.0.0.0:9224`.
### Configuration file
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/tree/master/onlyoffice/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/onlyoffice.yml`, `${HOME}/.ocis/onlyoffice.yml` or `$(pwd)/config/onlyoffice.yml`.
## Usage
The program provides a few sub-commands on execution. The available configuration methods have already been mentioned above. Generally you can always see a formatted help output if you execute the binary via `ocis-onlyoffice --help`.
### Server
The server command is used to start the http and debug server on two addresses within a single process. The http server is serving the general webservice while the debug server is used for health check, readiness check and to server the metrics mentioned below. For further help please execute:
{{< highlight txt >}}
ocis-onlyoffice server --help
{{< / highlight >}}
### Health
The health command is used to execute a health check, if the exit code equals zero the service should be up and running, if the exist code is greater than zero the service is not in a healthy state. Generally this command is used within our Docker containers, it could also be used within Kubernetes.
{{< highlight txt >}}
ocis-onlyoffice health --help
{{< / highlight >}}
## Metrics
This service provides some [Prometheus](https://prometheus.io/) metrics through the debug endpoint, you can optionally secure the metrics endpoint by some random token, which got to be configured through one of the flag `--debug-token` or the environment variable `ONLYOFFICE_DEBUG_TOKEN` mentioned above. By default the metrics endpoint is bound to `http://0.0.0.0:9224/metrics`.
go_gc_duration_seconds
: A summary of the GC invocation durations
go_gc_duration_seconds_sum
: A summary of the GC invocation durations
go_gc_duration_seconds_count
: A summary of the GC invocation durations
go_goroutines
: Number of goroutines that currently exist
go_info
: Information about the Go environment
go_memstats_alloc_bytes
: Number of bytes allocated and still in use
go_memstats_alloc_bytes_total
: Total number of bytes allocated, even if freed
go_memstats_buck_hash_sys_bytes
: Number of bytes used by the profiling bucket hash table
go_memstats_frees_total
: Total number of frees
go_memstats_gc_cpu_fraction
: The fraction of this program's available CPU time used by the GC since the program started
go_memstats_gc_sys_bytes
: Number of bytes used for garbage collection system metadata
go_memstats_heap_alloc_bytes
: Number of heap bytes allocated and still in use
go_memstats_heap_idle_bytes
: Number of heap bytes waiting to be used
go_memstats_heap_inuse_bytes
: Number of heap bytes that are in use
go_memstats_heap_objects
: Number of allocated objects
go_memstats_heap_released_bytes
: Number of heap bytes released to OS
go_memstats_heap_sys_bytes
: Number of heap bytes obtained from system
go_memstats_last_gc_time_seconds
: Number of seconds since 1970 of last garbage collection
go_memstats_lookups_total
: Total number of pointer lookups
go_memstats_mallocs_total
: Total number of mallocs
go_memstats_mcache_inuse_bytes
: Number of bytes in use by mcache structures
go_memstats_mcache_sys_bytes
: Number of bytes used for mcache structures obtained from system
go_memstats_mspan_inuse_bytes
: Number of bytes in use by mspan structures
go_memstats_mspan_sys_bytes
: Number of bytes used for mspan structures obtained from system
go_memstats_next_gc_bytes
: Number of heap bytes when next garbage collection will take place
go_memstats_other_sys_bytes
: Number of bytes used for other system allocations
go_memstats_stack_inuse_bytes
: Number of bytes in use by the stack allocator
go_memstats_stack_sys_bytes
: Number of bytes obtained from system for stack allocator
go_memstats_sys_bytes
: Number of bytes obtained from system
go_threads
: Number of OS threads created
promhttp_metric_handler_requests_in_flight
: Current number of scrapes being served
promhttp_metric_handler_requests_total
: Total number of scrapes by HTTP status code

10
docs/extensions/onlyoffice/license.md
View File

@@ -1,10 +0,0 @@
---
title: "License"
date: 2018-05-02T00:00:00+00:00
weight: 40
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/extensions/onlyoffice
geekdocFilePath: license.md
---
This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/blob/master/onlyoffice/LICENSE) license. For the license of the used libraries you have to check the respective sources.

2
docs/extensions/storage/_index.md
View File

@@ -26,7 +26,7 @@ The dashed lines in the diagram indicate requests that are made to authenticate
2. The gateway will verify the JWT signature of the `x-access-token` or try to authenticate the request itself, e.g. using a public link token.
{{< hint warning >}}
The bottom part is lighter because we will deprecate it in favor of using only the CS3 user and group providers after moving some account functionality into reva and glauth. The metadata storage is not registered in the reva gateway to seperate metadata necessary for running the service from data that is being served directly.
The bottom part is lighter because we will deprecate it in favor of using only the CS3 user and group providers after moving some account functionality into reva and glauth. The metadata storage is not registered in the reva gateway to separate metadata necessary for running the service from data that is being served directly.
{{< /hint >}}
## Endpoints and references

2
docs/extensions/storage/architecture.md
View File

@@ -89,7 +89,7 @@ The OCS service makes a stat request to the storage provider to get a [ResourceI
{{< hint >}}
The user and public share provider implementations identify the file using the [`ResourceId`](https://cs3org.github.io/cs3apis/#cs3.storage.provider.v1beta1.ResourceId). The [`ResourceInfo`](https://cs3org.github.io/cs3apis/#cs3.storage.provider.v1beta1.ResourceInfo) is passed so the share provider can also store who the owner of the resource is. The *path* is not part of the other API calls, e.g. when listing shares.
The OCM API takes an id based reference on the CS3 api, even if the OCM HTTP endpoint takes a path argument. *@jfd: Why? Does it not need the owner? It only stores the owner of the share, which is always the currently looged in user, when creating a share. Afterwards only the owner can update a share ... so collaborative management of shares is not possible. At least for OCM shares.*
The OCM API takes an id based reference on the CS3 api, even if the OCM HTTP endpoint takes a path argument. *@jfd: Why? Does it not need the owner? It only stores the owner of the share, which is always the currently logged in user, when creating a share. Afterwards only the owner can update a share ... so collaborative management of shares is not possible. At least for OCM shares.*
{{< /hint >}}
### User and Group provisioning

2
docs/extensions/storage/ports.md
View File

@@ -34,6 +34,8 @@ For now, the storage service uses these ports to preconfigure those services:
| 9159 | storage users debug |
| 9160 | groups |
| 9161 | groups debug |
| 9164 | storage appprovider |
| 9165 | storage appprovider debug |
| 9178 | storage public link |
| 9179 | storage public link data |
| 9215 | storage meta grpc |

8
docs/extensions/storage/terminology.md
View File

@@ -79,13 +79,13 @@ It maps the *path* and *id* based CS3 *references* to an appropriate [*storage s
{{< hint warning >}}
**Proposed Change**
iOS clients can only queue single requests to be executed in the background. The queue an upload and need to be able to identify the uploaded file after it has been uploaded to the server. The disconnected nature of the connection might cause worksflows or manual user interaction with the file on the server to move the file to a different place or changing the content while the device is offline. However, on the device users might have marked the file as favorite or added it to other iOS specific collections. To be able to reliably identify the file the client can generate a `uuid` and attach it to the file metadata during the upload. While it is not necessary to look up files by this `uuid` having a second file id that serves exactly the same purpose as the `file id` is redundant.
iOS clients can only queue single requests to be executed in the background. The queue an upload and need to be able to identify the uploaded file after it has been uploaded to the server. The disconnected nature of the connection might cause workflows or manual user interaction with the file on the server to move the file to a different place or changing the content while the device is offline. However, on the device users might have marked the file as favorite or added it to other iOS specific collections. To be able to reliably identify the file the client can generate a `uuid` and attach it to the file metadata during the upload. While it is not necessary to look up files by this `uuid` having a second file id that serves exactly the same purpose as the `file id` is redundant.
Another aspect for the `file id` / `uuid` is that it must be a logical identifier that can be set, at least by internal systems. Without a writeable fileid we cannot restore backups or migrate storage spaces from one storage provider to another storage provider.
Technically, this means that every storage driler needs to have a map of a `uuid` to in internal resource identifier. This internal resource identifier can be
Technically, this means that every storage driver needs to have a map of a `uuid` to in internal resource identifier. This internal resource identifier can be
- an eos fileid, because eos can look up files by id
- an inode if the filesystem and the storage driver support lookung up by inode
- an inode if the filesystem and the storage driver support looking up by inode
- a path if the storage driver has no way of looking up files by id.
- In this case other mechanisms like inotify, kernel audit or a fuse overlay might be used to keep the paths up to date.
- to prevent excessive writes when deep folders are renamed a reverse map might be used: it will map the `uuid` to `<parentuuid>:<childname>`, allowing to trade writes for reads
@@ -142,7 +142,7 @@ Examples would be every user's home storage space, project storage spaces or gro
Finally, a logical `storage space id` is not tied to a specific [*storage provider*]({{< ref "#storage-providers" >}}). If the [*storage driver*]({{< ref "#storage-drivers" >}}) supports it, we can import existing files including their `file id`, which makes it possible to move [*storage spaces*]({{< ref "#storage-spaces" >}}) between [*storage providers*]({{< ref "#storage-providers" >}}) to implement storage classes, e.g. with or without archival, workflows, on SSDs or HDDs.
## Shares
*To be clarified: we are aware that [*storage spaces*]({{< ref "#storage-spaces" >}}) may be too 'heavywheight' for ad hoc sharing with groups. That being said, there is no technical reason why group shares should not be treated like [*storage spaces*]({{< ref "#storage-spaces" >}}) that users can provision themselves. They would share the quota with the users home [*storage space*]({{< ref "#storage-spaces" >}}) and the share initiator would be the sole owner. Technically, the mechanism of treating a share like a new [*storage space*]({{< ref "#storage-spaces" >}}) would be the same. This obviously also extends to user shares and even file indvidual shares that would be wrapped in a virtual collection. It would also become possible to share collections of arbitrary files in a single storage space, e.g. the ten best pictures from a large album.*
*To be clarified: we are aware that [*storage spaces*]({{< ref "#storage-spaces" >}}) may be too 'heavyweight' for ad hoc sharing with groups. That being said, there is no technical reason why group shares should not be treated like [*storage spaces*]({{< ref "#storage-spaces" >}}) that users can provision themselves. They would share the quota with the users home [*storage space*]({{< ref "#storage-spaces" >}}) and the share initiator would be the sole owner. Technically, the mechanism of treating a share like a new [*storage space*]({{< ref "#storage-spaces" >}}) would be the same. This obviously also extends to user shares and even file individual shares that would be wrapped in a virtual collection. It would also become possible to share collections of arbitrary files in a single storage space, e.g. the ten best pictures from a large album.*
## Storage Systems

6
docs/ocis/_index.md
View File

@@ -15,11 +15,11 @@ Welcome to oCIS, the modern file-sync and share platform, which is based on our
### The idea of federated storage
To creata a truly federated storage architecture oCIS breaks down the old ownCloud 10 user specific namespace, which is assembled on the server side, and makes the individual parts accessible to clients as storage spaces and storage space registries.
To create a truly federated storage architecture oCIS breaks down the old ownCloud 10 user specific namespace, which is assembled on the server side, and makes the individual parts accessible to clients as storage spaces and storage space registries.
The below diagram shows the core conceps that are the foundation for the new architecture:
The below diagram shows the core concepts that are the foundation for the new architecture:
- End user devices can fetch the list of *storage spaces* a user has access to, by querying one or multiple *storage space registries*. The list contains a unique endpoint for every *storage space*.
- [*Storage space registries*]({{< ref "../extensions/storage/terminology#storage-space-registries" >}}) manage the list of storage spaces a user has access to. They may subscrible to *storage spaces* in order to receive notifications about changes on behalf of an end users mobile or desktop client.
- [*Storage space registries*]({{< ref "../extensions/storage/terminology#storage-space-registries" >}}) manage the list of storage spaces a user has access to. They may subscribe to *storage spaces* in order to receive notifications about changes on behalf of an end users mobile or desktop client.
- [*Storage spaces*]({{< ref "../extensions/storage/terminology#storage-spaces" >}}) represent a collection of files and folders. A users personal files are a *storage space*, a group or project drive is a *storage space*, and even incoming shares are treated and implemented as *storage spaces*. Each with properties like owners, permissions, quota and type.
- [*Storage providers*]({{< ref "../extensions/storage/terminology#storage-providers" >}}) can hold multiple *storage spaces*. At an oCIS instance, there might be a dedicated *storage provider* responsible for users personal storage spaces. There might be multiple, sharing the load or there might be just one, hosting all types of *storage spaces*.

2
docs/ocis/adr/0003-external-user-management.md
View File

@@ -69,7 +69,7 @@ Chosen option: "Move accounts functionality to GLAuth and name it accounts", by
### Negative Consequences
* If users want to store users in their IDM and at the same time guests in a seperate user management we need to implement GLAuth backends that support more than one LDAP server.
* If users want to store users in their IDM and at the same time guests in a separate user management we need to implement GLAuth backends that support more than one LDAP server.
## Pros and Cons of the Options

2
docs/ocis/adr/0004-support-hot-migration.md
View File

@@ -60,7 +60,7 @@ The migration happens while the service is offline. File metadata, blobs and sha
- Good, because oCIS can be tested in a staging system without writing to the production system.
- Good, because file layout on disk can be changed to support new storage driver capabilities.
- Bad, because the export and import might require significant amounts of storage.
- Bad, because a rollback to the state before the migration might cause data loss of the changes that happend in between.
- Bad, because a rollback to the state before the migration might cause data loss of the changes that happened in between.
- Bad, because the cold migration can mean significant downtime.
### Hot Migration

2
docs/ocis/adr/0005-cs3-api-account-management.md
View File

@@ -4,7 +4,7 @@ weight: 5
date: 2021-04-12T15:00:00+01:00
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/ocis/adr
geekdocFilePath: 0005-cs3api-user-management.md
geekdocFilePath: 0005-cs3-api-account-management.md
---
* Status: proposed

2
docs/ocis/adr/0006-service-discovery.md
View File

@@ -37,7 +37,7 @@ Chosen option: "Dynamic service registration". There were some drawbacks regardi
* Having dynamic service registration delegates the entire lifecycle of finding a process to the service registry.
* Removing a-priori knowledge of hostname + port for services.
* Marrying go-micro's registry and a newly defined registry abstraction on Reva.
* We will embrace go-micro interfaces by defining a third merger interface in order to marry go-micro registry and rega revistry.
* We will embrace go-micro interfaces by defining a third merger interface in order to marry go-micro registry and reva registry.
* The ability to fetch a service node relying only on its name (i.e: com.owncloud.proxy) and not on a tuple hostname + port that we rely on being preconfigured during runtime.
* Conceptually speaking, a better framework to tie all the services together. Referring to services by names is less overall confusing than having to add a service name + where it is running. A registry is agnostic to "where is it running" because it, by definition, keeps track of this specific question, so when speaking about design or functionality, it will ease communication.

4
docs/ocis/adr/0010-policy-enforcement.md
View File

@@ -23,11 +23,11 @@ There should be a way to impose certain limitations in areas of the code that re
## Considered Options
1. Build the evaluation engine in-house.
2. Use third party libraries such as Open Policy Agent (a CNCF aproved project written in Go)
2. Use third party libraries such as Open Policy Agent (a CNCF approved project written in Go)
## Decision Outcome
Chosen option: option 2; Use third party libraries such as Open Policy Agent (a CNCF aproved project written in Go)
Chosen option: option 2; Use third party libraries such as Open Policy Agent (a CNCF approved project written in Go)
### Positive Consequences

8
docs/ocis/adr/0011-global-url-format.md
View File

@@ -7,7 +7,7 @@ geekdocEditPath: edit/master/docs/ocis/adr
geekdocFilePath: 0011-global-url-format.md
---
* Status: proposed
* Status: accepted
* Deciders: @refs, @butonic, @micbar, @dragotin, @hodyroff, @pmaier1, @fschade, @tbsbdr, @kulmann
* Date: 2021-07-07
@@ -177,7 +177,7 @@ There is a customized ownCloud instance that uses path only based URLs:
{{< hint >}}
* `/#` is used by the current vue router.
* `/s` denotes that this is a space url.
* `<space_id>` and `<resource_id>` both consist of `<storage_id>:<node_id>`, but the `space_id` can be replaced with a shorter id or an alias. See furthor down below.
* `<space_id>` and `<resource_id>` both consist of `<storage_id>:<node_id>`, but the `space_id` can be replaced with a shorter id or an alias. See further down below.
* `<relative/path>` takes precedence over the `<resource_id>`, both are optional
{{< /hint >}}
@@ -245,11 +245,11 @@ When every space has a namespaced alias and a relative path we can build a globa
| `https://demo.owncloud.com/files/personal/einstein/relative/path/to/resource?id=b78c2044-5b51-446f-82f6-907a664d089c:194b4a97-597c-4461-ab56-afd4f5a21608` | sub folder `/relative/path/to/resource` |
| `https://demo.owncloud.com/files/shares/einstein/somesharename?id=b78c2044-5b51-446f-82f6-907a664d089c:194b4a97-597c-4461-ab56-afd4f5a21608` | shared URL for `/relative/path/to/resource` |
| `https://demo.owncloud.com/files/personal/einstein/marie is stupid/and richard as well/resource?id=b78c2044-5b51-446f-82f6-907a664d089c:194b4a97-597c-4461-ab56-afd4f5a21608` | sub folder `marie is stupid/and richard as well/resource` ... something einstein might not want to reveal |
| `https://demo.owncloud.com/files/shares/einstein/resource (2)?id=b78c2044-5b51-446f-82f6-907a664d089c:194b4a97-597c-4461-ab56-afd4f5a21608` | named link URL for `/marie is stupid/and richard as well/resource`, does not disclose the actual hierarchy, has an appended counter to avaid a collision |
| `https://demo.owncloud.com/files/shares/einstein/resource (2)?id=b78c2044-5b51-446f-82f6-907a664d089c:194b4a97-597c-4461-ab56-afd4f5a21608` | named link URL for `/marie is stupid/and richard as well/resource`, does not disclose the actual hierarchy, has an appended counter to avoid a collision |
| `https://demo.owncloud.com/files/shares/einstein/mybestfriends?id=b78c2044-5b51-446f-82f6-907a664d089c:194b4a97-597c-4461-ab56-afd4f5a21608` | named link URL for `/marie is stupid/and richard as well/resource`, does not disclose the actual hierarchy, has a custom alias for the share |
| `https://demo.owncloud.com/files/public/kcZVYaXr7oZ66bg/relative/path/to/resource` | sub folder `/relative/path/to/resource` in public link with token `kcZVYaXr7oZ66bg` |
| `https://demo.owncloud.com/files/public/kcZVYaXr7oZ66bg/relative/path/to/resource` | sub folder `/relative/path/to/resource` in public link with token `kcZVYaXr7oZ66bg` |
| `https://demo.owncloud.com/s/kcZVYaXr7oZ66bg/` | shortened link to a resource. This is needed to be able to copy a link to a resource whithout leaking any metadata. |
| `https://demo.owncloud.com/s/kcZVYaXr7oZ66bg/` | shortened link to a resource. This is needed to be able to copy a link to a resource without leaking any metadata. |
`</namespaced/alias></relative/path/to/resource>` is the global path in the CS3 api. The CS3 Storage Registry is responsible by managing the mount points.

49
docs/ocis/adr/0012-tracing.md Normal file
View File

@@ -0,0 +1,49 @@
---
title: "12. Tracing"
weight: 12
date: 2021-08-17T12:56:53+01:00
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/ocis/adr
geekdocFilePath: 0012-tracing.md
---
* Status: proposed
* Deciders: @butonic, @micbar, @dragotin, @mstingl, @pmaier1, @fschade
* Date: 2021-08-17
## Context and Problem Statement
At the time of this writing we are in a situation where our logs have too much verbosity, rendering impossible or rather difficult to debug an instance. For this reason we are giving some care to our traces by updating dependencies from OpenCensus to OpenTelemetry.
## Decision Drivers
- We don't want to rely only on logs to debug an instance.
- Logs are too verbose.
- Since we have micro-services, we want to holistically understand a request.
## Considered Options
- Trim down logs
- Use OpenCensus
- Migrate to OpenTelemetry
## Decision Outcome
Chosen option: option 3; Migrate to OpenTelemetry. OpenCensus is deprecated, and OpenTelemetry is the merger from OpenCensus and OpenTelemetry and the most recent up-to-date spec.
### Positive Consequences
- Fix the current state of the traces on Reva.
- Add more contextual information on a span for a given request.
- Per-request filtering with the `X-Request-Id` header.
- Group the supported tracing backends to support Jaeger only for simplicity.
## Chosen option approach
- A trace is a tree, and the proxy will create the root trace and propagate it downstream.
- The Root trace will log the request headers.
- The unit that ultimately does the work will log the result of the operation if success.
- The unit that ultimately does the work will change the state of the span to error if any occurred.
With this premises, this is by no means a fixed document and the more we learn about the usage of an instance the more context we can add to the traces.

4
docs/ocis/deployment/_index.md
View File

@@ -20,6 +20,7 @@ oCIS deployments are super simple, yet there are many configurations possible fo
- [oCIS setup with Traefik for SSL termination]({{< ref "ocis_traefik" >}})
- [oCIS setup with Keycloak as identity provider]({{< ref "ocis_keycloak" >}})
- [oCIS setup with WOPI server to open office documents in your browser]({{< ref "ocis_wopi" >}})
- [Parallel deployment of oC10 and oCIS]({{< ref "oc10_ocis_parallel" >}})
- [oCIS with S3 storage backend (MinIO)]({{< ref "ocis_s3" >}})
- [oCIS with the Hello extension example]({{< ref "ocis_hello" >}})
@@ -45,6 +46,9 @@ You can change it by setting the `STORAGE_TRANSFER_SECRET` environment variable
{{< hint info >}}
Before deleting the demo users mentioned below, you must create a new account for yourself and assign it to the administrator role.
To skip the generation of demo users in the first place, run the inital setup step with an additional environment variable.
`ACCOUNTS_DEMO_USERS_AND_GROUPS=false ./bin/ocis server` generates only the admin, and one user for IDP and Reva respectively.
{{< /hint >}}
oCIS ships with a few demo users besides the system users:

18
docs/ocis/deployment/continuous_deployment.md
View File

@@ -73,6 +73,24 @@ Credentials:
- oCIS: [ocis.ocis-keycloak.released.owncloud.works](https://ocis.ocis-keycloak.released.owncloud.works)
- Keycloak: [keycloak.ocis-keycloak.released.owncloud.works](https://keycloak.ocis-keycloak.released.owncloud.works)
# Parallel deployment of oC10 and oCIS
Credentials:
- oC10 / oCIS: see [default demo users]({{< ref "../getting-started#login-to-owncloud-web" >}})
- Keycloak:
- username: admin
- password: admin
- LDAP management:
- username: cn=admin,dc=owncloud,dc=com
- password: admin
## Latest
- oC10 / oCIS: [cloud.oc10-ocis-parallel.latest.owncloud.works](https://cloud.oc10-ocis-parallel.latest.owncloud.works)
- LDAP management: [ldap.oc10-ocis-parallel.latest.owncloud.works](https://ldap.oc10-ocis-parallel.latest.owncloud.works)
- Keycloak: [keycloak.oc10-ocis-parallel.latest.owncloud.works](https://keycloak.oc10-ocis-parallel.latest.owncloud.works)
# oCIS with Hello extension
Credentials:

256
docs/ocis/deployment/kubernetes.md Normal file
View File

@@ -0,0 +1,256 @@
---
title: "Kubernetes"
date: 2021-09-23T11:04:00+01:00
weight: 25
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/ocis/deployment
geekdocFilePath: kubernetes.md
---
{{< toc >}}
## What is Kubernetes
Formally described as:
> Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation.
_[source](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/)_
Without getting too deep in definitions, and for the purpose of compactness, Kubernetes can be summarized as a way of managing containers that run applications to ensure that there is no downtime and a optimal usage of resources. It provides with a framework in which to run distributed systems.
Kubernetes provides you with:
- **Service discovery and load balancing**: Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable.
- **Storage orchestration**: Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more.
- **Automated rollouts and rollbacks**: You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container.
- **Automatic bin packing**: You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.
- **Self-healing**: Kubernetes restarts containers that fail, replaces containers, kills containers that don't respond to your user-defined health check, and doesn't advertise them to clients until they are ready to serve.
- **Secret and configuration management**: Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.
_[extracted from k8s docs](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/#why-you-need-kubernetes-and-what-can-it-do)_
If that is still too abstract, [here is an ELI5 writeup](https://dev.to/miguelmota/comment/filh).
### References and further reads
- [Marcel Wunderlich's](https://github.com/Deaddy) [4 series articles](http://deaddy.net/introduction-to-kubernetes-pt-1.html) on Kubernetes clarifying its declarative nature, deep diving into ingress networking, storage and monitoring.
### How does oCIS fit in the Kubernetes model
oCIS was designed with running on Kubernetes in mind. We set up to adopt the [Twelve-Factor App](https://12factor.net/) principles regarding configuration, with almost every aspect of oCIS being modifiable via environment variables. This comes in handy when you especially have a look at how a helm chart's (we will introduce this concept shortly) [list of values](https://github.com/refs/ocis-charts/blob/d8735e3222d2050504303851d3461909c86fcc89/ocis/values.yaml) looks like.
## What is Minikube
[Minikube](https://minikube.sigs.k8s.io/docs/) lets you run a Kubernetes cluster locally. It is the most approachable way to test a deployment. It requires no extra configuration on any cloud platform, as everything runs on your local machine. For the purpose of these docs, this is the first approach we chose to run oCIS and will develop on how to set it up.
## What is `kubectl`
[kubectl](https://kubernetes.io/docs/tasks/tools/) is the command-line tool for Kubernetes. It allows users to run commands against a k8s cluster the user has access to. It supports for having multiple contexts for as many clusters as you have access to. In these docs we will setup 2 contexts, a minikube and a GCP context.
## What are Helm Charts, and why they are useful for oCIS
[Helm](https://helm.sh/) is the equivalent of a package manager for Kubernetes. It can be described as a layer on top of how you would write pods, deployments or any other k8s resource declaration.
### Installing Helm
[Follow the official installation guide](https://helm.sh/docs/intro/install/).
## Setting up Minikube
For a guide on how to set minikube up follow the [official minikube start guide](https://minikube.sigs.k8s.io/docs/start/) for your specific OS.
### Start minikube
First off, verify your installation is correct:
```console
~/code/refs/ocis-charts
❯ minikube status
minikube
type: Control Plane
host: Stopped
kubelet: Stopped
apiserver: Stopped
kubeconfig: Stopped
```
After that, start the cluster:
```console
~/code/refs/ocis-charts
❯ minikube start
😄 minikube v1.23.0 on Darwin 11.4
✨ Using the docker driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🔄 Restarting existing docker container for "minikube" ...
🐳 Preparing Kubernetes v1.22.1 on Docker 20.10.8 ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
```
_On these docs, we are using the Docker driver on Mac._
## Run a chart
The easiest way to run the entire package is by using the available charts on https://github.com/refs/ocis-charts. It is not the purpose of this guide to explain the inner working of Kubernetes or its resources, as Helm builds an abstraction oon top of it, letting you interact with a refined interface that roughly translates as "helm install" and "helm uninstall".
In order to host charts one can create a [charts repository](https://helm.sh/docs/topics/chart_repository/), but this is outside the scope of this documentation. Having said that, we will assume you have access to a cli and git.
### Requirements
1. minikube up and running.
2. `kubectl` installed. By [default you should be able to access the minikube's cluster](https://minikube.sigs.k8s.io/docs/handbook/kubectl/). If you chose not to install `kubectl`, minikube wraps `kubectl` as `minikube kubectl`.
3. helm cli installed.
4. git installed.
### Setup
1. clone the charts: `git clone https://github.com/refs/ocis-charts.git /var/tmp/ocis-charts`
2. cd into the charts root: `cd /var/tmp/ocis-charts/ocis`
3. install the package: `helm install ocis .`
4. verify the application is running in the cluster: `kubectl get pods`
```console
❯ kubectl get pods
NAME READY STATUS RESTARTS AGE
glauth-5fb678b9cb-zs5qh 1/1 Running 3 (10m ago) 3h33m
ocis-proxy-848f988687-g7fmb 1/1 Running 2 (10m ago) 130m
ocs-6bb8896dd6-t4bkx 1/1 Running 3 (10m ago) 3h33m
settings-6bf77f978d-27rdf 1/1 Running 3 (10m ago) 3h33m
storages-6b45f9c4-2j696 10/10 Running 23 (4m43s ago) 112m
store-cf79db94d-hvb7z 1/1 Running 3 (10m ago) 3h33m
web-8685fdd574-tmkfb 1/1 Running 2 (10m ago) 157m
webdav-f8d4dd7c6-vv4n7 1/1 Running 3 (10m ago) 3h33m
```
5. expose the proxy as a service to the host
```console
~/code/refs/ocis-charts
❯ minikube service proxy-service --url
🏃 Starting tunnel for service proxy-service.
|-----------|---------------|-------------|------------------------|
| NAMESPACE | NAME | TARGET PORT | URL |
|-----------|---------------|-------------|------------------------|
| default | proxy-service | | http://127.0.0.1:63633 |
|-----------|---------------|-------------|------------------------|
http://127.0.0.1:63633
❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.
```
6. attempt a `PROPFIND` WebDAV request to the storage: `curl -v -k -u einstein:relativity -H "depth: 0" -X PROPFIND https://127.0.0.1:63633/remote.php/dav/files/ | xmllint --format -`
If all is correctly setup, you should expect a response back:
```xml
<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
<d:response>
<d:href>/remote.php/dav/files/einstein/</d:href>
<d:propstat>
<d:prop>
<oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjZlMWIyMjdmLWZmYTQtNDU4Ny1iNjQ5LWE1YjBlYzFkMTNmYw==</oc:id>
<oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjZlMWIyMjdmLWZmYTQtNDU4Ny1iNjQ5LWE1YjBlYzFkMTNmYw==</oc:fileid>
<d:getetag>"92cc7f069c8496ee2ce33ad4f29de763"</d:getetag>
<oc:permissions>WCKDNVR</oc:permissions>
<d:resourcetype>
<d:collection/>
</d:resourcetype>
<d:getcontenttype>httpd/unix-directory</d:getcontenttype>
<oc:size>4096</oc:size>
<d:getlastmodified>Tue, 14 Sep 2021 12:45:29 +0000</d:getlastmodified>
<oc:favorite>0</oc:favorite>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
</d:multistatus>
```
## Setting up an external identity provider
The previous setup works because the proxy is configured to run using basic auth, but if we want to actually use the WebUI we will need an external identity provider. From here on the setup is composed of:
- keycloak
- traefik
- postgresql
Running on i.e: `https://keycloak.owncloud.works`. Because of this we have to adjust some of `values.yaml` key / values to:
```diff
diff --git a/ocis/values.yaml b/ocis/values.yaml
index fbc229c..5b36fbd 100644
--- a/ocis/values.yaml
+++ b/ocis/values.yaml
@@ -1,9 +1,9 @@
# when in local tunnel mode, ingressDomain is the proxy address.
# sadly when in combination with --set, anchors are lost.
-ingressDomain: &ingressDomain "https://stale-wasp-86.loca.lt"
+ingressDomain: &ingressDomain "https://keycloak.owncloud.works"
# base ocis image
-image: owncloud/ocis:1.0.0-rc8-linux-amd64
+image: owncloud/ocis:1.11.0-linux-amd64
# set of ocis services to create deployments objects.
services:
@@ -22,6 +22,8 @@ services:
value: "debug"
- name: "PROXY_REVA_GATEWAY_ADDR"
value: "storages-service:9142"
+ - name: "PROXY_OIDC_ISSUER"
+ value: "https://keycloak.ocis-keycloak.released.owncloud.works/auth/realms/oCIS"
- name: "PROXY_ENABLE_BASIC_AUTH"
value: "'true'" # see https://stackoverflow.com/a/44692213/2295410
volumeMounts:
@@ -81,34 +85,6 @@ services:
labels:
app: "glauth"
args: ["glauth"]
settings:
metadata:
name: "settings"
@@ -135,11 +111,11 @@ services:
args: ["web"]
env:
- name: "WEB_UI_CONFIG_SERVER"
- value: *ingressDomain
+ value: "https://127.0.0.1:51559/"
- name: "WEB_OIDC_METADATA_URL"
- value: *ingressDomain
+ value: "https://keycloak.owncloud.works/auth/realms/oCIS/.well-known/openid-configuration"
- name: "WEB_OIDC_AUTHORITY"
- value: *ingressDomain
+ value: "https://keycloak.owncloud.works/auth/realms/oCIS/.well-known/openid-configuration"
ports:
values:
- name: "http"
@@ -231,4 +207,4 @@ kubeServices:
- protocol: TCP
port: 9100
targetPort: 9100
```
NOTE: The IDP has to be properly configure with an oCIS realm and a `web` client configured. There are example config file that have to be adjusted depending on your environment on our [docker-compose examples](https://github.com/owncloud/ocis/tree/master/deployments/examples/ocis_keycloak/config/keycloak).
You might still need to adjust the IDP:
- Valid Redirect URIs (under clients > web)
- Web Origins (under clients > web)
## What is GCP
> Google Cloud Platform (GCP), offered by Google, is a suite of cloud computing services that runs on the same infrastructure that Google uses internally for its end-user products
One of such offered services are [Google Kubernetes Engines (GKE)](https://cloud.google.com/kubernetes-engine).
### Can Helm charts run on GCP?
Yes. The next logical step would be to deploy this charts on GKE. There is a pretty thorough guide [at shippable.com](http://docs.shippable.com/deploy/tutorial/deploy-to-gcp-gke-helm/) that, for the purposes of our docs, we are only interested on step 5, as we already explain the previous concepts, and provide with the Charts.
## TODOs
- While log-in works and creating folders work, uploading fails, most likely a configuration issue that has to be solved.

2
docs/ocis/deployment/monitoring-tracing.md
View File

@@ -31,6 +31,8 @@ For more information and how to deploy it, see [monitoring & tracing client](htt
## Monitoring & tracing server
A live version of the monitoring and tracing server for our demo instances is available here: [Grafana](https://grafana.infra.owncloud.works), [Prometheus](https://prometheus.infra.owncloud.works) and [Jaeger Query](https://jaeger.infra.owncloud.works).
The monitoring & tracing server is considered as shared infrastructure and is normally used for different services. This means that oCIS is not the only software whose metrics and traces are available on the monitoring server. It is also possible that data of multiple oCIS instances are available on the monitoring server.
Metrics are scraped, stored and can be queried with Prometheus. For the visualization of these metrics Grafana is used. Because Prometheus is scraping the metrics from the oCIS server (pull model instead of a push model), the Prometheus server must have access to the exposed endpoint of the Telegraf Prometheus output plugin.

172
docs/ocis/deployment/oc10_ocis_parallel.md Normal file
View File

@@ -0,0 +1,172 @@
---
title: "Parallel deployment of oC10 and oCIS"
date: 2020-10-12T14:04:00+01:00
weight: 24
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/ocis/deployment
geekdocFilePath: oc10_ocis_parallel.md
---
{{< toc >}}
{{< hint warning >}}
This deployment example currently has known issues. See [github.com/owncloud/ocis/issues/2549](https://github.com/owncloud/ocis/issues/2549) for more information.
{{< /hint >}}
## Overview
- This setup reflects [stage 6 of the oC10 to oCIS migration plan]({{< ref "migration#stage-6-parallel-deployment" >}})
- Traefik generating self signed certificates for local setup or obtaining valid SSL certificates for a server setup
- OpenLDAP server with demo users
- LDAP admin interface to edit users
- Keycloak as OpenID Connect provider in federation with the LDAP server
- ownCloud 10 with MariaDB and Redis
- ownCloud 10 is configured to synchronize users from the LDAP server
- ownCloud 10 is used to use OpenID Connect for authentication with Keycloak
- oCIS running behind Traefik as reverse proxy
- oCIS is using the ownCloud storage driver on the same files and same database as ownCloud 10
- oCIS is using Keycloak as OpenID Connect provider
- oCIS is using the LDAP server as user backend
- All requests to both oCIS and oC10 are routed through the oCIS proxy and will be routed based on an OIDC claim to one of them. Therefore admins can change on a user basis in the LDAP which backend is used.
[Find this example on GitHub](https://github.com/owncloud/ocis/tree/master/deployments/examples/oc10_ocis_parallel)
## Server Deployment
### Requirements
- Linux server with docker and docker-compose installed
- four domains set up and pointing to your server
- cloud.\* for serving oCIS
- keycloak.\* for serving Keycloak
- ldap .\* for serving the LDAP managment UI
- traefik.\* for serving the Traefik dashboard
See also [example server setup]({{< ref "preparing_server" >}})
### Install this example
- Clone oCIS repository
`git clone https://github.com/owncloud/ocis.git`
- Go to the deployment example
`cd ocis/deployment/examples/oc10_ocis_parallel`
- Open the `.env` file in a text editor
The file by default looks like this:
```bash
# If you're on a internet facing server please comment out following line.
# It skips certificate validation for various parts of oCIS and is needed if you use self signed certificates.
INSECURE=true
### Traefik settings ###
TRAEFIK_LOG_LEVEL=
# Serve Treafik dashboard. Defaults to "false".
TRAEFIK_DASHBOARD=
# Domain of Traefik, where you can find the dashboard. Defaults to "traefik.owncloud.test"
TRAEFIK_DOMAIN=
# Basic authentication for the dashboard. Defaults to user "admin" and password "admin"
TRAEFIK_BASIC_AUTH_USERS=
# Email address for obtaining LetsEncrypt certificates, needs only be changed if this is a public facing server
TRAEFIK_ACME_MAIL=
### shared oCIS / oC10 settings ###
# Domain of oCIS / oC10, where you can find the frontend. Defaults to "cloud.owncloud.test"
CLOUD_DOMAIN=
### oCIS settings ###
# oCIS version. Defaults to "latest"
OCIS_DOCKER_TAG=
# JWT secret which is used for the storage provider. Must be changed in order to have a secure oCIS. Defaults to "Pive-Fumkiu4"
OCIS_JWT_SECRET=
# JWT secret which is used for uploads to create transfer tokens. Must be changed in order to have a secure oCIS. Defaults to "replace-me-with-a-transfer-secret"
STORAGE_TRANSFER_SECRET=
### oCIS settings ###
# oC10 version. Defaults to "latest"
OC10_DOCKER_TAG=
# client secret which the openidconnect app uses to authenticate to Keycloak. Defaults to "oc10-oidc-secret"
OC10_OIDC_CLIENT_SECRET=
# app which will be shown when opening the ownCloud 10 UI. Defaults to "files" but also could be set to "web"
OWNCLOUD_DEFAULT_APP=
# if set to "false" (default) links will be opened in the classic UI, if set to "true" ownCloud Web is used
OWNCLOUD_WEB_REWRITE_LINKS=
### LDAP settings ###
# password for the LDAP admin user "cn=admin,dc=owncloud,dc=com", defaults to "admin"
LDAP_ADMIN_PASSWORD=
# Domain of the LDAP management frontend. Defaults to "ldap.owncloud.test"
LDAP_MANAGER_DOMAIN=
### Keycloak ###
# Domain of Keycloak, where you can find the managment and authentication frontend. Defaults to "keycloak.owncloud.test"
KEYCLOAK_DOMAIN=
# Realm which to be used with oC10 and oCIS. Defaults to "owncloud"
KEYCLOAK_REALM=
# Admin user login name. Defaults to "admin"
KEYCLOAK_ADMIN_USER=
# Admin user login password. Defaults to "admin"
KEYCLOAK_ADMIN_PASSWORD=
```
You are installing oCIS on a server and Traefik will obtain valid certificates for you so please remove `INSECURE=true` or set it to `false`.
If you want to use the Traefik dashboard, set TRAEFIK_DASHBOARD to `true` (default is `false` and therefore not active). If you activate it, you must set a domain for the Traefik dashboard in `TRAEFIK_DOMAIN=` eg. `TRAEFIK_DOMAIN=traefik.owncloud.test`.
The Traefik dashboard is secured by basic auth. Default credentials are the user `admin` with the password `admin`. To set your own credentials, generate a htpasswd (eg. by using [an online tool](https://htpasswdgenerator.de/) or a cli tool).
Traefik will issue certificates with LetsEncrypt and therefore you must set an email address in `TRAEFIK_ACME_MAIL=`.
By default oCIS will be started in the `latest` version. If you want to start a specific version of oCIS set the version to `OCIS_DOCKER_TAG=`. Available versions can be found on [Docker Hub](https://hub.docker.com/r/owncloud/ocis/tags?page=1&ordering=last_updated).
Set your domain for the oC10 and oCIS frontend in `CLOUD_DOMAIN=`, eg. `CLOUD_DOMAIN=cloud.owncloud.test`.
You also must override the default secrets in `STORAGE_TRANSFER_SECRET` and `OCIS_JWT_SECRET` in order to secure your oCIS instance. Choose some random strings eg. from the output of `openssl rand -base64 32`. For more information see [secure an oCIS instance]({{< ref "./#secure-an-ocis-instance" >}}).
By default ownCloud 10 will be started in the `latest` version. If you want to start a specific version of oCIS set the version to `OC10_DOCKER_TAG=`. Available versions can be found on [Docker Hub](https://hub.docker.com/r/owncloud/ocis/tags?page=1&ordering=last_updated).
You can switch the default application of ownCloud 10 by setting`OWNCLOUD_DEFAULT_APP=files` in oder to have the classic UI as frontend, which is also the default. If you prefer ownCloud Web as the default application in ownCloud 10 just set `OWNCLOUD_DEFAULT_APP=web`.
In oder to change the default link open action which defaults to the classic UI (`OWNCLOUD_WEB_REWRITE_LINKS=false`) you can set it to `OWNCLOUD_WEB_REWRITE_LINKS=true`. This will lead to links being opened in ownCloud Web.
The OpenLDAP server in this example deployment has an admin users, which is also used as bind user in order to keep theses examples simple. You can change the default password "admin" to a different one by setting it to `LDAP_ADMIN_PASSWORD=...`.
Set your domain for the LDAP manager UI in `LDAP_MANAGER_DOMAIN=`, eg. `ldap.owncloud.test`.
Set your domain for the Keycloak administration panel and authentication endpoints to `KEYCLOAK_DOMAIN=` eg. `KEYCLOAK_DOMAIN=keycloak.owncloud.test`.
Changing the used Keycloak realm can be done by setting `KEYCLOAK_REALM=`. This defaults to the ownCloud realm `KEYCLOAK_REALM=owncloud`. The ownCloud realm will be automatically imported on startup and includes our demo users.
You probably should secure your Keycloak admin account by setting `KEYCLOAK_ADMIN_USER=` and `KEYCLOAK_ADMIN_PASSWORD=` to values other than `admin`.
Now you have configured everything and can save the file.
- Start the docker stack
`docker-compose up -d`
- You now can visit the cloud, oC10 or oCIS depending on the user configuration. Marie defaults to oC10 and Richard and Einstein default to oCIS, but you can change the ownCloud selector at any time in the LDAP management UI.
## Local setup
For a more simple local ocis setup see [Getting started]({{< ref "../getting-started" >}})
This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer.
On Linux and macOS you can add them to your `/etc/hosts` files like this:
```
127.0.0.1 cloud.owncloud.test
127.0.0.1 keycloak.owncloud.test
127.0.0.1 ldap.owncloud.test
127.0.0.1 traefik.owncloud.test
```
After that you're ready to start the application stack:
`docker-compose up -d`
You now can visit the cloud, oC10 or oCIS depending on the user configuration. Marie defaults to oC10 and Richard and Einstein default to oCIS, but you can change the ownCloud selector at any time in the LDAP management UI.

8
docs/ocis/deployment/ocis_hello.md
View File

@@ -74,7 +74,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
# JWT secret which is used for the storage provider. Must be changed in order to have a secure oCIS. Defaults to "Pive-Fumkiu4"
OCIS_JWT_SECRET=
# JWT secret which is used for uploads to create transfer tokens. Must be changed in order to have a secure oCIS. Defaults to "replace-me-with-a-transfer-secret"
OCIS_TRANSFER_SECRET=
STORAGE_TRANSFER_SECRET=
### oCIS Hello settings ###
# oCIS Hello version. Defaults to "latest"
@@ -103,7 +103,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
`docker-compose up -d`
- You now can visit oCIS and are able to switch to the Hello extension by using the application switcher on the top right corner of ownCloud Web.
- You now can visit oCIS and are able to switch to the Hello extension by using the application switcher on the top right corner of ownCloud Web. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.
## Local setup
@@ -115,11 +115,11 @@ On Linux and macOS you can add them to your `/etc/hosts` files like this:
```
127.0.0.1 ocis.owncloud.test
127.0.0.1 traefik.owncloud.testt
127.0.0.1 traefik.owncloud.test
```
After that you're ready to start the application stack:
`docker-compose up -d`
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You are now able to switch to the Hello extension by using the application switcher on the top right corner of ownCloud Web.
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You are now able to switch to the Hello extension by using the application switcher on the top right corner of ownCloud Web. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.

8
docs/ocis/deployment/ocis_keycloak.md
View File

@@ -19,7 +19,7 @@ geekdocFilePath: ocis_keycloak.md
The docker stack consists 4 containers. One of them is Traefik, a proxy which is terminating ssl and forwards the requests to oCIS in the internal docker network.
Keykloak add two containers: Keycloak itself and a PostgreSQL as database. Keycloak will be configured as oCIS' IDP instead of the internal IDP [LibreGraph Connect]({{< ref "../../extensions/idp" >}})
Keycloak add two containers: Keycloak itself and a PostgreSQL as database. Keycloak will be configured as oCIS' IDP instead of the internal IDP [LibreGraph Connect]({{< ref "../../extensions/idp" >}})
The other container is oCIS itself running all extensions in one container. In this example oCIS uses [oCIS storage driver]({{< ref "../../extensions/storage/storages#storage-drivers" >}})
@@ -77,7 +77,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
# JWT secret which is used for the storage provider. Must be changed in order to have a secure oCIS. Defaults to "Pive-Fumkiu4"
OCIS_JWT_SECRET=
# JWT secret which is used for uploads to create transfer tokens. Must be changed in order to have a secure oCIS. Defaults to "replace-me-with-a-transfer-secret"
OCIS_TRANSFER_SECRET=
STORAGE_TRANSFER_SECRET=
### Keycloak ###
# Domain of Keycloak, where you can find the management and authentication frontend. Defaults to "keycloak.owncloud.test"
@@ -119,7 +119,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
`docker-compose up -d`
* You now can visit oCIS, Keycloak and Traefik dashboard on your configured domains
* You now can visit oCIS, Keycloak and Traefik dashboard on your configured domains. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.
## Local setup
For a more simple local ocis setup see [Getting started]({{< ref "../getting-started" >}})
@@ -139,4 +139,4 @@ After that you're ready to start the application stack:
Open https://keycloak.owncloud.test in your browser and accept the invalid certificate warning.
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You now can login to oCIS with the demo users.
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You now can login to oCIS with the demo users. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.

6
docs/ocis/deployment/ocis_s3.md
View File

@@ -76,7 +76,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
# JWT secret which is used for the storage provider. Must be changed in order to have a secure oCIS. Defaults to "Pive-Fumkiu4"
OCIS_JWT_SECRET=
# JWT secret which is used for uploads to create transfer tokens. Must be changed in order to have a secure oCIS. Defaults to "replace-me-with-a-transfer-secret"
OCIS_TRANSFER_SECRET=
STORAGE_TRANSFER_SECRET=
### MINIO / S3 settings ###
# Domain of MinIO where the Web UI is accessible. Defaults to "minio.owncloud.test".
@@ -116,7 +116,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
`docker-compose up -d`
* You now can visit oCIS and are able to use it just normally. If you log into the web UI of MinIO, you will see blobs of files you uploaded.
* You now can visit oCIS and are able to use it just normally. If you log into the web UI of MinIO, you will see blobs of files you uploaded. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.
## Local setup
For a more simple local ocis setup see [Getting started]({{< ref "../getting-started" >}})
@@ -134,4 +134,4 @@ After that you're ready to start the application stack:
`docker-compose up -d`
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You now can use oCIS normally and should now upload a file. Open https://minio.owncloud.test in your browser and accept the invalid certificate warning, after that you will see blobs of files you have uploaded to oCIS.
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You now can use oCIS normally and should now upload a file. Open https://minio.owncloud.test in your browser and accept the invalid certificate warning, after that you will see blobs of files you have uploaded to oCIS. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.

6
docs/ocis/deployment/ocis_traefik.md
View File

@@ -71,7 +71,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
# JWT secret which is used for the storage provider. Must be changed in order to have a secure oCIS. Defaults to "Pive-Fumkiu4"
OCIS_JWT_SECRET=
# JWT secret which is used for uploads to create transfer tokens. Must be changed in order to have a secure oCIS. Defaults to "replace-me-with-a-transfer-secret"
OCIS_TRANSFER_SECRET=
STORAGE_TRANSFER_SECRET=
```
You are installing oCIS on a server and Traefik will obtain valid certificates for you so please remove `INSECURE=true` or set it to `false`.
@@ -94,7 +94,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
`docker-compose up -d`
* You now can visit oCIS and Traefik dashboard on your configured domains
* You now can visit oCIS and Traefik dashboard on your configured domains. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.
## Local setup
For a more simple local ocis setup see [Getting started]({{< ref "../getting-started" >}})
@@ -111,4 +111,4 @@ After that you're ready to start the application stack:
`docker-compose up -d`
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You now can login to oCIS with the default users, which also can be found here: [Getting started]({{< ref "../getting-started#login-to-ocis-web" >}})
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You now can login to oCIS with the default users, which also can be found here: [Getting started]({{< ref "../getting-started#login-to-ocis-web" >}}). You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.

6
docs/ocis/deployment/ocis_wopi.md
View File

@@ -79,7 +79,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
# JWT secret which is used for the storage provider. Must be changed in order to have a secure oCIS. Defaults to "Pive-Fumkiu4"
OCIS_JWT_SECRET=
# JWT secret which is used for uploads to create transfer tokens. Must be changed in order to have a secure oCIS. Defaults to "replace-me-with-a-transfer-secret"
OCIS_TRANSFER_SECRET=
STORAGE_TRANSFER_SECRET=
### Wopi server settings ###
# oCIS Wopi server version. Defaults to "latest"
@@ -135,7 +135,7 @@ See also [example server setup]({{< ref "preparing_server" >}})
`docker-compose up -d`
* You now can visit oCIS and are able to open an office document in your browser.
* You now can visit oCIS and are able to open an office document in your browser. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.
## Local setup
For a more simple local ocis setup see [Getting started]({{< ref "../getting-started" >}})
@@ -156,4 +156,4 @@ After that you're ready to start the application stack:
Open https://collabora.owncloud.test and https://wopiserver.owncloud.test in your browser and accept the invalid certificate warning.
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You are now able to open an office document in your browser.
Open https://ocis.owncloud.test in your browser and accept the invalid certificate warning. You are now able to open an office document in your browser. You may need to wait some minutes until all services are fully ready, so make sure that you try to reload the pages from time to time.

68
docs/ocis/deployment/systemd.md Normal file
View File

@@ -0,0 +1,68 @@
---
title: "Systemd service"
date: 2020-09-27T06:00:00+01:00
weight: 16
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/ocis/deployment
geekdocFilePath: systemd.md
---
{{< toc >}}
## Install the oCIS binary
Download the oCIS binary of your preferred version and for your CPU architecture and operating system from [download.owncloud.com](https://download.owncloud.com/ocis/ocis).
Rename the downloaded binary to `ocis` and move it to `/usr/bin/`. As a next step, you need to mark it as executable with `chmod +x /usr/bin/ocis`.
When you now run `ocis help` on your command line, you should see the available options for the oCIS command.
## Systemd service definition
Create the Systemd service definition for oCIS in the file `/etc/systemd/system/ocis.service` with following content:
```
[Unit]
Description=OCIS server
[Service]
Type=simple
User=root
Group=root
EnvironmentFile=/etc/ocis/ocis.env
ExecStart=ocis server
Restart=always
[Install]
WantedBy=multi-user.target
```
For reasons of simplicity we are using the root user and group to run oCIS which is not recommended. Please use a non-root user in production environments and modify the oCIS service definition accordingly.
In the service definition we referenced `/etc/ocis/ocis.env` as our file containing environment variables for the oCIS process.
In order to create the file we need first to create the folder `/etc/ocis/` and than we can add the actual `/etc/ocis/ocis.env` with following content:
```
OCIS_URL=https://some-hostname-or-ip:9200
PROXY_HTTP_ADDR=0.0.0.0:9200
OCIS_LOG_LEVEL=error
GLAUTH_LDAPS_CERT=/etc/ocis/ldap/ldaps.crt
GLAUTH_LDAPS_KEY=/etc/ocis/ldap/ldaps.key
IDP_TRANSPORT_TLS_CERT=/etc/ocis/idp/server.crt
IDP_TRANSPORT_TLS_KEY=/etc/ocis/idp/server.key
PROXY_TRANSPORT_TLS_CERT=/etc/ocis/proxy/server.crt
PROXY_TRANSPORT_TLS_KEY=/etc/ocis/proxy/server.key
```
Please change your `OCIS_URL` in order to reflect your actual deployment.
## Starting the oCIS service
You can enable oCIS now by running `systemctl enable --now ocis`. It will ensure that oCIS also is restarted after a reboot of the host.
If you need to restart oCIS because of configuration changes in `/etc/ocis/ocis.env`, run `systemctl restart ocis`.
You can have a look at the logs of oCIS by issuing `journalctl -f -u ocis`.

2
docs/ocis/development/extensions.md
View File

@@ -7,7 +7,7 @@ geekdocEditPath: edit/master/docs/ocis/development
geekdocFilePath: extensions.md
---
oCIS is all about files, sync and share - but most of the time there is more you want to do with your files, e.g. having a different view on your photo collection or editing your offices files in an online file editor. ownCloud 10 faced the same problem and solved it with `applications`, which can extend the functionality of ownCloud 10 in a wide range. Since oCIS is different in its architecture compared to ownCloud 10, we had to come up with a similiar (yet slightly different) solution. To extend the functionality of oCIS, you can write or install `extensions`. An extension is basically any running code which integrates into oCIS and provides functionality to oCIS and its users. Because extensions are just microservices providing an API, you can technically choose any programming language you like - a huge improvement to ownCloud 10, where it was nearly impossible to use a different programming language than PHP.
oCIS is all about files, sync and share - but most of the time there is more you want to do with your files, e.g. having a different view on your photo collection or editing your offices files in an online file editor. ownCloud 10 faced the same problem and solved it with `applications`, which can extend the functionality of ownCloud 10 in a wide range. Since oCIS is different in its architecture compared to ownCloud 10, we had to come up with a similar (yet slightly different) solution. To extend the functionality of oCIS, you can write or install `extensions`. An extension is basically any running code which integrates into oCIS and provides functionality to oCIS and its users. Because extensions are just microservices providing an API, you can technically choose any programming language you like - a huge improvement to ownCloud 10, where it was nearly impossible to use a different programming language than PHP.
We will now introduce you to the oCIS extension system and show you how you can create a custom extension yourself.

2
docs/ocis/development/getting-started.md
View File

@@ -27,7 +27,7 @@ If you find tools needed besides the mentioned above, please feel free to open a
oCIS consists of multiple micro services, also called extensions. We started by having standalone repositories for each of them, but quickly noticed that this adds a time consuming overhead for developers. So we ended up with a monorepo housing all the extensions in one repository.
Each extension lives in a subfolder (eg. `accounts` or `settings`) within this respository as an independent Go module, following the [golang-standard project-layout](https://github.com/golang-standards/project-layout). They have common Makefile targets and can be used to change, build and run individual extensions. This allows us to version and release each extension independently.
Each extension lives in a subfolder (eg. `accounts` or `settings`) within this repository as an independent Go module, following the [golang-standard project-layout](https://github.com/golang-standards/project-layout). They have common Makefile targets and can be used to change, build and run individual extensions. This allows us to version and release each extension independently.
The `ocis` folder contains our [go-micro](https://github.com/asim/go-micro/) and [suture](https://github.com/thejerf/suture) based runtime. It is used to import all extensions and implements commands to manage them, similar to a small orchestrator. With the resulting oCIS binary you can start single extensions or even all extensions at the same time.

2
docs/ocis/flow-docs/request-flow.md
View File

@@ -50,7 +50,7 @@ sequenceDiagram
end
proxy->>+accounts: TODO API call to exchange sub@iss with account UUID
Note over proxy,accounts: does not autoprovision users. They are explicitly provsioned later.
Note over proxy,accounts: does not autoprovision users. They are explicitly provisioned later.
alt account exists or has been migrated

4
docs/ocis/getting-started/_index.md
View File

@@ -33,11 +33,11 @@ You can find the latest official release of oCIS at [our download mirror](https:
The latest build from the master branch can be found at [our download mirrors testing section](https://download.owncloud.com/ocis/ocis/testing/).
To run oCIS as binary you need to download it first and then run the following commands.
For this example, assuming version 1.10.0 of oCIS running on a Linux AMD64 host:
For this example, assuming version 1.12.0 of oCIS running on a Linux AMD64 host:
```console
# download
curl https://download.owncloud.com/ocis/ocis/1.10.0/ocis-1.10.0-linux-amd64 --output ocis
curl https://download.owncloud.com/ocis/ocis/1.12.0/ocis-1.12.0-linux-amd64 --output ocis
# make binary executable
chmod +x ocis

5
docs/ocis/getting-started/demo-users.md
View File

@@ -9,6 +9,11 @@ geekdocFilePath: demo-users.md
As long as oCIS is released as [technology preview]({{< ref "../release_roadmap#release_roadmap" >}}) it will come with default demo users. These enable you to do quick testing and developing.
{{< hint info >}}
To skip the generation of demo users, run the inital setup step with an additional environment variable.
`ACCOUNTS_DEMO_USERS_AND_GROUPS=false ./bin/ocis server` generates only the admin, and one user for IDP and Reva respectively.
{{< /hint >}}
Following users are available in the demo set:
| username | password | email | role | groups |

73
docs/ocis/migration.md
View File

@@ -78,7 +78,7 @@ _TODO @butonic add ADR for OpenID Connect_
#### User impact
When introducing OpenID Connect, the clients will detect the new authentication scheme when their current way of authenticating returns an error. Users will then have to
reauthorize at the OpenID Connecd IdP, which again, may be configured to skip the consent step for trusted clients.
reauthorize at the OpenID Connect IdP, which again, may be configured to skip the consent step for trusted clients.
#### Steps
1. There are multiple products that can be used as an OpenID Connect IdP. We test with [LibreGraph Connect](https://github.com/libregraph/lico), which is also [embedded in oCIS](https://github.com/owncloud/web/). Other alternatives include [Keycloak](https://www.keycloak.org/) or [Ping](https://www.pingidentity.com/). Please refer to the corresponding setup instructions for the product you intent to use.
@@ -106,7 +106,7 @@ Should there be problems with OpenID Connect at this point you can disable the a
<div style="break-after: avoid"></div>
Legacy clients relying on Basic auth or app passwords need to be migrated to OpenId Connect to work with oCIS. For a transition period Basic auth in oCIS can be enabled with `PROXY_ENABLE_BASIC_AUTH=true`, but we strongly recommend adopting OpenID Connect for other tools as well.
While OpenID Connect providers will send an `iss` and `sub` claim that relying parties (services like oCIS or ownCloud 10) can use to identify users we recommend introducing a dedicated, globally unique, persistent, non-reassignable user identifier like a UUID for every user. This `ownclouduuid` shold be sent as an additional claim to save additional lookups on the server side. It will become the user id in oCIS, e.g. when searching for recipients the `ownclouduuid` will be used to persist permissions with the share manager. It has a different purpose than the ownCloud 10 username, which is used to login. Using UUIDs we can not only mitigate username collisions when merging multiple instances but also allow renaming usernames after the migration to oCIS has been completed.
While OpenID Connect providers will send an `iss` and `sub` claim that relying parties (services like oCIS or ownCloud 10) can use to identify users we recommend introducing a dedicated, globally unique, persistent, non-reassignable user identifier like a UUID for every user. This `ownclouduuid` should be sent as an additional claim to save additional lookups on the server side. It will become the user id in oCIS, e.g. when searching for recipients the `ownclouduuid` will be used to persist permissions with the share manager. It has a different purpose than the ownCloud 10 username, which is used to login. Using UUIDs we can not only mitigate username collisions when merging multiple instances but also allow renaming usernames after the migration to oCIS has been completed.
<div class="editpage">
@@ -119,10 +119,10 @@ _Feel free to add your question as a PR to this document using the link at the t
### Stage 3: introduce oCIS interally
Befor letting oCIS handle end user requests we will first make it available in the internal network. By subsequently adding services we can add functionality and verify the services work as intended.
Before letting oCIS handle end user requests we will first make it available in the internal network. By subsequently adding services we can add functionality and verify the services work as intended.
Start oCIS backend and make read only tests on existing data using the `owncloudsql` storage driver which will read (and write)
- blobs from the same datadirectory layout as in ownCloud 10
- blobs from the same data directory layout as in ownCloud 10
- metadata from the ownCloud 10 database:
The oCIS share manager will read share information from the ownCloud database using an `owncloud` driver as well.
@@ -139,11 +139,11 @@ None, only administrators will be able to explore oCIS during this stage.
#### Steps and verifications
We are going to run and explore a series of services that will together handle the same requests as ownCloud 10. For initial exploration the oCIS binary is recommended. The services can later be deployed using a single oCIS runtime or in multiple cotainers.
We are going to run and explore a series of services that will together handle the same requests as ownCloud 10. For initial exploration the oCIS binary is recommended. The services can later be deployed using a single oCIS runtime or in multiple containers.
##### Storage provider for file metadata
1. Deploy OCIS storage provider with owncloudsql driver.
1. Deploy OCIS storage provider with the `owncloudsql` driver.
2. Set `read_only: true` in the storage provider config. <div class="editpage">_TODO @butonic add read only flag to storage drivers_</div>
3. Use cli tool to list files using the CS3 api
@@ -194,7 +194,7 @@ When reading the files from oCIS return the same `uuid`. It can be migrated to a
2. Use curl to list spaces using graph drives endpoint
##### owncloud flavoured WebDAV endpoint
1. Deploy Ocdav
1. Deploy ocdav
2. Use curl to send PROPFIND
##### data provider for up and download
@@ -205,13 +205,13 @@ When reading the files from oCIS return the same `uuid`. It can be migrated to a
Deploy ...
##### share manager
Deploy share manager with owncloud driver
Deploy share manager with ownCloud driver
##### reva gateway
1. Deploy gateway to authenticate requests? I guess we need that first... Or we need the to mint a token. Might be a good exercise.
##### automated deployment
Finally, deploy OCIS with a config to set up everything running in a single oCIS runtime or in multiple containers.
Finally, deploy oCIS with a config to set up everything running in a single oCIS runtime or in multiple containers.
#### Rollback
You can stop the oCIS process at any time.
@@ -280,7 +280,7 @@ The IP address of the ownCloud host changes. There is no change for the file syn
2. Verify the requests are routed based on the ownCloud 10 routing policy `oc10` by default
##### Test user based routing
1. Change the routing policy for a user or an early adoptors group to `ocis` <div class="editpage">_TODO @butonic currently, the migration selector will use the `ocis` policy for users that have been added to the accounts service. IMO we need to evaluate a claim from the IdP._</div>
1. Change the routing policy for a user or an early adopters group to `ocis` <div class="editpage">_TODO @butonic currently, the migration selector will use the `ocis` policy for users that have been added to the accounts service. IMO we need to evaluate a claim from the IdP._</div>
2. Verify the requests are routed based on the oCIS routing policy `oc10` for 'migrated' users.
At this point you are ready to rock & roll!
@@ -322,8 +322,8 @@ _TODO @butonic update performance comparisons nightly_
#### Steps
There are several options to move users to the oCIS backend:
- Use a canary app to let users decide thamselves
- Use an early adoptors group with an opt in
- Use a canary app to let users decide themselves
- Use an early adopters group with an opt in
- Force migrate users in batch or one by one at the administrators will
#### Verification
@@ -333,15 +333,14 @@ The same verification steps as for the internal testing stage apply. Just from t
Until now, the oCIS configuration mimics ownCloud 10 and uses the old data directory layout and the ownCloud 10 database. Users can seamlessly be switched from ownCloud 10 to oCIS and back again.
<div class="editpage">
_TODO @butonic we need a canary app that allows users to decide for themself which backend to use_
_TODO @butonic we need a canary app that allows users to decide for themselves which backend to use_
</div>
<div style="break-after: page"></div>
#### Notes
Running the two systems in parallel stage
Try to keep the duration of this stage short. Until now we only added services and made the system more complex. oCIS aims to reduce the maintenance cost of an ownCloud instance. You will not get there if you keep both systems alive.
Running the two systems in parallel requires additional maintenance effort. Try to keep the duration of this stage short. Until now, we only added services and made the system more complex. oCIS aims to reduce the maintenance cost of an ownCloud instance. You will not get there if you keep both systems alive.
<div class="editpage">
@@ -352,7 +351,29 @@ _Feel free to add your question as a PR to this document using the link at the t
<div style="break-after: page"></div>
### Stage-7: shut down ownCloud 10
### Stage-7: introduce spaces using ocis
To encourage users to switch you can promote the workspaces feature that is built into oCIS. The ownCloud 10 storage backend can be used for existing users. New users and group or project spaces can be provided by storage providers that better suit the underlying storage system.
#### Steps
First, the admin needs to
- deploy a storage provider with the storage driver that best fits the underlying storage system and requirements.
- register the storage in the storage registry with a new storage id (we recommend a uuid).
Then a user with the necessary create storage space role can create a storage space and assign Managers.
<div class="editpage">
_TODO @butonic a user with management permission needs to be presented with a list of storage spaces where he can see the amount of free space and decide on which storage provider the storage space should be created. For now a config option for the default storage provider for a specific type might be good enough._
</div>
#### Verification
The new storage space should show up in the `/graph/drives` endpoint for the managers and the creator of the space.
#### Notes
Depending on the requirements and acceptable tradeoffs, a database less deployment using the ocis or s3ng storage driver is possible. There is also a [cephfs driver](https://github.com/cs3org/reva/pull/1209) on the way, that directly works on the API level instead of POSIX.
### Stage-8: shut down ownCloud 10
Disable ownCloud 10 in the proxy, all requests are now handled by oCIS, shut down oc10 web servers and redis (or keep for calendar & contacts only? rip out files from oCIS?)
#### User impact
@@ -387,7 +408,7 @@ _Feel free to add your question as a PR to this document using the link at the t
<div style="break-after: page"></div>
### Stage 8: storage migration
### Stage 9: storage migration
To get rid of the database we will move the metadata from the old ownCloud 10 database into dedicated storage providers. This can happen in a user by user fashion. group drives can properly be migrated to group, project or workspaces in this stage.
#### User impact
@@ -401,15 +422,15 @@ Noticeable performance improvements because we effectively shard the storage log
_TODO @butonic implement `ownclouds3` based on `s3ng`_
_TODO @butonic implement tiered storage provider for seamless migration_
_TODO @butonic document how to manually do that until the storge registry can discover that on its own._
_TODO @butonic document how to manually do that until the storage registry can discover that on its own._
</div>
#### Verification
Start with a test user, then move to early adoptors and finally migrate all users.
Start with a test user, then move to early adopters and finally migrate all users.
#### Rollback
To switch the storage provider again the same storage space migration can be performed again: copy medatata and blob data using the CS3 api, then change the responsible storage provider in the storage registry.
To switch the storage provider again the same storage space migration can be performed again: copy metadata and blob data using the CS3 api, then change the responsible storage provider in the storage registry.
#### Notes
<div style="break-after: avoid"></div>
@@ -426,13 +447,13 @@ _Feel free to add your question as a PR to this document using the link at the t
<div style="break-after: page"></div>
### Stage-9: share metadata migration
### Stage-10: share metadata migration
Migrate share data to _yet to determine_ share manager backend and shut down ownCloud database.
The ownCloud 10 database still holds share information in the `oc_share` and `oc_share_external` tables. They are used to efficiently answer queries about who shared what with whom. In oCIS shares are persisted using a share manager and if desired these grants are also sent to the storage provider so it can set ACLs if possible. Only one system should be responsible for the shares, which in case of treating the storage as the primary source effectively turns the share manager into a cache.
#### User impact
Depending on chosen the share manager provider some sharing requests should be faster: listing incoming and outgoing shares is no longer bound to the ownCloud 10 database but to whatever technology is used by the share provdier:
Depending on chosen the share manager provider some sharing requests should be faster: listing incoming and outgoing shares is no longer bound to the ownCloud 10 database but to whatever technology is used by the share provider:
- For non HA scenarios they can be served from memory, backed by a simple json file.
- TODO: implement share manager with redis / nats / ... key value store backend: use the micro store interface please ...
@@ -452,7 +473,7 @@ _TODO for storage provider as source of truth persist ALL share data in the stor
</div>
#### Verification
After copying all metadata start a dedicated gateway and change the configuration to use the new share manager. Route a test user, a test group and early adoptors to the new gateway. When no problems occur you can stirt the desired number of share managers and roll out the change to all gateways.
After copying all metadata start a dedicated gateway and change the configuration to use the new share manager. Route a test user, a test group and early adopters to the new gateway. When no problems occur you can start the desired number of share managers and roll out the change to all gateways.
<div class="editpage">
@@ -465,8 +486,8 @@ To switch the share manager to the database one revert routing users to the new
<div class="editpage">
### Stage-10
Profit! Well, on the one hand you do not need to maintain a clustered database setup and can rely on the storage system. On the other hand you are now in microservice wonderland and will have to relearn how to identify bottlenecks and scale oCIS accordingly. The good thing is that tools like jaeger and prometheus have evolved and will help you understand what is going on. But this is a different Topic. See you on the other side!
### Stage-11
Profit! Well, on the one hand you do not need to maintain a clustered database setup and can rely on the storage system. On the other hand you are now in microservice wonderland and will have to relearn how to identify bottlenecks and scale oCIS accordingly. The good thing is that tools like jaeger and prometheus have evolved and will help you understand what is going on. But this is a different topic. See you on the other side!
#### FAQ
_Feel free to add your question as a PR to this document using the link at the top of this page!_
@@ -547,7 +568,7 @@ The `filecache` table itself has more metadata:
| Field | Type | Null | Key | Default | Extra | Comment | Migration |
|--------------------|---------------|------|-----|---------|----------------|----------------|----------------|
| `fileid` | bigint(20) | NO | PRI | NULL | auto_increment | | MUST become the oCIS `opaqueid` of a file reference. `ocis` driver stores it in extendet attributes and can use numbers as node ids on disk. for eos see note below table |
| `fileid` | bigint(20) | NO | PRI | NULL | auto_increment | | MUST become the oCIS `opaqueid` of a file reference. `ocis` driver stores it in extended attributes and can use numbers as node ids on disk. for eos see note below table |
| `storage` | int(11) | NO | MUL | 0 | | *the filecache holds metadata for multiple storages* | corresponds to an oCIS *storage space* |
| `path` | varchar(4000) | YES | | NULL | | *the path relative to the storages root* | MUST become the `path` relative to the storage root. `files` prefix needs to be trimmed. |
| `path_hash` | varchar(32) | NO | | | | *mysql once had problems indexing long paths, so we stored a hash for lookup by path. | - |

47
docs/ocis/release_notes.md
View File

@@ -7,6 +7,53 @@ geekdocEditPath: edit/master/docs/ocis
geekdocFilePath: release_notes.md
---
## ownCloud Infinite Scale 1.12.0 Technology Preview
Version 1.12.0 is a maintenance release with the foundations for the 'Spaces' feature and for viewer/editor application integrations. The Infinite Scale backend has been further hardened by fixing known issues, improving error handling and stabilizing existing features. Apart from bugfixing, ownCloud Web 4.2.0 has received a number of usability and design improvements for sharing and the file list.
The most prominent changes in ownCloud Infinite Scale 1.12.0 and ownCloud Web 4.2.0 comprise:
- The Infinite Scale backend now supports the first parts of the 'Spaces' feature
- Creating a new Space is now possible via Graph API [#2471](https://github.com/owncloud/ocis/pull/2471)
- A new sharing role, `Manager`, has been introduced for Spaces [cs3org/reva#2065](https://github.com/cs3org/reva/pull/2065)
- A capability for Spaces has been added [cs3org/reva#2015](https://github.com/cs3org/reva/pull/2015)
- Infinite Scale now provides an app provider and an app registry as a foundation for integrations with viewer/editor applications. [#2204](https://github.com/owncloud/ocis/pull/2204)
- ownCloud Web now has a re-designed sharing role selection. [#5632](https://github.com/owncloud/web/pull/5632)
- ownCloud Web now shows people in sharing as a collapsed list of avatars to save space. This can be expanded to show more details and the full list. [#5758](https://github.com/owncloud/web/pull/5758)
- ownCloud Web now shows sharing information in file/folder details. [#5735](https://github.com/owncloud/web/issues/5735)
- The file size calculation in ownCloud Web has been changed from base-2 (e.g., KB / Kibibyte) to base-10 (e.g., kB / Kilobyte) to match better with user expectations. [#5739](https://github.com/owncloud/web/pull/5739)
- The URL encoding/decoding in ownCloud Web has been improved. [#5714](https://github.com/owncloud/web/issues/5714)
- ownCloud Web now provides a robots.txt file. [#5762](https://github.com/owncloud/web/pull/5762)
You can also read the full [ownCloud Infinite Scale changelog](https://github.com/owncloud/ocis/releases/tag/v1.12.0) and [ownCloud Web changelog](https://github.com/owncloud/web/releases/tag/v4.2.0) for further details on what has changed.
### Breaking changes
{{< hint warning >}}
We are currently in a Tech Preview state and breaking changes may occur at any time. For more information see our [release roadmap]({{< ref "./release_roadmap" >}})
{{< /hint >}}
## ownCloud Infinite Scale 1.11.0 Technology Preview
Version 1.11.0 brings new features, usability improvements and bug fixes. ownCloud Web 4.1.0 now supports drag & drop and allows users to do actions (e.g., sharing) for the folder they are currently in.
The most prominent changes in ownCloud Infinite Scale 1.11.0 and ownCloud Web 4.1.0 comprise:
- ownCloud Web now supports drag & drop to move files/folders. [#5592](https://github.com/owncloud/web/issues/5592)
- The right sidebar in ownCloud Web can now be collapsed and expanded. This change also allows to open the sidebar without selecting a file/folder which will select the current folder and enable the user to do actions (e.g., sharing) for it. [#5165](https://github.com/owncloud/web/issues/5165)
- The right sidebar in ownCloud Web now presents details for multiple selected files/folders. [#5164](https://github.com/owncloud/web/issues/5164)
- The owncloud/ocis Docker image now uses a non-root user for improved security. This is a breaking change for existing Docker deployments. The permission on the files and folders in persistent volumes need to be changed to the UID and GID used for oCIS (default 1000:1000 if not changed by the user). [#2380](https://github.com/owncloud/ocis/pull/2380)
- Infinite Scale now supports request tracing through the whole stack to facilitate debugging. [#1984](https://github.com/cs3org/reva/pull/1984)
- Infinite Scale now provides a WebDAV endpoint for the new Spaces feature [#1803](https://github.com/cs3org/reva/pull/1803)
- The Infinite Scale backend has been further hardened by fixing known issues, improving error handling and stabilizing existing features.
- All test scenarios for file-related operations now pass in Infinite Scale (e.g., file operations, trash bin).
You can also read the full [ownCloud Infinite Scale changelog](https://github.com/owncloud/ocis/releases/tag/v1.11.0) and [ownCloud Web changelog](https://github.com/owncloud/web/releases/tag/v4.1.0) for further details on what has changed.
### Breaking changes
{{< hint warning >}}
We are currently in a Tech Preview state and breaking changes may occur at any time. For more information see our [release roadmap]({{< ref "./release_roadmap" >}})
{{< /hint >}}
## ownCloud Infinite Scale 1.10.0 Technology Preview
Version 1.10.0 brings new features, usability improvements and bug fixes. ownCloud Web 4.0.0 now supports ONLYOFFICE document editors and can search/filter files and folders. Furthermore it brings a new context menu for file actions that can be accessed via right click and comes with a big bunch of other notable improvements and fixes.

77
docs/ocis/storage-backends/cephfs.md Normal file
View File

@@ -0,0 +1,77 @@
---
title: "cephfs"
date: 2021-09-13T15:36:00+01:00
weight: 30
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/ocis/storage-backends/
geekdocFilePath: cephfs.md
---
{{< toc >}}
oCIS intends to make the aspects of existing storage systems available as transparently as possible, but the static sync algorithm of the desktop client relies on some form of recursive change time propagation on the server side to detect changes. While this can be bolted on top of existing file systems with inotify, the kernel audit or a fuse based overlay filesystem, a storage system that already implements this aspect is preferable. Aside from EOS, cephfs supports a recursive change time that oCIS can use to calculate an etag for the webdav API.
## Development
The cephfs development happens in a [Reva branch](https://github.com/cs3org/reva/pull/1209) and is currently driven by CERN.
## Architecture
In the original approach the driver was based on the [localfs](https://github.com/cs3org/reva/blob/a8c61401b662d8e09175416c0556da8ef3ba8ed6/pkg/storage/utils/localfs/localfs.go) driver, relying on a locally mounted cephfs. It would interface with it using the POSIX apis. This has been changed to directly call the Ceph API using https://github.com/ceph/go-ceph. It allows using the ceph admin APIs to create subvolumes for user homes and maintain a file id to path mapping using symlinks.
## Implemented Aspects
The recursive change time built ino cephfs is used to implement the etag propagation expected by the ownCloud clients. This allows oCIS to pick up changes that have been made by external tools, bypassing any oCIS APIs.
Like other filesystems cephfs uses inodes and like most other filesystems inodes are reused. To get stable file identifiers the current cephfs driver assigns every node a file id and maintains a custom fileid to path mapping in a system directory:
```
/tmp/cephfs $ tree -a
.
├── reva
│ └── einstein
│ ├── Pictures
│ └── welcome.txt
└── .reva_hidden
├── .fileids
│ ├── 50BC39D364A4703A20C58ED50E4EADC3_570078 -> /tmp/cephfs/reva/einstein
│ ├── 571EFB3F0ACAE6762716889478E40156_570081 -> /tmp/cephfs/reva/einstein/Pictures
│ └── C7A1397524D0419B38D04D539EA531F8_588108 -> /tmp/cephfs/reva/einstein/welcome.txt
└── .uploads
```
Versions are not file but snapshot based, a [native feature of cephfs](https://docs.ceph.com/en/latest/dev/cephfs-snapshots/). The driver maps entries in the native cephfs `.snap` folder to the CS3 api recycle bin concept and makes them available in the web UI using the versions sidebar. Snapshots can be triggered by users themselves or on a schedule.
Trash is not implemented, as cephfs has no native recycle bin and instead relies on the snapshot functionality that can be triggered by end users. It should be possible to automatically create a snapshot before deleting a file. This needs to be explored.
Shares [are be mapped to ACLs](https://github.com/cs3org/reva/pull/1209/files#diff-5e532e61f99bffb5754263bc6ce75f84a30c6f507a58ba506b0b487a50eda1d9R168-R224) supported by cephfs. The share manager is used to persist the intent of a share and can be used to periodically verify or reset the ACLs on cephfs.
## Future work
- The spaces concept matches cephfs subvolumes. We can implement the CreateStorageSpace call with that, keep track of the list of storage spaces using symlinks, like for the id based lookup.
- The share manager needs a persistence layer.
- Currently we persist using a single json file.
- As it basically provides two lists, *shared with me* and *shared with others*, we could persist them directly on cephfs!
- If needed for redundancy, the share manager can be run multiple times, backed by the same cephfs
- To save disk io the data can be cached in memory, and invalidated using stat requests.
- A good tradeoff would be a folder for each user with a json file for each list. That way, we only have to open and read a single file when the user want's to list the shares.
- To allow deprovisioning a user the data should by sharded by userid. That way all share information belonging to a user can easily be removed from the system. If necessary it can also be restored easily by copying the user specific folder back in place.
- For consistency over metadata any file blob data, backups can be done using snapshots.
- An example where einstein has shared a file with marie would look like this on disk:
```
/tmp/cephfs $ tree -a
.
├── reva
│ └── einstein
│ ├── Pictures
│ └── welcome.txt
├── .reva_hidden
│ ├── .fileids
│ │ ├── 50BC39D364A4703A20C58ED50E4EADC3_570078 -> /tmp/cephfs/reva/einstein
│ │ ├── 571EFB3F0ACAE6762716889478E40156_570081 -> /tmp/cephfs/reva/einstein/Pictures
│ │ └── C7A1397524D0419B38D04D539EA531F8_588108 -> /tmp/cephfs/reva/einstein/welcome.txt
│ └── .uploads
└── .reva_share_manager
├── einstein
│ └── sharedWithOthers.json
└── marie
└── sharedWithMe.json
```
- The fileids should [not be based on the path](https://github.com/cs3org/reva/pull/1209/files#diff-eba5c8b77ccdd1ac570c54ed86dfa7643b6b30e5625af191f789727874850172R125-R127) and instead use a uuid that is also persisted in the extended attributes to allow rebuilding the index from scratch if necessary.