mirror/opencloud
1
0
Fork 0
mirror of https://github.com/opencloud-eu/opencloud.git synced 2026-02-18 03:18:52 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into remove-default-insecure

Browse Source
This commit is contained in:
Willy Kloucek
2022-09-26 08:59:18 +02:00
parent 71790089fd fcff2b1724
commit 569ffd41f8
68 changed files with 4345 additions and 600 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.drone.env
View File

@@ -1,5 +1,5 @@
# The test runner source for API tests
CORE_COMMITID=67fad9d49fa7f3f42f77e882ee4cbb206464ca5d
CORE_COMMITID=abf37068ff0fb824fec331dce0a25c1d7ac85177
CORE_BRANCH=master
# The test runner source for UI tests

35
CHANGELOG.md
View File

@@ -6,14 +6,26 @@ The following sections list the changes for unreleased.
## Summary
* Bugfix - Fix the OIDC provider cache: [#4600](https://github.com/owncloud/ocis/pull/4600)
* Bugfix - Render webdav permissions as string in search report: [#4575](https://github.com/owncloud/ocis/issues/4575)
* Bugfix - Graph service now forwards trace context: [#4582](https://github.com/owncloud/ocis/pull/4582)
* Bugfix - Fix wopi access to public shares: [#4631](https://github.com/owncloud/ocis/pull/4631)
* Enhancement - Add Email templating: [#4564](https://github.com/owncloud/ocis/pull/4564)
* Enhancement - Add webURL to space root: [#4588](https://github.com/owncloud/ocis/pull/4588)
* Enhancement - Allow to configure applications in Web: [#4578](https://github.com/owncloud/ocis/pull/4578)
* Enhancement - Add thumbnails support for tiff and bmp files: [#4634](https://github.com/owncloud/ocis/pull/4634)
* Enhancement - Update reva: [#4588](https://github.com/owncloud/ocis/pull/4588)
## Details
* Bugfix - Fix the OIDC provider cache: [#4600](https://github.com/owncloud/ocis/pull/4600)
We've fixed the OIDC provider cache. It never had a cache hit before this fix. Under some
circumstances it could cause a painfully slow OCIS if the IDP wellknown endpoint takes some
time to respond.
https://github.com/owncloud/ocis/pull/4600
* Bugfix - Render webdav permissions as string in search report: [#4575](https://github.com/owncloud/ocis/issues/4575)
We now correctly render the `oc:permissions` of resources as a string.
@@ -25,6 +37,23 @@ The following sections list the changes for unreleased.
https://github.com/owncloud/ocis/pull/4582
* Bugfix - Fix wopi access to public shares: [#4631](https://github.com/owncloud/ocis/pull/4631)
I've added a request check to the public share authenticator middleware to allow wopi to access
public shares.
https://github.com/owncloud/ocis/issues/4382
https://github.com/owncloud/ocis/pull/4631
* Enhancement - Add Email templating: [#4564](https://github.com/owncloud/ocis/pull/4564)
We have added email templating to ocis. Which are send on the SpaceShared and ShareCreated
event.
https://github.com/owncloud/ocis/issues/4303
https://github.com/owncloud/ocis/pull/4564
https://github.com/cs3org/reva/pull/3252
* Enhancement - Add webURL to space root: [#4588](https://github.com/owncloud/ocis/pull/4588)
Add the web url to the space root on the graphAPI.
@@ -37,6 +66,12 @@ The following sections list the changes for unreleased.
https://github.com/owncloud/ocis/pull/4578
* Enhancement - Add thumbnails support for tiff and bmp files: [#4634](https://github.com/owncloud/ocis/pull/4634)
Support generating thumbnails for tiff and bmp files in the thumbnails service.
https://github.com/owncloud/ocis/pull/4634
* Enhancement - Update reva: [#4588](https://github.com/owncloud/ocis/pull/4588)
TBD

7
changelog/unreleased/add-email-templating.md Normal file
View File

@@ -0,0 +1,7 @@
Enhancement: Add Email templating
We have added email templating to ocis. Which are send on the SpaceShared and ShareCreated event.
https://github.com/owncloud/ocis/pull/4564
https://github.com/owncloud/ocis/issues/4303
https://github.com/cs3org/reva/pull/3252

6
changelog/unreleased/fix-oidc-provider-cache.md Normal file
View File

@@ -0,0 +1,6 @@
Bugfix: Fix the OIDC provider cache
We've fixed the OIDC provider cache. It never had a cache hit before this fix.
Under some circumstances it could cause a painfully slow OCIS if the IDP wellknown endpoint takes some time to respond.
https://github.com/owncloud/ocis/pull/4600

5
changelog/unreleased/thumbnails-tiff-bmp.md Normal file
View File

@@ -0,0 +1,5 @@
Enhancement: Add thumbnails support for tiff and bmp files
Support generating thumbnails for tiff and bmp files in the thumbnails service.
https://github.com/owncloud/ocis/pull/4634

6
changelog/unreleased/wopi-public-share.md Normal file
View File

@@ -0,0 +1,6 @@
Bugfix: Fix wopi access to public shares
I've added a request check to the public share authenticator middleware to allow wopi to access public shares.
https://github.com/owncloud/ocis/pull/4631
https://github.com/owncloud/ocis/issues/4382

4
docs/templates/ADOC.tmpl vendored
View File

@@ -13,9 +13,9 @@
`{{- $value }}`
{{- end }}
a| [subs=-attributes]
+{{.Type}}+
++{{.Type}}++
a| [subs=-attributes]
pass:[{{.DefaultValue}}]
++{{.DefaultValue}}++
a| [subs=-attributes]
{{.Description}}

36
go.mod
View File

@@ -7,11 +7,12 @@ require (
github.com/Masterminds/semver v1.5.0
github.com/MicahParks/keyfunc v1.2.2
github.com/ReneKroon/ttlcache/v2 v2.11.0
github.com/armon/go-radix v1.0.0
github.com/blevesearch/bleve/v2 v2.3.4
github.com/blevesearch/bleve_index_api v1.0.3
github.com/coreos/go-oidc/v3 v3.4.0
github.com/cs3org/go-cs3apis v0.0.0-20220818202316-e92afdddac6d
github.com/cs3org/reva/v2 v2.10.1-0.20220915095422-4b099c09a66c
github.com/cs3org/reva/v2 v2.10.1-0.20220921203558-038b633f66ad
github.com/disintegration/imaging v1.6.2
github.com/ggwhite/go-masker v1.0.9
github.com/go-chi/chi/v5 v5.0.7
@@ -33,10 +34,10 @@ require (
github.com/go-micro/plugins/v4/wrapper/monitoring/prometheus v1.1.0
github.com/go-micro/plugins/v4/wrapper/trace/opencensus v1.1.0
github.com/go-ozzo/ozzo-validation/v4 v4.3.0
github.com/gofrs/uuid v4.2.0+incompatible
github.com/gofrs/uuid v4.3.0+incompatible
github.com/golang-jwt/jwt/v4 v4.4.2
github.com/golang/protobuf v1.5.2
github.com/gookit/config/v2 v2.1.5
github.com/gookit/config/v2 v2.1.6
github.com/gorilla/mux v1.8.0
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3
github.com/justinas/alice v1.2.0
@@ -44,11 +45,11 @@ require (
github.com/libregraph/lico v0.54.1-0.20220325072321-31efc3995d63
github.com/mitchellh/mapstructure v1.5.0
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826
github.com/nats-io/nats-server/v2 v2.8.4
github.com/nats-io/nats-server/v2 v2.9.0
github.com/oklog/run v1.1.0
github.com/olekukonko/tablewriter v0.0.5
github.com/onsi/ginkgo v1.16.5
github.com/onsi/ginkgo/v2 v2.1.6
github.com/onsi/ginkgo/v2 v2.2.0
github.com/onsi/gomega v1.20.2
github.com/orcaman/concurrent-map v1.0.0
github.com/owncloud/libre-graph-api-go v0.17.0
@@ -65,11 +66,12 @@ require (
github.com/xhit/go-simple-mail/v2 v2.11.0
go-micro.dev/v4 v4.8.1
go.etcd.io/bbolt v1.3.6
go.etcd.io/etcd/client/v3 v3.5.5
go.opencensus.io v0.23.0
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.34.0
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.36.0
go.opentelemetry.io/otel v1.10.0
go.opentelemetry.io/otel/exporters/jaeger v1.9.0
go.opentelemetry.io/otel/sdk v1.9.0
go.opentelemetry.io/otel/exporters/jaeger v1.10.0
go.opentelemetry.io/otel/sdk v1.10.0
go.opentelemetry.io/otel/trace v1.10.0
golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90
golang.org/x/exp v0.0.0-20220518171630-0b5c67f07fdf
@@ -101,7 +103,6 @@ require (
github.com/alexedwards/argon2id v0.0.0-20211130144151-3585854a6387 // indirect
github.com/amoghe/go-crypt v0.0.0-20220222110647-20eada5f5964 // indirect
github.com/armon/go-metrics v0.3.10 // indirect
github.com/armon/go-radix v1.0.0 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/aws/aws-sdk-go v1.44.94 // indirect
github.com/beevik/etree v1.1.0 // indirect
@@ -177,7 +178,7 @@ require (
github.com/google/go-cmp v0.5.8 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/gookit/goutil v0.5.11 // indirect
github.com/gookit/goutil v0.5.12 // indirect
github.com/gorilla/handlers v1.5.1 // indirect
github.com/gorilla/schema v1.2.0 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
@@ -201,7 +202,7 @@ require (
github.com/json-iterator/go v1.1.12 // indirect
github.com/juliangruber/go-intersect v1.1.0 // indirect
github.com/kevinburke/ssh_config v1.1.0 // indirect
github.com/klauspost/compress v1.14.4 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/cpuid/v2 v2.0.9 // indirect
github.com/longsleep/go-metrics v1.0.0 // indirect
github.com/mattermost/xml-roundtrip-validator v0.1.0 // indirect
@@ -226,8 +227,8 @@ require (
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mschoch/smat v0.2.0 // indirect
github.com/nats-io/jwt/v2 v2.2.1-0.20220330180145-442af02fd36a // indirect
github.com/nats-io/nats.go v1.15.0 // indirect
github.com/nats-io/jwt/v2 v2.3.0 // indirect
github.com/nats-io/nats.go v1.16.1-0.20220906180156-a1017eec10b0 // indirect
github.com/nats-io/nkeys v0.3.0 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/nxadm/tail v1.4.8 // indirect
@@ -260,16 +261,15 @@ require (
github.com/wk8/go-ordered-map v1.0.0 // indirect
github.com/xanzy/ssh-agent v0.3.1 // indirect
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
go.etcd.io/etcd/api/v3 v3.5.2 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.2 // indirect
go.etcd.io/etcd/client/v3 v3.5.2 // indirect
go.etcd.io/etcd/api/v3 v3.5.5 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.5 // indirect
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.7.0 // indirect
go.uber.org/zap v1.19.1 // indirect
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 // indirect
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4 // indirect
golang.org/x/sys v0.0.0-20220829200755-d48e67d00261 // indirect
golang.org/x/time v0.0.0-20220224211638-0e9765cccd65 // indirect
golang.org/x/sys v0.0.0-20220906135438-9e1f76180b77 // indirect
golang.org/x/time v0.0.0-20220722155302-e5dcc9cfc0b9 // indirect
golang.org/x/tools v0.1.12 // indirect
google.golang.org/appengine v1.6.7 // indirect
gopkg.in/ini.v1 v1.66.2 // indirect

80
go.sum
View File

@@ -292,10 +292,8 @@ github.com/crewjam/saml v0.4.6 h1:XCUFPkQSJLvzyl4cW9OvpWUbRf0gE7VUpU8ZnilbeM4=
github.com/crewjam/saml v0.4.6/go.mod h1:ZBOXnNPFzB3CgOkRm7Nd6IVdkG+l/wF+0ZXLqD96t1A=
github.com/cs3org/go-cs3apis v0.0.0-20220818202316-e92afdddac6d h1:toyZ7IsXlUdEPZ/IG8fg7hbM8HcLPY0bkX4FKBmgLVI=
github.com/cs3org/go-cs3apis v0.0.0-20220818202316-e92afdddac6d/go.mod h1:UXha4TguuB52H14EMoSsCqDj7k8a/t7g4gVP+bgY5LY=
github.com/cs3org/reva/v2 v2.10.1-0.20220915071600-3358dc72a980 h1:siIHxgMHWCxERsHPYwHL7Pno27URyqjV2np9Mh1U84g=
github.com/cs3org/reva/v2 v2.10.1-0.20220915071600-3358dc72a980/go.mod h1:+BYVpRV8g1hL8wF3+3BunL9BKPsXVyJYmH8COxq/V7Y=
github.com/cs3org/reva/v2 v2.10.1-0.20220915095422-4b099c09a66c h1:pvbsnSl5WpS6PkSR4glwR8OJGrRdZASajAJtNwp9E+Y=
github.com/cs3org/reva/v2 v2.10.1-0.20220915095422-4b099c09a66c/go.mod h1:+BYVpRV8g1hL8wF3+3BunL9BKPsXVyJYmH8COxq/V7Y=
github.com/cs3org/reva/v2 v2.10.1-0.20220921203558-038b633f66ad h1:ug56A+3gPrzBaR1hyKj93vJ+CSZjMx80I8WBourC3a0=
github.com/cs3org/reva/v2 v2.10.1-0.20220921203558-038b633f66ad/go.mod h1:+BYVpRV8g1hL8wF3+3BunL9BKPsXVyJYmH8COxq/V7Y=
github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8 h1:Z9lwXumT5ACSmJ7WGnFl+OMLLjpz5uR2fyz7dC255FI=
github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8/go.mod h1:4abs/jPXcmJzYoYGF91JF9Uq9s/KL5n1jvFDix8KcqY=
github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4=
@@ -517,8 +515,9 @@ github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5x
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gofrs/uuid v4.2.0+incompatible h1:yyYWMnhkhrKwwr8gAOcOCYxOOscHgDS9yZgBrnJfGa0=
github.com/gofrs/uuid v4.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gofrs/uuid v4.3.0+incompatible h1:CaSVZxm5B+7o45rtab4jC2G37WGYX1zQfuU2i6DSvnc=
github.com/gofrs/uuid v4.3.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
@@ -639,19 +638,15 @@ github.com/googleapis/gax-go/v2 v2.2.0/go.mod h1:as02EH8zWkzwUoLbBaFeQ+arQaj/Oth
github.com/googleapis/gax-go/v2 v2.3.0/go.mod h1:b8LNqSzNabLiUpXKkY7HAR5jr6bIT99EXz9pXxye9YM=
github.com/googleapis/gax-go/v2 v2.4.0/go.mod h1:XOTVJ59hdnfJLIP/dh8n5CGryZR2LxK9wbMD5+iXC6c=
github.com/googleapis/go-type-adapters v1.0.0/go.mod h1:zHW75FOG2aur7gAO2B+MLby+cLsWGBF62rFAi7WjWO4=
github.com/gookit/color v1.5.0/go.mod h1:43aQb+Zerm/BWh2GnrgOQm7ffz7tvQXEKV6BFMl7wAo=
github.com/gookit/color v1.5.1/go.mod h1:wZFzea4X8qN6vHOSP2apMb4/+w/orMznEzYsIHPaqKM=
github.com/gookit/color v1.5.2 h1:uLnfXcaFjlrDnQDT+NCBcfhrXqYTx/rcCa6xn01Y8yI=
github.com/gookit/color v1.5.2/go.mod h1:w8h4bGiHeeBpvQVePTutdbERIUf3oJE5lZ8HM0UgXyg=
github.com/gookit/config/v2 v2.1.5 h1:XBzJZulch8Gul1KM3Sd4UW516TaEdFXmzpaCQ9ygj3Y=
github.com/gookit/config/v2 v2.1.5/go.mod h1:SJ8TJHua8S+Clrobxkd9OCncjfAdyyGcyLWbF+EinnI=
github.com/gookit/goutil v0.5.3/go.mod h1:W0rVeVN9EcoRV+ODq91TXvqoUA87BXGi36WFVykCKRI=
github.com/gookit/goutil v0.5.9/go.mod h1:iZLXpRhMqKGvKtJ9+b0cdls2gXRH4HaGWQfkf2mdHRQ=
github.com/gookit/goutil v0.5.11 h1:k7Z/5Y9c/CTj/V56p576s5lLMl93Lnp9/qckT4liVXQ=
github.com/gookit/goutil v0.5.11/go.mod h1:6vhWm/bSYXGE8poqFbFz6IGM7jV2r6qVhyK567SX/AI=
github.com/gookit/ini/v2 v2.1.1 h1:q2VtSSl/ivTOZMPvxhjWxO3f146NvWM84jBQZETj/1o=
github.com/gookit/ini/v2 v2.1.1/go.mod h1:zkMTCrnE2QgDW0izB/pRtgEKKjXmT/22dfk5eZg+IHo=
github.com/gookit/properties v0.1.0/go.mod h1:qg423zj2UMRsOilko6MAqyUBYQETTLjkk9NejTlMeBE=
github.com/gookit/config/v2 v2.1.6 h1:HZRj679RzHCkACwqUVYFKwUwCgBFBj/OCznG2YfG1Po=
github.com/gookit/config/v2 v2.1.6/go.mod h1:Aerfe2Hn5UREhswzixIr09f4vyyyLfLlUfbTer8jqoA=
github.com/gookit/goutil v0.5.12 h1:dcgIGLF1uBUTbjz0kBBL0LESMznthetAf1jQwFysNwU=
github.com/gookit/goutil v0.5.12/go.mod h1:6vhWm/bSYXGE8poqFbFz6IGM7jV2r6qVhyK567SX/AI=
github.com/gookit/ini/v2 v2.1.2 h1:EmlDqxTeP/a6erbQmAPE+OL4BDmkYTUO0/QbzF7oOhI=
github.com/gookit/ini/v2 v2.1.2/go.mod h1:5r9ypDH9eeQj8gRUMmj5NUiOL5UOLl6Ffmk1++5rGIM=
github.com/gookit/properties v0.2.0/go.mod h1:e+xw8PcXfPHUrBcb6fgB79HVCRBLJ6VYbtpeECJurow=
github.com/gophercloud/gophercloud v0.15.1-0.20210202035223-633d73521055/go.mod h1:wRtmUelyIIv3CSSDI47aUwbs075O6i+LY+pXsKCBsb4=
github.com/gophercloud/gophercloud v0.16.0/go.mod h1:wRtmUelyIIv3CSSDI47aUwbs075O6i+LY+pXsKCBsb4=
github.com/gophercloud/utils v0.0.0-20210216074907-f6de111f2eae/go.mod h1:wx8HMD8oQD0Ryhz6+6ykq75PJ79iPyEqYHfwZ4l7OsA=
@@ -797,8 +792,8 @@ github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvW
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.13.6/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/compress v1.14.4 h1:eijASRJcobkVtSt81Olfh7JX43osYLwy5krOJo6YEu4=
github.com/klauspost/compress v1.14.4/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/compress v1.15.9 h1:wKRjX6JRtDdrE9qwa4b/Cip7ACOshUI4smpCQanqjSY=
github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
@@ -942,12 +937,12 @@ github.com/mschoch/smat v0.2.0/go.mod h1:kc9mz7DoBKqDyiRL7VZN8KvXQMWeTaVnttLRXOl
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/namedotcom/go v0.0.0-20180403034216-08470befbe04/go.mod h1:5sN+Lt1CaY4wsPvgQH/jsuJi4XO2ssZbdsIizr4CVC8=
github.com/nats-io/jwt/v2 v2.2.1-0.20220330180145-442af02fd36a h1:lem6QCvxR0Y28gth9P+wV2K/zYUUAkJ+55U8cpS0p5I=
github.com/nats-io/jwt/v2 v2.2.1-0.20220330180145-442af02fd36a/go.mod h1:0tqz9Hlu6bCBFLWAASKhE5vUA4c24L9KPUUgvwumE/k=
github.com/nats-io/nats-server/v2 v2.8.4 h1:0jQzze1T9mECg8YZEl8+WYUXb9JKluJfCBriPUtluB4=
github.com/nats-io/nats-server/v2 v2.8.4/go.mod h1:8zZa+Al3WsESfmgSs98Fi06dRWLH5Bnq90m5bKD/eT4=
github.com/nats-io/nats.go v1.15.0 h1:3IXNBolWrwIUf2soxh6Rla8gPzYWEZQBUBK6RV21s+o=
github.com/nats-io/nats.go v1.15.0/go.mod h1:BPko4oXsySz4aSWeFgOHLZs3G4Jq4ZAyE6/zMCxRT6w=
github.com/nats-io/jwt/v2 v2.3.0 h1:z2mA1a7tIf5ShggOFlR1oBPgd6hGqcDYsISxZByUzdI=
github.com/nats-io/jwt/v2 v2.3.0/go.mod h1:0tqz9Hlu6bCBFLWAASKhE5vUA4c24L9KPUUgvwumE/k=
github.com/nats-io/nats-server/v2 v2.9.0 h1:DLWu+7/VgGOoChcDKytnUZPAmudpv7o/MhKmNrnH1RE=
github.com/nats-io/nats-server/v2 v2.9.0/go.mod h1:BWKY6217RvhI+FDoOLZ2BH+hOC37xeKRBlQ1Lz7teKI=
github.com/nats-io/nats.go v1.16.1-0.20220906180156-a1017eec10b0 h1:dPUKD6Iv8M1y9MU8PK6H4a4/12yx5/CbaYWz/Z1arY8=
github.com/nats-io/nats.go v1.16.1-0.20220906180156-a1017eec10b0/go.mod h1:BPko4oXsySz4aSWeFgOHLZs3G4Jq4ZAyE6/zMCxRT6w=
github.com/nats-io/nkeys v0.3.0 h1:cgM5tL53EvYRU+2YLXIK0G2mJtK12Ft9oeooSZMA2G8=
github.com/nats-io/nkeys v0.3.0/go.mod h1:gvUNGjVcM2IPr5rCsRsC6Wb3Hr2CQAm08dsxtV6A5y4=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
@@ -974,8 +969,8 @@ github.com/onsi/ginkgo v1.10.3/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+
github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
github.com/onsi/ginkgo/v2 v2.1.6 h1:Fx2POJZfKRQcM1pH49qSZiYeu319wji004qX+GDovrU=
github.com/onsi/ginkgo/v2 v2.1.6/go.mod h1:MEH45j8TBi6u9BMogfbp0stKC5cdGjumZj5Y7AG4VIk=
github.com/onsi/ginkgo/v2 v2.2.0 h1:3ZNA3L1c5FYDFTTxbFeVGGD8jYvjYauHD30YgLxVsNI=
github.com/onsi/ginkgo/v2 v2.2.0/go.mod h1:MEH45j8TBi6u9BMogfbp0stKC5cdGjumZj5Y7AG4VIk=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
@@ -1246,12 +1241,12 @@ go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
go.etcd.io/bbolt v1.3.6 h1:/ecaJf0sk1l4l6V4awd65v2C3ILy7MSj+s/x1ADCIMU=
go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4=
go.etcd.io/etcd/api/v3 v3.5.2 h1:tXok5yLlKyuQ/SXSjtqHc4uzNaMqZi2XsoSPr/LlJXI=
go.etcd.io/etcd/api/v3 v3.5.2/go.mod h1:5GB2vv4A4AOn3yk7MftYGHkUfGtDHnEraIjym4dYz5A=
go.etcd.io/etcd/client/pkg/v3 v3.5.2 h1:4hzqQ6hIb3blLyQ8usCU4h3NghkqcsohEQ3o3VetYxE=
go.etcd.io/etcd/client/pkg/v3 v3.5.2/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=
go.etcd.io/etcd/client/v3 v3.5.2 h1:WdnejrUtQC4nCxK0/dLTMqKOB+U5TP/2Ya0BJL+1otA=
go.etcd.io/etcd/client/v3 v3.5.2/go.mod h1:kOOaWFFgHygyT0WlSmL8TJiXmMysO/nNUlEsSsN6W4o=
go.etcd.io/etcd/api/v3 v3.5.5 h1:BX4JIbQ7hl7+jL+g+2j5UAr0o1bctCm6/Ct+ArBGkf0=
go.etcd.io/etcd/api/v3 v3.5.5/go.mod h1:KFtNaxGDw4Yx/BA4iPPwevUTAuqcsPxzyX8PHydchN8=
go.etcd.io/etcd/client/pkg/v3 v3.5.5 h1:9S0JUVvmrVl7wCF39iTQthdaaNIiAaQbmK75ogO6GU8=
go.etcd.io/etcd/client/pkg/v3 v3.5.5/go.mod h1:ggrwbk069qxpKPq8/FKkQ3Xq9y39kbFR4LnKszpRXeQ=
go.etcd.io/etcd/client/v3 v3.5.5 h1:q++2WTJbUgpQu4B6hCuT7VkdwaTP7Qz6Daak3WzbrlI=
go.etcd.io/etcd/client/v3 v3.5.5/go.mod h1:aApjR4WGlSumpnJ2kloS75h6aHUmAyaPLjHMxpc7E7c=
go.mongodb.org/mongo-driver v1.7.3/go.mod h1:NqaYOwnXWr5Pm7AOpO5QFxKJ503nbMse/R79oO62zWg=
go.mongodb.org/mongo-driver v1.7.5/go.mod h1:VXEWRZ6URJIkUq2SCAyapmhH0ZLRBP+FT4xhp5Zvxng=
go.mongodb.org/mongo-driver v1.8.3/go.mod h1:0sQWfOeY63QTntERDJJ/0SuKK0T1uVSgKCuAROlKEPY=
@@ -1265,14 +1260,14 @@ go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.opencensus.io v0.23.0 h1:gqCw0LfLxScz8irSi8exQc7fyQ0fKQU/qnC/X8+V/1M=
go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.34.0 h1:PNEMW4EvpNQ7SuoPFNkvbZqi1STkTPKq+8vfoMl/6AE=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.34.0/go.mod h1:fk1+icoN47ytLSgkoWHLJrtVTSQ+HgmkNgPTKrk/Nsc=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.36.0 h1:+jrwcA4gF8tIZmdKWgTUysKtYW2VIzywjkfgd/5OPEM=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.36.0/go.mod h1:h8TWwRAhQpOd0aM5nYsRD8+flnkj+526GEIVlarH7eY=
go.opentelemetry.io/otel v1.10.0 h1:Y7DTJMR6zs1xkS/upamJYk0SxxN4C9AqRd77jmZnyY4=
go.opentelemetry.io/otel v1.10.0/go.mod h1:NbvWjCthWHKBEUMpf0/v8ZRZlni86PpGFEMA9pnQSnQ=
go.opentelemetry.io/otel/exporters/jaeger v1.9.0 h1:gAEgEVGDWwFjcis9jJTOJqZNxDzoZfR12WNIxr7g9Ww=
go.opentelemetry.io/otel/exporters/jaeger v1.9.0/go.mod h1:hquezOLVAybNW6vanIxkdLXTXvzlj2Vn3wevSP15RYs=
go.opentelemetry.io/otel/sdk v1.9.0 h1:LNXp1vrr83fNXTHgU8eO89mhzxb/bbWAsHG6fNf3qWo=
go.opentelemetry.io/otel/sdk v1.9.0/go.mod h1:AEZc8nt5bd2F7BC24J5R0mrjYnpEgYHyTcM/vrSple4=
go.opentelemetry.io/otel/exporters/jaeger v1.10.0 h1:7W3aVVjEYayu/GOqOVF4mbTvnCuxF1wWu3eRxFGQXvw=
go.opentelemetry.io/otel/exporters/jaeger v1.10.0/go.mod h1:n9IGyx0fgyXXZ/i0foLHNxtET9CzXHzZeKCucvRBFgA=
go.opentelemetry.io/otel/sdk v1.10.0 h1:jZ6K7sVn04kk/3DNUdJ4mqRlGDiXAVuIG+MMENpTNdY=
go.opentelemetry.io/otel/sdk v1.10.0/go.mod h1:vO06iKzD5baltJz1zarxMCNHFpUlUiOy4s65ECtn6kE=
go.opentelemetry.io/otel/trace v1.10.0 h1:npQMbR8o7mum8uF95yFbOEJffhs1sbCOfDh8zAJiH5E=
go.opentelemetry.io/otel/trace v1.10.0/go.mod h1:Sij3YYczqAdz+EhmGhE6TpTxUO5/F/AzrK+kxfGqySM=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
@@ -1317,7 +1312,6 @@ golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201216223049-8b5274cf687f/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210314154223-e6e6c4f2bb5b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
@@ -1589,11 +1583,11 @@ golang.org/x/sys v0.0.0-20220624220833-87e55d714810/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220829200755-d48e67d00261 h1:v6hYoSR9T5oet+pMXwUWkbiVqx/63mlHjefrHmxwfeY=
golang.org/x/sys v0.0.0-20220829200755-d48e67d00261/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220906135438-9e1f76180b77 h1:C1tElbkWrsSkn3IRl1GCW/gETw1TywWIPgwZtXTZbYg=
golang.org/x/sys v0.0.0-20220906135438-9e1f76180b77/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20220722155259-a9ba230a4035 h1:Q5284mrmYTpACcm+eAKjKJH48BBwSyfJqmmGDTtT8Vc=
@@ -1614,8 +1608,9 @@ golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxb
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20201208040808-7e3f01d25324/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20220224211638-0e9765cccd65 h1:M73Iuj3xbbb9Uk1DYhzydthsj6oOd6l9bpuFcNoUvTs=
golang.org/x/time v0.0.0-20220224211638-0e9765cccd65/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20220722155302-e5dcc9cfc0b9 h1:ftMN5LMiBFjbzleLqtoBZk7KdJwhuybIU+FckUHgoyQ=
golang.org/x/time v0.0.0-20220722155302-e5dcc9cfc0b9/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -1857,6 +1852,7 @@ google.golang.org/grpc v1.39.0/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnD
google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE=
google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34=
google.golang.org/grpc v1.40.1/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34=
google.golang.org/grpc v1.41.0/go.mod h1:U3l9uK9J0sini8mHphKoXyaqDA/8VyGnDee1zzIUK6k=
google.golang.org/grpc v1.44.0/go.mod h1:k+4IHHFw41K8+bbowsex27ge2rCb65oeWqe4jJ590SU=
google.golang.org/grpc v1.45.0/go.mod h1:lN7owxKUQEqMfSyQikvvk5tf/6zMPsrK+ONuO11+0rQ=
google.golang.org/grpc v1.46.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk=

5
ocis-pkg/config/config.go
View File

@@ -57,8 +57,9 @@ type Runtime struct {
type Config struct {
*shared.Commons `yaml:"shared"`
Tracing *shared.Tracing `yaml:"tracing"`
Log *shared.Log `yaml:"log"`
Tracing *shared.Tracing `yaml:"tracing"`
Log *shared.Log `yaml:"log"`
CacheStore *shared.CacheStore `yaml:"cache_store"`
Mode Mode // DEPRECATED
File string

13
ocis-pkg/config/parser/parse.go
View File

@@ -48,6 +48,9 @@ func EnsureDefaults(cfg *config.Config) {
if cfg.TokenManager == nil {
cfg.TokenManager = &shared.TokenManager{}
}
if cfg.CacheStore == nil {
cfg.CacheStore = &shared.CacheStore{}
}
}
// EnsureCommons copies applicable parts of the oCIS config into the commons part
@@ -81,6 +84,16 @@ func EnsureCommons(cfg *config.Config) {
cfg.Commons.Tracing = &shared.Tracing{}
}
if cfg.CacheStore != nil {
cfg.Commons.CacheStore = &shared.CacheStore{
Type: cfg.CacheStore.Type,
Address: cfg.CacheStore.Address,
Size: cfg.CacheStore.Size,
}
} else {
cfg.Commons.CacheStore = &shared.CacheStore{}
}
// copy token manager to the commons part if set
if cfg.TokenManager != nil {
cfg.Commons.TokenManager = cfg.TokenManager

50
ocis-pkg/roles/manager.go
View File

@@ -2,16 +2,26 @@ package roles
import (
"context"
"time"
"github.com/owncloud/ocis/v2/ocis-pkg/log"
ocisstore "github.com/owncloud/ocis/v2/ocis-pkg/store"
settingsmsg "github.com/owncloud/ocis/v2/protogen/gen/ocis/messages/settings/v0"
settingssvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/settings/v0"
"go-micro.dev/v4/store"
"google.golang.org/protobuf/encoding/protojson"
)
const (
cacheDatabase = "ocis-pkg"
cacheTableName = "ocis-pkg/roles"
cacheTTL = time.Hour
)
// Manager manages a cache of roles by fetching unknown roles from the settings.RoleService.
type Manager struct {
logger log.Logger
cache cache
cache store.Store
roleService settingssvc.RoleService
}
@@ -19,8 +29,9 @@ type Manager struct {
func NewManager(o ...Option) Manager {
opts := newOptions(o...)
nStore := ocisstore.GetStore(opts.storeOptions)
return Manager{
cache: newCache(opts.size, opts.ttl),
cache: nStore,
roleService: opts.roleService,
}
}
@@ -31,10 +42,26 @@ func (m *Manager) List(ctx context.Context, roleIDs []string) []*settingsmsg.Bun
result := make([]*settingsmsg.Bundle, 0)
lookup := make([]string, 0)
for _, roleID := range roleIDs {
if hit := m.cache.get(roleID); hit == nil {
if records, err := m.cache.Read(roleID, store.ReadFrom(cacheDatabase, cacheTableName)); err != nil {
lookup = append(lookup, roleID)
} else {
result = append(result, hit)
role := &settingsmsg.Bundle{}
found := false
for _, record := range records {
if record.Key == roleID {
if err := protojson.Unmarshal(record.Value, role); err == nil {
// if we can unmarshal the role, append it to the result
// otherwise assume the role wasn't found (data was damaged and
// we need to get the role again)
result = append(result, role)
found = true
break
}
}
}
if !found {
lookup = append(lookup, roleID)
}
}
}
@@ -49,7 +76,20 @@ func (m *Manager) List(ctx context.Context, roleIDs []string) []*settingsmsg.Bun
return nil
}
for _, role := range res.Bundles {
m.cache.set(role.Id, role)
jsonbytes, _ := protojson.Marshal(role)
record := &store.Record{
Key: role.Id,
Value: jsonbytes,
Expiry: cacheTTL,
}
err := m.cache.Write(
record,
store.WriteTo(cacheDatabase, cacheTableName),
store.WriteTTL(cacheTTL),
)
if err != nil {
m.logger.Debug().Err(err).Msg("failed to cache roles")
}
result = append(result, role)
}
}

30
ocis-pkg/roles/option.go
View File

@@ -1,37 +1,21 @@
package roles
import (
"time"
"github.com/owncloud/ocis/v2/ocis-pkg/log"
ocisstore "github.com/owncloud/ocis/v2/ocis-pkg/store"
settingssvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/settings/v0"
)
// Options are all the possible options.
type Options struct {
size int
ttl time.Duration
logger log.Logger
roleService settingssvc.RoleService
storeOptions ocisstore.OcisStoreOptions
logger log.Logger
roleService settingssvc.RoleService
}
// Option mutates option
type Option func(*Options)
// CacheSize configures the size of the cache in items.
func CacheSize(s int) Option {
return func(o *Options) {
o.size = s
}
}
// CacheTTL rebuilds the cache after the configured duration.
func CacheTTL(ttl time.Duration) Option {
return func(o *Options) {
o.ttl = ttl
}
}
// Logger sets a preconfigured logger
func Logger(logger log.Logger) Option {
return func(o *Options) {
@@ -46,6 +30,12 @@ func RoleService(rs settingssvc.RoleService) Option {
}
}
func StoreOptions(storeOpts ocisstore.OcisStoreOptions) Option {
return func(o *Options) {
o.storeOptions = storeOpts
}
}
func newOptions(opts ...Option) Options {
o := Options{}

9
ocis-pkg/shared/shared_types.go
View File

@@ -34,11 +34,18 @@ type Reva struct {
Address string `yaml:"address" env:"REVA_GATEWAY" desc:"The CS3 gateway endpoint."`
}
// Commons holds configuration that are common to all services. Each service can then decide whether
type CacheStore struct {
Type string `yaml:"type" env:"OCIS_CACHE_STORE_TYPE" desc:"The type of the cache store. Valid options are \"noop\", \"ocmem\", \"etcd\" and \"memory\""`
Address string `yaml:"address" env:"OCIS_CACHE_STORE_ADDRESS" desc:"a comma-separated list of addresses to connect to. Only for etcd"`
Size int `yaml:"size" env:"OCIS_CACHE_STORE_SIZE" desc:"Maximum size for the cache store. Only ocmem will use this option, in number of items per table. The rest will ignore the option and can grow indefinitely"`
}
// Commons holds configuration that are common to all extensions. Each extension can then decide whether
// to overwrite its values.
type Commons struct {
Log *Log `yaml:"log"`
Tracing *Tracing `yaml:"tracing"`
CacheStore *CacheStore `yaml:"cache_store"`
OcisURL string `yaml:"ocis_url" env:"OCIS_URL" desc:"URL, where oCIS is reachable for users."`
TokenManager *TokenManager `mask:"struct" yaml:"token_manager"`
Reva *Reva `yaml:"reva"`

531
ocis-pkg/store/etcd/etcd.go Normal file
View File

@@ -0,0 +1,531 @@
package etcd
import (
"context"
"encoding/json"
"strings"
"time"
"go-micro.dev/v4/store"
clientv3 "go.etcd.io/etcd/client/v3"
"go.etcd.io/etcd/client/v3/namespace"
)
const (
prefixNS = ".prefix"
suffixNS = ".suffix"
)
type EtcdStore struct {
options store.Options
client *clientv3.Client
}
// Create a new go-micro store backed by etcd
func NewEtcdStore(opts ...store.Option) store.Store {
es := &EtcdStore{}
_ = es.Init(opts...)
return es
}
func (es *EtcdStore) getCtx() (context.Context, context.CancelFunc) {
currentCtx := es.options.Context
if currentCtx == nil {
currentCtx = context.TODO()
}
ctx, cancel := context.WithTimeout(currentCtx, 10*time.Second)
return ctx, cancel
}
// Setup the etcd client based on the current options. The old client (if any)
// will be closed.
// Currently, only the etcd nodes are configurable. If no node is provided,
// it will use the "127.0.0.1:2379" node.
// Context timeout is setup to 10 seconds, and dial timeout to 2 seconds
func (es *EtcdStore) setupClient() {
if es.client != nil {
es.client.Close()
}
endpoints := []string{"127.0.0.1:2379"}
if len(es.options.Nodes) > 0 {
endpoints = es.options.Nodes
}
cli, _ := clientv3.New(clientv3.Config{
DialTimeout: 2 * time.Second,
Endpoints: endpoints,
})
es.client = cli
}
// Initialize the go-micro store implementation.
// Currently, only the nodes are configurable, the rest of the options
// will be ignored.
func (es *EtcdStore) Init(opts ...store.Option) error {
optList := store.Options{}
for _, opt := range opts {
opt(&optList)
}
es.options = optList
es.setupClient()
return nil
}
// Get the store options
func (es *EtcdStore) Options() store.Options {
return es.options
}
// Get the effective TTL, as int64 number of seconds. It will prioritize
// the TTL set in the options, then the expiry time in the options, and
// finally the one set as part of the record
func getEffectiveTTL(r *store.Record, opts store.WriteOptions) int64 {
// set base ttl duration and expiration time based on the record
duration := r.Expiry
// overwrite ttl duration and expiration time based on options
if !opts.Expiry.IsZero() {
// options.Expiry is a time.Time, newRecord.Expiry is a time.Duration
duration = time.Until(opts.Expiry)
}
// TTL option takes precedence over expiration time
if opts.TTL != 0 {
duration = opts.TTL
}
// use milliseconds because it returns an int64 instead of a float64
return duration.Milliseconds() / 1000
}
// Write the record into the etcd. The record will be duplicated in order to
// find it by prefix or by suffix. This means that it will take double space.
// Note that this is an implementation detail and it will be handled
// transparently.
//
// Database and Table options will be used to provide a different prefix to
// the key. Each service using this store should use a different database+table
// combination in order to prevent key collisions.
//
// Due to how TTLs are implemented in etcd, the minimum valid TTL seems to
// be 2 secs. Using lower values or even negative values will force the etcd
// server to use the minimum value instead.
// In addition, getting a lease for the TTL and attach it to the target key
// are 2 different operations that can't be sent as part of a transaction.
// This means that it's possible to get a lease and have that lease expire
// before attaching it to the key. Errors are expected to happen if this is
// the case, and no key will be inserted.
// According to etcd documentation, the key is guaranteed to be available
// AT LEAST the TTL duration. This means that the key might be available for
// a longer period of time in special circumstances.
//
// It's recommended to use a minimum TTL of 10 secs or higher (or not to use
// TTL) in order to prevent problematic scenarios.
func (es *EtcdStore) Write(r *store.Record, opts ...store.WriteOption) error {
wopts := store.WriteOptions{}
for _, opt := range opts {
opt(&wopts)
}
prefix := buildPrefix(wopts.Database, wopts.Table, prefixNS)
suffix := buildPrefix(wopts.Database, wopts.Table, suffixNS)
kv := es.client.KV
jsonRecord, err := json.Marshal(r)
if err != nil {
return err
}
jsonStringRecord := string(jsonRecord)
effectiveTTL := getEffectiveTTL(r, wopts)
var opOpts []clientv3.OpOption
if effectiveTTL != 0 {
lease := es.client.Lease
ctx, cancel := es.getCtx()
gResp, gErr := lease.Grant(ctx, getEffectiveTTL(r, wopts))
cancel()
if gErr != nil {
return gErr
}
opOpts = []clientv3.OpOption{clientv3.WithLease(gResp.ID)}
} else {
opOpts = []clientv3.OpOption{clientv3.WithLease(0)}
}
ctx, cancel := es.getCtx()
_, err = kv.Txn(ctx).Then(
clientv3.OpPut(prefix+r.Key, jsonStringRecord, opOpts...),
clientv3.OpPut(suffix+reverseString(r.Key), jsonStringRecord, opOpts...),
).Commit()
cancel()
return err
}
// Process a Get response taking into account the provided offset
func processGetResponse(resp *clientv3.GetResponse, offset int64) ([]*store.Record, error) {
result := make([]*store.Record, 0, len(resp.Kvs))
for index, kvs := range resp.Kvs {
if int64(index) < offset {
// skip entries before the offset
continue
}
value := &store.Record{}
err := json.Unmarshal(kvs.Value, value)
if err != nil {
return nil, err
}
result = append(result, value)
}
return result, nil
}
// Process a List response taking into account the provided offset.
// The reverse flag will be used to reverse the keys found. For example,
// "zyxw" will be reversed to "wxyz". This is used for suffix searches,
// where the keys are stored reversed and need to be changed
func processListResponse(resp *clientv3.GetResponse, offset int64, reverse bool) ([]string, error) {
result := make([]string, 0, len(resp.Kvs))
for index, kvs := range resp.Kvs {
if int64(index) < offset {
// skip entries before the offset
continue
}
targetKey := string(kvs.Key)
if reverse {
targetKey = reverseString(targetKey)
}
result = append(result, targetKey)
}
return result, nil
}
// Perform an exact key read and return the result
func (es *EtcdStore) directRead(kv clientv3.KV, key string) ([]*store.Record, error) {
ctx, cancel := es.getCtx()
resp, err := kv.Get(ctx, key)
cancel()
if err != nil {
return nil, err
}
if len(resp.Kvs) == 0 {
return nil, store.ErrNotFound
}
return processGetResponse(resp, 0)
}
// Perform a prefix read with limit and offset. A limit of 0 will return all
// results. Usage of offset isn't recommended because those results must still
// be fethed from the server in order to be discarded.
func (es *EtcdStore) prefixRead(kv clientv3.KV, key string, limit, offset int64) ([]*store.Record, error) {
getOptions := []clientv3.OpOption{
clientv3.WithPrefix(),
}
if limit > 0 {
getOptions = append(getOptions, clientv3.WithLimit(limit+offset))
}
ctx, cancel := es.getCtx()
resp, err := kv.Get(ctx, key, getOptions...)
cancel()
if err != nil {
return nil, err
}
return processGetResponse(resp, offset)
}
// Perform a prefix + suffix read with limit and offset. A limit of 0 will
// return all results found. Usage of this function is discouraged because
// we'll have to request a prefix search and match the suffix manually. This
// means that even with a limit = 3 and offset = 0, there is no guarantee
// we'll find all the results we need within that range, and we'll likely
// need to request more data from the server. The number of requests we need
// to perform is unknown and might cause load.
func (es *EtcdStore) prefixSuffixRead(kv clientv3.KV, prefix, suffix string, limit, offset int64) ([]*store.Record, error) {
firstKeyOut := firstKeyOutOfPrefixString(prefix)
getOptions := []clientv3.OpOption{
clientv3.WithRange(firstKeyOut),
}
if limit > 0 {
// unlikely to find all the entries we need within offset + limit
getOptions = append(getOptions, clientv3.WithLimit((limit+offset)*2))
}
var currentRecordOffset int64
result := []*store.Record{}
initialKey := prefix
keepGoing := true
for keepGoing {
ctx, cancel := es.getCtx()
resp, respErr := kv.Get(ctx, initialKey, getOptions...)
cancel()
if respErr != nil {
return nil, respErr
}
records, err := processGetResponse(resp, 0)
if err != nil {
return nil, err
}
for _, record := range records {
if !strings.HasSuffix(record.Key, suffix) {
continue
}
if currentRecordOffset < offset {
currentRecordOffset++
continue
}
if !shouldFinish(int64(len(result)), limit) {
result = append(result, record)
if shouldFinish(int64(len(result)), limit) {
break
}
}
}
if !resp.More || shouldFinish(int64(len(result)), limit) {
keepGoing = false
} else {
initialKey = string(append(resp.Kvs[len(resp.Kvs)-1].Key, 0)) // append byte 0 (nul char) to the last key
}
}
return result, nil
}
// Read records from the etcd server based in the key. Database and Table
// options are highly recommended, otherwise we'll use a default one (which
// might not have the requested keys)
//
// If no prefix or suffix option is provided, we'll read the record matching
// the provided key. Note that a list of records will be provided anyway,
// likely with only one record (the one requested)
//
// Prefix and suffix options are supported and should perform fine even with
// a large amount of data. Note that the limit option should also be included
// in order to limit the amount of records we need to fetch.
//
// Note that using both prefix and suffix options at the same time is possible
// but discouraged. A prefix search will be send to the etcd server, and from
// there we'll manually pick the records matching the suffix. This might become
// very inefficient since we might need to request more data to the etcd
// multiple times in order to provide the results asked.
// Usage of the offset option is also discouraged because we'll have to request
// records that we'll have to skip manually on our side.
//
// Don't rely on any particular order of the keys. The records are expected to
// be sorted by key except if the suffix option (suffix without prefix) is
// used. In this case, the keys will be sorted based on the reversed key
func (es *EtcdStore) Read(key string, opts ...store.ReadOption) ([]*store.Record, error) {
ropts := store.ReadOptions{}
for _, opt := range opts {
opt(&ropts)
}
prefix := buildPrefix(ropts.Database, ropts.Table, prefixNS)
suffix := buildPrefix(ropts.Database, ropts.Table, suffixNS)
kv := es.client.KV
preKv := namespace.NewKV(kv, prefix)
sufKv := namespace.NewKV(kv, suffix)
if ropts.Prefix && ropts.Suffix {
return es.prefixSuffixRead(preKv, key, key, int64(ropts.Limit), int64(ropts.Offset))
}
if ropts.Prefix {
return es.prefixRead(preKv, key, int64(ropts.Limit), int64(ropts.Offset))
}
if ropts.Suffix {
return es.prefixRead(sufKv, reverseString(key), int64(ropts.Limit), int64(ropts.Offset))
}
return es.directRead(preKv, key)
}
// Delete the record containing the key provided. Database and Table
// options are highly recommended, otherwise we'll use a default one (which
// might not have the requested keys)
//
// Since the Write method inserts 2 entries for a given key, those both
// entries will also be removed using the same key. This is handled
// transparently.
func (es *EtcdStore) Delete(key string, opts ...store.DeleteOption) error {
dopts := store.DeleteOptions{}
for _, opt := range opts {
opt(&dopts)
}
prefix := buildPrefix(dopts.Database, dopts.Table, prefixNS)
suffix := buildPrefix(dopts.Database, dopts.Table, suffixNS)
kv := es.client.KV
ctx, cancel := es.getCtx()
_, err := kv.Txn(ctx).Then(
clientv3.OpDelete(prefix+key),
clientv3.OpDelete(suffix+reverseString(key)),
).Commit()
cancel()
return err
}
// List the keys based on the provided prefix. Use the empty string (and no
// limit nor offset) to list all keys available.
// Limit and offset options are available to limit the keys we need to return.
// The reverse option will reverse the keys before returning them. Use it when
// listing the keys from the suffix KV.
//
// Note that values for the keys won't be requested to the etcd server, that's
// why the reverse option is important
func (es *EtcdStore) listKeys(kv clientv3.KV, prefixKey string, limit, offset int64, reverse bool) ([]string, error) {
getOptions := []clientv3.OpOption{
clientv3.WithKeysOnly(),
clientv3.WithPrefix(),
}
if limit > 0 {
getOptions = append(getOptions, clientv3.WithLimit(limit+offset))
}
ctx, cancel := es.getCtx()
resp, err := kv.Get(ctx, prefixKey, getOptions...)
cancel()
if err != nil {
return nil, err
}
return processListResponse(resp, offset, reverse)
}
// List the keys matching both prefix and suffix, with the provided limit and
// offset. Usage of this function is discouraged because we'll have to match
// the suffix manually on our side, which means we'll likely need to perform
// additional requests to the etcd server to get more results matching all the
// requirements.
func (es *EtcdStore) prefixSuffixList(kv clientv3.KV, prefix, suffix string, limit, offset int64) ([]string, error) {
firstKeyOut := firstKeyOutOfPrefixString(prefix)
getOptions := []clientv3.OpOption{
clientv3.WithKeysOnly(),
clientv3.WithRange(firstKeyOut),
}
if firstKeyOut == "" {
// could happen of all bytes are "\xff"
getOptions = getOptions[:1] // remove the WithRange option
}
if limit > 0 {
// unlikely to find all the entries we need within offset + limit
getOptions = append(getOptions, clientv3.WithLimit((limit+offset)*2))
}
var currentRecordOffset int64
result := []string{}
initialKey := prefix
keepGoing := true
for keepGoing {
ctx, cancel := es.getCtx()
resp, respErr := kv.Get(ctx, initialKey, getOptions...)
cancel()
if respErr != nil {
return nil, respErr
}
keys, err := processListResponse(resp, 0, false)
if err != nil {
return nil, err
}
for _, key := range keys {
if !strings.HasSuffix(key, suffix) {
continue
}
if currentRecordOffset < offset {
currentRecordOffset++
continue
}
if !shouldFinish(int64(len(result)), limit) {
result = append(result, key)
if shouldFinish(int64(len(result)), limit) {
break
}
}
}
if !resp.More || shouldFinish(int64(len(result)), limit) {
keepGoing = false
} else {
initialKey = string(append(resp.Kvs[len(resp.Kvs)-1].Key, 0)) // append byte 0 (nul char) to the last key
}
}
return result, nil
}
// List the keys available in the etcd server. Database and Table
// options are highly recommended, otherwise we'll use a default one (which
// might not have the requested keys)
//
// With the Database and Table options, all the keys returned will be within
// that database and table. Each service is expected to use a different
// database + table, so using those options will list only the keys used by
// that particular service.
//
// Prefix and suffix options are available along with the limit and offset
// ones.
//
// Using prefix and suffix options at the same time is discourage because
// the suffix matching will be done on our side, and we'll likely need to
// perform multiple requests to get the requested results. Note that using
// just the suffix option is fine.
// In addition, using the offset option is also discouraged because we'll
// need to request additional keys that will be skipped on our side.
func (es *EtcdStore) List(opts ...store.ListOption) ([]string, error) {
lopts := store.ListOptions{}
for _, opt := range opts {
opt(&lopts)
}
prefix := buildPrefix(lopts.Database, lopts.Table, prefixNS)
suffix := buildPrefix(lopts.Database, lopts.Table, suffixNS)
kv := es.client.KV
preKv := namespace.NewKV(kv, prefix)
sufKv := namespace.NewKV(kv, suffix)
if lopts.Prefix != "" && lopts.Suffix != "" {
return es.prefixSuffixList(preKv, lopts.Prefix, lopts.Suffix, int64(lopts.Limit), int64(lopts.Offset))
}
if lopts.Prefix != "" {
return es.listKeys(preKv, lopts.Prefix, int64(lopts.Limit), int64(lopts.Offset), false)
}
if lopts.Suffix != "" {
return es.listKeys(sufKv, reverseString(lopts.Suffix), int64(lopts.Limit), int64(lopts.Offset), true)
}
return es.listKeys(preKv, "", int64(lopts.Limit), int64(lopts.Offset), false)
}
// Close the client
func (es *EtcdStore) Close() error {
return es.client.Close()
}
// Return the service name
func (es *EtcdStore) String() string {
return "Etcd"
}

65
ocis-pkg/store/etcd/utils.go Normal file
View File

@@ -0,0 +1,65 @@
package etcd
import (
"strings"
)
// Returns true if the limit isn't 0 AND is greater or equal to the number
// of results.
// If the limit is 0 or the number of items is less than the number of items,
// it will return false
func shouldFinish(numberOfResults, limit int64) bool {
if limit == 0 || numberOfResults < limit {
return false
}
return true
}
// Return the first key out of the prefix represented by the parameter,
// as a byte sequence. Note that it applies to byte sequences and not
// rune sequences, so it might be ill-suited for multi-byte chars
func firstKeyOutOfPrefix(src []byte) []byte {
dst := make([]byte, len(src))
copy(dst, src)
var i int
for i = len(dst) - 1; i >= 0; i-- {
if dst[i] < 255 {
dst[i]++
break
}
}
return dst[:i+1]
}
// Return the first key out of the prefix represented by the parameter.
// This function relies on the firstKeyOutOfPrefix one, which uses a byte
// sequence, so it might be ill-suited if the string contains multi-byte chars.
func firstKeyOutOfPrefixString(src string) string {
srcBytes := []byte(src)
dstBytes := firstKeyOutOfPrefix(srcBytes)
return string(dstBytes)
}
// Reverse the string based on the containing runes
func reverseString(s string) string {
r := []rune(s)
for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
r[i], r[j] = r[j], r[i]
}
return string(r)
}
// Build a string based on the parts, to be used as a prefix. Empty string is
// expected if no part is passed as parameter.
// The string will contain all the parts separated by '/'. The last char will
// also be '/'
//
// For example `buildPrefix(P1, P2, P3)` will return "P1/P2/P3/"
func buildPrefix(parts ...string) string {
var b strings.Builder
for _, part := range parts {
b.WriteString(part)
b.WriteRune('/')
}
return b.String()
}

513
ocis-pkg/store/memory/memstore.go Normal file
View File

@@ -0,0 +1,513 @@
package memory
import (
"container/list"
"context"
"strings"
"sync"
"time"
"github.com/armon/go-radix"
"go-micro.dev/v4/store"
)
// In-memory store implementation using radix tree for fast prefix and suffix
// searches.
// Insertions are expected to be a bit slow due to the data structures, but
// searches are expected to be fast, including exact key search, as well as
// prefix and suffix searches (based on the number of elements to be returned).
// Prefix+suffix search isn't optimized and will depend on how many items we
// need to skip.
// It's also recommended to use reasonable limits when using prefix or suffix
// searches because we'll need to traverse the data structures to provide the
// results. The traversal will stop a soon as we have the required number of
// results, so it will be faster if we use a short limit.
//
// The overall performance will depend on how the radix trees are built.
// The number of elements won't directly affect the performance but how the
// keys are dispersed. The more dispersed the keys are, the faster the search
// will be, regardless of the number of keys. This happens due to the number
// of hops we need to do to reach the target element.
// This also mean that if the keys are too similar, the performance might be
// slower than expected even if the number of elements isn't too big.
type MemStore struct {
preRadix *radix.Tree
sufRadix *radix.Tree
evictionList *list.List
options store.Options
lockGlob sync.RWMutex
lockEvicList sync.RWMutex // Read operation will modify the eviction list
}
type storeRecord struct {
Key string
Value []byte
Metadata map[string]interface{}
Expiry time.Duration
ExpiresAt time.Time
}
type contextKey string
var targetContextKey contextKey
// Prepare a context to be used with the memory implementation. The context
// is used to set up custom parameters to the specific implementation.
// In this case, you can configure the maximum capacity for the MemStore
// implementation as shown below.
// ```
// cache := NewMemStore(
// store.WithContext(
// NewContext(
// ctx,
// map[string]interface{}{
// "maxCap": 50,
// },
// ),
// ),
// )
// ```
//
// Available options for the MemStore are:
// * "maxCap" -> 512 (int) The maximum number of elements the cache will hold.
// Adding additional elements will remove old elements to ensure we aren't over
// the maximum capacity.
//
// For convenience, this can also be used for the MultiMemStore.
func NewContext(ctx context.Context, storeParams map[string]interface{}) context.Context {
return context.WithValue(ctx, targetContextKey, storeParams)
}
// Create a new MemStore instance
func NewMemStore(opts ...store.Option) store.Store {
m := &MemStore{}
_ = m.Init(opts...)
return m
}
// Get the maximum capacity configured. If no maxCap has been configured
// (via `NewContext`), 512 will be used as maxCap.
func (m *MemStore) getMaxCap() int {
maxCap := 512
ctx := m.options.Context
if ctx == nil {
return maxCap
}
ctxValue := ctx.Value(targetContextKey)
if ctxValue == nil {
return maxCap
}
additionalOpts := ctxValue.(map[string]interface{})
confCap, exists := additionalOpts["maxCap"]
if exists {
maxCap = confCap.(int)
}
return maxCap
}
// Initialize the MemStore. If the MemStore was used, this will reset
// all the internal structures and the new options (passed as parameters)
// will be used.
func (m *MemStore) Init(opts ...store.Option) error {
optList := store.Options{}
for _, opt := range opts {
opt(&optList)
}
m.lockGlob.Lock()
defer m.lockGlob.Unlock()
m.preRadix = radix.New()
m.sufRadix = radix.New()
m.evictionList = list.New()
m.options = optList
return nil
}
// Get the options being used
func (m *MemStore) Options() store.Options {
m.lockGlob.RLock()
defer m.lockGlob.RUnlock()
return m.options
}
// Write the record in the MemStore.
// Note that Database and Table options will be ignored.
// Expiration options will take the following precedence:
// TTL option > expiration option > TTL record
//
// New elements will take the last position in the eviction list. Updating
// an element will also move the element to the last position.
//
// Although not recommended, new elements might be inserted with an
// already-expired date
func (m *MemStore) Write(r *store.Record, opts ...store.WriteOption) error {
var element *list.Element
wopts := store.WriteOptions{}
for _, opt := range opts {
opt(&wopts)
}
cRecord := toStoreRecord(r, wopts)
m.lockGlob.Lock()
defer m.lockGlob.Unlock()
ele, exists := m.preRadix.Get(cRecord.Key)
if exists {
element = ele.(*list.Element)
element.Value = cRecord
m.evictionList.MoveToBack(element)
} else {
if m.evictionList.Len() >= m.getMaxCap() {
elementToDelete := m.evictionList.Front()
if elementToDelete != nil {
recordToDelete := elementToDelete.Value.(*storeRecord)
_, _ = m.preRadix.Delete(recordToDelete.Key)
_, _ = m.sufRadix.Delete(recordToDelete.Key)
m.evictionList.Remove(elementToDelete)
}
}
element = m.evictionList.PushBack(cRecord)
_, _ = m.preRadix.Insert(cRecord.Key, element)
_, _ = m.sufRadix.Insert(reverseString(cRecord.Key), element)
}
return nil
}
// Read the key from the MemStore. A list of records will be returned even if
// you're asking for the exact key (only one record is expected in that case).
//
// Reading the exact element will move such element to the last position of
// the eviction list. This WON'T apply for prefix and / or suffix reads.
//
// This method guarantees that no expired element will be returned. For the
// case of exact read, the element will be removed and a "not found" error
// will be returned.
// For prefix and suffix reads, all the elements that we traverse through
// will be removed. This includes the elements we need to skip as well as
// the elements that might have gotten into the the result. Note that the
// elements that are over the limit won't be touched
//
// All read options are supported except Database and Table.
//
// For prefix and prefix+suffix options, the records will be returned in
// alphabetical order on the keys.
// For the suffix option (just suffix, no prefix), the records will be
// returned in alphabetical order after reversing the keys. This means,
// reverse all the keys and then sort them alphabetically. This just affects
// the sorting order; the keys will be returned as expected.
// This means that ["aboz", "caaz", "ziuz"] will be sorted as ["caaz", "aboz", "ziuz"]
// for the key "z" as suffix.
//
// Note that offset are supported but not recommended. There is no direct access
// to the record X. We'd need to skip all the records until we reach the specified
// offset, which could be problematic.
// Performance for prefix and suffix searches should be good assuming we limit
// the number of results we need to return.
func (m *MemStore) Read(key string, opts ...store.ReadOption) ([]*store.Record, error) {
var element *list.Element
ropts := store.ReadOptions{}
for _, opt := range opts {
opt(&ropts)
}
if !ropts.Prefix && !ropts.Suffix {
m.lockGlob.RLock()
ele, exists := m.preRadix.Get(key)
if !exists {
m.lockGlob.RUnlock()
return nil, store.ErrNotFound
}
element = ele.(*list.Element)
record := element.Value.(*storeRecord)
if record.Expiry != 0 && record.ExpiresAt.Before(time.Now()) {
// record expired -> need to delete
m.lockGlob.RUnlock()
m.lockGlob.Lock()
defer m.lockGlob.Unlock()
m.evictionList.Remove(element)
_, _ = m.preRadix.Delete(key)
_, _ = m.sufRadix.Delete(reverseString(key))
return nil, store.ErrNotFound
}
m.lockEvicList.Lock()
m.evictionList.MoveToBack(element)
m.lockEvicList.Unlock()
foundRecords := []*store.Record{
fromStoreRecord(record),
}
m.lockGlob.RUnlock()
return foundRecords, nil
}
records := []*store.Record{}
expiredElements := make(map[string]*list.Element)
m.lockGlob.RLock()
if ropts.Prefix && ropts.Suffix {
// if we need to check both prefix and suffix, go through the
// prefix tree and skip elements without the right suffix. We
// don't need to check the suffix tree because the elements
// must be in both trees
m.preRadix.WalkPrefix(key, m.radixTreeCallBackCheckSuffix(ropts.Offset, ropts.Limit, key, &records, expiredElements))
} else {
if ropts.Prefix {
m.preRadix.WalkPrefix(key, m.radixTreeCallBack(ropts.Offset, ropts.Limit, &records, expiredElements))
}
if ropts.Suffix {
m.sufRadix.WalkPrefix(reverseString(key), m.radixTreeCallBack(ropts.Offset, ropts.Limit, &records, expiredElements))
}
}
m.lockGlob.RUnlock()
// if there are expired elements, get a write lock and delete the expired elements
if len(expiredElements) > 0 {
m.lockGlob.Lock()
for key, element := range expiredElements {
m.evictionList.Remove(element)
_, _ = m.preRadix.Delete(key)
_, _ = m.sufRadix.Delete(reverseString(key))
}
m.lockGlob.Unlock()
}
return records, nil
}
// Remove the record based on the key. It won't return any error if it's missing
//
// Database and Table options aren't supported
func (m *MemStore) Delete(key string, opts ...store.DeleteOption) error {
m.lockGlob.Lock()
defer m.lockGlob.Unlock()
ele, exists := m.preRadix.Get(key)
if exists {
element := ele.(*list.Element)
m.evictionList.Remove(element)
_, _ = m.preRadix.Delete(key)
_, _ = m.sufRadix.Delete(reverseString(key))
}
return nil
}
// List the keys currently used in the MemStore
//
// All options are supported except Database and Table
//
// For prefix and prefix+suffix options, the keys will be returned in
// alphabetical order.
// For the suffix option (just suffix, no prefix), the keys will be
// returned in alphabetical order after reversing the keys. This means,
// reverse all the keys and then sort them alphabetically. This just affects
// the sorting order; the keys will be returned as expected.
// This means that ["aboz", "caaz", "ziuz"] will be sorted as ["caaz", "aboz", "ziuz"]
func (m *MemStore) List(opts ...store.ListOption) ([]string, error) {
records := []string{}
expiredElements := make(map[string]*list.Element)
lopts := store.ListOptions{}
for _, opt := range opts {
opt(&lopts)
}
if lopts.Prefix == "" && lopts.Suffix == "" {
m.lockGlob.RLock()
m.preRadix.Walk(m.radixTreeCallBackKeysOnly(lopts.Offset, lopts.Limit, &records, expiredElements))
m.lockGlob.RUnlock()
// if there are expired elements, get a write lock and delete the expired elements
if len(expiredElements) > 0 {
m.lockGlob.Lock()
for key, element := range expiredElements {
m.evictionList.Remove(element)
_, _ = m.preRadix.Delete(key)
_, _ = m.sufRadix.Delete(reverseString(key))
}
m.lockGlob.Unlock()
}
return records, nil
}
m.lockGlob.RLock()
if lopts.Prefix != "" && lopts.Suffix != "" {
// if we need to check both prefix and suffix, go through the
// prefix tree and skip elements without the right suffix. We
// don't need to check the suffix tree because the elements
// must be in both trees
m.preRadix.WalkPrefix(lopts.Prefix, m.radixTreeCallBackKeysOnlyWithSuffix(lopts.Offset, lopts.Limit, lopts.Suffix, &records, expiredElements))
} else {
if lopts.Prefix != "" {
m.preRadix.WalkPrefix(lopts.Prefix, m.radixTreeCallBackKeysOnly(lopts.Offset, lopts.Limit, &records, expiredElements))
}
if lopts.Suffix != "" {
m.sufRadix.WalkPrefix(reverseString(lopts.Suffix), m.radixTreeCallBackKeysOnly(lopts.Offset, lopts.Limit, &records, expiredElements))
}
}
m.lockGlob.RUnlock()
// if there are expired elements, get a write lock and delete the expired elements
if len(expiredElements) > 0 {
m.lockGlob.Lock()
for key, element := range expiredElements {
m.evictionList.Remove(element)
_, _ = m.preRadix.Delete(key)
_, _ = m.sufRadix.Delete(reverseString(key))
}
m.lockGlob.Unlock()
}
return records, nil
}
func (m *MemStore) Close() error {
return nil
}
func (m *MemStore) String() string {
return "RadixMemStore"
}
func (m *MemStore) Len() (int, bool) {
eLen := m.evictionList.Len()
pLen := m.preRadix.Len()
sLen := m.sufRadix.Len()
if eLen == pLen && eLen == sLen {
return eLen, true
}
return 0, false
}
func (m *MemStore) radixTreeCallBack(offset, limit uint, result *[]*store.Record, expiredElements map[string]*list.Element) radix.WalkFn {
currentIndex := new(uint) // needs to be a pointer so the value persist across callback calls
maxIndex := new(uint) // needs to be a pointer so the value persist across callback calls
*maxIndex = offset + limit
return func(key string, value interface{}) bool {
element := value.(*list.Element)
record := element.Value.(*storeRecord)
if record.Expiry != 0 && record.ExpiresAt.Before(time.Now()) {
// record has expired -> add element to the expiredElements map
// and jump directly to the next element without increasing the index
expiredElements[record.Key] = element
return false
}
if *currentIndex >= offset && (*currentIndex < *maxIndex || *maxIndex == offset) {
// if it's within expected range, add a copy to the results
*result = append(*result, fromStoreRecord(record))
}
*currentIndex++
if *currentIndex < *maxIndex || *maxIndex == offset {
return false
}
return true
}
}
func (m *MemStore) radixTreeCallBackCheckSuffix(offset, limit uint, presuf string, result *[]*store.Record, expiredElements map[string]*list.Element) radix.WalkFn {
currentIndex := new(uint) // needs to be a pointer so the value persist across callback calls
maxIndex := new(uint) // needs to be a pointer so the value persist across callback calls
*maxIndex = offset + limit
return func(key string, value interface{}) bool {
if !strings.HasSuffix(key, presuf) {
return false
}
element := value.(*list.Element)
record := element.Value.(*storeRecord)
if record.Expiry != 0 && record.ExpiresAt.Before(time.Now()) {
// record has expired -> add element to the expiredElements map
// and jump directly to the next element without increasing the index
expiredElements[record.Key] = element
return false
}
if *currentIndex >= offset && (*currentIndex < *maxIndex || *maxIndex == offset) {
*result = append(*result, fromStoreRecord(record))
}
*currentIndex++
if *currentIndex < *maxIndex || *maxIndex == offset {
return false
}
return true
}
}
func (m *MemStore) radixTreeCallBackKeysOnly(offset, limit uint, result *[]string, expiredElements map[string]*list.Element) radix.WalkFn {
currentIndex := new(uint) // needs to be a pointer so the value persist across callback calls
maxIndex := new(uint) // needs to be a pointer so the value persist across callback calls
*maxIndex = offset + limit
return func(key string, value interface{}) bool {
element := value.(*list.Element)
record := element.Value.(*storeRecord)
if record.Expiry != 0 && record.ExpiresAt.Before(time.Now()) {
// record has expired -> add element to the expiredElements map
// and jump directly to the next element without increasing the index
expiredElements[record.Key] = element
return false
}
if *currentIndex >= offset && (*currentIndex < *maxIndex || *maxIndex == offset) {
*result = append(*result, record.Key)
}
*currentIndex++
if *currentIndex < *maxIndex || *maxIndex == offset {
return false
}
return true
}
}
func (m *MemStore) radixTreeCallBackKeysOnlyWithSuffix(offset, limit uint, presuf string, result *[]string, expiredElements map[string]*list.Element) radix.WalkFn {
currentIndex := new(uint) // needs to be a pointer so the value persist across callback calls
maxIndex := new(uint) // needs to be a pointer so the value persist across callback calls
*maxIndex = offset + limit
return func(key string, value interface{}) bool {
if !strings.HasSuffix(key, presuf) {
return false
}
element := value.(*list.Element)
record := element.Value.(*storeRecord)
if record.Expiry != 0 && record.ExpiresAt.Before(time.Now()) {
// record has expired -> add element to the expiredElements map
// and jump directly to the next element without increasing the index
expiredElements[record.Key] = element
return false
}
if *currentIndex >= offset && (*currentIndex < *maxIndex || *maxIndex == offset) {
*result = append(*result, record.Key)
}
*currentIndex++
if *currentIndex < *maxIndex || *maxIndex == offset {
return false
}
return true
}
}

1506
ocis-pkg/store/memory/memstore_test.go Normal file
View File

File diff suppressed because it is too large Load Diff

158
ocis-pkg/store/memory/multimemstore.go Normal file
View File

@@ -0,0 +1,158 @@
package memory
import (
"sync"
"go-micro.dev/v4/store"
)
// In-memory store implementation using multiple MemStore to provide support
// for multiple databases and tables.
// Each table will be mapped to its own MemStore, which will be completely
// isolated from the rest. In particular, each MemStore will have its own
// capacity, so it's possible to have 10 MemStores with full capacity (512
// by default)
//
// The options will be the same for all MemStores unless they're explicitly
// initialized otherwise.
//
// Since each MemStore is isolated, the required synchronization caused by
// concurrency will be minimal if the threads use different tables
type MultiMemStore struct {
storeMap map[string]*MemStore
storeMapLock sync.RWMutex
genOpts []store.Option
}
// Create a new MultiMemStore. A new MemStore will be mapped based on the options.
// A default MemStore will be mapped if no Database and Table aren't used.
func NewMultiMemStore(opts ...store.Option) store.Store {
m := &MultiMemStore{
storeMap: make(map[string]*MemStore),
genOpts: opts,
}
_ = m.Init(opts...)
return m
}
func (m *MultiMemStore) getMemStore(prefix string) *MemStore {
m.storeMapLock.RLock()
mStore, exists := m.storeMap[prefix]
if exists {
m.storeMapLock.RUnlock()
return mStore
}
m.storeMapLock.RUnlock()
// if not exists
newStore := NewMemStore(m.genOpts...).(*MemStore)
m.storeMapLock.Lock()
m.storeMap[prefix] = newStore
m.storeMapLock.Unlock()
return newStore
}
// Initialize the mapped MemStore based on the Database and Table values
// from the options with the same options. The target MemStore will be
// reinitialized if needed.
func (m *MultiMemStore) Init(opts ...store.Option) error {
optList := store.Options{}
for _, opt := range opts {
opt(&optList)
}
prefix := optList.Database + "/" + optList.Table
mStore := m.getMemStore(prefix)
return mStore.Init(opts...)
}
// Get the options used to create the MultiMemStore.
// Specific options for each MemStore aren't available
func (m *MultiMemStore) Options() store.Options {
optList := store.Options{}
for _, opt := range m.genOpts {
opt(&optList)
}
return optList
}
// Write the record in the target MemStore based on the Database and Table
// values from the options. A default MemStore will be used if no Database
// and Table options are provided.
// The write options will be forwarded to the target MemStore
func (m *MultiMemStore) Write(r *store.Record, opts ...store.WriteOption) error {
wopts := store.WriteOptions{}
for _, opt := range opts {
opt(&wopts)
}
prefix := wopts.Database + "/" + wopts.Table
mStore := m.getMemStore(prefix)
return mStore.Write(r, opts...)
}
// Read the matching records in the target MemStore based on the Database and Table
// values from the options. A default MemStore will be used if no Database
// and Table options are provided.
// The read options will be forwarded to the target MemStore.
//
// The expectations regarding the results (sort order, eviction policies, etc)
// will be the same as the target MemStore
func (m *MultiMemStore) Read(key string, opts ...store.ReadOption) ([]*store.Record, error) {
ropts := store.ReadOptions{}
for _, opt := range opts {
opt(&ropts)
}
prefix := ropts.Database + "/" + ropts.Table
mStore := m.getMemStore(prefix)
return mStore.Read(key, opts...)
}
// Delete the matching records in the target MemStore based on the Database and Table
// values from the options. A default MemStore will be used if no Database
// and Table options are provided.
//
// Matching records from other Tables won't be affected. In fact, we won't
// access to other Tables
func (m *MultiMemStore) Delete(key string, opts ...store.DeleteOption) error {
dopts := store.DeleteOptions{}
for _, opt := range opts {
opt(&dopts)
}
prefix := dopts.Database + "/" + dopts.Table
mStore := m.getMemStore(prefix)
return mStore.Delete(key, opts...)
}
// List the keys in the target MemStore based on the Database and Table
// values from the options. A default MemStore will be used if no Database
// and Table options are provided.
// The list options will be forwarded to the target MemStore.
func (m *MultiMemStore) List(opts ...store.ListOption) ([]string, error) {
lopts := store.ListOptions{}
for _, opt := range opts {
opt(&lopts)
}
prefix := lopts.Database + "/" + lopts.Table
mStore := m.getMemStore(prefix)
return mStore.List(opts...)
}
func (m *MultiMemStore) Close() error {
return nil
}
func (m *MultiMemStore) String() string {
return "MultiRadixMemStore"
}

172
ocis-pkg/store/memory/multimemstore_test.go Normal file
View File

@@ -0,0 +1,172 @@
package memory
import (
"context"
"strconv"
"testing"
"go-micro.dev/v4/store"
)
func TestWriteReadTables(t *testing.T) {
cache := NewMultiMemStore()
record1 := &store.Record{
Key: "sameKey",
Value: []byte("from record1"),
}
record2 := &store.Record{
Key: "sameKey",
Value: []byte("from record2"),
}
_ = cache.Write(record1)
_ = cache.Write(record2, store.WriteTo("DB02", "Table02"))
records1, _ := cache.Read("sameKey")
if len(records1) != 1 {
t.Fatalf("Wrong number of records, expected 1, got %d", len(records1))
}
if records1[0].Key != "sameKey" {
t.Errorf("Wrong key, expected \"sameKey\", got %s", records1[0].Key)
}
if string(records1[0].Value) != "from record1" {
t.Errorf("Wrong value, expected \"from record1\", got %s", string(records1[0].Value))
}
records2, _ := cache.Read("sameKey", store.ReadFrom("DB02", "Table02"))
if len(records2) != 1 {
t.Fatalf("Wrong number of records, expected 1, got %d", len(records2))
}
if records2[0].Key != "sameKey" {
t.Errorf("Wrong key, expected \"sameKey\", got %s", records2[0].Key)
}
if string(records2[0].Value) != "from record2" {
t.Errorf("Wrong value, expected \"from record2\", got %s", string(records2[0].Value))
}
}
func TestDeleteTables(t *testing.T) {
cache := NewMultiMemStore()
record1 := &store.Record{
Key: "sameKey",
Value: []byte("from record1"),
}
record2 := &store.Record{
Key: "sameKey",
Value: []byte("from record2"),
}
_ = cache.Write(record1)
_ = cache.Write(record2, store.WriteTo("DB02", "Table02"))
records1, _ := cache.Read("sameKey")
if len(records1) != 1 {
t.Fatalf("Wrong number of records, expected 1, got %d", len(records1))
}
if records1[0].Key != "sameKey" {
t.Errorf("Wrong key, expected \"sameKey\", got %s", records1[0].Key)
}
if string(records1[0].Value) != "from record1" {
t.Errorf("Wrong value, expected \"from record1\", got %s", string(records1[0].Value))
}
records2, _ := cache.Read("sameKey", store.ReadFrom("DB02", "Table02"))
if len(records2) != 1 {
t.Fatalf("Wrong number of records, expected 1, got %d", len(records2))
}
if records2[0].Key != "sameKey" {
t.Errorf("Wrong key, expected \"sameKey\", got %s", records2[0].Key)
}
if string(records2[0].Value) != "from record2" {
t.Errorf("Wrong value, expected \"from record2\", got %s", string(records2[0].Value))
}
_ = cache.Delete("sameKey")
if _, err := cache.Read("sameKey"); err != store.ErrNotFound {
t.Errorf("Key \"sameKey\" still exists after deletion")
}
records2, _ = cache.Read("sameKey", store.ReadFrom("DB02", "Table02"))
if len(records2) != 1 {
t.Fatalf("Wrong number of records, expected 1, got %d", len(records2))
}
if records2[0].Key != "sameKey" {
t.Errorf("Wrong key, expected \"sameKey\", got %s", records2[0].Key)
}
if string(records2[0].Value) != "from record2" {
t.Errorf("Wrong value, expected \"from record2\", got %s", string(records2[0].Value))
}
}
func TestListTables(t *testing.T) {
cache := NewMultiMemStore()
record1 := &store.Record{
Key: "key001",
Value: []byte("from record1"),
}
record2 := &store.Record{
Key: "key002",
Value: []byte("from record2"),
}
_ = cache.Write(record1)
_ = cache.Write(record2, store.WriteTo("DB02", "Table02"))
keys, _ := cache.List(store.ListFrom("DB02", "Table02"))
expectedKeys := []string{"key002"}
if len(keys) != 1 {
t.Fatalf("Wrong number of keys, expected 1, got %d", len(keys))
}
for index, key := range keys {
if expectedKeys[index] != key {
t.Errorf("Wrong key for index %d, expected %s, got %s", index, expectedKeys[index], key)
}
}
}
func TestWriteSizeLimit(t *testing.T) {
cache := NewMultiMemStore(
store.WithContext(
NewContext(
context.Background(),
map[string]interface{}{
"maxCap": 2,
},
),
),
)
record := &store.Record{}
for i := 0; i < 4; i++ {
v := strconv.Itoa(i)
record.Key = v
record.Value = []byte(v)
_ = cache.Write(record)
_ = cache.Write(record, store.WriteTo("DB02", "Table02"))
}
keys1, _ := cache.List()
expectedKeys1 := []string{"2", "3"}
if len(keys1) != 2 {
t.Fatalf("Wrong number of keys, expected 2, got %d", len(keys1))
}
for index, key := range keys1 {
if expectedKeys1[index] != key {
t.Errorf("Wrong key for index %d, expected %s, got %s", index, expectedKeys1[index], key)
}
}
keys2, _ := cache.List(store.ListFrom("DB02", "Table02"))
expectedKeys2 := []string{"2", "3"}
if len(keys2) != 2 {
t.Fatalf("Wrong number of keys, expected 2, got %d", len(keys2))
}
for index, key := range keys2 {
if expectedKeys2[index] != key {
t.Errorf("Wrong key for index %d, expected %s, got %s", index, expectedKeys2[index], key)
}
}
}

63
ocis-pkg/store/memory/utils.go Normal file
View File

@@ -0,0 +1,63 @@
package memory
import (
"time"
"go-micro.dev/v4/store"
)
func toStoreRecord(src *store.Record, options store.WriteOptions) *storeRecord {
newRecord := &storeRecord{}
newRecord.Key = src.Key
newRecord.Value = make([]byte, len(src.Value))
copy(newRecord.Value, src.Value)
// set base ttl duration and expiration time based on the record
newRecord.Expiry = src.Expiry
if src.Expiry != 0 {
newRecord.ExpiresAt = time.Now().Add(src.Expiry)
}
// overwrite ttl duration and expiration time based on options
if !options.Expiry.IsZero() {
// options.Expiry is a time.Time, newRecord.Expiry is a time.Duration
newRecord.Expiry = time.Until(options.Expiry)
newRecord.ExpiresAt = options.Expiry
}
// TTL option takes precedence over expiration time
if options.TTL != 0 {
newRecord.Expiry = options.TTL
newRecord.ExpiresAt = time.Now().Add(options.TTL)
}
newRecord.Metadata = make(map[string]interface{})
for k, v := range src.Metadata {
newRecord.Metadata[k] = v
}
return newRecord
}
func fromStoreRecord(src *storeRecord) *store.Record {
newRecord := &store.Record{}
newRecord.Key = src.Key
newRecord.Value = make([]byte, len(src.Value))
copy(newRecord.Value, src.Value)
if src.Expiry != 0 {
newRecord.Expiry = time.Until(src.ExpiresAt)
}
newRecord.Metadata = make(map[string]interface{})
for k, v := range src.Metadata {
newRecord.Metadata[k] = v
}
return newRecord
}
func reverseString(s string) string {
r := []rune(s)
for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
r[i], r[j] = r[j], r[i]
}
return string(r)
}

91
ocis-pkg/store/store.go Normal file
View File

@@ -0,0 +1,91 @@
package store
import (
"context"
"strings"
"github.com/owncloud/ocis/v2/ocis-pkg/store/etcd"
"github.com/owncloud/ocis/v2/ocis-pkg/store/memory"
"go-micro.dev/v4/store"
)
var (
storeEnv = "OCIS_STORE"
storeAddressEnv = "OCIS_STORE_ADDRESS"
storeOCMemSize = "OCIS_STORE_OCMEM_SIZE"
)
var ocMemStore *store.Store
type OcisStoreOptions struct {
Type string
Address string
Size int
}
// Get the configured key-value store to be used.
//
// Each microservice (or whatever piece is using the store) should use the
// options available in the interface's operations to choose the right database
// and table to prevent collisions with other microservices.
// Recommended approach is to use "services" or "ocis-pkg" for the database,
// and "services/<service-name>/" or "ocis-pkg/<pkg>/" for the package name.
//
// So far, only the name of the store and the node addresses are configurable
// via environment variables.
// Available options for "OCIS_STORE" are:
// * "noop", for a noop store (it does nothing)
// * "etcd", for etcd
// * "ocmem", custom in-memory implementation, with fixed size and optimized prefix
// and suffix search
// * "memory", for a in-memory implementation, which is the default if noone matches
//
// "OCIS_STORE_ADDRESS" is a comma-separated list of nodes that the store
// will use. This is currently usable only with the etcd implementation. If it
// isn't provided, "127.0.0.1:2379" will be the only node used.
//
// "OCIS_STORE_OCMEM_SIZE" will configure the maximum capacity of the cache for
// the "ocmem" implementation, in number of items that the cache can hold per table.
// You can use "OCIS_STORE_OCMEM_SIZE=5000" so the cache will hold up to 5000 elements.
// The parameter only affects to the "ocmem" implementation, the rest will ignore it.
// If an invalid value is used, the default will be used instead, so up to 512 elements
// the cache will hold.
func GetStore(ocisOpts OcisStoreOptions) store.Store {
var s store.Store
addresses := strings.Split(ocisOpts.Address, ",")
opts := []store.Option{
store.Nodes(addresses...),
}
switch ocisOpts.Type {
case "noop":
s = store.NewNoopStore(opts...)
case "etcd":
s = etcd.NewEtcdStore(opts...)
case "ocmem":
if ocMemStore == nil {
var memStore store.Store
sizeNum := ocisOpts.Size
if sizeNum <= 0 {
memStore = memory.NewMultiMemStore()
} else {
memStore = memory.NewMultiMemStore(
store.WithContext(
memory.NewContext(
context.Background(),
map[string]interface{}{
"maxCap": sizeNum,
},
)),
)
}
ocMemStore = &memStore
}
s = *ocMemStore
default:
s = store.NewMemoryStore(opts...)
}
return s
}

1
services/audit/pkg/types/events.go
View File

@@ -29,6 +29,7 @@ func RegisteredEvents() []events.Unmarshaller {
events.SpaceEnabled{},
events.SpaceDisabled{},
events.SpaceDeleted{},
events.SpaceShared{},
events.UserCreated{},
events.UserDeleted{},
events.UserFeatureChanged{},

8
services/graph/pkg/config/cachestore.go Normal file
View File

@@ -0,0 +1,8 @@
package config
// CacheStore defines the available configuration for the cache store
type CacheStore struct {
Type string `yaml:"type" env:"OCIS_CACHE_STORE_TYPE;GRAPH_CACHE_STORE_TYPE" desc:"The type of the cache store. Valid options are \"noop\", \"ocmem\", \"etcd\" and \"memory\""`
Address string `yaml:"address" env:"OCIS_CACHE_STORE_ADDRESS;GRAPH_CACHE_STORE_ADDRESS" desc:"a comma-separated list of addresses to connect to. Only for etcd"`
Size int `yaml:"size" env:"OCIS_CACHE_STORE_SIZE;GRAPH_CACHE_STORE_SIZE" desc:"Maximum size for the cache store. Only ocmem will use this option, in number of items per table. The rest will ignore the option and can grow indefinitely"`
}

7
services/graph/pkg/config/config.go
View File

@@ -12,9 +12,10 @@ type Config struct {
Service Service `yaml:"-"`
Tracing *Tracing `yaml:"tracing"`
Log *Log `yaml:"log"`
Debug Debug `yaml:"debug"`
Tracing *Tracing `yaml:"tracing"`
Log *Log `yaml:"log"`
CacheStore *CacheStore `yaml:"cache_store"`
Debug Debug `yaml:"debug"`
HTTP HTTP `yaml:"http"`

10
services/graph/pkg/config/defaults/defaultconfig.go
View File

@@ -96,6 +96,16 @@ func EnsureDefaults(cfg *config.Config) {
cfg.Tracing = &config.Tracing{}
}
if cfg.CacheStore == nil && cfg.Commons != nil && cfg.Commons.CacheStore != nil {
cfg.CacheStore = &config.CacheStore{
Type: cfg.Commons.CacheStore.Type,
Address: cfg.Commons.CacheStore.Address,
Size: cfg.Commons.CacheStore.Size,
}
} else if cfg.CacheStore == nil {
cfg.CacheStore = &config.CacheStore{}
}
if cfg.TokenManager == nil && cfg.Commons != nil && cfg.Commons.TokenManager != nil {
cfg.TokenManager = &config.TokenManager{
JWTSecret: cfg.Commons.TokenManager.JWTSecret,

10
services/graph/pkg/service/v0/service.go
View File

@@ -6,7 +6,6 @@ import (
"io/ioutil"
"net/http"
"strconv"
"time"
"github.com/ReneKroon/ttlcache/v2"
"github.com/cs3org/reva/v2/pkg/rgrpc/todo/pool"
@@ -16,6 +15,7 @@ import (
ocisldap "github.com/owncloud/ocis/v2/ocis-pkg/ldap"
"github.com/owncloud/ocis/v2/ocis-pkg/roles"
"github.com/owncloud/ocis/v2/ocis-pkg/service/grpc"
"github.com/owncloud/ocis/v2/ocis-pkg/store"
settingssvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/settings/v0"
"github.com/owncloud/ocis/v2/services/graph/pkg/identity"
"github.com/owncloud/ocis/v2/services/graph/pkg/identity/ldap"
@@ -144,9 +144,13 @@ func NewService(opts ...Option) Service {
roleManager := options.RoleManager
if roleManager == nil {
storeOptions := store.OcisStoreOptions{
Type: options.Config.CacheStore.Type,
Address: options.Config.CacheStore.Address,
Size: options.Config.CacheStore.Size,
}
m := roles.NewManager(
roles.CacheSize(1024),
roles.CacheTTL(time.Hour),
roles.StoreOptions(storeOptions),
roles.Logger(options.Logger),
roles.RoleService(svc.roleService),
)

18
services/notifications/pkg/channels/channels.go
View File

@@ -4,6 +4,7 @@ package channels
import (
"context"
"crypto/tls"
"fmt"
"strings"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
@@ -19,9 +20,9 @@ import (
// Channel defines the methods of a communication channel.
type Channel interface {
// SendMessage sends a message to users.
SendMessage(userIDs []string, msg string) error
SendMessage(userIDs []string, msg, subject, senderDisplayName string) error
// SendMessageToGroup sends a message to a group.
SendMessageToGroup(groupdID *groups.GroupId, msg string) error
SendMessageToGroup(groupdID *groups.GroupId, msg, subject, senderDisplayName string) error
}
// NewMailChannel instantiates a new mail communication channel.
@@ -100,7 +101,7 @@ func (m Mail) getMailClient() (*mail.SMTPClient, error) {
}
// SendMessage sends a message to all given users.
func (m Mail) SendMessage(userIDs []string, msg string) error {
func (m Mail) SendMessage(userIDs []string, msg, subject, senderDisplayName string) error {
if m.conf.Notifications.SMTP.Host == "" {
return nil
}
@@ -116,14 +117,19 @@ func (m Mail) SendMessage(userIDs []string, msg string) error {
}
email := mail.NewMSG()
email.SetFrom(m.conf.Notifications.SMTP.Sender).AddTo(to...)
if senderDisplayName != "" {
email.SetFrom(fmt.Sprintf("%s via owncloud <%s>", senderDisplayName, m.conf.Notifications.SMTP.Sender)).AddTo(to...)
} else {
email.SetFrom(m.conf.Notifications.SMTP.Sender).AddTo(to...)
}
email.SetBody(mail.TextPlain, msg)
email.SetSubject(subject)
return email.Send(smtpClient)
}
// SendMessageToGroup sends a message to all members of the given group.
func (m Mail) SendMessageToGroup(groupID *groups.GroupId, msg string) error {
func (m Mail) SendMessageToGroup(groupID *groups.GroupId, msg, subject, senderDisplayName string) error {
// TODO We need an authenticated context here...
res, err := m.gatewayClient.GetGroup(context.Background(), &groups.GetGroupRequest{GroupId: groupID})
if err != nil {
@@ -138,7 +144,7 @@ func (m Mail) SendMessageToGroup(groupID *groups.GroupId, msg string) error {
members = append(members, id.OpaqueId)
}
return m.SendMessage(members, msg)
return m.SendMessage(members, msg, subject, senderDisplayName)
}
func (m Mail) getReceiverAddresses(receivers []string) ([]string, error) {

10
services/notifications/pkg/command/server.go
View File

@@ -5,6 +5,7 @@ import (
"github.com/cs3org/reva/v2/pkg/events"
"github.com/cs3org/reva/v2/pkg/events/server"
"github.com/cs3org/reva/v2/pkg/rgrpc/todo/pool"
"github.com/go-micro/plugins/v4/events/natsjs"
"github.com/owncloud/ocis/v2/ocis-pkg/config/configlog"
"github.com/owncloud/ocis/v2/services/notifications/pkg/channels"
@@ -27,8 +28,10 @@ func Server(cfg *config.Config) *cli.Command {
Action: func(c *cli.Context) error {
logger := logging.Configure(cfg.Service.Name, cfg.Log)
// evs defines a list of events to subscribe to
evs := []events.Unmarshaller{
events.ShareCreated{},
events.SpaceShared{},
}
evtsCfg := cfg.Notifications.Events
@@ -47,7 +50,12 @@ func Server(cfg *config.Config) *cli.Command {
if err != nil {
return err
}
svc := service.NewEventsNotifier(evts, channel, logger)
gwclient, err := pool.GetGatewayServiceClient(cfg.Notifications.RevaGateway)
if err != nil {
logger.Fatal().Err(err).Str("addr", cfg.Notifications.RevaGateway).Msg("could not get reva client")
}
svc := service.NewEventsNotifier(evts, channel, logger, gwclient, cfg.Commons.MachineAuthAPIKey, cfg.Notifications.EmailTemplatePath)
return svc.Run()
},
}

1
services/notifications/pkg/config/config.go
View File

@@ -26,6 +26,7 @@ type Notifications struct {
Events Events `yaml:"events"`
RevaGateway string `yaml:"reva_gateway" env:"REVA_GATEWAY;NOTIFICATIONS_REVA_GATEWAY" desc:"CS3 gateway used to look up user metadata"`
MachineAuthAPIKey string `yaml:"machine_auth_api_key" env:"OCIS_MACHINE_AUTH_API_KEY;NOTIFICATIONS_MACHINE_AUTH_API_KEY" desc:"Machine auth API key used to validate internal requests necessary to access resources from other services."`
EmailTemplatePath string `yaml:"email_template_path" env:"OCIS_EMAIL_TEMPLATE_PATH;NOTIFICATIONS_EMAIL_TEMPLATE_PATH" desc:"Path to Email notification templates overriding embedded ones."`
}
// SMTP combines the smtp configuration options.

34
services/notifications/pkg/email/email.go Normal file
View File

@@ -0,0 +1,34 @@
package email
import (
"bytes"
"embed"
"html/template"
"path/filepath"
)
var (
//go:embed templates
templatesFS embed.FS
)
// RenderEmailTemplate renders the email template for a new share
func RenderEmailTemplate(templateName string, templateVariables map[string]string, emailTemplatePath string) (string, error) {
var err error
var tpl *template.Template
// try to lookup the files in the filesystem
tpl, err = template.ParseFiles(filepath.Join(emailTemplatePath, templateName))
if err != nil {
// template has not been found in the fs, or path has not been specified => use embed templates
tpl, err = template.ParseFS(templatesFS, filepath.Join("templates/", templateName))
if err != nil {
return "", err
}
}
var writer bytes.Buffer
err = tpl.Execute(&writer, templateVariables)
if err != nil {
return "", err
}
return writer.String(), nil
}

18
services/notifications/pkg/email/templates/shareCreated.email.tmpl Normal file
View File

@@ -0,0 +1,18 @@
Hello {{ .ShareGrantee }},
{{ .ShareSharer }} has shared {{ .ShareFolder }} with you.
Click here to view it: {{ .ShareLink }}
----------------------------------------------------------
Hallo {{ .Grantee }},
{{ .ShareSharer }} hat dich zu {{ .ShareFolder }} eingeladen.
Klicke hier zum Anzeigen: {{ .ShareLink }}
---
ownCloud - Store. Share. Work.
https://owncloud.com

18
services/notifications/pkg/email/templates/sharedSpace.email.tmpl Normal file
View File

@@ -0,0 +1,18 @@
Hello {{ .SpaceGrantee }},
{{ .SpaceSharer }} has invited you to join {{ .SpaceName }}.
Click here to view it: {{ .ShareLink }}
----------------------------------------------------------
Hallo {{ .SpaceGrantee }},
{{ .SpaceSharer }} hat dich in den Space {{ .SpaceName }} eingeladen.
Klicke hier zum Anzeigen: {{ .ShareLink }}
---
ownCloud - Store. Share. Work.
https://owncloud.com

299
services/notifications/pkg/service/service.go
View File

@@ -1,33 +1,53 @@
package service
import (
"context"
"fmt"
"net/url"
"os"
"os/signal"
"path"
"syscall"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
userv1beta1 "github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1"
rpcv1beta1 "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
providerv1beta1 "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
ctxpkg "github.com/cs3org/reva/v2/pkg/ctx"
"github.com/cs3org/reva/v2/pkg/events"
"github.com/cs3org/reva/v2/pkg/storagespace"
"github.com/owncloud/ocis/v2/ocis-pkg/log"
"github.com/owncloud/ocis/v2/services/notifications/pkg/channels"
"github.com/owncloud/ocis/v2/services/notifications/pkg/email"
"google.golang.org/grpc/metadata"
"google.golang.org/protobuf/types/known/fieldmaskpb"
)
type Service interface {
Run() error
}
func NewEventsNotifier(events <-chan interface{}, channel channels.Channel, logger log.Logger) Service {
// NewEventsNotifier provides a new eventsNotifier
func NewEventsNotifier(events <-chan interface{}, channel channels.Channel, logger log.Logger, gwClient gateway.GatewayAPIClient, machineAuthAPIKey, emailTemplatePath string) Service {
return eventsNotifier{
logger: logger,
channel: channel,
events: events,
signals: make(chan os.Signal, 1),
logger: logger,
channel: channel,
events: events,
signals: make(chan os.Signal, 1),
gwClient: gwClient,
machineAuthAPIKey: machineAuthAPIKey,
emailTemplatePath: emailTemplatePath,
}
}
type eventsNotifier struct {
logger log.Logger
channel channels.Channel
events <-chan interface{}
signals chan os.Signal
logger log.Logger
channel channels.Channel
events <-chan interface{}
signals chan os.Signal
gwClient gateway.GatewayAPIClient
machineAuthAPIKey string
emailTemplatePath string
}
func (s eventsNotifier) Run() error {
@@ -39,20 +59,10 @@ func (s eventsNotifier) Run() error {
case evt := <-s.events:
go func() {
switch e := evt.(type) {
case events.SpaceShared:
s.handleSpaceShared(e)
case events.ShareCreated:
msg := "You got a share!"
var err error
if e.GranteeUserID != nil {
err = s.channel.SendMessage([]string{e.GranteeUserID.OpaqueId}, msg)
} else if e.GranteeGroupID != nil {
err = s.channel.SendMessageToGroup(e.GranteeGroupID, msg)
}
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("failed to send a message")
}
s.handleShareCreated(e)
}
}()
case <-s.signals:
@@ -62,3 +72,248 @@ func (s eventsNotifier) Run() error {
}
}
}
func (s eventsNotifier) handleSpaceShared(e events.SpaceShared) {
sharerUserResponse, err := s.gwClient.GetUser(context.Background(), &userv1beta1.GetUserRequest{
UserId: e.Creator,
})
if err != nil || sharerUserResponse.Status.Code != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "SpaceCreated").
Msg("Could not get user response from gatway client")
return
}
granteeUserResponse, err := s.gwClient.GetUser(context.Background(), &userv1beta1.GetUserRequest{
UserId: e.GranteeUserID,
})
if err != nil || sharerUserResponse.Status.Code != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "SpaceCreated").
Msg("Could not get user response from gatway client")
return
}
// Get auth context
ownerCtx := ctxpkg.ContextSetUser(context.Background(), sharerUserResponse.User)
authRes, err := s.gwClient.Authenticate(ownerCtx, &gateway.AuthenticateRequest{
Type: "machine",
ClientId: "userid:" + e.Executant.OpaqueId,
ClientSecret: s.machineAuthAPIKey,
})
if err != nil {
s.logger.Error().
Err(err).
Str("event", "SpaceCreated").
Msg("Could not impersonate sharer")
return
}
if authRes.GetStatus().GetCode() != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "SpaceCreated").
Msg("could not get authenticated context for user")
return
}
ownerCtx = metadata.AppendToOutgoingContext(ownerCtx, ctxpkg.TokenHeader, authRes.Token)
resourceID, err := storagespace.ParseID(e.ID.OpaqueId)
if err != nil {
s.logger.Error().
Err(err).
Str("event", "SpaceCreated").
Str("itemid", e.ID.OpaqueId).
Msg("could not parse resourceid from ItemID ")
return
}
// TODO: maybe cache this stat to reduce storage iops
md, err := s.gwClient.Stat(ownerCtx, &providerv1beta1.StatRequest{
Ref: &providerv1beta1.Reference{
ResourceId: &resourceID,
},
// TODO: this filter needs to be implemented
//FieldMask: &fieldmaskpb.FieldMask{Paths: []string{"space.name"}},
})
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Str("itemid", e.ID.OpaqueId).
Msg("could not stat resource")
return
}
if md.Status.Code != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Str("itemid", e.ID.OpaqueId).
Str("rpc status", md.Status.Code.String()).
Msg("could not stat resource")
return
}
shareLink, err := urlJoinPath(e.Executant.Idp, "files/spaces/projects", storagespace.FormatResourceID(*e.ID))
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("could not create link to the share")
return
}
sharerDisplayName := sharerUserResponse.GetUser().DisplayName
msg, err := email.RenderEmailTemplate("sharedSpace.email.tmpl", map[string]string{
"SpaceGrantee": granteeUserResponse.GetUser().DisplayName,
"SpaceSharer": sharerDisplayName,
"SpaceName": md.GetInfo().GetSpace().Name,
"ShareLink": shareLink,
}, s.emailTemplatePath)
if err != nil {
s.logger.Error().
Err(err).
Str("event", "SpaceCreated").
Msg("Could not render E-Mail template for spaces")
}
emailSubject := fmt.Sprintf("%s invited you to join %s", sharerUserResponse.GetUser().DisplayName, md.GetInfo().GetSpace().Name)
if e.GranteeUserID != nil {
err = s.channel.SendMessage([]string{e.GranteeUserID.OpaqueId}, msg, emailSubject, sharerDisplayName)
} else if e.GranteeGroupID != nil {
err = s.channel.SendMessageToGroup(e.GranteeGroupID, msg, emailSubject, sharerDisplayName)
}
if err != nil {
s.logger.Error().
Err(err).
Str("event", "SpaceCreated").
Msg("failed to send a message")
}
}
func (s eventsNotifier) handleShareCreated(e events.ShareCreated) {
sharerUserResponse, err := s.gwClient.GetUser(context.Background(), &userv1beta1.GetUserRequest{
UserId: e.Sharer,
})
if err != nil || sharerUserResponse.Status.Code != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("Could not get user response from gatway client")
return
}
granteeUserResponse, err := s.gwClient.GetUser(context.Background(), &userv1beta1.GetUserRequest{
UserId: e.GranteeUserID,
})
if err != nil || sharerUserResponse.Status.Code != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("Could not get user response from gatway client")
return
}
// Get auth context
ownerCtx := ctxpkg.ContextSetUser(context.Background(), sharerUserResponse.User)
authRes, err := s.gwClient.Authenticate(ownerCtx, &gateway.AuthenticateRequest{
Type: "machine",
ClientId: "userid:" + e.Sharer.OpaqueId,
ClientSecret: s.machineAuthAPIKey,
})
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("Could not impersonate sharer")
return
}
if authRes.GetStatus().GetCode() != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("could not get authenticated context for user")
return
}
ownerCtx = metadata.AppendToOutgoingContext(ownerCtx, ctxpkg.TokenHeader, authRes.Token)
// TODO: maybe cache this stat to reduce storage iops
md, err := s.gwClient.Stat(ownerCtx, &providerv1beta1.StatRequest{
Ref: &providerv1beta1.Reference{
ResourceId: e.ItemID,
},
FieldMask: &fieldmaskpb.FieldMask{Paths: []string{"name"}},
})
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Str("itemid", e.ItemID.OpaqueId).
Msg("could not stat resource")
return
}
if md.Status.Code != rpcv1beta1.Code_CODE_OK {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Str("itemid", e.ItemID.OpaqueId).
Str("rpc status", md.Status.Code.String()).
Msg("could not stat resource")
return
}
shareLink, err := urlJoinPath(e.Executant.Idp, "files/shares/with-me")
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("could not create link to the share")
return
}
sharerDisplayName := sharerUserResponse.GetUser().DisplayName
msg, err := email.RenderEmailTemplate("shareCreated.email.tmpl", map[string]string{
"ShareGrantee": granteeUserResponse.GetUser().DisplayName,
"ShareSharer": sharerDisplayName,
"ShareFolder": md.GetInfo().Name,
"ShareLink": shareLink,
}, s.emailTemplatePath)
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("Could not render E-Mail template for shares")
}
emailSubject := fmt.Sprintf("%s shared %s with you", sharerUserResponse.GetUser().DisplayName, md.GetInfo().Name)
if e.GranteeUserID != nil {
err = s.channel.SendMessage([]string{e.GranteeUserID.OpaqueId}, msg, emailSubject, sharerDisplayName)
} else if e.GranteeGroupID != nil {
err = s.channel.SendMessageToGroup(e.GranteeGroupID, msg, emailSubject, sharerDisplayName)
}
if err != nil {
s.logger.Error().
Err(err).
Str("event", "ShareCreated").
Msg("failed to send a message")
}
}
// TODO: this function is a backport for go1.19 url.JoinPath, upon go bump, replace this
func urlJoinPath(base string, elements ...string) (string, error) {
u, err := url.Parse(base)
if err != nil {
return "", err
}
u.Path = path.Join(append([]string{u.Path}, elements...)...)
return u.String(), nil
}

8
services/ocs/pkg/config/cachestore.go Normal file
View File

@@ -0,0 +1,8 @@
package config
// CacheStore defines the available configuration for the cache store
type CacheStore struct {
Type string `yaml:"type" env:"OCIS_CACHE_STORE_TYPE;OCS_CACHE_STORE_TYPE" desc:"The type of the cache store. Valid options are \"noop\", \"ocmem\", \"etcd\" and \"memory\""`
Address string `yaml:"address" env:"OCIS_CACHE_STORE_ADDRESS;OCS_CACHE_STORE_ADDRESS" desc:"a comma-separated list of addresses to connect to. Only for etcd"`
Size int `yaml:"size" env:"OCIS_CACHE_STORE_SIZE;OCS_CACHE_STORE_SIZE" desc:"Maximum size for the cache store. Only ocmem will use this option, in number of items per table. The rest will ignore the option and can grow indefinitely"`
}

7
services/ocs/pkg/config/config.go
View File

@@ -12,9 +12,10 @@ type Config struct {
Service Service `yaml:"-"`
Tracing *Tracing `yaml:"tracing"`
Log *Log `yaml:"log"`
Debug Debug `yaml:"debug"`
Tracing *Tracing `yaml:"tracing"`
Log *Log `yaml:"log"`
CacheStore *CacheStore `yaml:"cache_store"`
Debug Debug `yaml:"debug"`
HTTP HTTP `yaml:"http"`

10
services/ocs/pkg/config/defaults/defaultconfig.go
View File

@@ -69,6 +69,16 @@ func EnsureDefaults(cfg *config.Config) {
cfg.Tracing = &config.Tracing{}
}
if cfg.CacheStore == nil && cfg.Commons != nil && cfg.Commons.CacheStore != nil {
cfg.CacheStore = &config.CacheStore{
Type: cfg.Commons.CacheStore.Type,
Address: cfg.Commons.CacheStore.Address,
Size: cfg.Commons.CacheStore.Size,
}
} else if cfg.CacheStore == nil {
cfg.CacheStore = &config.CacheStore{}
}
if cfg.Reva == nil && cfg.Commons != nil && cfg.Commons.Reva != nil {
cfg.Reva = &config.Reva{
Address: cfg.Commons.Reva.Address,

10
services/ocs/pkg/service/v0/service.go
View File

@@ -2,10 +2,10 @@ package svc
import (
"net/http"
"time"
"github.com/cs3org/reva/v2/pkg/rgrpc/todo/pool"
"github.com/owncloud/ocis/v2/ocis-pkg/service/grpc"
"github.com/owncloud/ocis/v2/ocis-pkg/store"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
@@ -42,9 +42,13 @@ func NewService(opts ...Option) Service {
}
roleManager := options.RoleManager
if roleManager == nil {
storeOptions := store.OcisStoreOptions{
Type: options.Config.CacheStore.Type,
Address: options.Config.CacheStore.Address,
Size: options.Config.CacheStore.Size,
}
m := roles.NewManager(
roles.CacheSize(1024),
roles.CacheTTL(time.Hour*24*7),
roles.StoreOptions(storeOptions),
roles.Logger(options.Logger),
roles.RoleService(roleService),
)

12
services/proxy/pkg/middleware/oidc_auth.go
View File

@@ -32,9 +32,9 @@ type OIDCProvider interface {
// NewOIDCAuthenticator returns a ready to use authenticator which can handle OIDC authentication.
func NewOIDCAuthenticator(logger log.Logger, tokenCacheTTL int, oidcHTTPClient *http.Client, oidcIss string, providerFunc func() (OIDCProvider, error),
jwksOptions config.JWKS, accessTokenVerifyMethod string) OIDCAuthenticator {
jwksOptions config.JWKS, accessTokenVerifyMethod string) *OIDCAuthenticator {
tokenCache := osync.NewCache(tokenCacheTTL)
return OIDCAuthenticator{
return &OIDCAuthenticator{
Logger: logger,
tokenCache: &tokenCache,
TokenCacheTTL: time.Duration(tokenCacheTTL),
@@ -66,7 +66,7 @@ type OIDCAuthenticator struct {
JWKS *keyfunc.JWKS
}
func (m OIDCAuthenticator) getClaims(token string, req *http.Request) (map[string]interface{}, error) {
func (m *OIDCAuthenticator) getClaims(token string, req *http.Request) (map[string]interface{}, error) {
var claims map[string]interface{}
hit := m.tokenCache.Load(token)
if hit == nil {
@@ -168,7 +168,7 @@ type jwksJSON struct {
JWKSURL string `json:"jwks_uri"`
}
func (m OIDCAuthenticator) getKeyfunc() *keyfunc.JWKS {
func (m *OIDCAuthenticator) getKeyfunc() *keyfunc.JWKS {
m.jwksLock.Lock()
defer m.jwksLock.Unlock()
if m.JWKS == nil {
@@ -219,7 +219,7 @@ func (m OIDCAuthenticator) getKeyfunc() *keyfunc.JWKS {
return m.JWKS
}
func (m OIDCAuthenticator) getProvider() OIDCProvider {
func (m *OIDCAuthenticator) getProvider() OIDCProvider {
m.providerLock.Lock()
defer m.providerLock.Unlock()
if m.provider == nil {
@@ -240,7 +240,7 @@ func (m OIDCAuthenticator) getProvider() OIDCProvider {
}
// Authenticate implements the authenticator interface to authenticate requests via oidc auth.
func (m OIDCAuthenticator) Authenticate(r *http.Request) (*http.Request, bool) {
func (m *OIDCAuthenticator) Authenticate(r *http.Request) (*http.Request, bool) {
// there is no bearer token on the request,
if !m.shouldServe(r) || isPublicPath(r.URL.Path) {
// The authentication of public path requests is handled by another authenticator.

11
services/proxy/pkg/middleware/public_share_auth.go
View File

@@ -38,9 +38,18 @@ func isPublicShareArchive(r *http.Request) bool {
return false
}
// The app open requests can be made in public share contexts. For that the PublicShareAuthenticator needs to
// augment the request context.
// The app open requests can also be made in authenticated context. In these cases the PublicShareAuthenticator
// needs to ignore the request.
func isPublicShareAppOpen(r *http.Request) bool {
return strings.HasPrefix(r.URL.Path, "/app/open") &&
(r.URL.Query().Get(headerShareToken) != "" || r.Header.Get(headerShareToken) != "")
}
// Authenticate implements the authenticator interface to authenticate requests via public share auth.
func (a PublicShareAuthenticator) Authenticate(r *http.Request) (*http.Request, bool) {
if !isPublicPath(r.URL.Path) && !isPublicShareArchive(r) {
if !isPublicPath(r.URL.Path) && !isPublicShareArchive(r) && !isPublicShareAppOpen(r) {
return nil, false
}

13
services/thumbnails/pkg/thumbnail/thumbnail.go
View File

@@ -13,11 +13,14 @@ import (
var (
// SupportedMimeTypes contains a all mimetypes which are supported by the thumbnailer.
SupportedMimeTypes = map[string]struct{}{
"image/png": {},
"image/jpg": {},
"image/jpeg": {},
"image/gif": {},
"text/plain": {},
"image/png": {},
"image/jpg": {},
"image/jpeg": {},
"image/gif": {},
"image/bmp": {},
"image/x-ms-bmp": {},
"image/tiff": {},
"text/plain": {},
}
)

17
services/web/pkg/config/defaults/defaultconfig.go
View File

@@ -47,9 +47,22 @@ func DefaultConfig() *config.Config {
ResponseType: "code",
Scope: "openid profile email",
},
Apps: []string{"files", "search", "preview", "text-editor", "pdf-viewer", "external", "user-management"},
Apps: []string{"files", "search", "text-editor", "pdf-viewer", "external", "user-management"},
ExternalApps: []config.ExternalApp{
{
ID: "preview",
Path: "web-app-preview",
Config: map[string]interface{}{
"mimeTypes": []string{
"image/tiff",
"image/bmp",
"image/x-ms-bmp",
},
},
},
},
Options: map[string]interface{}{
"previewFileMimeTypes": []string{"image/gif", "image/png", "image/jpeg", "text/plain"},
"previewFileMimeTypes": []string{"image/gif", "image/png", "image/jpeg", "text/plain", "image/tiff", "image/bmp", "image/x-ms-bmp"},
},
},
},

67
tests/acceptance/expected-failures-API-on-OCIS-storage.md
View File

@@ -15,7 +15,7 @@ Basic file management like up and download, move, copy, properties, trash, versi
#### [Custom dav properties with namespaces are rendered incorrectly](https://github.com/owncloud/ocis/issues/2140)
_ocdav: double check the webdav property parsing when custom namespaces are used_
_ocdav: double-check the webdav property parsing when custom namespaces are used_
- [apiWebdavProperties1/setFileProperties.feature:37](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/setFileProperties.feature#L37)
- [apiWebdavProperties1/setFileProperties.feature:38](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/setFileProperties.feature#L38)
@@ -60,7 +60,6 @@ Synchronization features like etag propagation, setting mtime and locking files
#### [Webdav LOCK operations](https://github.com/owncloud/ocis/issues/1284)
- [apiWebdavLocks/exclusiveLocks.feature:123](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavLocks/exclusiveLocks.feature#L123)
- [apiWebdavLocks/exclusiveLocks.feature:124](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavLocks/exclusiveLocks.feature#L124)
- [apiWebdavLocks/exclusiveLocks.feature:125](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavLocks/exclusiveLocks.feature#L125)
@@ -173,6 +172,7 @@ File and sync features in a shared scenario
#### [accepting matching name shared resources from different users/groups sets no serial identifiers on the resource name for the receiver](https://github.com/owncloud/ocis/issues/4289)
- [apiShareManagementToShares/acceptShares.feature:366](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementToShares/acceptShares.feature#L366)
- [apiShareManagementToShares/acceptShares.feature:402](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementToShares/acceptShares.feature#L402)
- [apiShareManagementToShares/acceptShares.feature:304](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementToShares/acceptShares.feature#L304)
@@ -185,9 +185,9 @@ File and sync features in a shared scenario
- [apiShareCreateSpecialToShares2/createShareReceivedInMultipleWays.feature:203](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareCreateSpecialToShares2/createShareReceivedInMultipleWays.feature#L203)
- [apiShareCreateSpecialToShares2/createShareReceivedInMultipleWays.feature:45](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareCreateSpecialToShares2/createShareReceivedInMultipleWays.feature#L45)
- [apiShareCreateSpecialToShares2/createShareReceivedInMultipleWays.feature:46](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareCreateSpecialToShares2/createShareReceivedInMultipleWays.feature#L46)
- [apiShareCreateSpecialToShares1/createShareUniqueReceivedNames.feature:15](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareCreateSpecialToShares1/createShareUniqueReceivedNames.feature#L15)
#### [sharing the shares folder to users exits with different status code than in oc10 backend](https://github.com/owncloud/ocis/issues/2215)
- [apiShareManagementBasicToShares/createShareToSharesFolder.feature:727](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L727)
- [apiShareManagementBasicToShares/createShareToSharesFolder.feature:728](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L728)
- [apiShareManagementBasicToShares/createShareToSharesFolder.feature:746](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L746)
@@ -195,7 +195,7 @@ File and sync features in a shared scenario
- [apiShareManagementBasicToShares/createShareToSharesFolder.feature:762](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L762)
- [apiShareManagementBasicToShares/createShareToSharesFolder.feature:763](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L763)
#### [file_target of a auto-renamed file is not correct directly after sharing](https://github.com/owncloud/core/issues/32322)
#### [file_target of an auto-renamed file is not correct directly after sharing](https://github.com/owncloud/core/issues/32322)
- [apiShareManagementToShares/mergeShare.feature:105](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementToShares/mergeShare.feature#L105)
@@ -314,15 +314,10 @@ cannot share a folder with create permission
- [apiShareManagementBasicToShares/createShareToSharesFolder.feature:500](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L500)
- [apiVersions/fileVersionsSharingToShares.feature:220](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiVersions/fileVersionsSharingToShares.feature#L220)
- [apiVersions/fileVersionsSharingToShares.feature:221](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiVersions/fileVersionsSharingToShares.feature#L221)
Scenario Outline: Moving a file into a shared folder as the sharee and as the sharer
- [apiWebdavMove2/moveShareOnOcis.feature:30](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavMove2/moveShareOnOcis.feature#L30)
- [apiWebdavMove2/moveShareOnOcis.feature:32](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavMove2/moveShareOnOcis.feature#L32)
Scenario Outline: Moving a folder into a shared folder as the sharee and as the sharer
- [apiWebdavMove2/moveShareOnOcis.feature:98](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavMove2/moveShareOnOcis.feature#L98)
- [apiWebdavMove2/moveShareOnOcis.feature:100](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavMove2/moveShareOnOcis.feature#L100)
Scenario Outline: Moving a file to a shared folder with no permissions
- [apiWebdavMove2/moveShareOnOcis.feature:169](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavMove2/moveShareOnOcis.feature#L169)
- [apiWebdavMove2/moveShareOnOcis.feature:170](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavMove2/moveShareOnOcis.feature#L170)
@@ -457,7 +452,6 @@ Scenario Outline: Moving a file into a shared folder as the sharee and as the sh
- [apiShareUpdateToShares/updateShare.feature:242](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareUpdateToShares/updateShare.feature#L242)
- [apiShareUpdateToShares/updateShare.feature:196](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareUpdateToShares/updateShare.feature#L196)
moving outside of the Shares folder gives 501 Not Implemented.
- [apiShareManagementToShares/mergeShare.feature:124](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementToShares/mergeShare.feature#L124)
#### [Sharing folder and sub-folder with same user but different permission,the permission of sub-folder is not obeyed ](https://github.com/owncloud/ocis/issues/2440)
@@ -473,7 +467,7 @@ Scenario Outline: Moving a file into a shared folder as the sharee and as the sh
- [apiShareCreateSpecialToShares2/createShareWithDisabledUser.feature:23](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareCreateSpecialToShares2/createShareWithDisabledUser.feature#L23)
#### [Edit user share response has an "name" field](https://github.com/owncloud/ocis/issues/1225)
#### [Edit user share response has a "name" field](https://github.com/owncloud/ocis/issues/1225)
- [apiShareUpdateToShares/updateShare.feature:288](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareUpdateToShares/updateShare.feature#L288)
- [apiShareUpdateToShares/updateShare.feature:289](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareUpdateToShares/updateShare.feature#L289)
@@ -523,37 +517,37 @@ API, search, favorites, config, capabilities, not existing endpoints, CORS and o
#### [no robots.txt available](https://github.com/owncloud/ocis/issues/1314)
- [apiMain/main.feature:5](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/main.feature#L5) Scenario: robots.txt file should be accessible
- [apiMain/main.feature:5](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/main.feature#L5)
#### [Ability to return error messages in Webdav response bodies](https://github.com/owncloud/ocis/issues/1293)
- [apiAuthOcs/ocsDELETEAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsDELETEAuth.feature#L10) Scenario: send DELETE requests to OCS endpoints as admin with wrong password
- [apiAuthOcs/ocsGETAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L10) Scenario: using OCS anonymously
- [apiAuthOcs/ocsGETAuth.feature:51](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L51) Scenario: using OCS with non-admin basic auth
- [apiAuthOcs/ocsGETAuth.feature:84](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L84) Scenario: using OCS as normal user with wrong password
- [apiAuthOcs/ocsGETAuth.feature:115](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L115) Scenario:using OCS with admin basic auth
- [apiAuthOcs/ocsGETAuth.feature:133](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L133) Scenario: using OCS as admin user with wrong password
- [apiAuthOcs/ocsPOSTAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsPOSTAuth.feature#L10) Scenario: send POST requests to OCS endpoints as normal user with wrong password
- [apiAuthOcs/ocsPUTAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsPUTAuth.feature#L10) Scenario: send PUT request to OCS endpoints as admin with wrong password
- [apiAuthOcs/ocsDELETEAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsDELETEAuth.feature#L10)
- [apiAuthOcs/ocsGETAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L10)
- [apiAuthOcs/ocsGETAuth.feature:51](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L51)
- [apiAuthOcs/ocsGETAuth.feature:84](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L84)
- [apiAuthOcs/ocsGETAuth.feature:115](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L115)
- [apiAuthOcs/ocsGETAuth.feature:133](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L133)
- [apiAuthOcs/ocsPOSTAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsPOSTAuth.feature#L10)
- [apiAuthOcs/ocsPUTAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsPUTAuth.feature#L10)
#### [sending MKCOL requests to another user's webDav endpoints as normal user gives 404 instead of 403 ](https://github.com/owncloud/ocis/issues/3872)
_ocdav: api compatibility, return correct status code_
- [apiAuthWebDav/webDavMKCOLAuth.feature:54](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMKCOLAuth.feature#L54) Scenario: send MKCOL requests to another user's webDav endpoints as normal user
- [apiAuthWebDav/webDavMKCOLAuth.feature:68](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMKCOLAuth.feature#L68) Scenario: send MKCOL requests to another user's webDav endpoints as normal user using the spaces WebDAV API
- [apiAuthWebDav/webDavMKCOLAuth.feature:54](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMKCOLAuth.feature#L54)
- [apiAuthWebDav/webDavMKCOLAuth.feature:68](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMKCOLAuth.feature#L68)
#### [trying to lock file of another user gives http 200](https://github.com/owncloud/ocis/issues/2176)
- [apiAuthWebDav/webDavLOCKAuth.feature:58](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavLOCKAuth.feature#L58) Scenario: send LOCK requests to another user's webDav endpoints as normal user
- [apiAuthWebDav/webDavLOCKAuth.feature:70](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavLOCKAuth.feature#L70) Scenario: send LOCK requests to another user's webDav endpoints as normal user using the spaces WebDAV API
- [apiAuthWebDav/webDavLOCKAuth.feature:58](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavLOCKAuth.feature#L58)
- [apiAuthWebDav/webDavLOCKAuth.feature:70](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavLOCKAuth.feature#L70)
#### [send (MOVE,COPY) requests to another user's webDav endpoints as normal user gives 400 instead of 403](https://github.com/owncloud/ocis/issues/3882)
_ocdav: api compatibility, return correct status code_
- [apiAuthWebDav/webDavMOVEAuth.feature:57](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMOVEAuth.feature#L57) Scenario: send MOVE requests to another user's webDav endpoints as normal user
- [apiAuthWebDav/webDavMOVEAuth.feature:66](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMOVEAuth.feature#L66) Scenario: send MOVE requests to another user's webDav endpoints as normal user using the spaces WebDAV API
- [apiAuthWebDav/webDavMOVEAuth.feature:57](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMOVEAuth.feature#L57)
- [apiAuthWebDav/webDavMOVEAuth.feature:66](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavMOVEAuth.feature#L66)
- [apiAuthWebDav/webDavCOPYAuth.feature:59](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavCOPYAuth.feature#L59)
- [apiAuthWebDav/webDavCOPYAuth.feature:68](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavCOPYAuth.feature#L68)
@@ -561,8 +555,8 @@ _ocdav: api compatibility, return correct status code_
_ocdav: api compatibility, return correct status code_
- [apiAuthWebDav/webDavPOSTAuth.feature:58](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavPOSTAuth.feature#L58) Scenario: send POST requests to another user's webDav endpoints as normal user
- [apiAuthWebDav/webDavPOSTAuth.feature:67](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavPOSTAuth.feature#L67) Scenario: send POST requests to another user's webDav endpoints as normal user using the spaces WebDAV API
- [apiAuthWebDav/webDavPOSTAuth.feature:58](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavPOSTAuth.feature#L58)
- [apiAuthWebDav/webDavPOSTAuth.feature:67](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavPOSTAuth.feature#L67)
#### Another users space literally does not exist because it is not listed as a space for him, 404 seems correct, expects 403
@@ -592,7 +586,7 @@ _ocdav: api compatibility, return correct status code_
#### [Difference in response content of status.php and default capabilities](https://github.com/owncloud/ocis/issues/1286)
- [apiCapabilities/capabilitiesWithNormalUser.feature:11](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiCapabilities/capabilitiesWithNormalUser.feature#L11) Scenario: getting default capabilities with normal user
- [apiCapabilities/capabilitiesWithNormalUser.feature:11](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiCapabilities/capabilitiesWithNormalUser.feature#L11)
#### [/webdav and spaces endpoint does not allow REPORT requests](https://github.com/owncloud/ocis/issues/4034)
@@ -609,7 +603,7 @@ _ocdav: api compatibility, return correct status code_
- [apiWebdavOperations/search.feature:245](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/search.feature#L245)
- [apiWebdavOperations/search.feature:270](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/search.feature#L270)
### [No permisions propertry in response while searching for files and folders on ocis with new webdav](https://github.com/owncloud/ocis/issues/4009)
### [No permissions property in response while searching for files and folders on ocis with new webdav](https://github.com/owncloud/ocis/issues/4009)
- [apiWebdavOperations/search.feature:208](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/search.feature#L208)
- [apiWebdavOperations/search.feature:240](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/search.feature#L240)
@@ -830,9 +824,9 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers
- [apiWebdavProperties1/copyFile.feature:122](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L122)
- [apiWebdavProperties1/copyFile.feature:123](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L123)
- [apiWebdavProperties1/copyFile.feature:128](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L128)
- [apiWebdavProperties1/createFolder.feature:95](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L95)
- [apiWebdavProperties1/createFolder.feature:96](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L96)
- [apiWebdavProperties1/createFolder.feature:101](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L101)
- [apiWebdavProperties1/createFileFolder.feature:98](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFileFolder.feature#L98)
- [apiWebdavProperties1/createFileFolder.feature:99](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFileFolder.feature#L99)
- [apiWebdavProperties1/createFileFolder.feature:104](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFileFolder.feature#L104)
- [apiWebdavUpload1/uploadFile.feature:181](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFile.feature#L181)
- [apiWebdavUpload1/uploadFile.feature:182](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFile.feature#L182)
- [apiWebdavUpload1/uploadFile.feature:187](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFile.feature#L187)
@@ -894,7 +888,7 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers
- [apiWebdavUpload1/uploadFileToBlacklistedName.feature:72](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFileToBlacklistedName.feature#L72)
- [apiWebdavUpload1/uploadFileToBlacklistedName.feature:77](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFileToBlacklistedName.feature#L77)
#### [Allow public link sharing only for certain groups feature not implemented]
#### [Allow public link sharing only for certain groups feature not implemented](https://github.com/owncloud/ocis/issues/4623)
- [apiSharePublicLink3/allowGroupToCreatePublicLinks.feature:35](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink3/allowGroupToCreatePublicLinks.feature#L35)
- [apiSharePublicLink3/allowGroupToCreatePublicLinks.feature:91](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink3/allowGroupToCreatePublicLinks.feature#L91)
@@ -959,6 +953,7 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers
- [apiWebdavOperations/search.feature:314](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/search.feature#L314)
#### [Cannot disable the dav propfind depth infinity for resources](https://github.com/owncloud/ocis/issues/3720)
- [apiWebdavOperations/listFiles.feature:398](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/listFiles.feature#L398)
- [apiWebdavOperations/listFiles.feature:399](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/listFiles.feature#L399)
- [apiWebdavOperations/listFiles.feature:404](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/listFiles.feature#L404)
@@ -987,6 +982,7 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers
- [apiWebdavOperations/refuseAccess.feature:41](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/refuseAccess.feature#L41)
#### [OCS status code zero](https://github.com/owncloud/ocis/issues/3621)
- [apiShareManagementToShares/moveReceivedShare.feature:32](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementToShares/moveReceivedShare.feature#L32)
#### [HTTP status code differ while listing the contents of another user's trash bin](https://github.com/owncloud/ocis/issues/3561)
@@ -998,9 +994,11 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers
- [apiTrashbin/trashbinFilesFolders.feature:283](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiTrashbin/trashbinFilesFolders.feature#L283)
#### [HTTP status code differ while deleting file of another user's trash bin](https://github.com/owncloud/ocis/issues/3544)
- [apiTrashbin/trashbinDelete.feature:108](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiTrashbin/trashbinDelete.feature#L108)
#### [Problem accessing trashbin with personal space id](https://github.com/owncloud/ocis/issues/3639)
- [apiTrashbin/trashbinDelete.feature:35](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiTrashbin/trashbinDelete.feature#L35)
- [apiTrashbin/trashbinDelete.feature:36](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiTrashbin/trashbinDelete.feature#L36)
- [apiTrashbin/trashbinDelete.feature:58](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiTrashbin/trashbinDelete.feature#L58)
@@ -1031,6 +1029,7 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers
- [apiTrashbin/trashbinFilesFolders.feature:475](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiTrashbin/trashbinFilesFolders.feature#L475)
#### [valid WebDAV (DELETE, COPY or MOVE) requests with body must exit with 415](https://github.com/owncloud/ocis/issues/4332)
- [apiAuthWebDav/webDavDELETEAuth.feature:188](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavDELETEAuth.feature#L188)
- [apiAuthWebDav/webDavDELETEAuth.feature:199](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavDELETEAuth.feature#L199)
- [apiAuthWebDav/webDavCOPYAuth.feature:166](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavCOPYAuth.feature#L166)

15
tests/acceptance/expected-failures-localAPI-on-OCIS-storage.md
View File

@@ -4,18 +4,17 @@ The expected failures in this file are from features in the owncloud/ocis repo.
#### [downloading an archive with invalid path returns HTTP/500](https://github.com/owncloud/ocis/issues/2768)
- [apiArchiver/downloadByPath.feature:69](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L69)
#### [Hardcoded call to /home/..., but /home no longer exists](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature)
#### [Downloading the archive of the resource (files | folder) using resource path is not possible](https://github.com/owncloud/ocis/issues/4637)
- [apiArchiver/downloadByPath.feature:26](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L26)
- [apiArchiver/downloadByPath.feature:27](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L27)
- [apiArchiver/downloadByPath.feature:44](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L44)
- [apiArchiver/downloadByPath.feature:45](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L45)
- [apiArchiver/downloadByPath.feature:48](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L48)
- [apiArchiver/downloadByPath.feature:69](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L69)
- [apiArchiver/downloadByPath.feature:74](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L74)
- [apiArchiver/downloadByPath.feature:132](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L132)
- [apiArchiver/downloadByPath.feature:133](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadByPath.feature#L133)
### Tries to download /Shares/ folder but it cannot be downloaded any more directly
### [Downloaded /Shares tar contains resource (files|folder) with leading / in Response](https://github.com/owncloud/ocis/issues/4636)
- [apiArchiver/downloadById.feature:134](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadById.feature#L134)
- [apiArchiver/downloadById.feature:135](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiArchiver/downloadById.feature#L135)
@@ -28,12 +27,16 @@ The expected failures in this file are from features in the owncloud/ocis repo.
- [apiGraph/createGroupCaseSensitive.feature:21](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiGraph/createGroupCaseSensitive.feature#L21)
### [PROPFIND on accepted shares with identical names containing brackets exit with 404](https://github.com/owncloud/ocis/issues/4421)
- [apiSpaces/changingFilesShare.feature:12](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/changingFilesShare.feature#L12)
### [copy to overwrite (file and folder) from Personal to Shares Jail behaves differently](https://github.com/owncloud/ocis/issues/4393)
- [apiSpaces/copySpaces.feature:487](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/copySpaces.feature#L487)
- [apiSpaces/copySpaces.feature:501](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/copySpaces.feature#L501)
- [apiSpaces/copySpaces.feature:488](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/copySpaces.feature#L488)
- [apiSpaces/copySpaces.feature:502](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/copySpaces.feature#L502)
### [search doesn't find the project space by name](https://github.com/owncloud/ocis/issues/4506)
- [apiSpaces/search.feature:95](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/search.feature#L95)
#### [PATCH request for TUS upload with wrong checksum gives incorrect response](https://github.com/owncloud/ocis/issues/1755)
- [apiSpaces/shareUploadTUS.feature:204](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/shareUploadTUS.feature#L204)
- [apiSpaces/shareUploadTUS.feature:219](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/shareUploadTUS.feature#L219)
- [apiSpaces/shareUploadTUS.feature:284](https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/apiSpaces/shareUploadTUS.feature#L284)

9
tests/acceptance/expected-failures-webUI-on-OCIS-storage-ocisSmokeTest.md
View File

@@ -17,12 +17,3 @@ Only the web scenarios tagged ocisSmokeTest are run by default in OCIS CI. This
### [impossible to navigate into a folder in the trashbin](https://github.com/owncloud/web/issues/1725)
- [webUITrashbinDelete/trashbinDelete.feature:29](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUITrashbinDelete/trashbinDelete.feature#L29)
### [Creating folder with & character makes the UI act weird](https://github.com/owncloud/web/issues/7528)
- [webUIUpload/uploadEdgecases.feature:48](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L48)
- [webUIUpload/uploadEdgecases.feature:49](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L49)
- [webUIUpload/uploadEdgecases.feature:50](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L50)
- [webUIMoveFilesFolders/moveFiles.feature:20](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIMoveFilesFolders/moveFiles.feature#L20)
- [webUIDeleteFilesFolders/deleteFilesFolders.feature:10](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIDeleteFilesFolders/deleteFilesFolders.feature#L10)
- [webUIRenameFolders/renameFolders.feature:25](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRenameFolders/renameFolders.feature#L25)
- [webUIRenameFiles/renameFiles.feature:25](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRenameFiles/renameFiles.feature#L25)

52
tests/acceptance/expected-failures-webUI-on-OCIS-storage.md
View File

@@ -7,10 +7,10 @@ Please follow this format for the actual expected failures.
Level-3 headings should be used for the references to the relevant issues. Include the issue title with a link to the issue in GitHub.
Other free text and markdown formatting can be used elsewhere in the document if needed. But if you want to explain something about the issue, then please post that in the issue itself.
Other free text and Markdown formatting can be used elsewhere in the document if needed. But if you want to explain something about the issue, then please post that in the issue itself.
### [Exit page re-appears in loop when logged in user is deleted](https://github.com/owncloud/web/issues/4677)
### [Exit page re-appears in loop when logged-in user is deleted](https://github.com/owncloud/web/issues/4677)
- [webUILogin/openidLogin.feature:50](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUILogin/openidLogin.feature#L50)
### [Support for favorites](https://github.com/owncloud/ocis/issues/1228)
@@ -55,14 +55,6 @@ Other free text and markdown formatting can be used elsewhere in the document if
- [webUIRestrictSharing/restrictSharing.feature:31](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRestrictSharing/restrictSharing.feature#L31)
- [webUIRestrictSharing/restrictSharing.feature:40](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRestrictSharing/restrictSharing.feature#L40)
- [webUIRestrictSharing/restrictSharing.feature:56](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRestrictSharing/restrictSharing.feature#L56)
### [Cannot create users with special characters](https://github.com/owncloud/ocis/issues/1417)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:37](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L37)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:38](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L38)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:39](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L39)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:40](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L40)
### [No occ command in ocis](https://github.com/owncloud/ocis/issues/1317)
- [webUISharingInternalUsersBlacklisted/shareWithUsers.feature:16](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalUsersBlacklisted/shareWithUsers.feature#L16)
- [webUISharingInternalUsersBlacklisted/shareWithUsers.feature:34](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalUsersBlacklisted/shareWithUsers.feature#L34)
- [webUISharingInternalUsersBlacklisted/shareWithUsers.feature:52](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalUsersBlacklisted/shareWithUsers.feature#L52)
@@ -70,7 +62,13 @@ Other free text and markdown formatting can be used elsewhere in the document if
- [webUISharingInternalUsersBlacklisted/shareWithUsers.feature:82](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalUsersBlacklisted/shareWithUsers.feature#L82)
- [webUISharingInternalGroups/shareWithGroups.feature:202](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalGroups/shareWithGroups.feature#L202)
### webUI-Private-Links
### [Cannot create users with special characters](https://github.com/owncloud/ocis/issues/1417)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:37](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L37)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:38](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L38)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:39](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L39)
- [webUISharingAutocompletion/shareAutocompletionSpecialChars.feature:40](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAutocompletion/shareAutocompletionSpecialChars.feature#L40)
### [webUI-Private-Links](https://github.com/owncloud/web/issues/6844)
- [webUIPrivateLinks/accessingPrivateLinks.feature:9](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIPrivateLinks/accessingPrivateLinks.feature#L9)
- [webUIPrivateLinks/accessingPrivateLinks.feature:17](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIPrivateLinks/accessingPrivateLinks.feature#L17)
- [webUIPrivateLinks/accessingPrivateLinks.feature:25](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIPrivateLinks/accessingPrivateLinks.feature#L25)
@@ -78,7 +76,6 @@ Other free text and markdown formatting can be used elsewhere in the document if
### [Share additional info](https://github.com/owncloud/ocis/issues/1253)
- [webUISharingInternalUsersShareWithPage/shareWithUsers.feature:140](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalUsersShareWithPage/shareWithUsers.feature#L140)
### [No occ command in ocis](https://github.com/owncloud/ocis/issues/1317)
### [Expiration date set is not implemented in user share](https://github.com/owncloud/ocis/issues/1250)
- [webUISharingInternalGroups/shareWithGroups.feature:279](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalGroups/shareWithGroups.feature#L279)
@@ -229,10 +226,6 @@ Other free text and markdown formatting can be used elsewhere in the document if
- [webUIWebdavLockProtection/delete.feature:74](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/delete.feature#L74)
- [webUIWebdavLockProtection/move.feature:123](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/move.feature#L123)
- [webUIWebdavLockProtection/move.feature:124](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/move.feature#L124)
- [webUIWebdavLockProtection/move.feature:146](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/move.feature#L146)
- [webUIWebdavLockProtection/move.feature:147](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/move.feature#L147)
- [webUIWebdavLockProtection/upload.feature:90](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/upload.feature#L90)
- [webUIWebdavLockProtection/upload.feature:91](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/upload.feature#L91)
### [Writing to locked files/folders give only a generic error message](https://github.com/owncloud/web/issues/5741)
- [webUIWebdavLockProtection/upload.feature:90](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIWebdavLockProtection/upload.feature#L90)
@@ -248,7 +241,7 @@ Other free text and markdown formatting can be used elsewhere in the document if
- [webUIUpload/upload.feature:159](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/upload.feature#L159)
- [webUIUpload/uploadEdgecases.feature:69](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L69)
### [browsing directly to a details 'tab' is not possible](https://github.com/owncloud/web/issues/5464)
### [browsing directly to a details tab is not possible](https://github.com/owncloud/web/issues/5464)
- [webUIFiles/browseDirectlyToDetailsTab.feature:21](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIFiles/browseDirectlyToDetailsTab.feature#L21)
- [webUIFiles/browseDirectlyToDetailsTab.feature:22](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIFiles/browseDirectlyToDetailsTab.feature#L22)
- [webUIFiles/browseDirectlyToDetailsTab.feature:31](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIFiles/browseDirectlyToDetailsTab.feature#L31)
@@ -286,28 +279,3 @@ Other free text and markdown formatting can be used elsewhere in the document if
### [PROPFIND to sub-folder of a shared resources with same name gives 404](https://github.com/owncloud/ocis/issues/3859)
- [webUISharingAcceptShares/acceptShares.feature:245](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingAcceptShares/acceptShares.feature#L245)
### [Creating folder with & character makes the UI act weird](https://github.com/owncloud/web/issues/7528)
- [webUICreateFilesFolders/createFolderEdgeCases.feature:26](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUICreateFilesFolders/createFolderEdgeCases.feature#L26)
- [webUIDeleteFilesFolders/deleteFilesFolders.feature:170](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIDeleteFilesFolders/deleteFilesFolders.feature#L170)
- [webUICreateFilesFolders/createFolders.feature:51](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUICreateFilesFolders/createFolders.feature#L51)
- [webUIDeleteFilesFolders/deleteFilesFolders.feature:54](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIDeleteFilesFolders/deleteFilesFolders.feature#L54)
- [webUIFilesCopy/copy.feature:73](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIFilesCopy/copy.feature#L73)
- [webUIDeleteFilesFolders/deleteFilesFolders.feature:150](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIDeleteFilesFolders/deleteFilesFolders.feature#L150)
- [webUIDeleteFilesFolders/deleteFilesFolders.feature:10](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIDeleteFilesFolders/deleteFilesFolders.feature#L10)
- [webUIFilesCopy/copy.feature:28](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIFilesCopy/copy.feature#L28)
- [webUIRenameFiles/renameFiles.feature:38](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRenameFiles/renameFiles.feature#L38)
- [webUIRenameFiles/renameFiles.feature:25](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRenameFiles/renameFiles.feature#L25)
- [webUIRenameFiles/renameFiles.feature:42](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRenameFiles/renameFiles.feature#L42)
- [webUIRenameFolders/renameFolders.feature:25](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRenameFolders/renameFolders.feature#L25)
- [webUIRenameFolders/renameFolders.feature:42](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIRenameFolders/renameFolders.feature#L42)
- [webUIUpload/uploadEdgecases.feature:15](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L15)
- [webUIUpload/uploadEdgecases.feature:53](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L53)
- [webUIUpload/uploadEdgecases.feature:50](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L50)
- [webUIUpload/uploadEdgecases.feature:48](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L48)
- [webUIUpload/uploadEdgecases.feature:49](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L49)
- [webUIUpload/uploadEdgecases.feature:96](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIUpload/uploadEdgecases.feature#L96)
- [webUISharingPublicDifferentRoles/shareByPublicLinkDifferentRoles.feature:114](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingPublicDifferentRoles/shareByPublicLinkDifferentRoles.feature#L114)
- [webUISharingPublicDifferentRoles/shareByPublicLinkDifferentRoles.feature:93](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingPublicDifferentRoles/shareByPublicLinkDifferentRoles.feature#L93)
- [webUIMoveFilesFolders/moveFiles.feature:82](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIMoveFilesFolders/moveFiles.feature#L82)
- [webUIMoveFilesFolders/moveFiles.feature:20](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIMoveFilesFolders/moveFiles.feature#L20)
- [webUIMoveFilesFolders/moveFolders.feature:69](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIMoveFilesFolders/moveFolders.feature#L69)

1
tests/acceptance/features/apiSpaces/changeSpaces.feature
View File

@@ -16,6 +16,7 @@ Feature: Change data of space
And user "Alice" has created a space "Project Jupiter" of type "project" with quota "20"
And user "Alice" has shared a space "Project Jupiter" to user "Brian" with role "editor"
And user "Alice" has shared a space "Project Jupiter" to user "Bob" with role "viewer"
And using spaces DAV path
Scenario Outline: Only space admin user can change the name of a Space via the Graph API

13
tests/acceptance/features/apiSpaces/copySpaces.feature
View File

@@ -9,6 +9,7 @@ Feature: copy file
| username |
| Alice |
| Brian |
And using spaces DAV path
Scenario Outline: Copying a file within a same space project with role manager and editor
@@ -527,9 +528,9 @@ Feature: copy file
Then the HTTP status code should be "204"
And as "Alice" folder "/Sample-Folder-A/sample-folder-b/sample-folder-c" should exist
And for user "Alice" folder "BRIAN-FOLDER" of the space "Shares Jail" should contain these entries:
| /second-level-folder/third-level-folder/sample-folder-c |
| /second-level-folder/third-level-folder/sample-folder-c/ |
And for user "Brian" folder "BRIAN-FOLDER" of the space "Personal" should contain these files:
| /second-level-folder/third-level-folder/sample-folder-c |
| /second-level-folder/third-level-folder/sample-folder-c/ |
And the response when user "Alice" gets the info of the last share should include
| file_target | /Shares/BRIAN-FOLDER |
@@ -594,7 +595,7 @@ Feature: copy file
And for user "Alice" folder "BRIAN-FOLDER" of the space "Shares Jail" should contain these entries:
| /second-level-folder/third-level-file.txt/third-level-folder |
And for user "Alice" folder "BRIAN-FOLDER" of the space "Shares Jail" should not contain these entries:
| /second-level-folder/second-level-folder |
| /second-level-folder/second-level-folder/ |
And the response when user "Alice" gets the info of the last share should include
| file_target | /Shares/BRIAN-FOLDER |
@@ -616,7 +617,7 @@ Feature: copy file
Then the HTTP status code should be "204"
And as "Alice" folder "/Sample-Folder-A/sample-folder-b/sample-folder-c" should exist
And for user "Alice" folder "BRIAN-FOLDER" of the space "Shares Jail" should contain these files:
| /second-level-folder/third-level-folder/sample-folder-c |
| /second-level-folder/third-level-folder/sample-folder-c/ |
And the response when user "Alice" gets the info of the last share should include
| file_target | /Shares/BRIAN-FOLDER |
@@ -684,8 +685,8 @@ Feature: copy file
When user "Alice" copies folder "/FOLDER/second-level-folder" from space "Personal" to "/BRIAN-FOLDER/second-level-folder/third-level-file.txt" inside space "Shares Jail" using the WebDAV API
Then the HTTP status code should be "204"
And for user "Alice" folder "BRIAN-FOLDER" of the space "Shares Jail" should contain these files:
| /second-level-folder/third-level-file.txt |
| /second-level-folder/third-level-file.txt/third-level-folder |
| /second-level-folder/third-level-file.txt/ |
| /second-level-folder/third-level-file.txt/third-level-folder/ |
And as "Alice" folder "FOLDER/second-level-folder/third-level-folder" should exist
And for user "Alice" folder "BRIAN-FOLDER" of the space "Shares Jail" should not contain these files:
| /second-level-folder/second-level-folder |

22
tests/acceptance/features/apiSpaces/createFileFolderWhenSharesSpaceExist.feature
View File

@@ -18,12 +18,12 @@ Feature: create file or folder named similar to Shares folder
When user "Brian" creates folder "<folder_name>" using the WebDAV API
Then the HTTP status code should be "201"
And for user "Brian" the space "Personal" should contain these entries:
| /<folder_name> |
| <folder_name>/ |
Examples:
| folder_name |
| /Share |
| /shares |
| /Share1 |
| Share |
| shares |
| Share1 |
Scenario Outline: create a file with a name similar to Shares
Given using spaces DAV path
@@ -31,25 +31,25 @@ Feature: create file or folder named similar to Shares folder
Then the HTTP status code should be "201"
And the content of file "<file_name>" for user "Brian" should be "some text"
And for user "Brian" the space "Personal" should contain these entries:
| /<file_name> |
| <file_name> |
And for user "Brian" the space "Shares Jail" should contain these entries:
| /FOLDER |
| FOLDER/ |
Examples:
| file_name |
| /Share |
| /shares |
| /Share1 |
| Share |
| shares |
| Share1 |
Scenario: try to create a folder named Shares
Given using spaces DAV path
When user "Brian" creates folder "/Shares" using the WebDAV API
Then the HTTP status code should be "201"
And for user "Brian" the space "Shares Jail" should contain these entries:
| /FOLDER |
| FOLDER/ |
Scenario: try to create a file named Shares
Given using spaces DAV path
When user "Brian" uploads file with content "some text" to "/Shares" using the WebDAV API
Then the HTTP status code should be "201"
And for user "Brian" the space "Shares Jail" should contain these entries:
| /FOLDER |
| FOLDER/ |

1
tests/acceptance/features/apiSpaces/download.feature
View File

@@ -11,6 +11,7 @@ Feature: Download file in project space
| Alice |
| Brian |
| Bob |
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "download file" with the default quota using the GraphApi
And user "Alice" has uploaded a file inside space "download file" with content "some content" to "file.txt"

1
tests/acceptance/features/apiSpaces/editPublicLinkOfSpace.feature
View File

@@ -14,6 +14,7 @@ Feature: A manager of the space can edit public link
| username |
| Alice |
| Brian |
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "edit space" with the default quota using the GraphApi
And user "Alice" has created a public link share of the space "edit space" with settings:

1
tests/acceptance/features/apiSpaces/filePreviews.feature
View File

@@ -11,6 +11,7 @@ Feature: Preview file in project space
| Alice |
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "previews of the files" with the default quota using the GraphApi
And using spaces DAV path
Scenario Outline: An user can preview created txt files in the project space

1
tests/acceptance/features/apiSpaces/listSpaces.feature
View File

@@ -8,6 +8,7 @@ Feature: List and create spaces
Background:
Given user "Alice" has been created with default attributes and without skeleton files
And using spaces DAV path
Scenario: An ordinary user can request information about their Space via the Graph API

1
tests/acceptance/features/apiSpaces/moveSpaces.feature
View File

@@ -9,6 +9,7 @@ Feature: move (rename) file
| username |
| Alice |
| Brian |
And using spaces DAV path
Scenario Outline: Moving a file within same space project with role manager and editor
Given the administrator has given "Brian" the role "Space Admin" using the settings api

1
tests/acceptance/features/apiSpaces/quota.feature
View File

@@ -14,6 +14,7 @@ Feature: State of the quota
Background:
Given user "Alice" has been created with default attributes and without skeleton files
And the administrator has given "Alice" the role "Space Admin" using the settings api
And using spaces DAV path
Scenario Outline: Quota information is returned in the list of spaces returned via the Graph API

1
tests/acceptance/features/apiSpaces/removeSpaceObjects.feature
View File

@@ -13,6 +13,7 @@ Feature: Remove files, folder
| username |
| Alice |
| Brian |
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "delete objects" with the default quota using the GraphApi
And user "Alice" has created a folder "folderForDeleting/sub1/sub2" in space "delete objects"

1
tests/acceptance/features/apiSpaces/restoreSpaceObjects.feature
View File

@@ -14,6 +14,7 @@ Feature: Restore files, folder
| Brian |
| Bob |
| Carol |
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" creates a space "restore objects" of type "project" with the default quota using the GraphApi
And user "Alice" has created a folder "newFolder" in space "restore objects"

1
tests/acceptance/features/apiSpaces/restoreSpaces.feature
View File

@@ -16,6 +16,7 @@ Feature: Restoring space
| Bob |
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "restore a space" of type "project" with quota "10"
And using spaces DAV path
Scenario: An owner can restore a Space via the Graph API

10
tests/acceptance/features/apiSpaces/search.feature
View File

@@ -10,12 +10,12 @@ Feature: Search
| username |
| Alice |
| Brian |
And using new DAV path
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "find data" with the default quota using the GraphApi
And user "Alice" has created a folder "folder/SubFolder1/subFOLDER2" in space "find data"
And user "Alice" has uploaded a file inside space "find data" with content "some content" to "folder/SubFolder1/subFOLDER2/insideTheFolder.txt"
And using new DAV path
Scenario: Alice can find data from the project space
When user "Alice" searches for "fol" using the WebDAV API
@@ -51,7 +51,7 @@ Feature: Search
| /SubFolder1/subFOLDER2 |
| /SubFolder1/subFOLDER2/insideTheFolder.txt |
And for user "Brian" the search result should contain space "mountpoint/folder"
Scenario: User can find hidden file
Given user "Alice" has created a folder ".space" in space "find data"
@@ -72,7 +72,7 @@ Feature: Search
| /SubFolder1/subFOLDER2 |
| /SubFolder1/subFOLDER2/insideTheFolder.txt |
Scenario: User cannot find declined folder
Given user "Alice" shares the following entity "folder" inside of space "find data" with user "Brian" with role "viewer"
And user "Brian" has declined share "/folder" offered by user "Alice"
@@ -90,7 +90,7 @@ Feature: Search
When user "Alice" searches for "folder" using the WebDAV API
Then the HTTP status code should be "207"
And the search result should contain "0" entries
Scenario: User can find project space by name
When user "Alice" searches for "find data" using the WebDAV API

1
tests/acceptance/features/apiSpaces/shareSpaces.feature
View File

@@ -14,6 +14,7 @@ Feature: Share spaces
| Bob |
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "share space" with the default quota using the GraphApi
And using spaces DAV path
Scenario Outline:: A user can share a space to another user

1
tests/acceptance/features/apiSpaces/shareSpacesViaLink.feature
View File

@@ -11,6 +11,7 @@ Feature: Share spaces via link
| username |
| Alice |
| Brian |
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "share space" with the default quota using the GraphApi
And user "Alice" has uploaded a file inside space "share space" with content "some content" to "test.txt"

6
tests/acceptance/features/apiSpaces/shareSubItemOfSpace.feature
View File

@@ -13,13 +13,15 @@ Feature: Share a file or folder that is inside a space
| Alice |
| Brian |
| Bob |
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "share sub-item" with the default quota using the GraphApi
And user "Alice" has created a folder "folder" in space "share sub-item"
And user "Alice" has uploaded a file inside space "share sub-item" with content "some content" to "file.txt"
And using new DAV path
Scenario Outline: A manager of the space can share an entity inside project space to another user with role:
Scenario Outline: A manager of the space can share an entity inside project space to another user with role
When user "Alice" shares the following entity "<entity>" inside of space "share sub-item" with user "Brian" with role "<role>"
Then the HTTP status code should be "200"
And the OCS status code should be "200"

1
tests/acceptance/features/apiSpaces/shareSubItemOfSpaceViaPublicLink.feature
View File

@@ -23,6 +23,7 @@ Feature: Share a file or folder that is inside a space via public link
| username |
| Alice |
| Brian |
And using spaces DAV path
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "share sub-item" with the default quota using the GraphApi
And user "Alice" has created a folder "folder" in space "share sub-item"

299
tests/acceptance/features/apiSpaces/shareUploadTUS.feature Normal file
View File

@@ -0,0 +1,299 @@
@api @skipOnOcV10
Feature: upload resources on share using TUS protocol
As a user
I want to be able to upload files
So that I can store and share files between multiple client systems
Background:
Given using spaces DAV path
And these users have been created with default attributes and without skeleton files:
| username |
| Alice |
| Brian |
Scenario: upload file with mtime to a received share
Given user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file "filesForUpload/textfile.txt" to "toShare/file.txt" with mtime "Thu, 08 Aug 2012 04:18:13 GMT" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Brian" folder "toShare" of the space "Shares Jail" should contain these entries:
| file.txt |
And as "Brian" the mtime of the file "/toShare/file.txt" in space "Shares Jail" should be "Thu, 08 Aug 2012 04:18:13 GMT"
And as "Alice" the mtime of the file "/toShare/file.txt" in space "Personal" should be "Thu, 08 Aug 2012 04:18:13 GMT"
Scenario: upload file with mtime to a sent share
Given user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Alice" uploads a file "filesForUpload/textfile.txt" to "toShare/file.txt" with mtime "Thu, 08 Aug 2012 04:18:13 GMT" via TUS inside of the space "Personal" using the WebDAV API
Then for user "Alice" folder "toShare" of the space "Personal" should contain these entries:
| file.txt |
And as "Alice" the mtime of the file "/toShare/file.txt" in space "Personal" should be "Thu, 08 Aug 2012 04:18:13 GMT"
And as "Brian" the mtime of the file "/toShare/file.txt" in space "Shares Jail" should be "Thu, 08 Aug 2012 04:18:13 GMT"
Scenario: overwriting a file with mtime in a received share
Given user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
And user "Alice" has uploaded file with content "uploaded content" to "/toShare/file.txt"
When user "Brian" uploads a file "filesForUpload/textfile.txt" to "toShare/file.txt" with mtime "Thu, 08 Aug 2012 04:18:13 GMT" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Brian" folder "toShare" of the space "Shares Jail" should contain these entries:
| file.txt |
And as "Brian" the mtime of the file "/toShare/file.txt" in space "Shares Jail" should be "Thu, 08 Aug 2012 04:18:13 GMT"
And as "Alice" the mtime of the file "/toShare/file.txt" in space "Personal" should be "Thu, 08 Aug 2012 04:18:13 GMT"
Scenario: overwriting a file with mtime in a sent share
Given user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
And user "Brian" has uploaded a file inside space "Shares Jail" with content "uploaded content" to "toShare/file.txt"
When user "Alice" uploads a file "filesForUpload/textfile.txt" to "toShare/file.txt" with mtime "Thu, 08 Aug 2012 04:18:13 GMT" via TUS inside of the space "Personal" using the WebDAV API
Then for user "Alice" folder "toShare" of the space "Personal" should contain these entries:
| file.txt |
And as "Alice" the mtime of the file "/toShare/file.txt" in space "Personal" should be "Thu, 08 Aug 2012 04:18:13 GMT"
And as "Brian" the mtime of the file "/toShare/file.txt" in space "Shares Jail" should be "Thu, 08 Aug 2012 04:18:13 GMT"
Scenario: attempt to upload a file into a nonexistent folder within correctly received share
Given using OCS API version "1"
And user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file with content "uploaded content" to "/toShare/nonExistentFolder/file.txt" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Brian" folder "toShare" of the space "Shares Jail" should not contain these entries:
| nonExistentFolder/file.txt |
Scenario: attempt to upload a file into a nonexistent folder within correctly received read only share
Given using OCS API version "1"
And user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian" with permissions "read"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file with content "uploaded content" to "/toShare/nonExistentFolder/file.txt" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Brian" folder "toShare" of the space "Shares Jail" should not contain these entries:
| nonExistentFolder/file.txt |
Scenario: Uploading a file to a received share folder
Given user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file with content "uploaded content" to "/toShare/file.txt" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Alice" folder "toShare" of the space "Personal" should contain these entries:
| file.txt |
And for user "Alice" the content of the file "toShare/file.txt" of the space "Personal" should be "uploaded content"
Scenario: Uploading a file to a user read/write share folder
Given user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian" with permissions "change"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file with content "uploaded content" to "/toShare/file.txt" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Alice" folder "toShare" of the space "Personal" should contain these entries:
| file.txt |
And for user "Alice" the content of the file "toShare/file.txt" of the space "Personal" should be "uploaded content"
Scenario: Uploading a file into a group share as a share receiver
Given group "grp1" has been created
And user "Brian" has been added to group "grp1"
And user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "toShare" with group "grp1" with permissions "change"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file with content "uploaded content" to "/toShare/file.txt" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Alice" folder "toShare" of the space "Personal" should contain these entries:
| file.txt |
And for user "Alice" the content of the file "toShare/file.txt" of the space "Personal" should be "uploaded content"
Scenario: Overwrite file to a received share folder
Given user "Alice" has created folder "/toShare"
And user "Alice" has uploaded file with content "original content" to "/toShare/file.txt"
And user "Alice" has shared folder "/toShare" with user "Brian"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file with content "overwritten content" to "/toShare/file.txt" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Alice" folder "toShare" of the space "Personal" should contain these entries:
| file.txt |
And for user "Alice" the content of the file "toShare/file.txt" of the space "Personal" should be "overwritten content"
Scenario: attempt to upload a file into a folder within correctly received read only share
Given user "Alice" has created folder "/toShare"
And user "Alice" has shared folder "/toShare" with user "Brian" with permissions "read"
And user "Brian" has accepted share "/toShare" offered by user "Alice"
When user "Brian" uploads a file with content "uploaded content" to "/toShare/file.txt" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Brian" folder "toShare" of the space "Shares Jail" should not contain these entries:
| file.txt |
Scenario: Upload a file to shared folder with checksum should return the checksum in the propfind for sharee
Given user "Alice" has created folder "/FOLDER"
And user "Alice" has shared folder "/FOLDER" with user "Brian"
And user "Brian" has accepted share "/FOLDER" offered by user "Alice"
And user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 5 |
# L0ZPTERFUi90ZXh0RmlsZS50eHQ= is the base64 encode of /FOLDER/textFile.txt
| Upload-Metadata | filename L0ZPTERFUi90ZXh0RmlsZS50eHQ= |
| Tus-Resumable | 1.0.0 |
And user "Alice" has uploaded file with checksum "SHA1 8cb2237d0679ca88db6464eac60da96345513964" to the last created TUS Location with offset "0" and content "12345" via TUS inside of the space "Personal" using the WebDAV API
When user "Brian" requests the checksum of file "/FOLDER/textFile.txt" in space "Shares Jail" via propfind using the WebDAV API
Then the HTTP status code should be "207"
And the webdav checksum should match "SHA1:8cb2237d0679ca88db6464eac60da96345513964 MD5:827ccb0eea8a706c4c34a16891f84e7b ADLER32:02f80100"
Scenario: Upload a file to shared folder with checksum should return the checksum in the download header for sharee
Given user "Alice" has created folder "/FOLDER"
And user "Alice" has shared folder "/FOLDER" with user "Brian"
And user "Brian" has accepted share "/FOLDER" offered by user "Alice"
And user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 5 |
# L0ZPTERFUi90ZXh0RmlsZS50eHQ= is the base64 encode of /FOLDER/textFile.txt
| Upload-Metadata | filename L0ZPTERFUi90ZXh0RmlsZS50eHQ= |
| Tus-Resumable | 1.0.0 |
And user "Alice" has uploaded file with checksum "SHA1 8cb2237d069ca88db6464eac60da96345513964" to the last created TUS Location with offset "0" and content "12345" via TUS inside of the space "Personal" using the WebDAV API
When user "Brian" downloads the file "/FOLDER/textFile.txt" of the space "Shares Jail" using the WebDAV API
Then the header checksum should match "SHA1:8cb2237d0679ca88db6464eac60da96345513964"
Scenario: Sharer shares a file with correct checksum should return the checksum in the propfind for sharee
Given user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 5 |
# dGV4dEZpbGUudHh0 is the base64 encode of textFile.txt
| Upload-Metadata | filename dGV4dEZpbGUudHh0 |
| Tus-Resumable | 1.0.0 |
And user "Alice" has uploaded file with checksum "SHA1 8cb2237d0679ca88db6464eac60da96345513964" to the last created TUS Location with offset "0" and content "12345" via TUS inside of the space "Personal" using the WebDAV API
And user "Alice" has shared file "/textFile.txt" with user "Brian"
And user "Brian" has accepted share "/textFile.txt" offered by user "Alice"
When user "Brian" requests the checksum of file "/textFile.txt" in space "Shares Jail" via propfind using the WebDAV API
Then the HTTP status code should be "207"
And the webdav checksum should match "SHA1:8cb2237d0679ca88db6464eac60da96345513964 MD5:827ccb0eea8a706c4c34a16891f84e7b ADLER32:02f80100"
Scenario: Sharer shares a file with correct checksum should return the checksum in the download header for sharee
Given user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 5 |
# dGV4dEZpbGUudHh0 is the base64 encode of textFile.txt
| Upload-Metadata | filename dGV4dEZpbGUudHh0 |
| Tus-Resumable | 1.0.0 |
And user "Alice" has uploaded file with checksum "SHA1 8cb2237d0679ca88db6464eac60da96345513964" to the last created TUS Location with offset "0" and content "12345" via TUS inside of the space "Personal" using the WebDAV API
And user "Alice" has shared file "/textFile.txt" with user "Brian"
And user "Brian" has accepted share "/textFile.txt" offered by user "Alice"
When user "Brian" downloads the file "/textFile.txt" of the space "Shares Jail" using the WebDAV API
Then the header checksum should match "SHA1:8cb2237d0679ca88db6464eac60da96345513964"
Scenario: Sharee uploads a file to a received share folder with correct checksum
Given user "Alice" has created folder "/FOLDER"
And user "Alice" has shared folder "/FOLDER" with user "Brian"
And user "Brian" has accepted share "/FOLDER" offered by user "Alice"
When user "Brian" creates a new TUS resource for the space "Shares Jail" using the WebDAV API with these headers:
| Upload-Length | 16 |
# L0ZPTERFUi90ZXh0RmlsZS50eHQ= is the base64 encode of /FOLDER/textFile.txt
| Upload-Metadata | filename L0ZPTERFUi90ZXh0RmlsZS50eHQ= |
| Tus-Resumable | 1.0.0 |
And user "Brian" uploads file with checksum "MD5 827ccb0eea8a706c4c34a16891f84e7b" to the last created TUS Location with offset "0" and content "uploaded content" via TUS inside of the space "Shares Jail" using the WebDAV API
Then for user "Alice" folder "FOLDER" of the space "Personal" should contain these entries:
| textFile.txt |
And for user "Alice" the content of the file "FOLDER/textFile.txt" of the space "Personal" should be "uploaded content"
Scenario: Sharee uploads a file to a received share folder with wrong checksum should not work
Given user "Alice" has created folder "/FOLDER"
And user "Alice" has shared folder "/FOLDER" with user "Brian"
And user "Brian" has accepted share "/FOLDER" offered by user "Alice"
When user "Brian" creates a new TUS resource for the space "Shares Jail" using the WebDAV API with these headers:
| Upload-Length | 16 |
# L0ZPTERFUi90ZXh0RmlsZS50eHQ= is the base64 encode of /FOLDER/textFile.txt
| Upload-Metadata | filename L0ZPTERFUi90ZXh0RmlsZS50eHQ= |
| Tus-Resumable | 1.0.0 |
And user "Brian" uploads file with checksum "MD5 827ccb0eea8a706c4c34a16891f84e8c" to the last created TUS Location with offset "0" and content "uploaded content" via TUS inside of the space "Shares Jail" using the WebDAV API
Then the HTTP status code should be "406"
And for user "Alice" folder "FOLDER" of the space "Personal" should not contain these entries:
| textFile.txt |
Scenario: Sharer uploads a file to shared folder with wrong checksum should not work
Given user "Alice" has created folder "/FOLDER"
And user "Alice" has shared folder "/FOLDER" with user "Brian"
And user "Brian" has accepted share "/FOLDER" offered by user "Alice"
And user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 5 |
# L0ZPTERFUi90ZXh0RmlsZS50eHQ= is the base64 encode of /FOLDER/textFile.txt
| Upload-Metadata | filename L0ZPTERFUi90ZXh0RmlsZS50eHQ= |
| Tus-Resumable | 1.0.0 |
When user "Alice" uploads file with checksum "SHA1 8cb2237d0679ca88db6464eac60da96345513954" to the last created TUS Location with offset "0" and content "uploaded content" via TUS inside of the space "Personal" using the WebDAV API
Then the HTTP status code should be "406"
And for user "Alice" folder "FOLDER" of the space "Personal" should not contain these entries:
| textFile.txt |
And for user "Brian" folder "FOLDER" of the space "Shares Jail" should not contain these entries:
| textFile.txt |
Scenario: Sharer uploads a chunked file with correct checksum and share it with sharee should work
Given user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 10 |
# dGV4dEZpbGUudHh0 is the base64 encode of textFile.txt
| Upload-Metadata | filename dGV4dEZpbGUudHh0 |
| Tus-Resumable | 1.0.0 |
When user "Alice" sends a chunk to the last created TUS Location with offset "0" and data "01234" with checksum "MD5 4100c4d44da9177247e44a5fc1546778" via TUS inside of the space "Personal" using the WebDAV API
And user "Alice" sends a chunk to the last created TUS Location with offset "5" and data "56789" with checksum "MD5 099ebea48ea9666a7da2177267983138" via TUS inside of the space "Personal" using the WebDAV API
And user "Alice" shares file "textFile.txt" with user "Brian" using the sharing API
And user "Brian" accepts share "/textFile.txt" offered by user "Alice" using the sharing API
Then for user "Brian" the content of the file "/textFile.txt" of the space "Shares Jail" should be "0123456789"
Scenario: Sharee uploads a chunked file with correct checksum to a received share folder should work
Given user "Alice" has created folder "/FOLDER"
And user "Alice" has shared folder "/FOLDER" with user "Brian"
And user "Brian" has accepted share "/FOLDER" offered by user "Alice"
And user "Brian" has created a new TUS resource for the space "Shares Jail" using the WebDAV API with these headers:
| Upload-Length | 10 |
# L0ZPTERFUi90ZXh0RmlsZS50eHQ= is the base64 encode of /FOLDER/textFile.txt
| Upload-Metadata | filename L0ZPTERFUi90ZXh0RmlsZS50eHQ= |
| Tus-Resumable | 1.0.0 |
When user "Brian" sends a chunk to the last created TUS Location with offset "0" and data "01234" with checksum "MD5 4100c4d44da9177247e44a5fc1546778" via TUS inside of the space "Shares Jail" using the WebDAV API
And user "Brian" sends a chunk to the last created TUS Location with offset "5" and data "56789" with checksum "MD5 099ebea48ea9666a7da2177267983138" via TUS inside of the space "Shares Jail" using the WebDAV API
Then the HTTP status code should be "204"
And for user "Alice" folder "FOLDER" of the space "Personal" should contain these entries:
| textFile.txt |
And for user "Alice" the content of the file "/FOLDER/textFile.txt" of the space "Personal" should be "0123456789"
Scenario: Sharer uploads a file with checksum and as a sharee overwrites the shared file with new data and correct checksum
Given user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 16 |
# dGV4dEZpbGUudHh0 is the base64 encode of textFile.txt
| Upload-Metadata | filename dGV4dEZpbGUudHh0 |
| Tus-Resumable | 1.0.0 |
And user "Alice" has uploaded file with checksum "SHA1 c1dab0c0864b6ac9bdd3743a1408d679f1acd823" to the last created TUS Location with offset "0" and content "original content" via TUS inside of the space "Personal" using the WebDAV API
And user "Alice" has shared file "/textFile.txt" with user "Brian"
And user "Brian" has accepted share "/textFile.txt" offered by user "Alice"
When user "Brian" overwrites recently shared file with offset "0" and data "overwritten content" with checksum "SHA1 fe990d2686a0fc86004efc31f5bf2475a45d4905" via TUS inside of the space "Shares Jail" using the WebDAV API with these headers:
| Upload-Length | 19 |
# dGV4dEZpbGUudHh0 is the base64 encode of textFile.txt
| Upload-Metadata | filename dGV4dEZpbGUudHh0 |
| Tus-Resumable | 1.0.0 |
Then the HTTP status code should be "204"
And for user "Alice" the content of the file "/textFile.txt" of the space "Personal" should be "overwritten content"
Scenario: Sharer uploads a file with checksum and as a sharee overwrites the shared file with new data and invalid checksum
Given user "Alice" has created a new TUS resource for the space "Personal" using the WebDAV API with these headers:
| Upload-Length | 16 |
# dGV4dEZpbGUudHh0 is the base64 encode of textFile.txt
| Upload-Metadata | filename dGV4dEZpbGUudHh0 |
| Tus-Resumable | 1.0.0 |
And user "Alice" has uploaded file with checksum "SHA1 c1dab0c0864b6ac9bdd3743a1408d679f1acd823" to the last created TUS Location with offset "0" and content "original content" via TUS inside of the space "Personal" using the WebDAV API
And user "Alice" has shared file "/textFile.txt" with user "Brian"
And user "Brian" has accepted share "/textFile.txt" offered by user "Alice"
When user "Brian" overwrites recently shared file with offset "0" and data "overwritten content" with checksum "SHA1 fe990d2686a0fc86004efc31f5bf2475a45d4906" via TUS inside of the space "Shares Jail" using the WebDAV API with these headers:
| Upload-Length | 19 |
# dGV4dEZpbGUudHh0 is the base64 encode of textFile.txt
| Upload-Metadata | filename dGV4dEZpbGUudHh0 |
| Tus-Resumable | 1.0.0 |
Then the HTTP status code should be "406"
And for user "Alice" the content of the file "/textFile.txt" of the space "Personal" should be "original content"

3
tests/acceptance/features/apiSpaces/tusUpload.feature
View File

@@ -15,8 +15,7 @@ Feature: upload resources using TUS protocol
Scenario: upload a file within the set quota to a project space
Given user "Alice" has created a space "Project Jupiter" of type "project" with quota "10000"
When user "Alice" uploads a file with content "uploaded content" to "/upload.txt" via TUS inside of the space "Project Jupiter" using the WebDAV API
Then the HTTP status code should be "200"
And for user "Alice" the space "Project Jupiter" should contain these entries:
Then for user "Alice" the space "Project Jupiter" should contain these entries:
| upload.txt |

1
tests/acceptance/features/apiSpaces/uploadSpaces.feature
View File

@@ -14,6 +14,7 @@ Feature: Upload files into a space
| Bob |
And the administrator has given "Alice" the role "Space Admin" using the settings api
And user "Alice" has created a space "Project Ceres" of type "project" with quota "2000"
And using spaces DAV path
Scenario Outline: An user creates a folder in the Space via the Graph API

385
tests/acceptance/features/bootstrap/SpacesContext.php
View File

@@ -69,7 +69,10 @@ class SpacesContext implements Context {
* @var ChecksumContext
*/
private ChecksumContext $checksumContext;
/**
* @var FilesVersionsContext
*/
private FilesVersionsContext $filesVersionsContext;
/**
* @var string
*/
@@ -107,18 +110,6 @@ class SpacesContext implements Context {
*/
private $storedEtags = [];
/**
*
* @var string[][]
*/
private $tokenOfLastLock = [];
private $etagPropfindBody = '<?xml version="1.0"?>'
. '<d:propfind xmlns:d="DAV:" '
. 'xmlns:oc="http://owncloud.org/ns" '
. 'xmlns:ocs="http://open-collaboration-services.org/ns">'
. '<d:prop><d:getetag/></d:prop></d:propfind>';
/**
* @param string $spaceName
*
@@ -192,22 +183,6 @@ class SpacesContext implements Context {
*/
private string $responseSpaceId;
/**
* @param string $responseSpaceId
*
* @return void
*/
public function setResponseSpaceId(string $responseSpaceId): void {
$this->responseSpaceId = $responseSpaceId;
}
/**
* @return string
*/
public function getResponseSpaceId(): string {
return $this->responseSpaceId;
}
/**
* Get SpaceId by Name
*
@@ -285,6 +260,8 @@ class SpacesContext implements Context {
*/
public function setSpaceIDByName(string $user, string $spaceName): void {
$space = $this->getSpaceByName($user, $spaceName);
Assert::assertIsArray($space, "Space with name $spaceName not found");
Assert::assertNotEmpty($space["root"]["webDavUrl"], "WebDavUrl for space with name $spaceName not found");
WebDavHelper::$SPACE_ID_FROM_OCIS = $space['id'];
}
@@ -340,29 +317,6 @@ class SpacesContext implements Context {
return $this->featureContext->getResponse();
}
/**
* Download a file using user password
*
* @param string $fullUrl
* @param string $user
*
* @return void
*/
public function downloadFileAsUserUsingPassword(
string $fullUrl,
string $user
):void {
$this->featureContext->setResponse(
HttpRequestHelper::sendRequest(
$fullUrl,
$this->featureContext->getStepLineRef(),
'GET',
$user,
$this->featureContext->getPasswordForUser($user),
)
);
}
/**
* The method returns fileId
*
@@ -456,6 +410,7 @@ class SpacesContext implements Context {
$this->webDavPropertiesContext = $environment->getContext('WebDavPropertiesContext');
$this->favoritesContext = $environment->getContext('FavoritesContext');
$this->checksumContext = $environment->getContext('ChecksumContext');
$this->filesVersionsContext = $environment->getContext('FilesVersionsContext');
// Run the BeforeScenario function in OCSContext to set it up correctly
$this->ocsContext->before($scope);
$this->baseUrl = \trim($this->featureContext->getBaseUrl(), "/");
@@ -542,31 +497,6 @@ class SpacesContext implements Context {
return HttpRequestHelper::sendRequest($fullUrl, $xRequestId, 'PROPFIND', $user, $password, $headers, $body);
}
/**
* Send Put Request to Url
*
* @param string $fullUrl
* @param string $user
* @param string $password
* @param string $xRequestId
* @param array $headers
* @param string $content
*
* @return ResponseInterface
*
* @throws GuzzleException
*/
public function sendPutRequestToUrl(
string $fullUrl,
string $user,
string $password,
string $xRequestId = '',
array $headers = [],
string $content = ""
): ResponseInterface {
return HttpRequestHelper::sendRequest($fullUrl, $xRequestId, 'PUT', $user, $password, $headers, $content);
}
/**
* Send POST Request to url
*
@@ -856,24 +786,20 @@ class SpacesContext implements Context {
string $spaceName,
string $foldersPath = ''
): void {
$space = $this->getSpaceByName($user, $spaceName);
Assert::assertIsArray($space);
Assert::assertNotEmpty($spaceId = $space["id"]);
Assert::assertNotEmpty($spaceWebDavUrl = $space["root"]["webDavUrl"]);
$headers['Depth'] = 'infinity';
$this->setSpaceIDByName($user, $spaceName);
$this->featureContext->setResponse(
$this->sendPropfindRequestToUrl(
$spaceWebDavUrl . '/' . $foldersPath,
WebDavHelper::propfind(
$this->featureContext->getBaseUrl(),
$user,
$this->featureContext->getPasswordForUser($user),
$foldersPath,
[],
'',
$headers
'infinity',
'files',
WebDavHelper::DAV_VERSION_SPACES
)
);
$this->setResponseSpaceId($spaceId);
$this->setResponseXml(
HttpRequestHelper::parseResponseAsXml($this->featureContext->getResponse())
);
}
/**
@@ -890,10 +816,11 @@ class SpacesContext implements Context {
string $shouldOrNot,
TableNode $expectedFiles
): void {
$this->propfindResultShouldContainEntries(
$this->featureContext->propfindResultShouldContainEntries(
$shouldOrNot,
$expectedFiles,
);
WebDavHelper::$SPACE_ID_FROM_OCIS = '';
}
/**
@@ -912,14 +839,18 @@ class SpacesContext implements Context {
string $shouldOrNot,
TableNode $expectedFiles
): void {
$spaceCreator = $this->getSpaceCreator($spaceName);
$space = $this->getSpaceByName($spaceCreator, $spaceName);
$this->theUserListsTheContentOfAPersonalSpaceRootUsingTheWebDAvApi(
$this->getSpaceCreator($spaceName),
$spaceCreator,
$spaceName
);
$this->propfindResultShouldContainEntries(
WebDavHelper::$SPACE_ID_FROM_OCIS = $space['id'];
$this->featureContext->propfindResultShouldContainEntries(
$shouldOrNot,
$expectedFiles,
);
WebDavHelper::$SPACE_ID_FROM_OCIS = '';
}
/**
@@ -940,14 +871,14 @@ class SpacesContext implements Context {
string $shouldOrNot,
TableNode $expectedFiles
): void {
$space = $this->getSpaceByName($user, $spaceName);
$this->theUserListsTheContentOfAPersonalSpaceRootUsingTheWebDAvApi(
$user,
$spaceName
);
$this->propfindResultShouldContainEntries(
$shouldOrNot,
$expectedFiles
);
WebDavHelper::$SPACE_ID_FROM_OCIS = $space['id'];
$this->featureContext->propfindResultShouldContainEntries($shouldOrNot, $expectedFiles, $user, 'PROPFIND');
WebDavHelper::$SPACE_ID_FROM_OCIS = '';
}
/**
@@ -970,16 +901,21 @@ class SpacesContext implements Context {
string $shouldOrNot,
TableNode $expectedFiles
): void {
$space = $this->getSpaceByName($user, $spaceName);
$this->theUserListsTheContentOfAPersonalSpaceRootUsingTheWebDAvApi(
$user,
$spaceName,
$folderPath
);
$this->propfindResultShouldContainEntries(
WebDavHelper::$SPACE_ID_FROM_OCIS = $space['id'];
$this->featureContext->propfindResultShouldContainEntries(
$shouldOrNot,
$expectedFiles,
$this->featureContext->getActualUsername($user),
'PROPFIND',
$folderPath
);
WebDavHelper::$SPACE_ID_FROM_OCIS = '';
}
/**
@@ -1214,43 +1150,6 @@ class SpacesContext implements Context {
}
}
/**
* @param string $shouldOrNot (not|)
* @param TableNode $expectedFiles
* @param string $folderPath
*
* @return void
*
* @throws Exception
*/
public function propfindResultShouldContainEntries(
string $shouldOrNot,
TableNode $expectedFiles,
string $folderPath = ''
): void {
$this->verifyTableNodeColumnsCount($expectedFiles, 1);
$elementRows = $expectedFiles->getRows();
$should = ($shouldOrNot !== "not");
foreach ($elementRows as $expectedFile) {
$fileFound = $this->findEntryFromPropfindResponse(
$expectedFile[0],
$folderPath
);
if ($should) {
Assert::assertNotEmpty(
$fileFound,
"response does not contain the entry '$expectedFile[0]'"
);
} else {
Assert::assertEmpty(
$fileFound,
"response does contain the entry '$expectedFile[0]' but should not"
);
}
}
}
/**
* Verify that the tableNode contains expected number of columns
*
@@ -1288,72 +1187,6 @@ class SpacesContext implements Context {
return \str_replace([" ", "(", ")"], ["%20", "%28", "%29"], $path);
}
/**
* parses a PROPFIND response from $this->response into xml
* and returns found search results if found else returns false
*
* @param string|null $entryNameToSearch
* @param string $folderPath
*
* @return array
* string if $entryNameToSearch is given and is found
* array if $entryNameToSearch is not given
* boolean false if $entryNameToSearch is given and is not found
*/
public function findEntryFromPropfindResponse(
string $entryNameToSearch = null,
string $folderPath = ''
): array {
$spaceId = $this->getResponseSpaceId();
//if we are using that step the second time in a scenario e.g. 'But ... should not'
//then don't parse the result again, because the result is in a ResponseInterface
if (empty($this->getResponseXml())) {
$this->setResponseXml(
HttpRequestHelper::parseResponseAsXml($this->featureContext->getResponse())
);
}
Assert::assertNotEmpty($this->getResponseXml(), __METHOD__ . ' Response is empty');
Assert::assertNotEmpty($spaceId, __METHOD__ . ' SpaceId is empty');
// trim any leading "/" passed by the caller, we can just match the "raw" name
$trimmedEntryNameToSearch = \trim($entryNameToSearch, "/");
// url encode for spaces and brackets that may appear in the filePath
$folderPath = $this->escapePath($folderPath);
// topWebDavPath should be something like /remote.php/webdav/ or
// /remote.php/dav/files/alice/
$topWebDavPath = "/" . "dav/spaces/" . $spaceId . "/" . $folderPath;
Assert::assertIsArray(
$this->responseXml,
__METHOD__ . " responseXml for space $spaceId is not an array"
);
Assert::assertArrayHasKey(
"value",
$this->responseXml,
__METHOD__ . " responseXml for space $spaceId does not have key 'value'"
);
$multistatusResults = $this->responseXml["value"];
$results = [];
if ($multistatusResults !== null) {
foreach ($multistatusResults as $multistatusResult) {
$entryPath = \urldecode($multistatusResult['value'][0]['value']);
$entryName = \str_replace($topWebDavPath, "", $entryPath);
$entryName = \rawurldecode($entryName);
$entryName = \trim($entryName, "/");
if ($trimmedEntryNameToSearch === $entryName) {
return $multistatusResult;
}
$results[] = $entryName;
}
}
if ($entryNameToSearch === null) {
return $results;
}
return [];
}
/**
* @When /^user "([^"]*)" creates a (?:folder|subfolder) "([^"]*)" in space "([^"]*)" using the WebDav Api$/
*
@@ -1423,19 +1256,8 @@ class SpacesContext implements Context {
if ($ownerUser === '') {
$ownerUser = $user;
}
$space = $this->getSpaceByName($ownerUser, $spaceName);
$fullUrl = $this->baseUrl . "/dav/spaces/" . $space['id'] . '/' . $folder;
$this->featureContext->setResponse(
$this->sendCreateFolderRequest(
$fullUrl,
"MKCOL",
$user,
$this->featureContext->getPasswordForUser($user)
)
);
$this->setSpaceIDByName($ownerUser, $spaceName);
$this->featureContext->userCreatesFolder($user, $folder);
}
/**
@@ -1456,20 +1278,8 @@ class SpacesContext implements Context {
string $content,
string $destination
): void {
$space = $this->getSpaceByName($user, $spaceName);
Assert::assertIsArray($space, "Space with name $spaceName not found");
Assert::assertNotEmpty($space["root"]["webDavUrl"], "WebDavUrl for space with name $spaceName not found");
$this->featureContext->setResponse(
$this->sendPutRequestToUrl(
$space["root"]["webDavUrl"] . "/" . $destination,
$user,
$this->featureContext->getPasswordForUser($user),
"",
[],
$content
)
);
$this->setSpaceIDByName($user, $spaceName);
$this->featureContext->uploadFileWithContent($user, $content, $destination);
}
/**
@@ -1492,20 +1302,8 @@ class SpacesContext implements Context {
string $content,
string $destination
): void {
$space = $this->getSpaceByName($ownerUser, $spaceName);
Assert::assertIsArray($space, "Space with name $spaceName not found");
Assert::assertNotEmpty($space["root"]["webDavUrl"], "WebDavUrl for space with name $spaceName not found");
$this->featureContext->setResponse(
$this->sendPutRequestToUrl(
$space["root"]["webDavUrl"] . "/" . $destination,
$user,
$this->featureContext->getPasswordForUser($user),
"",
[],
$content
)
);
$this->setSpaceIDByName($ownerUser, $spaceName);
$this->featureContext->uploadFileWithContent($user, $content, $destination);
}
/**
@@ -1917,7 +1715,7 @@ class SpacesContext implements Context {
):void {
$space = $this->getSpaceByName($user, $spaceName);
$fullUrl = $space["root"]["webDavUrl"] . '/' . ltrim($fileName, "/");
$this->downloadFileAsUserUsingPassword($fullUrl, $user);
$this->featureContext->downloadFileAsUserUsingPassword($user, $fileName, $this->featureContext->getPasswordForUser($user));
Assert::assertGreaterThanOrEqual(
400,
$this->featureContext->getResponse()->getStatusCode(),
@@ -2017,22 +1815,7 @@ class SpacesContext implements Context {
string $destination
): void {
$this->theUserListsAllHisAvailableSpacesUsingTheGraphApi($user);
$space = $this->getSpaceByName($user, $spaceName);
Assert::assertIsArray($space, "Space with name $spaceName not found");
Assert::assertNotEmpty($space["root"]["webDavUrl"], "WebDavUrl for space with name $spaceName not found");
$this->featureContext->setResponse(
$this->sendPutRequestToUrl(
$space["root"]["webDavUrl"] . "/" . $destination,
$user,
$this->featureContext->getPasswordForUser($user),
"",
[],
$fileContent
)
);
$this->theUserUploadsAFileToSpace($user, $spaceName, $fileContent, $destination);
$this->featureContext->theHTTPStatusCodeShouldBeOr(201, 204);
}
@@ -2098,7 +1881,6 @@ class SpacesContext implements Context {
];
$fullUrl = $this->baseUrl . $this->ocsApiUrl;
$this->featureContext->setResponse(
$this->sendPostRequestToUrl(
$fullUrl,
@@ -2671,20 +2453,8 @@ class SpacesContext implements Context {
string $fileName,
string $spaceName
): void {
$space = $this->getSpaceByName($user, $spaceName);
$fullUrl = $this->baseUrl . $this->davSpacesUrl . $space['id'] . '/' . $fileName;
$this->featureContext->setResponse(
HttpRequestHelper::sendRequest(
$fullUrl,
"",
'HEAD',
$user,
$this->featureContext->getPasswordForUser($user),
[],
""
)
);
$this->setSpaceIDByName($user, $spaceName);
$this->featureContext->downloadFileAsUserUsingPassword($user, $fileName, $this->featureContext->getPasswordForUser($user));
}
/**
@@ -2746,56 +2516,9 @@ class SpacesContext implements Context {
string $index,
string $spaceName
): void {
$fileVersion = $this->listFileVersion($user, $fileName, $spaceName);
if (!isset($fileVersion[$index])) {
Assert::fail(
'could not find version of file "' . $fileName . '" with index "' . $index . '"'
);
}
$url = $this->baseUrl . $fileVersion[$index][0];
$this->featureContext->setResponse(
HttpRequestHelper::sendRequest(
$url,
"",
'HEAD',
$user,
$this->featureContext->getPasswordForUser($user),
[],
""
)
);
}
/**
* Method returns an array with url values from the propfind request
* like: /remote.php/dav/meta/spaceUuid%fileUuid/v/fileUuid.REV.2022-05-17T10:39:49.672285951Z
*
* @param string $user
* @param string $fileName
* @param string $spaceName
*
* @return array
* @throws GuzzleException
*/
public function listFileVersion(
string $user,
string $fileName,
string $spaceName
): array {
$fileId = $this->getFileId($user, $spaceName, $fileName);
$fullUrl = $this->baseUrl . '/remote.php/dav/meta/' . $fileId . '/v';
$this->featureContext->setResponse(
$this->sendPropfindRequestToUrl(
$fullUrl,
$user,
$this->featureContext->getPasswordForUser($user)
)
);
$responseXml = HttpRequestHelper::getResponseXml($this->featureContext->getResponse());
return $responseXml->xpath("//d:response/d:href");
$this->setSpaceIDByName($user, $spaceName);
$this->filesVersionsContext->downloadVersion($user, $fileName, $index);
WebDavHelper::$SPACE_ID_FROM_OCIS = '';
}
/**
@@ -2809,20 +2532,8 @@ class SpacesContext implements Context {
* @throws GuzzleException
*/
public function userGetsEtagOfElementInASpace(string $user, string $space, string $path) {
$user = $this->featureContext->getActualUsername($user);
$space = $this->getSpaceByName($user, $space);
$fullUrl = $space['root']['webDavUrl'] . '/' . ltrim($path, '/');
$response = $this->sendPropfindRequestToUrl(
$fullUrl,
$user,
$this->featureContext->getPasswordForUser($user),
$this->featureContext->getStepLineRef(),
[],
$this->etagPropfindBody
);
$responseXml = HttpRequestHelper::getResponseXml($response);
$this->featureContext->setResponseXmlObject($responseXml);
$this->setSpaceIDByName($user, $space);
$this->webDavPropertiesContext->storeEtagOfElement($user, $path);
return $this->featureContext->getEtagFromResponseXmlObject();
}

141
tests/acceptance/features/bootstrap/SpacesTUSContext.php
View File

@@ -1,6 +1,7 @@
<?php
declare(strict_types=1);
/**
* ownCloud
*
@@ -166,4 +167,144 @@ class SpacesTUSContext implements Context {
$this->spacesContext->setSpaceIDByName($user, $spaceName);
$this->tusContext->userUploadsAFileWithContentToUsingTus($user, $content, $resource);
}
/**
* @When /^user "([^"]*)" uploads a file "([^"]*)" to "([^"]*)" with mtime "([^"]*)" via TUS inside of the space "([^"]*)" using the WebDAV API$/
*
* @param string $user
* @param string $source
* @param string $destination
* @param string $mtime Time in human readable format is taken as input which is converted into milliseconds that is used by API
* @param string $spaceName
*
* @return void
*
* @throws Exception
* @throws GuzzleException
*/
public function userUploadsAFileToWithMtimeViaTusInsideOfTheSpaceUsingTheWebdavApi(
string $user,
string $source,
string $destination,
string $mtime,
string $spaceName
): void {
$this->spacesContext->setSpaceIDByName($user, $spaceName);
$this->tusContext->userUploadsFileWithContentToWithMtimeUsingTUS($user, $source, $destination, $mtime);
}
/**
* @Given /^user "([^"]*)" has uploaded file with checksum "([^"]*)" to the last created TUS Location with offset "([^"]*)" and content "([^"]*)" via TUS inside of the space "([^"]*)" using the WebDAV API$/
*
* @param string $user
* @param string $checksum
* @param string $offset
* @param string $content
* @param string $spaceName
*
* @return void
* @throws Exception|GuzzleException
*/
public function userHasUploadedFileWithChecksumToTheLastCreatedTusLocationWithOffsetAndContentViaTusInsideOfTheSpaceUsingTheWebdavApi(
string $user,
string $checksum,
string $offset,
string $content,
string $spaceName
): void {
$this->spacesContext->setSpaceIDByName($user, $spaceName);
$this->tusContext->userHasUploadedFileWithChecksum($user, $checksum, $offset, $content);
}
/**
* @Given /^user "([^"]*)" uploads file with checksum "([^"]*)" to the last created TUS Location with offset "([^"]*)" and content "([^"]*)" via TUS inside of the space "([^"]*)" using the WebDAV API$/
*
* @param string $user
* @param string $checksum
* @param string $offset
* @param string $content
* @param string $spaceName
*
* @return void
* @throws Exception|GuzzleException
*/
public function userUploadsFileWithChecksumToTheLastCreatedTusLocationWithOffsetAndContentViaTusInsideOfTheSpaceUsingTheWebdavApi(
string $user,
string $checksum,
string $offset,
string $content,
string $spaceName
): void {
$this->spacesContext->setSpaceIDByName($user, $spaceName);
$this->tusContext->userUploadsFileWithChecksum($user, $checksum, $offset, $content);
}
/**
* @When /^user "([^"]*)" sends a chunk to the last created TUS Location with offset "([^"]*)" and data "([^"]*)" with checksum "([^"]*)" via TUS inside of the space "([^"]*)" using the WebDAV API$/
*
* @param string $user
* @param string $offset
* @param string $data
* @param string $checksum
* @param string $spaceName
*
* @return void
* @throws Exception|GuzzleException
*/
public function userSendsAChunkToTheLastCreatedTusLocationWithOffsetAndDataWithChecksumViaTusInsideOfTheSpaceUsingTheWebdavApi(
string $user,
string $offset,
string $data,
string $checksum,
string $spaceName
): void {
$this->spacesContext->setSpaceIDByName($user, $spaceName);
$this->tusContext->userUploadsChunkFileWithChecksum($user, $offset, $data, $checksum);
}
/**
* @When /^user "([^"]*)" overwrites recently shared file with offset "([^"]*)" and data "([^"]*)" with checksum "([^"]*)" via TUS inside of the space "([^"]*)" using the WebDAV API with these headers:$/
*
* @param string $user
* @param string $offset
* @param string $data
* @param string $checksum
* @param string $spaceName
* @param TableNode $headers
*
* @return void
* @throws GuzzleException
*/
public function userOverwritesRecentlySharedFileWithOffsetAndDataWithChecksumViaTusInsideOfTheSpaceUsingTheWebdavApiWithTheseHeaders(
string $user,
string $offset,
string $data,
string $checksum,
string $spaceName,
TableNode $headers
): void {
$this->spacesContext->setSpaceIDByName($user, $spaceName);
$this->tusContext->userOverwritesFileWithChecksum($user, $offset, $data, $checksum, $headers);
}
/**
* @Then /^as "([^"]*)" the mtime of the file "([^"]*)" in space "([^"]*)" should be "([^"]*)"$/
*
* @param string $user
* @param string $resource
* @param string $spaceName
* @param string $mtime
*
* @return void
* @throws Exception|GuzzleException
*/
public function theMtimeOfTheFileInSpaceShouldBe(
string $user,
string $resource,
string $spaceName,
string $mtime
): void {
$this->spacesContext->setSpaceIDByName($user, $spaceName);
$this->featureContext->theMtimeOfTheFileShouldBe($user, $resource, $mtime);
}
}