Commit Graph

136 Commits

Author SHA1 Message Date
Jörn Friedrich Dreyer
6bec87f582 Proxy accesstoken cache store (#5829)
* refactor middleware options

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use ocmemstore micro store implementaiton for token cache

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* refactor ocis store options, support redis sentinel

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* align cache configuration

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* database and tabe are used to build prefixes for inmemory stores

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add global persistent store options to userlog config

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* log cache errors but continue

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* drup unnecessary type conversion

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Better description for the default userinfo ttl

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use global cache options for even more caches

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* don't log userinfo cache misses

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* default to stock memory store

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use correct mem store typo string

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* split cache options, doc cleanup

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* mint and write userinfo to cache async

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use hashed token as key

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* go mod tidy

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update docs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update cache store naming

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bring back depreceted ocis-pkg/store package for backwards compatability

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: kobergj <jkoberg@owncloud.com>

* revert ocis-pkg/cache to store rename

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add waiting for each step 50 milliseconds

* starlack check

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: kobergj <jkoberg@owncloud.com>
Co-authored-by: Viktor Scharf <scharf.vi@gmail.com>
2023-03-22 15:21:57 +01:00
Ralf Haferkamp
7e6b690ccf proxy: update README with notes about role assignment (#5878) 2023-03-20 21:03:41 +01:00
Jörn Friedrich Dreyer
474c4b848d upgrade to go1.20 and auto set go mem limit (#5732)
* upgrade to go1.19 and set go mem limit

* create ocis-pkg memlimit package

* use std automemlimit import

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* import automemlimit in every ocis service, drop ocis-pkg/memlimit package

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump go to 1.20

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* drop unused config options and env vars

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update all version numbers, add doc

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update bingo and mockery

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump golangci-lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix selector test

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Update changelog/unreleased/enhancement-memlimit.md

Co-authored-by: kobergj <juliankoberg@googlemail.com>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Willy Kloucek <wkloucek@owncloud.com>
Co-authored-by: kobergj <juliankoberg@googlemail.com>
2023-03-20 17:22:02 +01:00
Ralf Haferkamp
98b4a6a23a Address some linter warnings
Response bodies need to be closed
2023-03-20 15:12:09 +01:00
Jörn Friedrich Dreyer
7f8e54b1c2 fix link 2023-03-20 15:12:09 +01:00
Ralf Haferkamp
1287bbc10d proxy: Cache the role Mapping for some time
Avoid torturing the settings service with "ListRoles" request for
every incoming request to the proxy.
The role Mapping is refreshed if cached data is older than 5 minutes.
2023-03-20 15:12:09 +01:00
Ralf Haferkamp
96ed144745 proxy: Add some missing doc strings 2023-03-20 15:12:09 +01:00
Ralf Haferkamp
a448c75c75 Extract role assignments from claims
Add a UserRoleAssigner implementation that extract role names from the
users' claims and creates role assignments in the settings service based
on a configured mapping of claim values to ocis role names.

Closes: #5669
2023-03-20 15:12:09 +01:00
Ralf Haferkamp
d57d52b33d proxy: Add an interface for user role assignment
This moves the lookup and the creation of the users' role assignemt out
of the user backend into its own interface. This makes the user backend
a bit simpler and allows to provide different implemenation for the user
role assignment more easily.
2023-03-20 15:12:09 +01:00
Ralf Haferkamp
490a835a3a proxy: Move CS3 User backend options to "functional" options
This will make it easier to add/remove options to the backend in the
future.
2023-03-20 15:12:09 +01:00
Ralf Haferkamp
88e8bb1b72 account_resolver: Handle user roles separately from user lookup
This removes the "withRoles" flag from the GetUserByClaims lookup and move the
functionality into a separate method. This should make the code a bit more readable
in preparation for maintaining the RoleAssignments from OIDC claims.
2023-03-20 15:12:09 +01:00
Ralf Haferkamp
f5cfa7e126 proxy: Move to mockery for generating mocks
To align with what we're using everywhere else.
2023-03-20 15:12:09 +01:00
Florian Schade
f38a9f4385 Introduce Policies-Service (#5716)
* add policies service
add policies proxy middleware
add policies event service
add policies grpc service
prepare ci and git environments (ci, make, readme, doc)

* add webfinger to the drone conf

* fix docs
remove not used virus scan postprocessing step

* relocate example rego file
implicitly enable and disable proxy and postprocessing policy checking by setting the query.
update configuration descriptions

* move policies
update readme

* use converter func to convert pp environment to actual environment
expose and test custom rego functions
add engine unit tests
add opa unit tests
update policies readme

Co-authored-by: Martin <github@diemattels.at>

* relocate sample policies to the deployments folder
change and document policies service port

* update index.md and small fix

* add health command
add version command
add debug server

---------

Co-authored-by: Martin <github@diemattels.at>
2023-03-14 16:08:22 +01:00
Ralf Haferkamp
a34a3b2a98 Cleanup some oidc related bits (#5751)
* Remove unused code from oidc module

* Use already existing Metadata type for jwks discovery

ocis-pkg/oidc already provides a type for the oidc metadata. Switch to
that instead of defining yet another custom type.

* oidc: Add helper to get IDP metadata
2023-03-07 14:43:42 +01:00
David Christofas
924c2ca346 add missing comments 2023-03-06 19:01:16 +01:00
David Christofas
63fa35fa50 reduce duplication in configuration code 2023-03-06 16:52:44 +01:00
Martin
fd5790d8dd Merge pull request #5610 from kobergj/UserlogService
Userlog Service
2023-02-23 15:33:12 +01:00
jkoberg
a9561d85c8 sharpen userlog service
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-02-23 14:21:56 +01:00
mmattel
7f706f38cc update quota text 2023-02-23 10:19:07 +01:00
David Christofas
3fbb6ca2fe Merge pull request #5617 from owncloud/role-quota
add config option to set default quota per role
2023-02-22 17:08:16 +01:00
David Christofas
67549b4ebd add config option to set default quota per role 2023-02-22 16:22:45 +01:00
jkoberg
d56565555b introduce userlog service
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-02-21 14:25:21 +01:00
Ralf Haferkamp
791c8a6c4f proxy: Fix provsioning new users with the default role
With 078698fdf4 the semantics of the
ListRoleAssignments Call in the settings service change. It no
no longer returns a "not found" error when there is not RoleAssignment
for a user. We'll just get an empty list as the result.

This changes the behaviour of the default role assignment to work with
the new semantics.
2023-02-21 11:19:44 +01:00
David Christofas
7c17ddb0b0 add a permission check to the logo upload 2023-02-13 13:48:39 +01:00
David Christofas
13dfc06f63 add proxy route for the branding endpoint 2023-02-13 13:36:52 +01:00
Jörn Friedrich Dreyer
2c98d3246c minimal webfinger (#5373)
* initial webfinger stub

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add webfinger to proxy, return current host

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* some cleanup

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* allow passing multiple rel params

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* introduce interfaces

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* parse oidc auth token

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add templating, drop chain, use map of relation providers

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix ocis url yaml

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix typos

Co-authored-by: Dominik Schmidt <dschmidt@owncloud.com>

* switch to userinfo claims

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* readme cleanup

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add TODO.md with ideas

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* replace subject on authenticated request responses

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* markdown lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* return a 401 when bearer token expired, some more docs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* fix docs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* clarify env var

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* extract handler func

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use correct service in reflex.conf

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* test relations

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Update services/webfinger/pkg/config/config.go

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Dominik Schmidt <dschmidt@owncloud.com>
Co-authored-by: Martin <github@diemattels.at>
2023-02-13 11:05:20 +01:00
Willy Kloucek
cd20bc82bf improve the description of PROXY_USER_OIDC_CLAIM 2023-02-10 11:06:09 +01:00
Benedikt Kulmann
3525302160 Merge pull request #5463 from owncloud/kill-the-settings-ui
[full-ci] change: remove the settings ui
2023-02-06 14:30:10 +01:00
Benedikt Kulmann
8c033e1936 change: remove the settings ui
ownCloud Web recently transitioned to Vue3. The settings ui is still
written in Vue2. Since it's pretty much unused we won't take the efforts
of upgrading it to Vue3.
2023-02-03 10:27:05 +01:00
mmattel
210cd54817 fix some envvar descriptions 2023-02-03 09:40:35 +01:00
Jörn Friedrich Dreyer
3aa864aecc Remove Staticcheck warnings (#5394)
* remove deprecated ioutil imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove duplicate imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove unused values

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove some unused types

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove deprecated ioutil import

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-01-13 15:29:14 +01:00
Jörn Friedrich Dreyer
53d15d329e remove deprecated use of ioutil (#5205)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-12-08 14:44:53 +01:00
Willy Kloucek
0ba134dbf6 discard errors 2022-11-24 17:07:01 +01:00
Willy Kloucek
c6b61cd347 remove any limits 2022-11-24 16:48:25 +01:00
Willy Kloucek
2e2e0cd4b6 fix HTTP1.1 RFC 2616 for bodies smaller than 1GB 2022-11-24 14:09:02 +01:00
mmattel
3cad3fbf2d env descriptions that needed a fix 2022-11-24 09:56:52 +01:00
Ralf Haferkamp
ee54a4c30c Update/align a few TLS variable description
Fixes: 5073
2022-11-23 13:07:29 +01:00
Ralf Haferkamp
07a78287a7 Allow initial self-assignemnt of UserRole
When using an external user management we need to allow users to self-assign
the default role. This adds an explicit check for that to the settings service.
This also means we no longer need to fiddle with the account id in the proxy
upon first login.

Fixes: #5045
2022-11-16 16:23:32 +01:00
Jörn Friedrich Dreyer
0f7dba53fb use min tls 1.2 (#4969)
* use min tls 1.2

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-11-03 16:08:56 +01:00
Ralf Haferkamp
b24d126b30 Introduce TLS Settings for go-micro based http services
TLS for the services can be configure by setting the "OCIS_HTTP_TLS_ENABLED",
"OCIS_HTTP_TLS_CERTIFICATE" and "OCIS_HTTP_TLS_KEY" environment variables.
Currently the ocis proxy is this only service that directly accesses backend
services. It determines whether to use TLS or not by looking a the new registry
metadata "use_tls". As specific CA Cert for certificate verification
can be set with the "PROXY_HTTPS_CACERT" environment variable.
2022-11-03 11:58:53 +01:00
Ralf Haferkamp
a4f5682851 Switch proxy defaults to use go-micro registry everywhere
This makes the default configuration somewhat easier. There doesn't
seem to be a specific reason why some services were still using a
static backend configuration.
2022-11-03 11:58:53 +01:00
Ralf Haferkamp
ee974afebf [full-ci] Introduce TLS Settings for go-micro based grpc services and clients (#4901)
* Introduce TLS Settings for go-micro based grpc services and clients

TLS for the services can be configure by setting the OCIS_MICRO_GRPC_TLS_ENABLED"
"OCIS_MICRO_GRPC_TLS_CERTIFICATE" and "OCIS_MICRO_GRPC_TLS_KEY"
enviroment variables.

TLS for the clients can configured by setting the "OCIS_MICRO_GRPC_CLIENT_TLS_MODE"
and "OCIS_MICRO_GRPC_CLIENT_TLS_CACERT" variables.

By default TLS is disabled.

Co-authored-by: Martin <github@diemattels.at>

* Unify TLS configuration for all grpc services

All grpc service (whether they're based on reva) or go-micro use the
same set of config vars now.

TLS for the services can be configure by setting the OCIS_GRPC_TLS_ENABLED,
OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY enviroment variables.

TLS for the clients can configured by setting the OCIS_GRPC_CLIENT_TLS_MODE
and OCIS_MICRO_GRPC_CLIENT_TLS_CACERT variables.

There are no individual per service config vars currently. If really
needed, per service tls configurations can be specified via config file.

Co-authored-by: Martin <github@diemattels.at>

Co-authored-by: Martin <github@diemattels.at>
2022-11-03 10:17:08 +01:00
Martin
bc06dfb20a Merge pull request #4911 from owncloud/proxy-readme
add proxy README
2022-10-28 14:33:00 +02:00
mmattel
bdcbe3e0c2 Proxy env var description rephrasing 2022-10-27 13:09:37 +02:00
Martin
3d5a6c36fc Update services/proxy/README.md 2022-10-27 12:56:06 +02:00
Martin
0f2b427f03 Update services/proxy/README.md 2022-10-27 12:11:00 +02:00
David Christofas
453acf3d66 add proxy README 2022-10-27 11:53:13 +02:00
Ralf Haferkamp
3d57f5cc21 Introduce TLS Settings for all reva grpc services and clients 2022-10-25 11:50:08 +02:00
Ralf Haferkamp
e373e48383 Get rid of duplicated Reva config struct
Consolidate all services to use the Reva config struct for the shared package.
This works because all services (except 'notifications', 'thumbnails' and
'webdav') where using the same config keys and environment variables for
setting the reva gateway.
2022-10-25 11:50:08 +02:00
Michael Barz
d965f23fd9 Merge pull request #4815 from owncloud/requestID
harden drives api and improve logging
2022-10-17 20:23:50 +02:00