Commit Graph

499 Commits

Author SHA1 Message Date
kobergj
8e5fcf8482 Merge pull request #5846 from kobergj/Antivirus
Antivirus
2023-03-23 10:04:12 +01:00
Jörn Friedrich Dreyer
6bec87f582 Proxy accesstoken cache store (#5829)
* refactor middleware options

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use ocmemstore micro store implementaiton for token cache

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* refactor ocis store options, support redis sentinel

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* align cache configuration

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* database and tabe are used to build prefixes for inmemory stores

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add global persistent store options to userlog config

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* log cache errors but continue

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* drup unnecessary type conversion

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Better description for the default userinfo ttl

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use global cache options for even more caches

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* don't log userinfo cache misses

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* default to stock memory store

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use correct mem store typo string

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* split cache options, doc cleanup

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* mint and write userinfo to cache async

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use hashed token as key

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* go mod tidy

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update docs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update cache store naming

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bring back depreceted ocis-pkg/store package for backwards compatability

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: kobergj <jkoberg@owncloud.com>

* revert ocis-pkg/cache to store rename

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add waiting for each step 50 milliseconds

* starlack check

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: kobergj <jkoberg@owncloud.com>
Co-authored-by: Viktor Scharf <scharf.vi@gmail.com>
2023-03-22 15:21:57 +01:00
jkoberg
86980441fe move debughandlers to their own package
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-22 13:59:54 +01:00
jkoberg
f4ba4e0f64 backport antivirus from experimental
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-21 10:04:11 +01:00
Jörn Friedrich Dreyer
474c4b848d upgrade to go1.20 and auto set go mem limit (#5732)
* upgrade to go1.19 and set go mem limit

* create ocis-pkg memlimit package

* use std automemlimit import

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* import automemlimit in every ocis service, drop ocis-pkg/memlimit package

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump go to 1.20

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* drop unused config options and env vars

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update all version numbers, add doc

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update bingo and mockery

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump golangci-lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix selector test

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Update changelog/unreleased/enhancement-memlimit.md

Co-authored-by: kobergj <juliankoberg@googlemail.com>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Willy Kloucek <wkloucek@owncloud.com>
Co-authored-by: kobergj <juliankoberg@googlemail.com>
2023-03-20 17:22:02 +01:00
Ralf Haferkamp
8b704085ce Allow ADMIN_USER_ID being empty (#5842)
For certain setups we don't need the ADMIN_USER_ID to be set. It is
mainly needed for bootstrapping the internal idm and the initial role
assignment.  If roles are assigned by other means (e.g. OIDC claims
in the future) we don't need it.

This makes the ADMIN_USER_ID optional, also if ADMIN_USER_ID is unset
we don't need to configure a password for the admin user. We will still
generated the admin_id and password when running 'ocis init', but it is
ok to run manual setups without those settings.
2023-03-15 16:15:18 +01:00
Florian Schade
f38a9f4385 Introduce Policies-Service (#5716)
* add policies service
add policies proxy middleware
add policies event service
add policies grpc service
prepare ci and git environments (ci, make, readme, doc)

* add webfinger to the drone conf

* fix docs
remove not used virus scan postprocessing step

* relocate example rego file
implicitly enable and disable proxy and postprocessing policy checking by setting the query.
update configuration descriptions

* move policies
update readme

* use converter func to convert pp environment to actual environment
expose and test custom rego functions
add engine unit tests
add opa unit tests
update policies readme

Co-authored-by: Martin <github@diemattels.at>

* relocate sample policies to the deployments folder
change and document policies service port

* update index.md and small fix

* add health command
add version command
add debug server

---------

Co-authored-by: Martin <github@diemattels.at>
2023-03-14 16:08:22 +01:00
Daniel Swärd
6b11f0bfe4 graph: Fix problem with unescaped semicolon and such causing problems. 2023-03-14 15:21:58 +01:00
Michael Barz
b5bfdb86d0 bump version 2023-03-08 11:36:45 +01:00
Ralf Haferkamp
a34a3b2a98 Cleanup some oidc related bits (#5751)
* Remove unused code from oidc module

* Use already existing Metadata type for jwks discovery

ocis-pkg/oidc already provides a type for the oidc metadata. Switch to
that instead of defining yet another custom type.

* oidc: Add helper to get IDP metadata
2023-03-07 14:43:42 +01:00
David Christofas
63fa35fa50 reduce duplication in configuration code 2023-03-06 16:52:44 +01:00
Daniel Swärd
c17add76c4 graph: Move token middleware and return proper JSON response on error.
Fixes #5661
2023-03-06 10:24:44 +01:00
jkoberg
d56565555b introduce userlog service
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-02-21 14:25:21 +01:00
jkoberg
c9dfb778a9 sharpen eventhistory service
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-02-21 10:51:32 +01:00
David Christofas
45d1ba25c0 add logo reset endpoint
when resetting the logo we are falling back to the embedded logo asset
2023-02-13 13:48:44 +01:00
David Christofas
20e4e56d28 implement first prototype of the logo upload API 2023-02-13 13:36:30 +01:00
Jörn Friedrich Dreyer
2c98d3246c minimal webfinger (#5373)
* initial webfinger stub

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add webfinger to proxy, return current host

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* some cleanup

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* allow passing multiple rel params

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* introduce interfaces

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* parse oidc auth token

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add templating, drop chain, use map of relation providers

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix ocis url yaml

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix typos

Co-authored-by: Dominik Schmidt <dschmidt@owncloud.com>

* switch to userinfo claims

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* readme cleanup

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add TODO.md with ideas

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* replace subject on authenticated request responses

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* markdown lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* return a 401 when bearer token expired, some more docs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* fix docs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* clarify env var

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* extract handler func

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use correct service in reflex.conf

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* test relations

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Update services/webfinger/pkg/config/config.go

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Dominik Schmidt <dschmidt@owncloud.com>
Co-authored-by: Martin <github@diemattels.at>
2023-02-13 11:05:20 +01:00
mmattel
bc6241a3dd Improve the description of the OCIS_ADMIN_USER_ID envvar 2023-02-09 12:50:01 +01:00
Daniel Swärd
1a01d778ca Set line info logging to info log level and lower. 2023-01-31 10:27:34 +01:00
Daniel Swärd
77c27826b3 Make line number info available on all log levels. 2023-01-30 11:28:29 +01:00
Daniel Swärd
f7b51d6dfa Skipping regex path removal. 2023-01-30 11:14:00 +01:00
Daniel Swärd
cca0c8d44d Add line number and file to log messages when at debug level. 2023-01-30 11:14:00 +01:00
Jörn Friedrich Dreyer
3aa864aecc Remove Staticcheck warnings (#5394)
* remove deprecated ioutil imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove duplicate imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove unused values

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove some unused types

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove deprecated ioutil import

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-01-13 15:29:14 +01:00
kobergj
341cdc9dd3 Merge pull request #5207 from kobergj/AsyncPostprocessing
[full-ci] Async Postprocessing
2022-12-16 11:58:24 +01:00
Jörn Friedrich Dreyer
70a6323c42 token middleware: use constant time comparison (#5225)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-12-14 14:04:02 +01:00
Jörn Friedrich Dreyer
53d15d329e remove deprecated use of ioutil (#5205)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-12-08 14:44:53 +01:00
jkoberg
06f3f5adb3 start pp service in single binary
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2022-12-08 11:49:04 +01:00
Michael Barz
ab57daf462 prepare release, bump version 2022-11-29 23:16:49 +01:00
Michael Barz
65849896b7 prepare release, bump version 2022-11-25 17:46:14 +01:00
Ralf Haferkamp
ee54a4c30c Update/align a few TLS variable description
Fixes: 5073
2022-11-23 13:07:29 +01:00
Willy Kloucek
3e4068e2a0 fix OCIS_GRPC_CLIENT_TLS_CACERT annotation 2022-11-15 14:49:31 +01:00
Michael Barz
736f513cfc prepare changelog, bump version 2022-11-05 10:53:41 +01:00
Ralf Haferkamp
b24d126b30 Introduce TLS Settings for go-micro based http services
TLS for the services can be configure by setting the "OCIS_HTTP_TLS_ENABLED",
"OCIS_HTTP_TLS_CERTIFICATE" and "OCIS_HTTP_TLS_KEY" environment variables.
Currently the ocis proxy is this only service that directly accesses backend
services. It determines whether to use TLS or not by looking a the new registry
metadata "use_tls". As specific CA Cert for certificate verification
can be set with the "PROXY_HTTPS_CACERT" environment variable.
2022-11-03 11:58:53 +01:00
Ralf Haferkamp
cbe41fb85f Add method for creating in-memory certificates 2022-11-03 11:58:53 +01:00
Ralf Haferkamp
ee974afebf [full-ci] Introduce TLS Settings for go-micro based grpc services and clients (#4901)
* Introduce TLS Settings for go-micro based grpc services and clients

TLS for the services can be configure by setting the OCIS_MICRO_GRPC_TLS_ENABLED"
"OCIS_MICRO_GRPC_TLS_CERTIFICATE" and "OCIS_MICRO_GRPC_TLS_KEY"
enviroment variables.

TLS for the clients can configured by setting the "OCIS_MICRO_GRPC_CLIENT_TLS_MODE"
and "OCIS_MICRO_GRPC_CLIENT_TLS_CACERT" variables.

By default TLS is disabled.

Co-authored-by: Martin <github@diemattels.at>

* Unify TLS configuration for all grpc services

All grpc service (whether they're based on reva) or go-micro use the
same set of config vars now.

TLS for the services can be configure by setting the OCIS_GRPC_TLS_ENABLED,
OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY enviroment variables.

TLS for the clients can configured by setting the OCIS_GRPC_CLIENT_TLS_MODE
and OCIS_MICRO_GRPC_CLIENT_TLS_CACERT variables.

There are no individual per service config vars currently. If really
needed, per service tls configurations can be specified via config file.

Co-authored-by: Martin <github@diemattels.at>

Co-authored-by: Martin <github@diemattels.at>
2022-11-03 10:17:08 +01:00
Ralf Haferkamp
89f2f2a87f Add documentation tags for grpc server TLS options 2022-10-25 11:50:08 +02:00
Ralf Haferkamp
3d57f5cc21 Introduce TLS Settings for all reva grpc services and clients 2022-10-25 11:50:08 +02:00
Michael Barz
f84fbdd6ff corrections from code review 2022-10-17 17:22:33 +02:00
Michael Barz
0b7297d1b4 harden drives api and improve logging 2022-10-16 00:38:44 +02:00
David Christofas
4623b6c8e7 Nats tls (#4781)
* use tls for nats connections

* add config options for nats client tls config

* add nats tls config to CI

* add function to create a certpool

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* configure nats clients in reva to use tls
2022-10-12 14:56:47 +02:00
Ralf Haferkamp
b5bb6980e2 Fix condition for registry cache
We want caching for all registries except "memory".
2022-10-12 09:28:05 +02:00
Jörn Friedrich Dreyer
e9e9947f88 bring back CORS env vars (#4723)
* bring back CORS env vars

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update CORS descriptions

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* align writing of 'A comma-separated ...'

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix some desc quotes

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* Apply more suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* Apply final suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Martin <github@diemattels.at>
2022-10-04 16:56:05 +02:00
David Christofas
3babaf1a4d remove the graph-explorer servier from ocis (#4733) 2022-10-04 15:55:30 +02:00
Ralf Haferkamp
c6664cfe61 Fix share/space link in notification mail
Use OCIS_URL as the link base. This change also makes sure that the
top-level OcisURL config value gets a default assigned, even when
OCIS_URL is unset.

Fixes: #4688
2022-09-29 16:26:21 +02:00
Ralf Haferkamp
8f2bf0ed9c Default to in-memory registry in single-binary mode
This avoids various issues of the old "mdns" default. At least for the simple
single process setup (#3134, #4597). When starting the services individually we
still default to "mdns".
2022-09-26 16:25:56 +02:00
Ralf Haferkamp
01650a5023 Use lazy initialization for the default grpc client
This avoids using a public global variable. It allows us to initialize
the default client a bit later (outside of init()). That way we can e.g.
properly initialize the in-memory registry.
2022-09-26 16:25:56 +02:00
Juan Pablo Villafañez
6ee4a084a2 Use go-micro store to cache the roles (#4337)
* Use go-micro store to cache the roles

Add custom in-memory implementation

* replace redis with custom etcd implementation

* adjust table name for the cache in the roles manager

* Fix tests

* Fix sonarcloud issues

* Refactor for sonarcloud

* Allow configuration of cache per service

* Reuse parent context in etcd implementation
2022-09-16 15:42:47 +02:00
Ralf Haferkamp
fffd49fc4e Introduce "noop" go-micro broker
This introduces a no op broker for go-micro. It is set as the default
broker for all ocis http services. To avoid starting the default http
broker, which opens an unused random http port.

Fixes: https://github.com/owncloud/ocis/issues/3829
2022-09-14 12:22:23 +02:00
Michael Barz
826d45877c prepare release, bump version 2022-09-13 10:17:43 +02:00
Phil Davis
b79de5f356 Merge pull request #4462 from owncloud/fix-typo-2022-08-26
[docs-only] Fix typo 2022-08-26
2022-08-26 18:33:53 +05:45