From c235e073206d6c0564cdf5909dc18df6a63893a4 Mon Sep 17 00:00:00 2001 From: Willy Kloucek Date: Tue, 26 Apr 2022 14:39:11 +0200 Subject: [PATCH 01/13] reference windows' host file also on all other deployment examples --- docs/ocis/deployment/oc10_ocis_parallel.md | 2 +- docs/ocis/deployment/ocis_hello.md | 2 +- docs/ocis/deployment/ocis_individual_services.md | 3 ++- docs/ocis/deployment/ocis_keycloak.md | 3 ++- docs/ocis/deployment/ocis_ldap.md | 2 +- docs/ocis/deployment/ocis_s3.md | 1 + docs/ocis/deployment/ocis_traefik.md | 3 ++- docs/ocis/deployment/ocis_wopi.md | 3 ++- 8 files changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/ocis/deployment/oc10_ocis_parallel.md b/docs/ocis/deployment/oc10_ocis_parallel.md index b771e76133..d5ff6e72e4 100644 --- a/docs/ocis/deployment/oc10_ocis_parallel.md +++ b/docs/ocis/deployment/oc10_ocis_parallel.md @@ -154,7 +154,7 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` files like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: ``` 127.0.0.1 cloud.owncloud.test diff --git a/docs/ocis/deployment/ocis_hello.md b/docs/ocis/deployment/ocis_hello.md index 5b4dc1fa1f..afc6e441c1 100644 --- a/docs/ocis/deployment/ocis_hello.md +++ b/docs/ocis/deployment/ocis_hello.md @@ -113,7 +113,7 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` files like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: ``` 127.0.0.1 ocis.owncloud.test diff --git a/docs/ocis/deployment/ocis_individual_services.md b/docs/ocis/deployment/ocis_individual_services.md index 198af51d8d..ab2bebe9b3 100644 --- a/docs/ocis/deployment/ocis_individual_services.md +++ b/docs/ocis/deployment/ocis_individual_services.md @@ -108,7 +108,8 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` files like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: + ``` 127.0.0.1 ocis.owncloud.test 127.0.0.1 traefik.owncloud.test diff --git a/docs/ocis/deployment/ocis_keycloak.md b/docs/ocis/deployment/ocis_keycloak.md index 8980bd1b9c..7495f661f5 100644 --- a/docs/ocis/deployment/ocis_keycloak.md +++ b/docs/ocis/deployment/ocis_keycloak.md @@ -129,7 +129,8 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` files like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: + ``` 127.0.0.1 ocis.owncloud.test 127.0.0.1 traefik.owncloud.test diff --git a/docs/ocis/deployment/ocis_ldap.md b/docs/ocis/deployment/ocis_ldap.md index e39396b5e3..7b0bea3354 100644 --- a/docs/ocis/deployment/ocis_ldap.md +++ b/docs/ocis/deployment/ocis_ldap.md @@ -113,7 +113,7 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` files like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: ``` 127.0.0.1 cloud.owncloud.test diff --git a/docs/ocis/deployment/ocis_s3.md b/docs/ocis/deployment/ocis_s3.md index 4e4ef41217..4c2ff90b8d 100644 --- a/docs/ocis/deployment/ocis_s3.md +++ b/docs/ocis/deployment/ocis_s3.md @@ -126,6 +126,7 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. On Linux and macOS you can add them to your `/etc/hosts` files like this: + ``` 127.0.0.1 ocis.owncloud.test 127.0.0.1 traefik.owncloud.test diff --git a/docs/ocis/deployment/ocis_traefik.md b/docs/ocis/deployment/ocis_traefik.md index 14a60e1ec3..a672577e2c 100644 --- a/docs/ocis/deployment/ocis_traefik.md +++ b/docs/ocis/deployment/ocis_traefik.md @@ -103,7 +103,8 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: + ``` 127.0.0.1 ocis.owncloud.test 127.0.0.1 traefik.owncloud.test diff --git a/docs/ocis/deployment/ocis_wopi.md b/docs/ocis/deployment/ocis_wopi.md index 01e070c9b7..10c5fd04d3 100644 --- a/docs/ocis/deployment/ocis_wopi.md +++ b/docs/ocis/deployment/ocis_wopi.md @@ -159,7 +159,8 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` files like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: + ``` 127.0.0.1 ocis.owncloud.test 127.0.0.1 traefik.owncloud.test From 0daaf08f3a76e05239c9f34b36a60e0e7b78fbe4 Mon Sep 17 00:00:00 2001 From: Willy Kloucek Date: Tue, 26 Apr 2022 15:00:56 +0200 Subject: [PATCH 02/13] add windows hosts file hint also to s3 example --- docs/ocis/deployment/ocis_s3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ocis/deployment/ocis_s3.md b/docs/ocis/deployment/ocis_s3.md index 4c2ff90b8d..d98617aee6 100644 --- a/docs/ocis/deployment/ocis_s3.md +++ b/docs/ocis/deployment/ocis_s3.md @@ -125,7 +125,7 @@ For a more simple local ocis setup see [Getting started]({{< ref "../getting-sta This docker stack can also be run locally. One downside is that Traefik can not obtain valid SSL certificates and therefore will create self-signed ones. This means that your browser will show scary warnings. Another downside is that you can not point DNS entries to your localhost. So you have to add static host entries to your computer. -On Linux and macOS you can add them to your `/etc/hosts` files like this: +On Linux and macOS you can add them to your `/etc/hosts` file and on Windows to `C:\Windows\System32\Drivers\etc\hosts` file like this: ``` 127.0.0.1 ocis.owncloud.test From 1a38f3623c69a9986eeed417652556c620b88ec8 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Wed, 9 Mar 2022 11:42:51 +0100 Subject: [PATCH 03/13] switch default config to idm - The accounts and glauth service are turned off by default - proxy is switch from "accounts" to "cs3" for the account backend - The LDAP configuration (graph, idp, storage) of all services now points to idm instead of glauth --- extensions/auth-basic/pkg/command/command.go | 2 ++ extensions/auth-basic/pkg/config/config.go | 2 ++ .../pkg/config/defaults/defaultconfig.go | 24 ++++++++++--------- .../pkg/config/defaults/defaultconfig.go | 16 ++++++------- extensions/group/pkg/command/command.go | 2 ++ extensions/group/pkg/config/config.go | 2 ++ .../pkg/config/defaults/defaultconfig.go | 24 ++++++++++--------- .../idp/pkg/config/defaults/defaultconfig.go | 12 +++++----- .../ocs/pkg/config/defaults/defaultconfig.go | 2 +- .../pkg/config/defaults/defaultconfig.go | 2 +- .../pkg/config/defaults/defaultconfig.go | 22 ++++++++--------- extensions/user/pkg/command/command.go | 2 ++ extensions/user/pkg/config/config.go | 2 ++ .../user/pkg/config/defaults/defaultconfig.go | 24 ++++++++++--------- ocis/pkg/runtime/service/service.go | 8 +++++++ 15 files changed, 86 insertions(+), 60 deletions(-) diff --git a/extensions/auth-basic/pkg/command/command.go b/extensions/auth-basic/pkg/command/command.go index 44745e4825..7835e9f09f 100644 --- a/extensions/auth-basic/pkg/command/command.go +++ b/extensions/auth-basic/pkg/command/command.go @@ -193,6 +193,8 @@ func ldapConfigFromString(cfg config.LDAPProvider) map[string]interface{} { "group_base_dn": cfg.GroupBaseDN, "user_filter": cfg.UserFilter, "group_filter": cfg.GroupFilter, + "user_scope": cfg.UserScope, + "group_scope": cfg.GroupScope, "user_objectclass": cfg.UserObjectClass, "group_objectclass": cfg.GroupObjectClass, "login_attributes": cfg.LoginAttributes, diff --git a/extensions/auth-basic/pkg/config/config.go b/extensions/auth-basic/pkg/config/config.go index 8557e3e7f1..e3706473cc 100644 --- a/extensions/auth-basic/pkg/config/config.go +++ b/extensions/auth-basic/pkg/config/config.go @@ -66,6 +66,8 @@ type LDAPProvider struct { BindPassword string `env:"LDAP_BIND_PASSWORD;AUTH_BASIC_LDAP_BIND_PASSWORD"` UserBaseDN string `env:"LDAP_USER_BASE_DN;AUTH_BASIC_LDAP_USER_BASE_DN"` GroupBaseDN string `env:"LDAP_GROUP_BASE_DN;AUTH_BASIC_LDAP_GROUP_BASE_DN"` + UserScope string `env:"LDAP_USER_SCOPE;AUTH_BASIC_LDAP_USER_SCOPE"` + GroupScope string `env:"LDAP_GROUP_SCOPE;AUTH_BASIC_LDAP_GROUP_SCOPE"` UserFilter string `env:"LDAP_USERFILTER;AUTH_BASIC_LDAP_USERFILTER"` GroupFilter string `env:"LDAP_GROUPFILTER;AUTH_BASIC_LDAP_USERFILTER"` UserObjectClass string `env:"LDAP_USER_OBJECTCLASS;AUTH_BASIC_LDAP_USER_OBJECTCLASS"` diff --git a/extensions/auth-basic/pkg/config/defaults/defaultconfig.go b/extensions/auth-basic/pkg/config/defaults/defaultconfig.go index 4d23247193..309b132446 100644 --- a/extensions/auth-basic/pkg/config/defaults/defaultconfig.go +++ b/extensions/auth-basic/pkg/config/defaults/defaultconfig.go @@ -35,31 +35,33 @@ func DefaultConfig() *config.Config { AuthProvider: "ldap", AuthProviders: config.AuthProviders{ LDAP: config.LDAPProvider{ - URI: "ldaps://localhost:9126", - CACert: filepath.Join(defaults.BaseDataPath(), "ldap", "ldap.crt"), + URI: "ldaps://localhost:9235", + CACert: filepath.Join(defaults.BaseDataPath(), "idm", "ldap.crt"), Insecure: false, - UserBaseDN: "dc=ocis,dc=test", - GroupBaseDN: "dc=ocis,dc=test", - LoginAttributes: []string{"cn", "mail"}, + UserBaseDN: "ou=users,o=libregraph-idm", + GroupBaseDN: "ou=groups,o=libregraph-idm", + UserScope: "sub", + GroupScope: "sub", + LoginAttributes: []string{"uid", "mail"}, UserFilter: "", GroupFilter: "", - UserObjectClass: "posixAccount", - GroupObjectClass: "posixGroup", - BindDN: "cn=reva,ou=sysusers,dc=ocis,dc=test", + UserObjectClass: "inetOrgPerson", + GroupObjectClass: "groupOfNames", + BindDN: "uid=reva,ou=sysusers,o=libregraph-idm", BindPassword: "reva", IDP: "https://localhost:9200", UserSchema: config.LDAPUserSchema{ ID: "ownclouduuid", Mail: "mail", DisplayName: "displayname", - Username: "cn", + Username: "uid", }, GroupSchema: config.LDAPGroupSchema{ - ID: "cn", + ID: "ownclouduuid", Mail: "mail", DisplayName: "cn", Groupname: "cn", - Member: "cn", + Member: "member", }, }, JSON: config.JSONProvider{}, diff --git a/extensions/graph/pkg/config/defaults/defaultconfig.go b/extensions/graph/pkg/config/defaults/defaultconfig.go index 49cd9916b5..512fa68d10 100644 --- a/extensions/graph/pkg/config/defaults/defaultconfig.go +++ b/extensions/graph/pkg/config/defaults/defaultconfig.go @@ -33,15 +33,15 @@ func DefaultConfig() *config.Config { Insecure: false, }, Identity: config.Identity{ - Backend: "cs3", + Backend: "ldap", LDAP: config.LDAP{ - URI: "ldap://localhost:9125", - Insecure: false, - BindDN: "", - BindPassword: "", + URI: "ldaps://localhost:9235", + Insecure: true, + BindDN: "uid=libregraph,ou=sysusers,o=libregraph-idm", + BindPassword: "idm", UseServerUUID: false, - WriteEnabled: false, - UserBaseDN: "ou=users,dc=ocis,dc=test", + WriteEnabled: true, + UserBaseDN: "ou=users,o=libregraph-idm", UserSearchScope: "sub", UserFilter: "", UserObjectClass: "inetOrgPerson", @@ -51,7 +51,7 @@ func DefaultConfig() *config.Config { // FIXME: switch this to some more widely available attribute by default // ideally this needs to be constant for the lifetime of a users UserIDAttribute: "owncloudUUID", - GroupBaseDN: "ou=groups,dc=ocis,dc=test", + GroupBaseDN: "ou=groups,o=libregraph-idm", GroupSearchScope: "sub", GroupFilter: "", GroupObjectClass: "groupOfNames", diff --git a/extensions/group/pkg/command/command.go b/extensions/group/pkg/command/command.go index ab71caef11..0f2162e0a3 100644 --- a/extensions/group/pkg/command/command.go +++ b/extensions/group/pkg/command/command.go @@ -190,6 +190,8 @@ func ldapConfigFromString(cfg config.LDAPDriver) map[string]interface{} { "bind_password": cfg.BindPassword, "user_base_dn": cfg.UserBaseDN, "group_base_dn": cfg.GroupBaseDN, + "user_scope": cfg.UserScope, + "group_scope": cfg.GroupScope, "user_filter": cfg.UserFilter, "group_filter": cfg.GroupFilter, "user_objectclass": cfg.UserObjectClass, diff --git a/extensions/group/pkg/config/config.go b/extensions/group/pkg/config/config.go index 9940bd7f26..c0eb2a4a71 100644 --- a/extensions/group/pkg/config/config.go +++ b/extensions/group/pkg/config/config.go @@ -67,6 +67,8 @@ type LDAPDriver struct { BindPassword string `env:"LDAP_BIND_PASSWORD;GROUPS_LDAP_BIND_PASSWORD"` UserBaseDN string `env:"LDAP_USER_BASE_DN;GROUPS_LDAP_USER_BASE_DN"` GroupBaseDN string `env:"LDAP_GROUP_BASE_DN;GROUPS_LDAP_GROUP_BASE_DN"` + UserScope string `env:"LDAP_USER_SCOPE;GROUPS_LDAP_USER_SCOPE"` + GroupScope string `env:"LDAP_GROUP_SCOPE;GROUPS_LDAP_GROUP_SCOPE"` UserFilter string `env:"LDAP_USERFILTER;GROUPS_LDAP_USERFILTER"` GroupFilter string `env:"LDAP_GROUPFILTER;GROUPS_LDAP_USERFILTER"` UserObjectClass string `env:"LDAP_USER_OBJECTCLASS;GROUPS_LDAP_USER_OBJECTCLASS"` diff --git a/extensions/group/pkg/config/defaults/defaultconfig.go b/extensions/group/pkg/config/defaults/defaultconfig.go index d7b0d988a8..372bcc6952 100644 --- a/extensions/group/pkg/config/defaults/defaultconfig.go +++ b/extensions/group/pkg/config/defaults/defaultconfig.go @@ -36,31 +36,33 @@ func DefaultConfig() *config.Config { Driver: "ldap", Drivers: config.Drivers{ LDAP: config.LDAPDriver{ - URI: "ldaps://localhost:9126", - CACert: filepath.Join(defaults.BaseDataPath(), "ldap", "ldap.crt"), + URI: "ldaps://localhost:9235", + CACert: filepath.Join(defaults.BaseDataPath(), "idm", "ldap.crt"), Insecure: false, - UserBaseDN: "dc=ocis,dc=test", - GroupBaseDN: "dc=ocis,dc=test", - LoginAttributes: []string{"cn", "mail"}, + UserBaseDN: "ou=users,o=libregraph-idm", + GroupBaseDN: "ou=groups,o=libregraph-idm", + UserScope: "sub", + GroupScope: "sub", + LoginAttributes: []string{"uid", "mail"}, UserFilter: "", GroupFilter: "", - UserObjectClass: "posixAccount", - GroupObjectClass: "posixGroup", - BindDN: "cn=reva,ou=sysusers,dc=ocis,dc=test", + UserObjectClass: "inetOrgPerson", + GroupObjectClass: "groupOfNames", + BindDN: "uid=reva,ou=sysusers,o=libregraph-idm", BindPassword: "reva", IDP: "https://localhost:9200", UserSchema: config.LDAPUserSchema{ ID: "ownclouduuid", Mail: "mail", DisplayName: "displayname", - Username: "cn", + Username: "uid", }, GroupSchema: config.LDAPGroupSchema{ - ID: "cn", + ID: "ownclouduuid", Mail: "mail", DisplayName: "cn", Groupname: "cn", - Member: "cn", + Member: "member", }, }, JSON: config.JSONDriver{}, diff --git a/extensions/idp/pkg/config/defaults/defaultconfig.go b/extensions/idp/pkg/config/defaults/defaultconfig.go index 2be18b92a4..fe328b2bb3 100644 --- a/extensions/idp/pkg/config/defaults/defaultconfig.go +++ b/extensions/idp/pkg/config/defaults/defaultconfig.go @@ -42,7 +42,7 @@ func DefaultConfig() *config.Config { SignedOutURI: "", AuthorizationEndpointURI: "", EndsessionEndpointURI: "", - Insecure: false, + Insecure: true, TrustedProxy: nil, AllowScope: nil, AllowClientGuests: false, @@ -68,18 +68,18 @@ func DefaultConfig() *config.Config { DyamicClientSecretDurationSeconds: 0, }, Ldap: config.Ldap{ - URI: "ldap://localhost:9125", - BindDN: "cn=idp,ou=sysusers,dc=ocis,dc=test", + URI: "ldaps://localhost:9235", + BindDN: "uid=idp,ou=sysusers,o=libregraph-idm", BindPassword: "idp", - BaseDN: "ou=users,dc=ocis,dc=test", + BaseDN: "ou=users,o=libregraph-idm", Scope: "sub", - LoginAttribute: "cn", + LoginAttribute: "uid", EmailAttribute: "mail", NameAttribute: "displayName", UUIDAttribute: "uid", UUIDAttributeType: "text", Filter: "", - ObjectClass: "posixAccount", + ObjectClass: "inetOrgPerson", }, } } diff --git a/extensions/ocs/pkg/config/defaults/defaultconfig.go b/extensions/ocs/pkg/config/defaults/defaultconfig.go index 90edea71eb..6038e0c8d5 100644 --- a/extensions/ocs/pkg/config/defaults/defaultconfig.go +++ b/extensions/ocs/pkg/config/defaults/defaultconfig.go @@ -41,7 +41,7 @@ func DefaultConfig() *config.Config { TokenManager: config.TokenManager{ JWTSecret: "Pive-Fumkiu4", }, - AccountBackend: "accounts", + AccountBackend: "cs3", Reva: config.Reva{ Address: "127.0.0.1:9142", }, diff --git a/extensions/proxy/pkg/config/defaults/defaultconfig.go b/extensions/proxy/pkg/config/defaults/defaultconfig.go index 487f9f09ab..d646436d81 100644 --- a/extensions/proxy/pkg/config/defaults/defaultconfig.go +++ b/extensions/proxy/pkg/config/defaults/defaultconfig.go @@ -45,7 +45,7 @@ func DefaultConfig() *config.Config { AllowedHTTPMethods: []string{"GET"}, Enabled: true, }, - AccountBackend: "accounts", + AccountBackend: "cs3", UserOIDCClaim: "email", UserCS3Claim: "mail", MachineAuthAPIKey: "change-me-please", diff --git a/extensions/storage/pkg/config/defaults/defaultconfig.go b/extensions/storage/pkg/config/defaults/defaultconfig.go index 95cc5c6cd2..c14ac52f0d 100644 --- a/extensions/storage/pkg/config/defaults/defaultconfig.go +++ b/extensions/storage/pkg/config/defaults/defaultconfig.go @@ -44,35 +44,35 @@ func DefaultConfig() *config.Config { IDClaim: "preferred_username", }, LDAP: config.LDAP{ - URI: "ldaps://localhost:9126", - CACert: path.Join(defaults.BaseDataPath(), "ldap", "ldap.crt"), + URI: "ldaps://localhost:9235", + CACert: path.Join(defaults.BaseDataPath(), "idm", "ldap.crt"), Insecure: false, - UserBaseDN: "dc=ocis,dc=test", - GroupBaseDN: "dc=ocis,dc=test", + UserBaseDN: "ou=users,o=libregraph-idm", + GroupBaseDN: "ou=groups,o=libregraph-idm", UserScope: "sub", GroupScope: "sub", - LoginAttributes: []string{"cn", "mail"}, + LoginAttributes: []string{"uid", "mail"}, UserFilter: "", GroupFilter: "", - UserObjectClass: "posixAccount", - GroupObjectClass: "posixGroup", - BindDN: "cn=reva,ou=sysusers,dc=ocis,dc=test", + UserObjectClass: "inetOrgPerson", + GroupObjectClass: "groupOfNames", + BindDN: "uid=reva,ou=sysusers,o=libregraph-idm", BindPassword: "reva", IDP: defaultPublicURL, UserSchema: config.LDAPUserSchema{ ID: "ownclouduuid", Mail: "mail", DisplayName: "displayname", - Username: "cn", + Username: "uid", UIDNumber: "uidnumber", GIDNumber: "gidnumber", }, GroupSchema: config.LDAPGroupSchema{ - ID: "cn", + ID: "ownclouduuid", Mail: "mail", DisplayName: "cn", Groupname: "cn", - Member: "cn", + Member: "member", GIDNumber: "gidnumber", }, }, diff --git a/extensions/user/pkg/command/command.go b/extensions/user/pkg/command/command.go index 31035acda1..a77f23f4c1 100644 --- a/extensions/user/pkg/command/command.go +++ b/extensions/user/pkg/command/command.go @@ -209,6 +209,8 @@ func ldapConfigFromString(cfg config.LDAPDriver) map[string]interface{} { "bind_password": cfg.BindPassword, "user_base_dn": cfg.UserBaseDN, "group_base_dn": cfg.GroupBaseDN, + "user_scope": cfg.UserScope, + "group_scope": cfg.GroupScope, "user_filter": cfg.UserFilter, "group_filter": cfg.GroupFilter, "user_objectclass": cfg.UserObjectClass, diff --git a/extensions/user/pkg/config/config.go b/extensions/user/pkg/config/config.go index efdcd54430..fdb08f931d 100644 --- a/extensions/user/pkg/config/config.go +++ b/extensions/user/pkg/config/config.go @@ -67,6 +67,8 @@ type LDAPDriver struct { BindPassword string `env:"LDAP_BIND_PASSWORD;USERS_LDAP_BIND_PASSWORD"` UserBaseDN string `env:"LDAP_USER_BASE_DN;USERS_LDAP_USER_BASE_DN"` GroupBaseDN string `env:"LDAP_GROUP_BASE_DN;USERS_LDAP_GROUP_BASE_DN"` + UserScope string `env:"LDAP_USER_SCOPE;USERS_LDAP_USER_SCOPE"` + GroupScope string `env:"LDAP_GROUP_SCOPE;USERS_LDAP_GROUP_SCOPE"` UserFilter string `env:"LDAP_USERFILTER;USERS_LDAP_USERFILTER"` GroupFilter string `env:"LDAP_GROUPFILTER;USERS_LDAP_USERFILTER"` UserObjectClass string `env:"LDAP_USER_OBJECTCLASS;USERS_LDAP_USER_OBJECTCLASS"` diff --git a/extensions/user/pkg/config/defaults/defaultconfig.go b/extensions/user/pkg/config/defaults/defaultconfig.go index 09f4abe003..20a486f47a 100644 --- a/extensions/user/pkg/config/defaults/defaultconfig.go +++ b/extensions/user/pkg/config/defaults/defaultconfig.go @@ -36,31 +36,33 @@ func DefaultConfig() *config.Config { Driver: "ldap", Drivers: config.Drivers{ LDAP: config.LDAPDriver{ - URI: "ldaps://localhost:9126", - CACert: filepath.Join(defaults.BaseDataPath(), "ldap", "ldap.crt"), + URI: "ldaps://localhost:9235", + CACert: filepath.Join(defaults.BaseDataPath(), "idm", "ldap.crt"), Insecure: false, - UserBaseDN: "dc=ocis,dc=test", - GroupBaseDN: "dc=ocis,dc=test", - LoginAttributes: []string{"cn", "mail"}, + UserBaseDN: "ou=users,o=libregraph-idm", + GroupBaseDN: "ou=groups,o=libregraph-idm", + UserScope: "sub", + GroupScope: "sub", + LoginAttributes: []string{"uid", "mail"}, UserFilter: "", GroupFilter: "", - UserObjectClass: "posixAccount", - GroupObjectClass: "posixGroup", - BindDN: "cn=reva,ou=sysusers,dc=ocis,dc=test", + UserObjectClass: "inetOrgPerson", + GroupObjectClass: "groupOfNames", + BindDN: "uid=reva,ou=sysusers,o=libregraph-idm", BindPassword: "reva", IDP: "https://localhost:9200", UserSchema: config.LDAPUserSchema{ ID: "ownclouduuid", Mail: "mail", DisplayName: "displayname", - Username: "cn", + Username: "uid", }, GroupSchema: config.LDAPGroupSchema{ - ID: "cn", + ID: "ownclouduuid", Mail: "mail", DisplayName: "cn", Groupname: "cn", - Member: "cn", + Member: "member", }, }, JSON: config.JSONDriver{}, diff --git a/ocis/pkg/runtime/service/service.go b/ocis/pkg/runtime/service/service.go index e7ca1fc71e..7215d8098d 100644 --- a/ocis/pkg/runtime/service/service.go +++ b/ocis/pkg/runtime/service/service.go @@ -254,10 +254,18 @@ func (s *Service) generateRunSet(cfg *ociscfg.Config) { } for name := range s.ServicesRegistry { + // don't run glauth by default but keep the possiblity to start it via cfg.Runtime.Extensions for now + if name == "glauth" { + continue + } runset = append(runset, name) } for name := range s.Delayed { + // don't run accounts by default but keep the possiblity to start it via cfg.Runtime.Extensions for now + if name == "accounts" { + continue + } runset = append(runset, name) } } From 45f0940071bd0208f6afdbf35a8449e9cd8bb658 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Wed, 16 Mar 2022 12:40:57 +0100 Subject: [PATCH 04/13] Turn of "insecure" of built-in IDP Setup idp to verify the LDAP server certificate. As this certificate might be generated on startup, this also moved the IDP to the "delayed" set of services. So it starts after "idm". --- extensions/idp/pkg/config/config.go | 3 ++- extensions/idp/pkg/config/defaults/defaultconfig.go | 3 ++- extensions/idp/pkg/service/v0/service.go | 4 ++++ ocis/pkg/runtime/service/service.go | 2 +- 4 files changed, 9 insertions(+), 3 deletions(-) diff --git a/extensions/idp/pkg/config/config.go b/extensions/idp/pkg/config/config.go index 83bd84554d..4979fb0f38 100644 --- a/extensions/idp/pkg/config/config.go +++ b/extensions/idp/pkg/config/config.go @@ -27,7 +27,8 @@ type Config struct { // Ldap defines the available LDAP configuration. type Ldap struct { - URI string `yaml:"uri" env:"LDAP_URI;IDP_LDAP_URI"` + URI string `yaml:"uri" env:"LDAP_URI;IDP_LDAP_URI"` + TLSCACert string `yaml:"cacert" env:"LDAP_CACERT;IDP_LDAP_TLS_CACERT"` BindDN string `yaml:"bind_dn" env:"LDAP_BIND_DN;IDP_LDAP_BIND_DN"` BindPassword string `yaml:"bind_password" env:"LDAP_BIND_PASSWORD;IDP_LDAP_BIND_PASSWORD"` diff --git a/extensions/idp/pkg/config/defaults/defaultconfig.go b/extensions/idp/pkg/config/defaults/defaultconfig.go index fe328b2bb3..d9b68fb506 100644 --- a/extensions/idp/pkg/config/defaults/defaultconfig.go +++ b/extensions/idp/pkg/config/defaults/defaultconfig.go @@ -42,7 +42,7 @@ func DefaultConfig() *config.Config { SignedOutURI: "", AuthorizationEndpointURI: "", EndsessionEndpointURI: "", - Insecure: true, + Insecure: false, TrustedProxy: nil, AllowScope: nil, AllowClientGuests: false, @@ -69,6 +69,7 @@ func DefaultConfig() *config.Config { }, Ldap: config.Ldap{ URI: "ldaps://localhost:9235", + TLSCACert: path.Join(defaults.BaseDataPath(), "idm", "ldap.crt"), BindDN: "uid=idp,ou=sysusers,o=libregraph-idm", BindPassword: "idp", BaseDN: "ou=users,o=libregraph-idm", diff --git a/extensions/idp/pkg/service/v0/service.go b/extensions/idp/pkg/service/v0/service.go index 71270f0c44..1b1a8bf1d0 100644 --- a/extensions/idp/pkg/service/v0/service.go +++ b/extensions/idp/pkg/service/v0/service.go @@ -142,6 +142,10 @@ func initLicoInternalEnvVars(ldap *config.Ldap) error { "LDAP_FILTER": filter, } + if ldap.TLSCACert != "" { + defaults["LDAP_TLS_CACERT"] = ldap.TLSCACert + } + for k, v := range defaults { if err := os.Setenv(k, v); err != nil { return fmt.Errorf("could not set env var %s=%s", k, v) diff --git a/ocis/pkg/runtime/service/service.go b/ocis/pkg/runtime/service/service.go index 7215d8098d..f791543a45 100644 --- a/ocis/pkg/runtime/service/service.go +++ b/ocis/pkg/runtime/service/service.go @@ -112,7 +112,6 @@ func NewService(options ...Option) (*Service, error) { s.ServicesRegistry["glauth"] = glauth.NewSutureService s.ServicesRegistry["graph"] = graph.NewSutureService s.ServicesRegistry["graph-explorer"] = graphExplorer.NewSutureService - s.ServicesRegistry["idp"] = idp.NewSutureService s.ServicesRegistry["idm"] = idm.NewSutureService s.ServicesRegistry["ocs"] = ocs.NewSutureService s.ServicesRegistry["store"] = store.NewSutureService @@ -137,6 +136,7 @@ func NewService(options ...Option) (*Service, error) { s.Delayed["accounts"] = accounts.NewSutureService s.Delayed["proxy"] = proxy.NewSutureService s.Delayed["ocdav"] = ocdav.NewOCDav + s.Delayed["idp"] = idp.NewSutureService return s, nil } From 72cb96079052609eda88225685ab03453bd372db Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Thu, 21 Apr 2022 14:34:14 +0200 Subject: [PATCH 05/13] Adjust drone config for defaulting to libreidm/graph --- .drone.star | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/.drone.star b/.drone.star index 20aa53b7b5..bb288b03a4 100644 --- a/.drone.star +++ b/.drone.star @@ -76,16 +76,16 @@ config = { }, "uiTests": { "filterTags": "@ocisSmokeTest", - "skip": False, + "skip": True, "skipExceptParts": [], "earlyFail": True, }, "accountsUITests": { - "skip": False, + "skip": True, "earlyFail": True, }, "settingsUITests": { - "skip": False, + "skip": True, "earlyFail": True, }, "parallelApiTests": { @@ -111,7 +111,7 @@ config = { }, }, "graphApiTests": { - "skip": False, + "skip": True, "earlyFali": False, "numberOfParts": 10, "skipExceptParts": [], @@ -505,6 +505,9 @@ def localApiTests(ctx, storage, suite, accounts_hash_difficulty = 4): "name": "localApiTests-%s-%s" % (suite, storage), "image": OC_CI_PHP % DEFAULT_PHP_VERSION, "environment": { + "TEST_WITH_GRAPH_API": "true", + "PATH_TO_OCIS": "/drone/src", + "PATH_TO_CORE": "/srv/app/testrunner", "TEST_SERVER_URL": "https://ocis-server:9200", "OCIS_REVA_DATA_ROOT": "%s" % ("/srv/app/tmp/ocis/owncloud/data/" if storage == "owncloud" else ""), "SKELETON_DIR": "/srv/app/tmp/testing/data/apiSkeleton", @@ -513,8 +516,7 @@ def localApiTests(ctx, storage, suite, accounts_hash_difficulty = 4): "SEND_SCENARIO_LINE_REFERENCES": "true", "STORAGE_DRIVER": storage, "BEHAT_SUITE": suite, - "BEHAT_FILTER_TAGS": "~@skip&&~@skipOnOcis-%s-Storage" % ("OC" if storage == "owncloud" else "OCIS"), - "PATH_TO_CORE": "/srv/app/testrunner", + "BEHAT_FILTER_TAGS": "~@skip&&~@skipOnGraph&&~@skipOnOcis-%s-Storage" % ("OC" if storage == "owncloud" else "OCIS"), "EXPECTED_FAILURES_FILE": "/drone/src/tests/acceptance/expected-failures-localAPI-on-%s-storage.md" % (storage.upper()), "UPLOAD_DELETE_WAIT_TIME": "1" if storage == "owncloud" else 0, }, @@ -570,6 +572,8 @@ def cs3ApiTests(ctx, storage, accounts_hash_difficulty = 4): def coreApiTests(ctx, part_number = 1, number_of_parts = 1, storage = "ocis", accounts_hash_difficulty = 4): early_fail = config["apiTests"]["earlyFail"] if "earlyFail" in config["apiTests"] else False + filterTags = "~@skipOnGraph&&~@skipOnOcis&&~@notToImplementOnOCIS&&~@toImplementOnOCIS&&~comments-app-required&&~@federation-app-required&&~@notifications-app-required&&~systemtags-app-required&&~@local_storage&&~@skipOnOcis-%s-Storage&&~@issue-ocis-3023" % ("OC" if storage == "owncloud" else "OCIS") + expectedFailuresFile = "/drone/src/tests/acceptance/expected-failures-graphAPI-on-%s-storage.md" % (storage.upper()) return { "kind": "pipeline", @@ -586,6 +590,9 @@ def coreApiTests(ctx, part_number = 1, number_of_parts = 1, storage = "ocis", ac "name": "oC10ApiTests-%s-storage-%s" % (storage, part_number), "image": OC_CI_PHP % DEFAULT_PHP_VERSION, "environment": { + "TEST_WITH_GRAPH_API": "true", + "PATH_TO_OCIS": "/drone/src", + "PATH_TO_CORE": "/srv/app/testrunner", "TEST_SERVER_URL": "https://ocis-server:9200", "OCIS_REVA_DATA_ROOT": "%s" % ("/srv/app/tmp/ocis/owncloud/data/" if storage == "owncloud" else ""), "SKELETON_DIR": "/srv/app/tmp/testing/data/apiSkeleton", @@ -593,10 +600,10 @@ def coreApiTests(ctx, part_number = 1, number_of_parts = 1, storage = "ocis", ac "TEST_OCIS": "true", "SEND_SCENARIO_LINE_REFERENCES": "true", "STORAGE_DRIVER": storage, - "BEHAT_FILTER_TAGS": "~@skipOnOcis&&~@notToImplementOnOCIS&&~@toImplementOnOCIS&&~comments-app-required&&~@federation-app-required&&~@notifications-app-required&&~systemtags-app-required&&~@local_storage&&~@skipOnOcis-%s-Storage&&~@issue-ocis-3023" % ("OC" if storage == "owncloud" else "OCIS"), + "BEHAT_FILTER_TAGS": filterTags, "DIVIDE_INTO_NUM_PARTS": number_of_parts, "RUN_PART": part_number, - "EXPECTED_FAILURES_FILE": "/drone/src/tests/acceptance/expected-failures-API-on-%s-storage.md" % (storage.upper()), + "EXPECTED_FAILURES_FILE": expectedFailuresFile, "UPLOAD_DELETE_WAIT_TIME": "1" if storage == "owncloud" else 0, }, "commands": [ @@ -1696,7 +1703,6 @@ def ocisServer(storage, accounts_hash_difficulty = 4, volumes = [], depends_on = "OCIS_LOG_LEVEL": "error", "SETTINGS_DATA_PATH": "/srv/app/tmp/ocis/settings", "OCIS_INSECURE": "true", - "ACCOUNTS_DEMO_USERS_AND_GROUPS": True, # deprecated, remove after switching to LibreIDM "IDM_CREATE_DEMO_USERS": True, } wait_for_ocis = { @@ -1710,6 +1716,8 @@ def ocisServer(storage, accounts_hash_difficulty = 4, volumes = [], depends_on = else: user = "33:33" environment = { + "GRAPH_IDENTITY_BACKEND": "cs3", + "GRAPH_LDAP_SERVER_WRITE_ENABLED": "false", # Keycloak IDP specific configuration "PROXY_OIDC_ISSUER": "https://keycloak/auth/realms/owncloud", "LDAP_IDP": "https://keycloak/auth/realms/owncloud", @@ -1781,6 +1789,7 @@ def ocisServer(storage, accounts_hash_difficulty = 4, volumes = [], depends_on = "OCIS_MACHINE_AUTH_API_KEY": "change-me-please", "OCIS_INSECURE": "true", "PROXY_ENABLE_BASIC_AUTH": "true", + "IDM_CREATE_DEMO_USERS": True, } wait_for_ocis = { "name": "wait-for-ocis-server", @@ -2508,7 +2517,7 @@ def graphApiTests(ctx, part_number = 1, number_of_parts = 1): }, "steps": skipIfUnchanged(ctx, "acceptance-tests") + restoreBuildArtifactCache(ctx, "ocis-binary-amd64", "ocis/bin/ocis") + - ocisServerWithIdp() + + ocisServer() + cloneCoreRepos() + [ { "name": "Graph-oC10ApiTests-%s-storage-%s" % (storage, part_number), From 5d4800b570cdfc111de558f78d5c43bc03642fb2 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Mon, 25 Apr 2022 11:35:50 +0200 Subject: [PATCH 06/13] move test away from accounts api, use graph to get userid --- .../features/bootstrap/SpacesContext.php | 52 ++++++++----------- 1 file changed, 21 insertions(+), 31 deletions(-) diff --git a/tests/acceptance/features/bootstrap/SpacesContext.php b/tests/acceptance/features/bootstrap/SpacesContext.php index bbe9eda598..8200217e2f 100644 --- a/tests/acceptance/features/bootstrap/SpacesContext.php +++ b/tests/acceptance/features/bootstrap/SpacesContext.php @@ -30,6 +30,7 @@ use GuzzleHttp\Exception\GuzzleException; use Psr\Http\Message\ResponseInterface; use TestHelpers\HttpRequestHelper; use TestHelpers\SetupHelper; +use TestHelpers\GraphHelper; use PHPUnit\Framework\Assert; require_once 'bootstrap.php'; @@ -285,31 +286,23 @@ class SpacesContext implements Context { * @return string */ public function getUserIdByUserName(string $userName): string { - $fullUrl = $this->baseUrl . "/api/v0/accounts/accounts-list"; - $this->featureContext->setResponse( - HttpRequestHelper::post( - $fullUrl, - "", - $this->featureContext->getAdminUsername(), - $this->featureContext->getAdminPassword(), - [], - "{}" - ) - ); + $this->featureContext->setResponse(GraphHelper::getUser( + $this->featureContext->getBaseUrl(), + $this->featureContext->getStepLineRef(), + $this->featureContext->getAdminUsername(), + $this->featureContext->getAdminPassword(), + $userName + )); if ($this->featureContext->getResponse()) { $rawBody = $this->featureContext->getResponse()->getBody()->getContents(); $response = \json_decode($rawBody, true, 512, JSON_THROW_ON_ERROR); - if (isset($response["accounts"])) { - $accounts = $response["accounts"]; + if (isset($response["id"])) { + $user = $response; } else { throw new Exception(__METHOD__ . " accounts-list is empty"); } } - foreach ($accounts as $account) { - if ($account["preferredName"] === $userName) { - return $account["id"]; - } - } + return $user["id"]; throw new Exception(__METHOD__ . " user with name $userName not found"); } @@ -607,7 +600,6 @@ class SpacesContext implements Context { $password = $this->featureContext->getAdminPassword(); $headers = []; $bundles = []; - $accounts = []; $assignment = []; // get the roles list first @@ -628,22 +620,20 @@ class SpacesContext implements Context { } Assert::assertNotEmpty($roleToAssign, "The selected role $role could not be found"); - // get the accounts list first - $fullUrl = $this->baseUrl . "/api/v0/accounts/accounts-list"; - $this->featureContext->setResponse(HttpRequestHelper::post($fullUrl, "", $admin, $password, $headers, "{}")); + $this->featureContext->setResponse(GraphHelper::getUser( + $this->featureContext->getBaseUrl(), + $this->featureContext->getStepLineRef(), + $this->featureContext->getAdminUsername(), + $this->featureContext->getAdminPassword(), + $user + )); if ($this->featureContext->getResponse()) { $rawBody = $this->featureContext->getResponse()->getBody()->getContents(); - if (isset(\json_decode($rawBody, true, 512, JSON_THROW_ON_ERROR)["accounts"])) { - $accounts = \json_decode($rawBody, true, 512, JSON_THROW_ON_ERROR)["accounts"]; - } - } - $accountToChange = ""; - foreach ($accounts as $account) { - // find the selected user - if ($account["preferredName"] === $user) { - $accountToChange = $account; + if (isset(\json_decode($rawBody, true, 512, JSON_THROW_ON_ERROR)["id"])) { + $accountToChange = \json_decode($rawBody, true, 512, JSON_THROW_ON_ERROR); } } + Assert::assertNotEmpty($accountToChange, "The selected account $user does not exist"); // set the new role From 095e3c3f779700ed913753f20bd30fba96ef11c0 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Thu, 21 Apr 2022 17:37:07 +0200 Subject: [PATCH 07/13] skip test that requires ocs provisioning API When using the GraphAPI skip ocs specific test --- .../features/apiAccountsHashDifficulty/addUser.feature | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/acceptance/features/apiAccountsHashDifficulty/addUser.feature b/tests/acceptance/features/apiAccountsHashDifficulty/addUser.feature index eb271388b9..869a9eada5 100644 --- a/tests/acceptance/features/apiAccountsHashDifficulty/addUser.feature +++ b/tests/acceptance/features/apiAccountsHashDifficulty/addUser.feature @@ -1,4 +1,4 @@ -@api @provisioning_api-app-required @skipOnLDAP +@api @provisioning_api-app-required @skipOnLDAP @skipOnGraph Feature: add user As an admin I want to be able to add users and store their password with the full hash difficulty From 33cb8a4e63171a939f580ee9b77d531fc6bfee2b Mon Sep 17 00:00:00 2001 From: Viktor Scharf Date: Mon, 25 Apr 2022 13:19:46 +0200 Subject: [PATCH 08/13] addSteps: clean project space after test --- .../features/bootstrap/SpacesContext.php | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/tests/acceptance/features/bootstrap/SpacesContext.php b/tests/acceptance/features/bootstrap/SpacesContext.php index 8200217e2f..61d8f20b0d 100644 --- a/tests/acceptance/features/bootstrap/SpacesContext.php +++ b/tests/acceptance/features/bootstrap/SpacesContext.php @@ -24,6 +24,7 @@ declare(strict_types=1); use Behat\Behat\Context\Context; use Behat\Behat\Hook\Scope\BeforeScenarioScope; +use Behat\Behat\Hook\Call\AfterScenario; use Behat\Gherkin\Node\TableNode; use Behat\Testwork\Environment\Environment; use GuzzleHttp\Exception\GuzzleException; @@ -333,6 +334,55 @@ class SpacesContext implements Context { ); } + /** + * @AfterScenario + * + * @return void + * + * @throws Exception + */ + public function cleanDataAfterTests(): void + { + $this->deleteAllSpacesOfTheType('project'); + } + + /** + * The method first disables and then deletes spaces + * @param string $driveType + * + * @return void + * + * @throws Exception + */ + public function deleteAllSpacesOfTheType(string $driveType): void + { + $query = "\$filter=driveType eq $driveType"; + $userAdmin = $this->featureContext->getAdminUsername(); + + for ($i = 0; $i < 2; ++$i) { + $this->theUserListsAllHisAvailableSpacesUsingTheGraphApiWithFilter( + $userAdmin, + $query + ); + + $rawBody = $this->featureContext->getResponse()->getBody()->getContents(); + $drives = json_decode($rawBody, true, 512, JSON_THROW_ON_ERROR); + if (isset($drives["value"])) { + $drives = $drives["value"]; + } + + if (!empty($drives)) { + foreach ($drives as $value) { + if (!array_key_exists("deleted", $value["root"])) { + $this->sendDisableSpaceRequest($userAdmin, $value["name"]); + } else { + $this->sendDeleteSpaceRequest($userAdmin, $value["name"]); + } + } + } + } + } + /** * Send Graph List My Spaces Request * From aa7d75c84583e51fcf93815b6ae992087c94d1a5 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Tue, 26 Apr 2022 08:29:10 +0200 Subject: [PATCH 09/13] Run UI test with accounts service enabled until the ui tests are able to switch to the Graph API for user provisioning --- .drone.star | 72 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 31 deletions(-) diff --git a/.drone.star b/.drone.star index bb288b03a4..e5794bea7a 100644 --- a/.drone.star +++ b/.drone.star @@ -76,7 +76,7 @@ config = { }, "uiTests": { "filterTags": "@ocisSmokeTest", - "skip": True, + "skip": False, "skipExceptParts": [], "earlyFail": True, }, @@ -698,7 +698,7 @@ def uiTestPipeline(ctx, filterTags, early_fail, runPart = 1, numberOfParts = 1, "arch": "amd64", }, "steps": skipIfUnchanged(ctx, "acceptance-tests") + restoreBuildArtifactCache(ctx, "ocis-binary-amd64", "ocis/bin/ocis") + - ocisServer(storage, accounts_hash_difficulty, [stepVolumeOC10Tests]) + waitForSeleniumService() + waitForMiddlewareService() + [ + ocisServerWithAccounts(storage, accounts_hash_difficulty, [stepVolumeOC10Tests]) + waitForSeleniumService() + waitForMiddlewareService() + [ { "name": "webUITests", "image": OC_CI_NODEJS % DEFAULT_NODEJS_VERSION, @@ -1629,40 +1629,50 @@ def notify(ctx): }, } -def ocisServerWithIdp(): +def ocisServerWithAccounts(storage, accounts_hash_difficulty = 4, volumes = [], depends_on = []): environment = { - "GRAPH_IDENTITY_BACKEND": "ldap", - "GRAPH_LDAP_SERVER_WRITE_ENABLED": "true", - "LDAP_URI": "ldaps://0.0.0.0:9235", + "GRAPH_IDENTITY_BACKEND": "cs3", + "GRAPH_LDAP_SERVER_WRITE_ENABLED": "false", + "LDAP_URI": "ldaps://0.0.0.0:9126", "LDAP_INSECURE": "true", - "GRAPH_LDAP_BIND_DN": "uid=libregraph,ou=sysusers,o=libregraph-idm", - "GRAPH_LDAP_BIND_PASSWORD": "idm", - "LDAP_USER_BASE_DN": "ou=users,o=libregraph-idm", + "LDAP_BIND_DN": "cn=admin,dc=ocis,dc=test", + "LDAP_BIND_PASSWORD": "admin", + "LDAP_USER_BASE_DN": "dc=ocis,dc=test", "LDAP_USER_SCHEMA_ID": "ownclouduuid", "LDAP_USER_SCHEMA_MAIL": "mail", - "LDAP_USER_SCHEMA_USERNAME": "uid", - "LDAP_USER_OBJECTCLASS": "inetOrgPerson", - "LDAP_GROUP_BASE_DN": "ou=groups,o=libregraph-idm", - "LDAP_GROUP_SCHEMA_ID": "ownclouduuid", + "LDAP_USER_SCHEMA_USERNAME": "cn", + "LDAP_USER_OBJECTCLASS": "posixAccount", + "LDAP_GROUP_BASE_DN": "dc=ocis,dc=test", + "LDAP_GROUP_SCHEMA_ID": "cn", "LDAP_GROUP_SCHEMA_MAIL": "mail", "LDAP_GROUP_SCHEMA_GROUPNAME": "cn", - "LDAP_GROUP_SCHEMA_MEMBER": "member", - "LDAP_GROUP_OBJECTCLASS": "groupOfNames", - "IDP_INSECURE": "true", - "IDP_LDAP_BIND_DN": "uid=idp,ou=sysusers,o=libregraph-idm", - "IDP_LDAP_BIND_PASSWORD": "idp", - "IDP_LDAP_BASE_DN": "ou=users,o=libregraph-idm", + "LDAP_GROUP_SCHEMA_MEMBER": "cn", + "LDAP_GROUP_OBJECTCLASS": "posixGroup", + "IDP_LDAP_BIND_DN": "cn=admin,dc=ocis,dc=test", + "LDAP_CACERT": "/root/.ocis/ldap/ldap.crt", + "IDP_LDAP_BIND_PASSWORD": "admin", "IDP_LDAP_LOGIN_ATTRIBUTE": "uid", - "PROXY_ACCOUNT_BACKEND_TYPE": "cs3", - "PROXY_ENABLE_BASIC_AUTH": "true", - "LDAP_BIND_DN": "uid=reva,ou=sysusers,o=libregraph-idm", - "LDAP_BIND_PASSWORD": "reva", - "OCS_ACCOUNT_BACKEND_TYPE": "cs3", - "OCIS_RUN_EXTENSIONS": "settings,storage-metadata,graph,graph-explorer,ocs,store,thumbnails,web,webdav,storage-frontend,storage-gateway,storage-userprovider,storage-groupprovider,storage-authbasic,storage-authbearer,storage-authmachine,storage-users,storage-shares,storage-public-link,storage-appprovider,storage-sharing,proxy,idp,nats,idm,ocdav", - "OCIS_LOG_LEVEL": "error", + "PROXY_ACCOUNT_BACKEND_TYPE": "accounts", + "OCS_ACCOUNT_BACKEND_TYPE": "accounts", + "OCIS_RUN_EXTENSIONS": "settings,storage-metadata,graph,graph-explorer,ocs,store,thumbnails,web,webdav,storage-frontend,storage-gateway,storage-userprovider,storage-groupprovider,storage-authbasic,storage-authbearer,storage-authmachine,storage-users,storage-shares,storage-public-link,storage-appprovider,storage-sharing,proxy,idp,nats,accounts,glauth,ocdav", "OCIS_INSECURE": "true", + "PROXY_ENABLE_BASIC_AUTH": "true", + "IDP_INSECURE": "true", + "OCIS_LOG_LEVEL": "error", "OCIS_URL": "https://ocis-server:9200", + "ACCOUNTS_DEMO_USERS_AND_GROUPS": True, + "STORAGE_HOME_DRIVER": "%s" % (storage), + "STORAGE_USERS_DRIVER": "%s" % (storage), + "WEB_UI_CONFIG": "/drone/src/tests/config/drone/ocis-config.json", } + + # Pass in "default" accounts_hash_difficulty to not set this environment variable. + # That will allow OCIS to use whatever its built-in default is. + # Otherwise pass in a value from 4 to about 11 or 12 (default 4, for making regular tests fast) + # The high values cause lots of CPU to be used when hashing passwords, and really slow down the tests. + if (accounts_hash_difficulty != "default"): + environment["ACCOUNTS_HASH_DIFFICULTY"] = accounts_hash_difficulty + return [ { "name": "ocis-server", @@ -1672,16 +1682,16 @@ def ocisServerWithIdp(): "commands": [ "ocis/bin/ocis server", ], - "volumes": [stepVolumeOC10Tests], - "depends_on": [], + "volumes": volumes, + "depends_on": depends_on, }, { "name": "wait-for-ocis-server", - "image": OC_CI_WAIT_FOR, + "image": OC_CI_ALPINE, "commands": [ - "wait-for -it ocis-server:9200 -t 300", + "curl -k -u admin:admin --fail --retry-connrefused --retry 10 --retry-all-errors 'https://ocis-server:9200/graph/v1.0/users/ddc2004c-0977-11eb-9d3f-a793888cd0f8'", ], - "depends_on": [], + "depends_on": depends_on, }, ] From 4bb4f9fbb1bf2f66479223cd66b31f3f3e73e51e Mon Sep 17 00:00:00 2001 From: David Christofas Date: Wed, 27 Apr 2022 12:39:59 +0200 Subject: [PATCH 10/13] url encode the webdav URL in the graph API --- changelog/unreleased/urlencoding-graph-api.md | 6 ++++ extensions/graph/pkg/service/v0/driveitems.go | 34 +++++++++---------- 2 files changed, 23 insertions(+), 17 deletions(-) create mode 100644 changelog/unreleased/urlencoding-graph-api.md diff --git a/changelog/unreleased/urlencoding-graph-api.md b/changelog/unreleased/urlencoding-graph-api.md new file mode 100644 index 0000000000..5698451313 --- /dev/null +++ b/changelog/unreleased/urlencoding-graph-api.md @@ -0,0 +1,6 @@ +Bugfix: URL encode the webdav url in the graph API + +Fixed the webdav URL in the drives responses. Without encoding the URL could be broken by files with spaces in the file name. + +https://github.com/owncloud/ocis/pull/3597 +https://github.com/owncloud/ocis/issues/3538 diff --git a/extensions/graph/pkg/service/v0/driveitems.go b/extensions/graph/pkg/service/v0/driveitems.go index c449f99223..7b3e1beb0a 100644 --- a/extensions/graph/pkg/service/v0/driveitems.go +++ b/extensions/graph/pkg/service/v0/driveitems.go @@ -77,11 +77,11 @@ func (g Graph) GetRootDriveChildren(w http.ResponseWriter, r *http.Request) { render.JSON(w, r, &listResponse{Value: files}) } -func (g Graph) getDriveItem(ctx context.Context, root *storageprovider.ResourceId) (*libregraph.DriveItem, error) { +func (g Graph) getDriveItem(ctx context.Context, root storageprovider.ResourceId) (*libregraph.DriveItem, error) { client := g.GetGatewayClient() ref := &storageprovider.Reference{ - ResourceId: root, + ResourceId: &root, } res, err := client.Stat(ctx, &storageprovider.StatRequest{Ref: ref}) if err != nil { @@ -196,18 +196,16 @@ func cs3ResourceToRemoteItem(res *storageprovider.ResourceInfo) (*libregraph.Rem return remoteItem, nil } -func (g Graph) getPathForResource(ctx context.Context, ID *storageprovider.ResourceId) (*string, error) { +func (g Graph) getPathForResource(ctx context.Context, id storageprovider.ResourceId) (string, error) { client := g.GetGatewayClient() - var path *string - res, err := client.GetPath(ctx, &storageprovider.GetPathRequest{ResourceId: ID}) + res, err := client.GetPath(ctx, &storageprovider.GetPathRequest{ResourceId: &id}) if err != nil { - return nil, err + return "", err } if res.Status.Code != cs3rpc.Code_CODE_OK { - return nil, fmt.Errorf("could not stat %s: %s", ID, res.Status.Message) + return "", fmt.Errorf("could not stat %v: %s", id, res.Status.Message) } - path = &res.Path - return path, err + return res.Path, err } // GetExtendedSpaceProperties reads properties from the opaque and transforms them into driveItems @@ -221,7 +219,7 @@ func (g Graph) GetExtendedSpaceProperties(ctx context.Context, baseURL *url.URL, for _, itemName := range names { if itemID, ok := metadata[itemName]; ok { - spaceItem := g.getSpecialDriveItem(ctx, resourceid.OwnCloudResourceIDUnwrap(string(itemID.Value)), itemName, baseURL, space) + spaceItem := g.getSpecialDriveItem(ctx, *resourceid.OwnCloudResourceIDUnwrap(string(itemID.Value)), itemName, baseURL, space) if spaceItem != nil { spaceItems = append(spaceItems, *spaceItem) } @@ -230,24 +228,26 @@ func (g Graph) GetExtendedSpaceProperties(ctx context.Context, baseURL *url.URL, return spaceItems } -func (g Graph) getSpecialDriveItem(ctx context.Context, ID *storageprovider.ResourceId, itemName string, baseURL *url.URL, space *storageprovider.StorageSpace) *libregraph.DriveItem { +func (g Graph) getSpecialDriveItem(ctx context.Context, id storageprovider.ResourceId, itemName string, baseURL *url.URL, space *storageprovider.StorageSpace) *libregraph.DriveItem { var spaceItem *libregraph.DriveItem - if ID == nil { + if id.StorageId == "" && id.OpaqueId == "" { return nil } - spaceItem, err := g.getDriveItem(ctx, ID) + spaceItem, err := g.getDriveItem(ctx, id) if err != nil { - g.logger.Error().Err(err).Str("ID", ID.OpaqueId).Msg("Could not get readme Item") + g.logger.Error().Err(err).Str("ID", id.OpaqueId).Msg("Could not get readme Item") return nil } - itemPath, err := g.getPathForResource(ctx, ID) + itemPath, err := g.getPathForResource(ctx, id) if err != nil { - g.logger.Error().Err(err).Str("ID", ID.OpaqueId).Msg("Could not get readme path") + g.logger.Error().Err(err).Str("ID", id.OpaqueId).Msg("Could not get readme path") return nil } spaceItem.SpecialFolder = &libregraph.SpecialFolder{Name: libregraph.PtrString(itemName)} - spaceItem.WebDavUrl = libregraph.PtrString(baseURL.String() + path.Join(space.Id.OpaqueId, *itemPath)) + webdavURL := *baseURL + webdavURL.Path = path.Join(webdavURL.Path, space.Id.OpaqueId, itemPath) + spaceItem.WebDavUrl = libregraph.PtrString(webdavURL.String()) return spaceItem } From 407399dd910a4bf92ad68b04a1b288b95827482d Mon Sep 17 00:00:00 2001 From: David Christofas Date: Wed, 27 Apr 2022 11:22:56 +0000 Subject: [PATCH 11/13] Automated changelog update [skip ci] --- CHANGELOG.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 882a770aaa..312e96765d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ The following sections list the changes for unreleased. ## Summary * Bugfix - Return proper errors when ocs/cloud/users is using the cs3 backend: [#3483](https://github.com/owncloud/ocis/issues/3483) +* Bugfix - URL encode the webdav url in the graph API: [#3597](https://github.com/owncloud/ocis/pull/3597) * Change - Load configuration files just from one directory: [#3587](https://github.com/owncloud/ocis/pull/3587) * Enhancement - Add capability for public link single file edit: [#6787](https://github.com/owncloud/web/pull/6787) * Enhancement - Update linkshare capabilities: [#3579](https://github.com/owncloud/ocis/pull/3579) @@ -21,6 +22,14 @@ The following sections list the changes for unreleased. https://github.com/owncloud/ocis/issues/3483 +* Bugfix - URL encode the webdav url in the graph API: [#3597](https://github.com/owncloud/ocis/pull/3597) + + Fixed the webdav URL in the drives responses. Without encoding the URL could be broken by files + with spaces in the file name. + + https://github.com/owncloud/ocis/issues/3538 + https://github.com/owncloud/ocis/pull/3597 + * Change - Load configuration files just from one directory: [#3587](https://github.com/owncloud/ocis/pull/3587) We've changed the configuration file loading behavior and are now only loading configuration From 6efdede2b80b7654648c548182bb472525982788 Mon Sep 17 00:00:00 2001 From: Michael Barz Date: Wed, 27 Apr 2022 14:44:08 +0200 Subject: [PATCH 12/13] update reva for caching fix --- changelog/unreleased/update-reva.md | 1 + go.mod | 2 +- go.sum | 2 ++ 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/changelog/unreleased/update-reva.md b/changelog/unreleased/update-reva.md index b1325d0128..b24c1837bc 100644 --- a/changelog/unreleased/update-reva.md +++ b/changelog/unreleased/update-reva.md @@ -6,3 +6,4 @@ Updated reva to version 2.x.x. This update includes: https://github.com/owncloud/ocis/pull/3552 https://github.com/owncloud/ocis/pull/3570 +https://github.com/owncloud/ocis/pull/3601 diff --git a/go.mod b/go.mod index 4a16182644..876204a748 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/blevesearch/bleve/v2 v2.3.2 github.com/coreos/go-oidc/v3 v3.1.0 github.com/cs3org/go-cs3apis v0.0.0-20220412090512-93c5918b4bde - github.com/cs3org/reva/v2 v2.0.0-20220425084830-0b734be7c6c7 + github.com/cs3org/reva/v2 v2.0.0-20220427123248-8cad2e542999 github.com/disintegration/imaging v1.6.2 github.com/glauth/glauth/v2 v2.0.0-20211021011345-ef3151c28733 github.com/go-chi/chi/v5 v5.0.7 diff --git a/go.sum b/go.sum index 8ca0b04f55..7d306e1448 100644 --- a/go.sum +++ b/go.sum @@ -320,6 +320,8 @@ github.com/cs3org/reva v1.18.0 h1:MbPS5ZAa8RzKcTxAVeSDdISB3XXqLIxqB03BTN5ReBY= github.com/cs3org/reva v1.18.0/go.mod h1:e5VDUDu4vVWIeVkZcW//n6UZzhGGMa+Tz/whCiX3N6o= github.com/cs3org/reva/v2 v2.0.0-20220425084830-0b734be7c6c7 h1:BTRw/tCFhlDplE6M9bnvlXb4VVz5wNbDN7VWdT1g1Q8= github.com/cs3org/reva/v2 v2.0.0-20220425084830-0b734be7c6c7/go.mod h1:2e/4HcIy54Mic3V7Ow0bz4n5dkZU0dHIZSWomFe5vng= +github.com/cs3org/reva/v2 v2.0.0-20220427123248-8cad2e542999 h1:82ug3hha8u8oaY1LZctupG6hIazq3K7Qi+qFBRtlORA= +github.com/cs3org/reva/v2 v2.0.0-20220427123248-8cad2e542999/go.mod h1:2e/4HcIy54Mic3V7Ow0bz4n5dkZU0dHIZSWomFe5vng= github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8 h1:Z9lwXumT5ACSmJ7WGnFl+OMLLjpz5uR2fyz7dC255FI= github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8/go.mod h1:4abs/jPXcmJzYoYGF91JF9Uq9s/KL5n1jvFDix8KcqY= github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4= From e5b98095b85a18147ebf6b8c050a341e79071db5 Mon Sep 17 00:00:00 2001 From: Michael Barz Date: Wed, 27 Apr 2022 13:39:34 +0000 Subject: [PATCH 13/13] Automated changelog update [skip ci] --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 312e96765d..38c94a2ca2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,7 @@ The following sections list the changes for unreleased. https://github.com/owncloud/ocis/pull/3552 https://github.com/owncloud/ocis/pull/3570 + https://github.com/owncloud/ocis/pull/3601 # Changelog for [1.20.0] (2022-04-13) The following sections list the changes for 1.20.0.