Merge remote-tracking branch 'upstream/master' into search

CHANGELOG.md

@@ -6,15 +6,25 @@ The following sections list the changes for unreleased.
 
 ## Summary
 
+* Bugfix - Idp: Check if CA certificate if present: [#3623](https://github.com/owncloud/ocis/issues/3623)
 * Bugfix - Return proper errors when ocs/cloud/users is using the cs3 backend: [#3483](https://github.com/owncloud/ocis/issues/3483)
 * Bugfix - URL encode the webdav url in the graph API: [#3597](https://github.com/owncloud/ocis/pull/3597)
 * Change - Load configuration files just from one directory: [#3587](https://github.com/owncloud/ocis/pull/3587)
+* Change - Switched default configuration to use libregraph/idm: [#3331](https://github.com/owncloud/ocis/pull/3331)
 * Enhancement - Add capability for public link single file edit: [#6787](https://github.com/owncloud/web/pull/6787)
 * Enhancement - Update linkshare capabilities: [#3579](https://github.com/owncloud/ocis/pull/3579)
 * Enhancement - Update reva to v2.x.x: [#3552](https://github.com/owncloud/ocis/pull/3552)
 
 ## Details
 
+* Bugfix - Idp: Check if CA certificate if present: [#3623](https://github.com/owncloud/ocis/issues/3623)
+
+   Upon first start with the default configurtation the idm service creates a server
+   certificate, that might not be finished before the idp service is starting. Add a check to idp
+   similar to what the user, group, and auth-providers implement.
+
+   https://github.com/owncloud/ocis/issues/3623
+
 * Bugfix - Return proper errors when ocs/cloud/users is using the cs3 backend: [#3483](https://github.com/owncloud/ocis/issues/3483)
 
    The ocs API was just exiting with a fatal error on any update request, when configured for the cs3
@@ -42,6 +52,15 @@ The following sections list the changes for unreleased.
 
    https://github.com/owncloud/ocis/pull/3587
 
+* Change - Switched default configuration to use libregraph/idm: [#3331](https://github.com/owncloud/ocis/pull/3331)
+
+   We switched the default configuration of oCIS to use the "idm" service (based on
+   libregraph/idm) as the standard source for user and group information. The accounts and
+   glauth services are no longer enabled by default and will be removed with an upcoming release.
+
+   https://github.com/owncloud/ocis/pull/3331
+   https://github.com/owncloud/ocis/pull/3633
+
 * Enhancement - Add capability for public link single file edit: [#6787](https://github.com/owncloud/web/pull/6787)
 
    It is now possible to share a single file by link with edit permissions. Therefore we need a

changelog/unreleased/idp-cert-wait.md

@@ -0,0 +1,8 @@
+Bugfix: idp: Check if CA certificate if present
+
+Upon first start with the default configurtation the idm service creates
+a server certificate, that might not be finished before the idp service
+is starting. Add a check to idp similar to what the user, group, and
+auth-providers implement.
+
+https://github.com/owncloud/ocis/issues/3623

changelog/unreleased/libregraph-idm-switch.md

@@ -0,0 +1,9 @@
+Change: Switched default configuration to use libregraph/idm
+
+We switched the default configuration of oCIS to use the "idm" service (based
+on libregraph/idm) as the standard source for user and group information. The
+accounts and glauth services are no longer enabled by default and will be
+removed with an upcoming release.
+
+https://github.com/owncloud/ocis/pull/3331
+https://github.com/owncloud/ocis/pull/3633

docs/extensions/idm/_index.md

@@ -10,6 +10,20 @@ geekdocCollapseSection: true
 
 ## Abstract
 
+The IDM service provides a minimal LDAP Service (based on https://github.com/libregraph/idm) for oCIS. It is started as part of
+the default configuration and serves as a central place for storing user and group information.
+
+It is mainly targeted at small oCIS installations. For larger setups it is recommended to replace IDM with a "real" LDAP server
+or to switch to an external Identity Management Solution.
+
+IDM listens on port 9325 by default. In the default configuration it only accepts TLS protected connections (LDAPS). The BaseDN
+of the LDAP tree is `o=libregraph-idm`. IDM gives LDAP write permissions to a single user 
+(DN: `uid=libregraph,ou=sysusers,o=libregraph-idm`). Any other authenticated user has read-only access. IDM stores its data in a
+[boltdb](https://github.com/etcd-io/bbolt) file `idm/ocis.boltdb` inside the oCIS base data directory.
+
+Note: IDM is limited in its functionality. It only supports a subset of the LDAP operations (namely BIND, SEARCH, ADD, MODIFY, DELETE).
+Also IDM currently does not do any schema verification (e.g. structural vs. auxiliary object classes, require and option attributes,
+syntax checks, ...). So it's not meant as a general purpose LDAP server.
 
 ## Table of Contents
 

docs/extensions/idm/configuration_hints.md

@@ -0,0 +1,49 @@
+---
+title: Configuration Hints
+date: 2022-04-27:00:00+00:00
+weight: 20
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/idm
+geekdocFilePath: configuration_hints.md
+geekdocCollapseSection: true
+---
+
+## TLS Server Certificates
+By default IDM generates a self-signed certificate and key on first startup to be
+able to provide TLS protected services. The certificate is stored in
+`idm/ldap.crt` inside the oCIS base data directory. The key is in
+`idm/ldap.key` in the same directory. You can use a custom server
+certificate by setting the `IDM_LDAPS_CERT` and `IDM_LDAPS_KEY`.
+
+## Default / Demo Users
+On startup IDM creates a set of default services users that are needed
+internally to provide access to IDM to other oCIS services. These users are stored
+in a separate subtree. The base DN of that subtree is:
+`ou=sysusers,o=libregraph-idm`. The service users are:
+
+* `uid=libregraph,ou=sysusers,o=libregraph-idm`: This is the only user with write
+  access to the LDAP tree. It is used by the Graph service to look up, create, delete and
+  modify users and groups.
+* `uid=idp,ou=sysusers,o=libregraph-idm`: This user is used by the IDP service to
+  perform user lookups for authentication.
+* `uid=reva,ou=sysusers,o=libregraph-idm`: This user is used by the "reva" services
+  `user`, `group` and `auth-basic`.
+
+IDM is also able to create [Demo Users](../../../ocis/getting-started/demo-users)
+upon startup. 
+
+## Access via LDAP command line tools
+For testing purposes it is sometimes helpful to query IDM using the ldap
+command line clients. To e.g. list all users, this command can be used:
+
+```
+ldapsearch -x -H ldaps://127.0.0.1:9235 -x -D uid=libregraph,ou=sysusers,o=libregraph-idm -w idm -b o=libregraph-idm objectclass=inetorgperson
+```
+
+When using the default configuration with the self-signed server certificate,
+you might need to switch off the certificate validation using the `LDAPTL_REQCERT` env
+variable:
+
+```
+LDAPTLS_REQCERT=never ldapsearch -x -H ldaps://127.0.0.1:9235 -x -D uid=libregraph,ou=sysusers,o=libregraph-idm -w idm -b o=libregraph-idm objectclass=inetorgperson
+```

docs/extensions/idm/setup.md

@@ -1,50 +0,0 @@
----
-title: Service Setup
-date: 2022-03-22T00:00:00+00:00
-weight: 20
-geekdocRepo: https://github.com/owncloud/ocis
-geekdocEditPath: edit/master/docs/extensions/idm
-geekdocFilePath: setup.md
-geekdocCollapseSection: true
----
-
-{{< toc >}}
-
-## Using ocis with libregraph/idm
-
-Currently, oCIS still runs the accounts and glauth services to manage users. Until the default is switched
-to libregraph/idm, oCIS has to be started with a custom configuration in order to use libregraph/idm as
-the users and groups backend (this setup also disables the glauth and accounts service):
-
-
-```
-export GRAPH_IDENTITY_BACKEND=ldap
-export LDAP_URI=ldaps://localhost:9235
-export LDAP_INSECURE="true"
-export LDAP_USER_BASE_DN="ou=users,o=libregraph-idm"
-export LDAP_USER_SCHEMA_ID="ownclouduuid"
-export LDAP_USER_SCHEMA_MAIL="mail"
-export LDAP_USER_SCHEMA_USERNAME="uid"
-export LDAP_USER_OBJECTCLASS="inetOrgPerson"
-export LDAP_GROUP_BASE_DN="ou=groups,o=libregraph-idm"
-export LDAP_GROUP_SCHEMA_ID="ownclouduuid"
-export LDAP_GROUP_SCHEMA_MAIL="mail"
-export LDAP_GROUP_SCHEMA_GROUPNAME="cn"
-export LDAP_GROUP_SCHEMA_MEMBER="member"
-export LDAP_GROUP_OBJECTCLASS="groupOfNames"
-export GRAPH_LDAP_BIND_DN="uid=libregraph,ou=sysusers,o=libregraph-idm"
-export GRAPH_LDAP_BIND_PASSWORD=idm
-export GRAPH_LDAP_SERVER_WRITE_ENABLED="true"
-export IDP_INSECURE="true"
-export IDP_LDAP_BIND_DN="uid=idp,ou=sysusers,o=libregraph-idm"
-export IDP_LDAP_BIND_PASSWORD="idp"
-export IDP_LDAP_LOGIN_ATTRIBUTE=uid
-export PROXY_ACCOUNT_BACKEND_TYPE=cs3
-export OCS_ACCOUNT_BACKEND_TYPE=cs3
-export STORAGE_LDAP_BIND_DN="uid=reva,ou=sysusers,o=libregraph-idm"
-export STORAGE_LDAP_BIND_PASSWORD=reva
-export OCIS_RUN_EXTENSIONS=settings,storage-metadata,graph,graph-explorer,ocs,store,thumbnails,web,webdav,storage-frontend,storage-gateway,storage-userprovider,storage-groupprovider,storage-authbasic,storage-authbearer,storage-authmachine,storage-users,storage-shares,storage-public-link,storage-appprovider,storage-sharing,proxy,idp,nats,idm,ocdav
-export OCIS_INSECURE=true
-bin/ocis server
-```
-

docs/ocis/getting-started/demo-users.md

@@ -11,7 +11,7 @@ oCIS has the option to create demo users during the first startup. These enable
 
 {{< hint info >}}
 To create the demo users, run the initial setup step with an additional environment variable.
-`ACCOUNTS_DEMO_USERS_AND_GROUPS=true ./bin/ocis server` will generate the demo users listed in the table below. By default, it only generates the admin and one user for IDP and Reva respectively.
+`IDM_CREATE_DEMO_USERS=true ./bin/ocis server` will generate the demo users listed in the table below. By default, it only generates the admin and one user for IDP and Reva respectively.
 {{< /hint >}}
 
 Following users are available in the demo set:

extensions/idp/pkg/service/v0/service.go

@@ -22,6 +22,7 @@ import (
 	"github.com/owncloud/ocis/extensions/idp/pkg/assets"
 	"github.com/owncloud/ocis/extensions/idp/pkg/config"
 	"github.com/owncloud/ocis/extensions/idp/pkg/middleware"
+	"github.com/owncloud/ocis/ocis-pkg/ldap"
 	"github.com/owncloud/ocis/ocis-pkg/log"
 	"stash.kopano.io/kgol/rndm"
 )
@@ -41,6 +42,14 @@ func NewService(opts ...Option) Service {
 		assets.Config(options.Config),
 	)
 
+	if err := ldap.WaitForCA(options.Logger, options.Config.IDP.Insecure, options.Config.Ldap.TLSCACert); err != nil {
+		logger.Fatal().Err(err).Msg("The configured LDAP CA cert does not exist")
+	}
+	if options.Config.IDP.Insecure {
+		// force CACert to be empty to avoid lico try to load it
+		options.Config.Ldap.TLSCACert = ""
+	}
+
 	if err := initLicoInternalEnvVars(&options.Config.Ldap); err != nil {
 		logger.Fatal().Err(err).Msg("could not initialize env vars")
 	}
@@ -56,7 +65,6 @@ func NewService(opts ...Option) Service {
 
 	// https://play.golang.org/p/Mh8AVJCd593
 	idpSettings := bootstrap.Settings(options.Config.IDP)
-
 	bs, err := bootstrap.Boot(ctx, &idpSettings, &licoconfig.Config{
 		Logger: log.LogrusWrap(logger),
 	})

extensions/web/pkg/service/v0/service.go

@@ -79,10 +79,6 @@ func (p Web) getPayload() (payload []byte, err error) {
 					ID:   "settings",
 					Path: "/settings.js",
 				},
-				{
-					ID:   "accounts",
-					Path: "/accounts.js",
-				},
 			}
 		}