Commit Graph

87 Commits

Author SHA1 Message Date
André Duffeck
f033182e09 Log search durations 2023-05-24 13:12:41 +02:00
André Duffeck
cfb0152372 Fix boolean queries 2023-05-24 12:01:35 +02:00
André Duffeck
b1bfb93230 Fix test expectations 2023-05-24 12:01:35 +02:00
André Duffeck
e8c9d16576 Do not choke on : searches
Fixes #4744
2023-05-24 12:01:35 +02:00
André Duffeck
9fe1225b53 Make field base search case insensitive
Fixes part of #6366
2023-05-24 12:01:35 +02:00
André Duffeck
6275b7b124 Do not include full-text and tag data in the composite field
This commit removes the full-text and tag data from the composite field
("_all") which can be used for default term queries which we don't do
anyways.
2023-05-19 16:13:30 +02:00
André Duffeck
55a6716d49 Trigger a space reindex when the space has been renamed
Fixes #6289 (in combination with https://github.com/cs3org/reva/pull/3889)
2023-05-15 16:21:53 +02:00
André Duffeck
e9d4e07a08 constify the "trashed" space state 2023-05-12 10:37:43 +02:00
André Duffeck
82590632b6 Always use UTC for doc timestamps 2023-05-12 10:37:43 +02:00
André Duffeck
e01ef9b8c6 Search spaces concurrently 2023-05-12 10:37:43 +02:00
André Duffeck
a9b139af97 Do not consider disabled spaces during search
Fixes #6201
2023-05-12 10:37:43 +02:00
André Duffeck
63b2ac2368 Do not use prefix queries which gets slow with big indexes
Co-authored-by: Florian Schade <f.schade@icloud.com>
2023-05-12 10:37:43 +02:00
André Duffeck
ca637de017 Fix detecting resources that haven't changed when reindexing spaces
This is a regression introduced with the experimental search backport.
2023-04-19 17:44:16 +02:00
mmattel
8ed34926ef [docs-only] Correct the removal version according semver to 4.0.0 2023-04-19 10:16:06 +02:00
Daniel Swärd
46e50b3a70 Reapply name change, without adding files by mistake. 2023-04-05 13:26:29 +02:00
Daniel Swärd
a693a18b99 Revert "Change new name to address PR comment."
This reverts commit 8911ca2e98.
2023-04-05 13:25:04 +02:00
Daniel Swärd
8911ca2e98 Change new name to address PR comment. 2023-04-05 12:26:56 +02:00
Daniel Swärd
afbfd2d6b2 Deprecating and renaming many environment variables
Fixes #5767
Fixes #5905
Fixes #5532
2023-04-05 11:05:48 +02:00
Roman Perekhod
fcfb2ab9fa Introduced global OCIS_EVENTS_ENDPOINT and OCIS_EVENTS_CLUSTER #5901 2023-04-04 15:43:03 +02:00
Dennis Sieben
e28603e633 - fixed typos 2023-03-27 08:54:51 +02:00
David Christofas
924c2ca346 add missing comments 2023-03-06 19:01:16 +01:00
David Christofas
63fa35fa50 reduce duplication in configuration code 2023-03-06 16:52:44 +01:00
Jörn Friedrich Dreyer
9b00f02383 bump reva to ec27f5f8feb3
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-02-15 15:26:07 +00:00
mmattel
210cd54817 fix some envvar descriptions 2023-02-03 09:40:35 +01:00
Daniël Franke
d2076941f4 Add tests.
Also update mocks with mockery.
2023-01-23 14:38:12 +01:00
Florian Schade
1afd7c2a4b use event executant to index tags (#5405) 2023-01-13 17:31:34 +01:00
Jörn Friedrich Dreyer
3aa864aecc Remove Staticcheck warnings (#5394)
* remove deprecated ioutil imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove duplicate imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove unused values

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove some unused types

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove deprecated ioutil import

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-01-13 15:29:14 +01:00
jkoberg
837eff9017 adjust for tags pkg
Signed-off-by: jkoberg <jkoberg@owncloud.com>

tmp

Signed-off-by: jkoberg <jkoberg@owncloud.com>
2022-12-23 10:55:57 +01:00
jkoberg
b29ffeb3a9 use stream pkg instead server pkg
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2022-12-20 09:53:11 +01:00
Florian Schade
1db03dd512 [full-ci] experimental tags backport (#5227)
* add tags to search service resource
add tags getTags, AssignTags and UnassignTags endpoint to graph
use and prefer search event spaceOwner over executant
add tags to search report response
update libre graph api
update reva

Co-authored-by: David Christofas <dchristofas@owncloud.com>
2022-12-19 15:44:02 +01:00
Florian Schade
2404eff48e [full-ci] experimental search backport (#5221)
* experimental search backport
fix basic extractor resource name
move escapeQuery regex into global variable
minor pr review changes
rename DebounceDuration env variable
add document title and content when rebuilding bleve resource

Co-authored-by: David Christofas <dchristofas@owncloud.com>
2022-12-13 14:22:41 +01:00
Jörn Friedrich Dreyer
b2b4f9893c align revactx package import (#5206)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-12-08 14:45:30 +01:00
Christian Richter
080a37784f Fix tests
Co-authored-by: André Duffeck <aduffeck@firondu.de>
Signed-off-by: Christian Richter <crichter@owncloud.com>
2022-11-24 15:11:38 +01:00
Christian Richter
724cf40c0e Add traces to search + indexer
Co-authored-by: André Duffeck <aduffeck@firondu.de?
Signed-off-by: Christian Richter <crichter@owncloud.com>
2022-11-24 12:38:09 +01:00
Andre Duffeck
60b979d1d4 Find spaces (#5044)
* Use the node name for the name field in the index instead of the path

* Add changelog

* Fix tests

* Adapt expected failures
2022-11-15 16:52:55 +01:00
Willy Kloucek
d74b879d4f update tooling (#5038)
* update tooling

* update nodejs version
2022-11-11 15:01:45 +01:00
Andre Duffeck
a0762e248e Filter hidden files (#5018)
* Filter hidden files from search results

* Add changelog

* Do not filter hidden files by default

* Set the hidden fields when converting fields to an entity

* Fix test
2022-11-10 17:05:43 +01:00
Andre Duffeck
925790d64b Dont reindex twice (#5001)
* Only reindex a space once at a time

* Add changelog
2022-11-08 15:20:38 +01:00
Jörn Friedrich Dreyer
0f7dba53fb use min tls 1.2 (#4969)
* use min tls 1.2

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-11-03 16:08:56 +01:00
André Duffeck
cfb4f45f8d Fix indentation 2022-11-03 13:22:44 +01:00
André Duffeck
12bdf29ac4 Change the default reindex debounce duration to 1s.
Also make it configurable using an env var.
2022-11-03 13:15:54 +01:00
Ralf Haferkamp
ee974afebf [full-ci] Introduce TLS Settings for go-micro based grpc services and clients (#4901)
* Introduce TLS Settings for go-micro based grpc services and clients

TLS for the services can be configure by setting the OCIS_MICRO_GRPC_TLS_ENABLED"
"OCIS_MICRO_GRPC_TLS_CERTIFICATE" and "OCIS_MICRO_GRPC_TLS_KEY"
enviroment variables.

TLS for the clients can configured by setting the "OCIS_MICRO_GRPC_CLIENT_TLS_MODE"
and "OCIS_MICRO_GRPC_CLIENT_TLS_CACERT" variables.

By default TLS is disabled.

Co-authored-by: Martin <github@diemattels.at>

* Unify TLS configuration for all grpc services

All grpc service (whether they're based on reva) or go-micro use the
same set of config vars now.

TLS for the services can be configure by setting the OCIS_GRPC_TLS_ENABLED,
OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY enviroment variables.

TLS for the clients can configured by setting the OCIS_GRPC_CLIENT_TLS_MODE
and OCIS_MICRO_GRPC_CLIENT_TLS_CACERT variables.

There are no individual per service config vars currently. If really
needed, per service tls configurations can be specified via config file.

Co-authored-by: Martin <github@diemattels.at>

Co-authored-by: Martin <github@diemattels.at>
2022-11-03 10:17:08 +01:00
Ralf Haferkamp
3d57f5cc21 Introduce TLS Settings for all reva grpc services and clients 2022-10-25 11:50:08 +02:00
Ralf Haferkamp
e373e48383 Get rid of duplicated Reva config struct
Consolidate all services to use the Reva config struct for the shared package.
This works because all services (except 'notifications', 'thumbnails' and
'webdav') where using the same config keys and environment variables for
setting the reva gateway.
2022-10-25 11:50:08 +02:00
Michael Barz
ec764ea5b1 improve the description of the help texts 2022-10-21 12:50:03 +02:00
David Christofas
45b92fc56f add config option to enable or disable TLS for nats 2022-10-21 12:40:21 +02:00
David Christofas
bbcbf69283 fix shareroot path in REPORT responses 2022-10-20 10:26:10 +02:00
Andre Duffeck
5e220920f0 [full-ci] Rescan spaces (#4777)
* Increase the timeout for indexing spaces

* Allow for making queries that are not constrained to a RootID

* Use nano precision for the Mtime

* Add a SpaceDebouncer

The debouncer can be used to delay operations on spaces until things
have settled down to avoid doing the same operation multiple times.

* Do not index subtrees until they have changed (i.e. the mtime differs)

* Also pass a user to the space debouncer func

* Trigger a rescan of the according space when an event is received

* Improve wording/logging

* Add changelog

* Get rid of superfluous GetUserByClaim call

* Fix tests

* Fix reindexing triggered by shares by using the event's SpaceOwner

* Bump reva to pull in the space owner events changes

* Fix changelog

* Fix linter issues

* Fall back to the executant if no owner was received from the event

* Bump reva and go-cs3apis

* Fix go.sum

* Bump core

* Adapt expected failures

* Tweak debounce settings
2022-10-14 17:03:05 +02:00
mmattel
6edb44fa83 Improving the description for several environemt variables 2022-10-13 12:07:46 +02:00
David Christofas
4623b6c8e7 Nats tls (#4781)
* use tls for nats connections

* add config options for nats client tls config

* add nats tls config to CI

* add function to create a certpool

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* configure nats clients in reva to use tls
2022-10-12 14:56:47 +02:00