mirror/opencloud
1
0
Fork 0
mirror of https://github.com/opencloud-eu/opencloud.git synced 2026-02-24 22:49:06 -06:00
Code Issues Packages Projects Releases Wiki Activity

[full-ci] experimental tags backport (#5227)

Browse Source 
* add tags to search service resource
add tags getTags, AssignTags and UnassignTags endpoint to graph
use and prefer search event spaceOwner over executant
add tags to search report response
update libre graph api
update reva

Co-authored-by: David Christofas <dchristofas@owncloud.com>
This commit is contained in:
Florian Schade
2022-12-19 15:44:02 +01:00
committed by GitHub
parent 2443744599
commit 1db03dd512
30 changed files with 434 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
changelog/unreleased/add-global-vars-extrator.md
View File

@@ -3,5 +3,5 @@ Enhancement: add global env variable extractor
We have added a little tool that will extract global env vars, that are loaded
only through os.Getenv for documentation purposes
https://github.com/owncloud/ocis/issues/4916
https://github.com/owncloud/ocis/pull/5164
https://github.com/owncloud/ocis/issues/4916

5
changelog/unreleased/enhancement-search.md
View File

@@ -1,8 +1,9 @@
Bugfix: Enhancement search
Enhancement: extended search
Provides multiple enhancement to the current search implementation.
Provides multiple enhancement to the search implementation.
* content extraction, search now supports apache tika to extract resource contents.
* search engine, underlying search engine is swappable now.
* event consumers, the number of event consumers can now be set, which improves the speed of the individual tasks
https://github.com/owncloud/ocis/pull/5221
https://github.com/owncloud/ocis/issues/5184

8
changelog/unreleased/enhancement-tags.md Normal file
View File

@@ -0,0 +1,8 @@
Enhancement: resource tags
We've added the ability to tag resources via the graph api.
Tags can be added (put request) and removed (delete request) from a resource,
a list of available tags can also be requested by sending a get request to the graph endpoint.
https://github.com/owncloud/ocis/pull/5227
https://github.com/owncloud/ocis/issues/5184

4
go.mod
View File

@@ -11,7 +11,7 @@ require (
github.com/blevesearch/bleve/v2 v2.3.5
github.com/coreos/go-oidc/v3 v3.4.0
github.com/cs3org/go-cs3apis v0.0.0-20221012090518-ef2996678965
github.com/cs3org/reva/v2 v2.12.1-0.20221214090401-47e0591bb902
github.com/cs3org/reva/v2 v2.12.1-0.20221215082748-05a97d63f308
github.com/disintegration/imaging v1.6.2
github.com/ggwhite/go-masker v1.0.9
github.com/go-chi/chi/v5 v5.0.7
@@ -54,7 +54,7 @@ require (
github.com/onsi/ginkgo/v2 v2.5.0
github.com/onsi/gomega v1.24.1
github.com/orcaman/concurrent-map v1.0.0
github.com/owncloud/libre-graph-api-go v1.0.0
github.com/owncloud/libre-graph-api-go v1.0.1-0.20221216081114-57ab57ed98b0
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.14.0
github.com/rs/zerolog v1.28.0

8
go.sum
View File

@@ -343,8 +343,8 @@ github.com/crewjam/saml v0.4.6 h1:XCUFPkQSJLvzyl4cW9OvpWUbRf0gE7VUpU8ZnilbeM4=
github.com/crewjam/saml v0.4.6/go.mod h1:ZBOXnNPFzB3CgOkRm7Nd6IVdkG+l/wF+0ZXLqD96t1A=
github.com/cs3org/go-cs3apis v0.0.0-20221012090518-ef2996678965 h1:y4n2j68LLnvac+zw/al8MfPgO5aQiIwLmHM/JzYN8AM=
github.com/cs3org/go-cs3apis v0.0.0-20221012090518-ef2996678965/go.mod h1:UXha4TguuB52H14EMoSsCqDj7k8a/t7g4gVP+bgY5LY=
github.com/cs3org/reva/v2 v2.12.1-0.20221214090401-47e0591bb902 h1:r8K9y0RMFXjQlrbx17iQziWYhNyAYmh70ixaXbQHsHY=
github.com/cs3org/reva/v2 v2.12.1-0.20221214090401-47e0591bb902/go.mod h1:GpocVB1w6yxeSr1VBsO9jztmt1SyNC4lCwudLwDzxHQ=
github.com/cs3org/reva/v2 v2.12.1-0.20221215082748-05a97d63f308 h1:9MePXhcJ39iQGnI7ojNlqrFrPv8B+dCa4EnlGtQEbn4=
github.com/cs3org/reva/v2 v2.12.1-0.20221215082748-05a97d63f308/go.mod h1:GpocVB1w6yxeSr1VBsO9jztmt1SyNC4lCwudLwDzxHQ=
github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8 h1:Z9lwXumT5ACSmJ7WGnFl+OMLLjpz5uR2fyz7dC255FI=
github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8/go.mod h1:4abs/jPXcmJzYoYGF91JF9Uq9s/KL5n1jvFDix8KcqY=
github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4=
@@ -1054,8 +1054,8 @@ github.com/oracle/oci-go-sdk v24.3.0+incompatible/go.mod h1:VQb79nF8Z2cwLkLS35uk
github.com/orcaman/concurrent-map v1.0.0 h1:I/2A2XPCb4IuQWcQhBhSwGfiuybl/J0ev9HDbW65HOY=
github.com/orcaman/concurrent-map v1.0.0/go.mod h1:Lu3tH6HLW3feq74c2GC+jIMS/K2CFcDWnWD9XkenwhI=
github.com/ovh/go-ovh v1.1.0/go.mod h1:AxitLZ5HBRPyUd+Zl60Ajaag+rNTdVXWIkzfrVuTXWA=
github.com/owncloud/libre-graph-api-go v1.0.0 h1:3uu5Thr9uxRkF6yak91TKydtvvvG8j9LbfirYIh+hcM=
github.com/owncloud/libre-graph-api-go v1.0.0/go.mod h1:579sFrPP7aP24LZXGPopLfvE+hAka/2DYHk0+Ij+w+U=
github.com/owncloud/libre-graph-api-go v1.0.1-0.20221216081114-57ab57ed98b0 h1:47N8o/5MmQ5781dF44fk1lQtyFmTAvu6/5hPG3rM1TQ=
github.com/owncloud/libre-graph-api-go v1.0.1-0.20221216081114-57ab57ed98b0/go.mod h1:579sFrPP7aP24LZXGPopLfvE+hAka/2DYHk0+Ij+w+U=
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c h1:rp5dCmg/yLR3mgFuSOe4oEnDDmGLROTvMragMUXpTQw=
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c/go.mod h1:X07ZCGwUbLaax7L0S3Tw4hpejzu63ZrrQiUe6W0hcy0=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=

35
protogen/gen/ocis/messages/search/v0/search.pb.go
View File

@@ -156,6 +156,7 @@ type Entity struct {
Deleted bool `protobuf:"varint,10,opt,name=deleted,proto3" json:"deleted,omitempty"`
ShareRootName string `protobuf:"bytes,11,opt,name=shareRootName,proto3" json:"shareRootName,omitempty"`
ParentId *ResourceID `protobuf:"bytes,12,opt,name=parent_id,json=parentId,proto3" json:"parent_id,omitempty"`
Tags []string `protobuf:"bytes,13,rep,name=tags,proto3" json:"tags,omitempty"`
}
func (x *Entity) Reset() {
@@ -274,6 +275,13 @@ func (x *Entity) GetParentId() *ResourceID {
return nil
}
func (x *Entity) GetTags() []string {
if x != nil {
return x.Tags
}
return nil
}
type Match struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -352,7 +360,7 @@ var file_ocis_messages_search_v0_search_proto_rawDesc = []byte{
0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76,
0x30, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x44, 0x52, 0x0a, 0x72, 0x65,
0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x70, 0x61, 0x74, 0x68,
0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x70, 0x61, 0x74, 0x68, 0x22, 0xce, 0x03, 0x0a,
0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x70, 0x61, 0x74, 0x68, 0x22, 0xe2, 0x03, 0x0a,
0x06, 0x45, 0x6e, 0x74, 0x69, 0x74, 0x79, 0x12, 0x34, 0x0a, 0x03, 0x72, 0x65, 0x66, 0x18, 0x01,
0x20, 0x01, 0x28, 0x0b, 0x32, 0x22, 0x2e, 0x6f, 0x63, 0x69, 0x73, 0x2e, 0x6d, 0x65, 0x73, 0x73,
0x61, 0x67, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x52,
@@ -381,18 +389,19 @@ var file_ocis_messages_search_v0_search_proto_rawDesc = []byte{
0x61, 0x72, 0x65, 0x6e, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x0c, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x23,
0x2e, 0x6f, 0x63, 0x69, 0x73, 0x2e, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x73, 0x2e, 0x73,
0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63,
0x65, 0x49, 0x44, 0x52, 0x08, 0x70, 0x61, 0x72, 0x65, 0x6e, 0x74, 0x49, 0x64, 0x22, 0x56, 0x0a,
0x05, 0x4d, 0x61, 0x74, 0x63, 0x68, 0x12, 0x37, 0x0a, 0x06, 0x65, 0x6e, 0x74, 0x69, 0x74, 0x79,
0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1f, 0x2e, 0x6f, 0x63, 0x69, 0x73, 0x2e, 0x6d, 0x65,
0x73, 0x73, 0x61, 0x67, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30,
0x2e, 0x45, 0x6e, 0x74, 0x69, 0x74, 0x79, 0x52, 0x06, 0x65, 0x6e, 0x74, 0x69, 0x74, 0x79, 0x12,
0x14, 0x0a, 0x05, 0x73, 0x63, 0x6f, 0x72, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x02, 0x52, 0x05,
0x73, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x42, 0x5a, 0x40, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e,
0x63, 0x6f, 0x6d, 0x2f, 0x6f, 0x77, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x6f, 0x63, 0x69,
0x73, 0x2f, 0x76, 0x32, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x67, 0x65, 0x6e, 0x2f, 0x67, 0x65,
0x6e, 0x2f, 0x6f, 0x63, 0x69, 0x73, 0x2f, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x73, 0x2f,
0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x76, 0x30, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x33,
0x65, 0x49, 0x44, 0x52, 0x08, 0x70, 0x61, 0x72, 0x65, 0x6e, 0x74, 0x49, 0x64, 0x12, 0x12, 0x0a,
0x04, 0x74, 0x61, 0x67, 0x73, 0x18, 0x0d, 0x20, 0x03, 0x28, 0x09, 0x52, 0x04, 0x74, 0x61, 0x67,
0x73, 0x22, 0x56, 0x0a, 0x05, 0x4d, 0x61, 0x74, 0x63, 0x68, 0x12, 0x37, 0x0a, 0x06, 0x65, 0x6e,
0x74, 0x69, 0x74, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1f, 0x2e, 0x6f, 0x63, 0x69,
0x73, 0x2e, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63,
0x68, 0x2e, 0x76, 0x30, 0x2e, 0x45, 0x6e, 0x74, 0x69, 0x74, 0x79, 0x52, 0x06, 0x65, 0x6e, 0x74,
0x69, 0x74, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x73, 0x63, 0x6f, 0x72, 0x65, 0x18, 0x02, 0x20, 0x01,
0x28, 0x02, 0x52, 0x05, 0x73, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x42, 0x5a, 0x40, 0x67, 0x69, 0x74,
0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x6f, 0x77, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64,
0x2f, 0x6f, 0x63, 0x69, 0x73, 0x2f, 0x76, 0x32, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x67, 0x65,
0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x2f, 0x6f, 0x63, 0x69, 0x73, 0x2f, 0x6d, 0x65, 0x73, 0x73, 0x61,
0x67, 0x65, 0x73, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x76, 0x30, 0x62, 0x06, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (

2
protogen/gen/ocis/messages/settings/v0/settings.pb.go
View File

@@ -1193,6 +1193,7 @@ type Value struct {
AccountUuid string `protobuf:"bytes,4,opt,name=account_uuid,json=accountUuid,proto3" json:"account_uuid,omitempty"`
Resource *Resource `protobuf:"bytes,5,opt,name=resource,proto3" json:"resource,omitempty"`
// Types that are assignable to Value:
//
// *Value_BoolValue
// *Value_IntValue
// *Value_StringValue
@@ -1383,6 +1384,7 @@ type ListOptionValue struct {
unknownFields protoimpl.UnknownFields
// Types that are assignable to Option:
//
// *ListOptionValue_StringValue
// *ListOptionValue_IntValue
Option isListOptionValue_Option `protobuf_oneof:"option"`

6
protogen/gen/ocis/services/search/v0/search.swagger.json
View File

@@ -198,6 +198,12 @@
},
"parentId": {
"$ref": "#/definitions/v0ResourceID"
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
}
}
},

1
protogen/proto/ocis/messages/search/v0/search.proto
View File

@@ -30,6 +30,7 @@ message Entity {
bool deleted = 10;
string shareRootName = 11;
ResourceID parent_id = 12;
repeated string tags = 13;
}
message Match {

2
services/graph/pkg/server/http/server.go
View File

@@ -17,6 +17,7 @@ import (
"github.com/owncloud/ocis/v2/ocis-pkg/service/grpc"
"github.com/owncloud/ocis/v2/ocis-pkg/service/http"
"github.com/owncloud/ocis/v2/ocis-pkg/version"
searchsvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/search/v0"
settingssvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/settings/v0"
graphMiddleware "github.com/owncloud/ocis/v2/services/graph/pkg/middleware"
svc "github.com/owncloud/ocis/v2/services/graph/pkg/service/v0"
@@ -129,6 +130,7 @@ func Server(opts ...Option) (http.Service, error) {
svc.WithRoleService(roleService),
svc.WithRequireAdminMiddleware(requireAdminMiddleware),
svc.WithGatewayClient(gatewayClient),
svc.WithSearchService(searchsvc.NewSearchProviderService("com.owncloud.api.search", grpc.DefaultClient())),
)
if handle == nil {

3
services/graph/pkg/service/v0/graph.go
View File

@@ -12,6 +12,7 @@ import (
"github.com/go-chi/chi/v5"
"github.com/jellydator/ttlcache/v2"
"github.com/owncloud/ocis/v2/ocis-pkg/log"
searchsvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/search/v0"
settingssvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/settings/v0"
"github.com/owncloud/ocis/v2/services/graph/pkg/config"
"github.com/owncloud/ocis/v2/services/graph/pkg/identity"
@@ -57,6 +58,7 @@ type GatewayClient interface {
// MUST return CODE_NOT_FOUND if the reference does not exist
// MUST return CODE_RESOURCE_EXHAUSTED on exceeded quota limits.
GetQuota(ctx context.Context, in *gateway.GetQuotaRequest, opts ...grpc.CallOption) (*provider.GetQuotaResponse, error)
SetArbitraryMetadata(ctx context.Context, request *provider.SetArbitraryMetadataRequest, opts ...grpc.CallOption) (*provider.SetArbitraryMetadataResponse, error)
}
// Publisher is the interface for events publisher
@@ -97,6 +99,7 @@ type Graph struct {
permissionsService Permissions
spacePropertiesCache *ttlcache.Cache
eventsPublisher events.Publisher
searchService searchsvc.SearchProviderService
}
// ServeHTTP implements the Service interface.

15
services/graph/pkg/service/v0/instrument.go
View File

@@ -133,3 +133,18 @@ func (i instrument) CreateDrive(w http.ResponseWriter, r *http.Request) {
func (i instrument) GetRootDriveChildren(w http.ResponseWriter, r *http.Request) {
i.next.GetRootDriveChildren(w, r)
}
// GetTags implements the Service interface.
func (i instrument) GetTags(w http.ResponseWriter, r *http.Request) {
i.next.GetTags(w, r)
}
// AssignTags implements the Service interface.
func (i instrument) AssignTags(w http.ResponseWriter, r *http.Request) {
i.next.AssignTags(w, r)
}
// UnassignTags implements the Service interface.
func (i instrument) UnassignTags(w http.ResponseWriter, r *http.Request) {
i.next.UnassignTags(w, r)
}

15
services/graph/pkg/service/v0/logging.go
View File

@@ -133,3 +133,18 @@ func (l logging) CreateDrive(w http.ResponseWriter, r *http.Request) {
func (l logging) GetRootDriveChildren(w http.ResponseWriter, r *http.Request) {
l.next.GetRootDriveChildren(w, r)
}
// GetTags implements the Service interface.
func (l logging) GetTags(w http.ResponseWriter, r *http.Request) {
l.next.GetTags(w, r)
}
// AssignTags implements the Service interface.
func (l logging) AssignTags(w http.ResponseWriter, r *http.Request) {
l.next.AssignTags(w, r)
}
// UnassignTags implements the Service interface.
func (l logging) UnassignTags(w http.ResponseWriter, r *http.Request) {
l.next.UnassignTags(w, r)
}

9
services/graph/pkg/service/v0/option.go
View File

@@ -6,6 +6,7 @@ import (
"github.com/cs3org/reva/v2/pkg/events"
"github.com/owncloud/ocis/v2/ocis-pkg/log"
"github.com/owncloud/ocis/v2/ocis-pkg/roles"
searchsvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/search/v0"
settingssvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/settings/v0"
"github.com/owncloud/ocis/v2/services/graph/pkg/config"
"github.com/owncloud/ocis/v2/services/graph/pkg/identity"
@@ -26,6 +27,7 @@ type Options struct {
PermissionService Permissions
RoleManager *roles.Manager
EventsPublisher events.Publisher
SearchService searchsvc.SearchProviderService
}
// newOptions initializes the available default options.
@@ -88,6 +90,13 @@ func WithRoleService(val RoleService) Option {
}
}
// WithRoleService provides a function to set the RoleService option.
func WithSearchService(val searchsvc.SearchProviderService) Option {
return func(o *Options) {
o.SearchService = val
}
}
// PermissionService provides a function to set the PermissionService option.
func PermissionService(val settingssvc.PermissionService) Option {
return func(o *Options) {

10
services/graph/pkg/service/v0/service.go
View File

@@ -53,6 +53,10 @@ type Service interface {
CreateDrive(w http.ResponseWriter, r *http.Request)
UpdateDrive(w http.ResponseWriter, r *http.Request)
DeleteDrive(w http.ResponseWriter, r *http.Request)
GetTags(w http.ResponseWriter, r *http.Request)
AssignTags(w http.ResponseWriter, r *http.Request)
UnassignTags(w http.ResponseWriter, r *http.Request)
}
// NewService returns a service implementation for Service.
@@ -69,6 +73,7 @@ func NewService(opts ...Option) Service {
spacePropertiesCache: ttlcache.NewCache(),
eventsPublisher: options.EventsPublisher,
gatewayClient: options.GatewayClient,
searchService: options.SearchService,
}
if options.IdentityBackend == nil {
@@ -167,6 +172,11 @@ func NewService(opts ...Option) Service {
m.Route(options.Config.HTTP.Root, func(r chi.Router) {
r.Use(middleware.StripSlashes)
r.Route("/v1.0", func(r chi.Router) {
r.Route("/extensions/org.libregraph", func(r chi.Router) {
r.Get("/tags", svc.GetTags)
r.Put("/tags", svc.AssignTags)
r.Delete("/tags", svc.UnassignTags)
})
r.Route("/me", func(r chi.Router) {
r.Get("/", svc.GetMe)
r.Get("/drives", svc.GetDrives)

228
services/graph/pkg/service/v0/tags.go Normal file
View File

@@ -0,0 +1,228 @@
package svc
import (
"encoding/json"
"net/http"
"strings"
rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
revaCtx "github.com/cs3org/reva/v2/pkg/ctx"
"github.com/cs3org/reva/v2/pkg/events"
"github.com/cs3org/reva/v2/pkg/storagespace"
"github.com/cs3org/reva/v2/pkg/tags"
"github.com/go-chi/render"
libregraph "github.com/owncloud/libre-graph-api-go"
searchsvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/search/v0"
"github.com/owncloud/ocis/v2/services/graph/pkg/service/v0/errorcode"
"go-micro.dev/v4/metadata"
)
// GetTags returns all available tags
func (g Graph) GetTags(w http.ResponseWriter, r *http.Request) {
th := r.Header.Get(revaCtx.TokenHeader)
ctx := revaCtx.ContextSetToken(r.Context(), th)
ctx = metadata.Set(ctx, revaCtx.TokenHeader, th)
sr, err := g.searchService.Search(ctx, &searchsvc.SearchRequest{
Query: "Tags:*",
PageSize: -1,
})
if err != nil {
g.logger.Error().Err(err).Msg("Could not search for tags")
w.WriteHeader(http.StatusInternalServerError)
return
}
tagList := tags.FromList("")
for _, match := range sr.Matches {
for _, tag := range match.Entity.Tags {
tagList.AddList(tag)
}
}
tagCollection := libregraph.NewCollectionOfTags()
tagCollection.Value = tagList.AsSlice()
render.Status(r, http.StatusOK)
render.JSON(w, r, tagCollection)
}
// AssignTags assigns a tag to a resource
func (g Graph) AssignTags(w http.ResponseWriter, r *http.Request) {
var (
assignment libregraph.TagAssignment
ctx = r.Context()
)
if err := json.NewDecoder(r.Body).Decode(&assignment); err != nil {
g.logger.Debug().Err(err).Interface("body", r.Body).Msg("could not decode tag assignment request")
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid body schema definition")
return
}
rid, err := storagespace.ParseID(assignment.ResourceId)
if err != nil {
g.logger.Debug().Err(err).Msg("could not parse resourceId")
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid resourceId")
return
}
sres, err := g.gatewayClient.Stat(ctx, &provider.StatRequest{
Ref: &provider.Reference{ResourceId: &rid},
})
if err != nil {
g.logger.Error().Err(err).Msg("error stating file")
w.WriteHeader(http.StatusInternalServerError)
return
}
if sres.GetStatus().GetCode() != rpc.Code_CODE_OK {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "can't stat resource")
return
}
pm := sres.GetInfo().GetPermissionSet()
if pm == nil {
g.logger.Error().Err(err).Msg("no permissionset on file")
w.WriteHeader(http.StatusInternalServerError)
return
}
// it says we need "write access" to set tags. One of those should do
if !pm.InitiateFileUpload && !pm.CreateContainer {
g.logger.Info().Msg("no permission to create a tag")
w.WriteHeader(http.StatusForbidden)
return
}
var currentTags string
if m := sres.GetInfo().GetArbitraryMetadata().GetMetadata(); m != nil {
currentTags = m["tags"]
}
allTags := tags.FromList(currentTags)
newTags := strings.Join(assignment.Tags, ",")
if !allTags.AddList(newTags) {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "no new tags in createtagsrequest or maximum reached")
return
}
resp, err := g.gatewayClient.SetArbitraryMetadata(ctx, &provider.SetArbitraryMetadataRequest{
Ref: &provider.Reference{ResourceId: &rid},
ArbitraryMetadata: &provider.ArbitraryMetadata{
Metadata: map[string]string{
"tags": allTags.AsList(),
},
},
})
if err != nil || resp.GetStatus().GetCode() != rpc.Code_CODE_OK {
g.logger.Error().Err(err).Msg("error setting tags")
w.WriteHeader(http.StatusInternalServerError)
return
}
if g.eventsPublisher != nil {
ev := events.TagsAdded{
Tags: newTags,
Ref: &provider.Reference{
ResourceId: &rid,
Path: ".",
},
SpaceOwner: sres.Info.Owner,
Executant: revaCtx.ContextMustGetUser(r.Context()).Id,
}
if err := events.Publish(g.eventsPublisher, ev); err != nil {
g.logger.Error().Err(err).Msg("Failed to publish TagsAdded event")
}
}
}
// UnassignTags removes a tag from a resource
func (g Graph) UnassignTags(w http.ResponseWriter, r *http.Request) {
var (
unassignment libregraph.TagUnassignment
ctx = r.Context()
)
if err := json.NewDecoder(r.Body).Decode(&unassignment); err != nil {
g.logger.Debug().Err(err).Interface("body", r.Body).Msg("could not decode tag assignment request")
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid body schema definition")
return
}
rid, err := storagespace.ParseID(unassignment.ResourceId)
if err != nil {
g.logger.Debug().Err(err).Msg("could not parse resourceId")
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid resourceId")
return
}
sres, err := g.gatewayClient.Stat(ctx, &provider.StatRequest{
Ref: &provider.Reference{ResourceId: &rid},
})
if err != nil {
g.logger.Error().Err(err).Msg("error stating file")
w.WriteHeader(http.StatusInternalServerError)
return
}
if sres.GetStatus().GetCode() != rpc.Code_CODE_OK {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "can't stat resource")
return
}
pm := sres.GetInfo().GetPermissionSet()
if pm == nil {
g.logger.Error().Err(err).Msg("no permissionset on file")
w.WriteHeader(http.StatusInternalServerError)
return
}
// it says we need "write access" to set tags. One of those should do
if !pm.InitiateFileUpload && !pm.CreateContainer {
g.logger.Info().Msg("no permission to create a tag")
w.WriteHeader(http.StatusForbidden)
return
}
var currentTags string
if m := sres.GetInfo().GetArbitraryMetadata().GetMetadata(); m != nil {
currentTags = m["tags"]
}
allTags := tags.FromList(currentTags)
toDelete := strings.Join(unassignment.Tags, ",")
if !allTags.RemoveList(toDelete) {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "no new tags in createtagsrequest or maximum reached")
return
}
resp, err := g.gatewayClient.SetArbitraryMetadata(ctx, &provider.SetArbitraryMetadataRequest{
Ref: &provider.Reference{ResourceId: &rid},
ArbitraryMetadata: &provider.ArbitraryMetadata{
Metadata: map[string]string{
"tags": allTags.AsList(),
},
},
})
if err != nil || resp.GetStatus().GetCode() != rpc.Code_CODE_OK {
g.logger.Error().Err(err).Msg("error setting tags")
w.WriteHeader(http.StatusInternalServerError)
return
}
if g.eventsPublisher != nil {
ev := events.TagsRemoved{
Tags: toDelete,
Ref: &provider.Reference{
ResourceId: &rid,
Path: ".",
},
SpaceOwner: sres.Info.Owner,
Executant: revaCtx.ContextMustGetUser(r.Context()).Id,
}
if err := events.Publish(g.eventsPublisher, ev); err != nil {
g.logger.Error().Err(err).Msg("Failed to publish TagsAdded event")
}
}
}

15
services/graph/pkg/service/v0/tracing.go
View File

@@ -129,3 +129,18 @@ func (t tracing) CreateDrive(w http.ResponseWriter, r *http.Request) {
func (t tracing) GetRootDriveChildren(w http.ResponseWriter, r *http.Request) {
t.next.GetRootDriveChildren(w, r)
}
// GetTags implements the Service interface.
func (t tracing) GetTags(w http.ResponseWriter, r *http.Request) {
t.next.GetTags(w, r)
}
// AssignTags implements the Service interface.
func (t tracing) AssignTags(w http.ResponseWriter, r *http.Request) {
t.next.AssignTags(w, r)
}
// UnassignTags implements the Service interface.
func (t tracing) UnassignTags(w http.ResponseWriter, r *http.Request) {
t.next.UnassignTags(w, r)
}

12
services/search/pkg/content/basic.go
View File

@@ -5,7 +5,7 @@ import (
"time"
storageProvider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
//"github.com/cs3org/reva/v2/pkg/tags"
"github.com/cs3org/reva/v2/pkg/tags"
"github.com/owncloud/ocis/v2/ocis-pkg/log"
)
@@ -27,11 +27,11 @@ func (b Basic) Extract(_ context.Context, ri *storageProvider.ResourceInfo) (Doc
MimeType: ri.MimeType,
}
//if m := ri.ArbitraryMetadata.GetMetadata(); m != nil {
//if t, ok := m["tags"]; ok {
//doc.Tags = tags.FromList(t).AsSlice()
//}
//}
if m := ri.ArbitraryMetadata.GetMetadata(); m != nil {
if t, ok := m["tags"]; ok {
doc.Tags = tags.FromList(t).AsSlice()
}
}
if ri.Mtime != nil {
doc.Mtime = time.Unix(int64(ri.Mtime.Seconds), int64(ri.Mtime.Nanos)).UTC().Format(time.RFC3339)

4
services/search/pkg/content/basic_test.go
View File

@@ -40,7 +40,7 @@ var _ = Describe("Basic", func() {
Expect(doc.MimeType).To(Equal(ri.MimeType))
})
/*It("adds tags", func() {
It("adds tags", func() {
for _, data := range []struct {
tags string
expect []string
@@ -63,7 +63,7 @@ var _ = Describe("Basic", func() {
Expect(doc).ToNot(BeNil())
Expect(doc.Tags).To(Equal(data.expect))
}
})*/
})
It("RFC3339 mtime", func() {
for _, data := range []struct {

2
services/search/pkg/content/content.go
View File

@@ -9,5 +9,5 @@ type Document struct {
Size uint64
Mtime string
MimeType string
//Tags []string
Tags []string
}

2
services/search/pkg/content/mocks/extractor.go
View File

@@ -1,4 +1,4 @@
// Code generated by mockery v2.14.1. DO NOT EDIT.
// Code generated by mockery v2.15.0. DO NOT EDIT.
package mocks

2
services/search/pkg/content/mocks/retriever.go
View File

@@ -1,4 +1,4 @@
// Code generated by mockery v2.14.1. DO NOT EDIT.
// Code generated by mockery v2.15.0. DO NOT EDIT.
package mocks

6
services/search/pkg/engine/bleve.go
View File

@@ -71,7 +71,7 @@ func BuildBleveMapping() (mapping.IndexMapping, error) {
docMapping := bleve.NewDocumentMapping()
docMapping.AddFieldMappingsAt("Name", lowercaseMapping)
//docMapping.AddFieldMappingsAt("Tags", lowercaseMapping)
docMapping.AddFieldMappingsAt("Tags", lowercaseMapping)
docMapping.AddFieldMappingsAt("Content", fulltextFieldMapping)
indexMapping := bleve.NewIndexMapping()
@@ -185,7 +185,7 @@ func (b *Bleve) Search(_ context.Context, sir *searchService.SearchIndexRequest)
Type: uint64(getValue[float64](hit.Fields, "Type")),
MimeType: getValue[string](hit.Fields, "MimeType"),
Deleted: getValue[bool](hit.Fields, "Deleted"),
//Tags: getSliceValue[string](hit.Fields, "Tags"),
Tags: getSliceValue[string](hit.Fields, "Tags"),
},
}
@@ -302,7 +302,7 @@ func (b *Bleve) getResource(id string) (*Resource, error) {
Mtime: getValue[string](fields, "Mtime"),
MimeType: getValue[string](fields, "MimeType"),
Content: getValue[string](fields, "Content"),
//Tags: getSliceValue[string](fields, "Tags"),
Tags: getSliceValue[string](fields, "Tags"),
},
}, nil
}

4
services/search/pkg/engine/bleve_test.go
View File

@@ -98,7 +98,7 @@ var _ = Describe("Bleve", func() {
Describe("Search", func() {
Context("by other fields than filename", func() {
/*It("finds files by tags", func() {
It("finds files by tags", func() {
parentResource.Document.Tags = []string{"foo", "bar"}
err := eng.Upsert(parentResource.ID, parentResource)
Expect(err).ToNot(HaveOccurred())
@@ -109,7 +109,7 @@ var _ = Describe("Bleve", func() {
assertDocCount(rootResource.ID, "Tags:foo Tags:bar Tags:baz", 1)
assertDocCount(rootResource.ID, "Tags:foo Tags:bar Tags:baz", 1)
assertDocCount(rootResource.ID, "Tags:baz", 0)
})*/
})
It("finds files by size", func() {
parentResource.Document.Size = 12345

51
services/search/pkg/engine/engine.go
View File

@@ -60,29 +60,28 @@ func getValue[T any](m map[string]interface{}, key string) (out T) {
return
}
// TODO comment back in when re-adding the tags features
// func getSliceValue[T any](m map[string]interface{}, key string) (out []T) {
// iv := getValue[interface{}](m, key)
// add := func(v interface{}) {
// cv, ok := v.(T)
// if !ok {
// return
// }
//
// out = append(out, cv)
// }
//
// // bleve tend to convert []string{"foo"} to type string if slice contains only one value
// // bleve: []string{"foo"} -> "foo"
// // bleve: []string{"foo", "bar"} -> []string{"foo", "bar"}
// switch v := iv.(type) {
// case T:
// add(v)
// case []interface{}:
// for _, rv := range v {
// add(rv)
// }
// }
//
// return
// }
func getSliceValue[T any](m map[string]interface{}, key string) (out []T) {
iv := getValue[interface{}](m, key)
add := func(v interface{}) {
cv, ok := v.(T)
if !ok {
return
}
out = append(out, cv)
}
// bleve tend to convert []string{"foo"} to type string if slice contains only one value
// bleve: []string{"foo"} -> "foo"
// bleve: []string{"foo", "bar"} -> []string{"foo", "bar"}
switch v := iv.(type) {
case T:
add(v)
case []interface{}:
for _, rv := range v {
add(rv)
}
}
return
}

2
services/search/pkg/engine/mocks/engine.go
View File

@@ -1,4 +1,4 @@
// Code generated by mockery v2.14.1. DO NOT EDIT.
// Code generated by mockery v2.15.0. DO NOT EDIT.
package mocks

53
services/search/pkg/search/events.go
View File

@@ -21,12 +21,12 @@ func HandleEvents(s Searcher, bus events.Consumer, logger log.Logger, cfg *confi
events.ContainerCreated{},
events.FileTouched{},
events.FileVersionRestored{},
//events.TagsAdded{},
//events.TagsRemoved{},
events.TagsAdded{},
events.TagsRemoved{},
}
if cfg.Events.AsyncUploads {
// evts = append(evts, events.UploadReady{})
evts = append(evts, events.UploadReady{})
} else {
evts = append(evts, events.FileUploaded{})
}
@@ -40,7 +40,7 @@ func HandleEvents(s Searcher, bus events.Consumer, logger log.Logger, cfg *confi
cfg.Events.NumConsumers = 1
}
spaceID := func(ref *provider.Reference) *provider.StorageSpaceId {
getSpaceID := func(ref *provider.Reference) *provider.StorageSpaceId {
return &provider.StorageSpaceId{
OpaqueId: storagespace.FormatResourceID(
provider.ResourceId{
@@ -51,6 +51,18 @@ func HandleEvents(s Searcher, bus events.Consumer, logger log.Logger, cfg *confi
}
}
getUser := func(users ...*user.UserId) *user.UserId {
for _, u := range users {
if u == nil {
continue
}
return u
}
return nil
}
indexSpaceDebouncer := NewSpaceDebouncer(time.Duration(cfg.Events.DebounceDuration)*time.Millisecond, func(id *provider.StorageSpaceId, userID *user.UserId) {
if err := s.IndexSpace(id, userID); err != nil {
logger.Error().Err(err).Interface("spaceID", id).Interface("userID", userID).Msg("error while indexing a space")
@@ -66,28 +78,31 @@ func HandleEvents(s Searcher, bus events.Consumer, logger log.Logger, cfg *confi
switch ev := e.(type) {
case events.ItemTrashed:
u := getUser(ev.SpaceOwner, ev.Executant)
s.TrashItem(ev.ID)
indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
indexSpaceDebouncer.Debounce(getSpaceID(ev.Ref), u)
case events.ItemMoved:
s.MoveItem(ev.Ref, ev.Executant)
indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
u := getUser(ev.SpaceOwner, ev.Executant)
s.MoveItem(ev.Ref, u)
indexSpaceDebouncer.Debounce(getSpaceID(ev.Ref), getUser(ev.SpaceOwner, ev.Executant))
case events.ItemRestored:
s.RestoreItem(ev.Ref, ev.Executant)
indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
u := getUser(ev.SpaceOwner, ev.Executant)
s.RestoreItem(ev.Ref, u)
indexSpaceDebouncer.Debounce(getSpaceID(ev.Ref), u)
case events.ContainerCreated:
indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
indexSpaceDebouncer.Debounce(getSpaceID(ev.Ref), getUser(ev.SpaceOwner, ev.Executant))
case events.FileTouched:
indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
indexSpaceDebouncer.Debounce(getSpaceID(ev.Ref), getUser(ev.SpaceOwner, ev.Executant))
case events.FileVersionRestored:
indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
//case events.TagsAdded:
// indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
//case events.TagsRemoved:
//indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
indexSpaceDebouncer.Debounce(getSpaceID(ev.Ref), getUser(ev.SpaceOwner, ev.Executant))
case events.TagsAdded:
s.UpsertItem(ev.Ref, getUser(ev.SpaceOwner, ev.Executant))
case events.TagsRemoved:
s.UpsertItem(ev.Ref, getUser(ev.SpaceOwner, ev.Executant))
case events.FileUploaded:
indexSpaceDebouncer.Debounce(spaceID(ev.Ref), ev.Executant)
//case events.UploadReady:
//indexSpaceDebouncer.Debounce(spaceID(ev.FileRef), ev.ExecutingUser.Id)
indexSpaceDebouncer.Debounce(getSpaceID(ev.Ref), getUser(ev.SpaceOwner, ev.Executant))
case events.UploadReady:
indexSpaceDebouncer.Debounce(getSpaceID(ev.FileRef), getUser(ev.SpaceOwner, ev.ExecutingUser.Id))
}
if err != nil {

7
services/search/pkg/search/events_test.go
View File

@@ -1,6 +1,7 @@
package search_test
import (
userv1beta1 "github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1"
"github.com/cs3org/reva/v2/pkg/events"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
@@ -46,8 +47,8 @@ var _ = DescribeTable("events",
Entry("ContainerCreated", []string{"IndexSpace"}, events.ContainerCreated{}, false),
Entry("FileTouched", []string{"IndexSpace"}, events.FileTouched{}, false),
Entry("FileVersionRestored", []string{"IndexSpace"}, events.FileVersionRestored{}, false),
//Entry("TagsAdded", []string{"IndexSpace"}, events.TagsAdded{}, false),
//Entry("TagsRemoved", []string{"IndexSpace"}, events.TagsRemoved{}, false),
Entry("TagsAdded", []string{"UpsertItem"}, events.TagsAdded{}, false),
Entry("TagsRemoved", []string{"UpsertItem"}, events.TagsRemoved{}, false),
Entry("FileUploaded", []string{"IndexSpace"}, events.FileUploaded{}, false),
//Entry("UploadReady", []string{"IndexSpace"}, events.UploadReady{ExecutingUser: &userv1beta1.User{}}, true),
Entry("UploadReady", []string{"IndexSpace"}, events.UploadReady{ExecutingUser: &userv1beta1.User{}}, true),
)

2
services/search/pkg/search/mocks/searcher.go
View File

@@ -1,4 +1,4 @@
// Code generated by mockery v2.14.1. DO NOT EDIT.
// Code generated by mockery v2.15.0. DO NOT EDIT.
package mocks

7
services/webdav/pkg/service/v0/search.go
View File

@@ -14,6 +14,7 @@ import (
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
revactx "github.com/cs3org/reva/v2/pkg/ctx"
"github.com/cs3org/reva/v2/pkg/storagespace"
"github.com/cs3org/reva/v2/pkg/tags"
"github.com/cs3org/reva/v2/pkg/utils"
searchmsg "github.com/owncloud/ocis/v2/protogen/gen/ocis/messages/search/v0"
searchsvc "github.com/owncloud/ocis/v2/protogen/gen/ocis/services/search/v0"
@@ -189,6 +190,12 @@ func matchToPropResponse(ctx context.Context, match *searchmsg.Match) (*propfind
propstatOK.Prop = append(propstatOK.Prop, prop.Escaped("d:getcontenttype", match.Entity.MimeType))
propstatOK.Prop = append(propstatOK.Prop, prop.Escaped("oc:permissions", match.Entity.Permissions))
t := tags.FromList("")
for _, tag := range match.Entity.Tags {
t.AddList(tag)
}
propstatOK.Prop = append(propstatOK.Prop, prop.Escaped("oc:tags", t.AsList()))
// those seem empty - bug?
propstatOK.Prop = append(propstatOK.Prop, prop.Escaped("d:getetag", match.Entity.Etag))