Commit Graph

286 Commits

Author SHA1 Message Date
fschade 82e75e19c1 enhancement(search): move bleve engine into its own package and clean up the search batch processing implementation 2025-08-28 15:53:29 +02:00
André Duffeck 1003734b45 Use batches when restoring or moving items 2025-08-28 15:50:44 +02:00
André Duffeck 9f9e03794d Do not intertwine different batch operations 2025-08-28 15:50:44 +02:00
fschade ca0493b286 enhancement(search): add support for testcontainers to run local tests 2025-08-28 09:32:26 +02:00
fschade ad866b8ce3 refactor(search): unify osu request and params naming 2025-08-28 09:32:05 +02:00
fschade 8c509263b7 refactor(search): simplify osu builder interface and make use of a base for the requests 2025-08-28 09:32:05 +02:00
fschade 7fe5383d61 refactor(search): rename test-testdata helper 2025-08-28 09:32:05 +02:00
fschade 8d850b1f4a refactor(search): move index management from the osu to the opensearch package 2025-08-28 09:32:05 +02:00
fschade e00fdc6ba3 refactor(search): remove samber/lo 2025-08-28 09:32:05 +02:00
fschade 42b794e01a refactor(search): cleanup for review 2025-08-28 09:32:05 +02:00
fschade 8795284a76 fix(search): potential nil slice entries 2025-08-28 09:32:05 +02:00
fschade f3750f32c9 refactor(search):
- introduce path_hierarchy analyzer and tokenizer
- optimize performance by using the os painless script api to restore, purge and delete documents
2025-08-28 09:32:05 +02:00
fschade a9d21bbb15 enhancement(search): allow to set the resource index name 2025-08-28 09:32:05 +02:00
fschade 1586f7fcbb enhancement(search): implement index manager and remove the use of index templates 2025-08-28 09:32:04 +02:00
fschade 9faa09e4c6 enhancement(search): implement search match highlighting for the content field 2025-08-28 09:31:38 +02:00
fschade 85e436b2bb fix(search): open-search engine interface compatibility 2025-08-28 09:31:38 +02:00
fschade 63e71b5bc4 enhancement(search): allow to configure open-search as search backend 2025-08-28 09:30:59 +02:00
fschade 2857e54975 fix(search): use recursion to request more search results if the searchResponse results are paginated 2025-08-28 09:30:59 +02:00
fschade d761e8b3f0 enhancement(search): implement search backend recursive move and update restore and delete to be recursive too 2025-08-28 09:30:59 +02:00
fschade f6144e6cdd enhancement(search): implement kql ast expansion helper and remove similar parts from the to os dsl query transpiler 2025-08-28 09:30:59 +02:00
fschade d97b2a6410 enhancement(search): implement kql NOT operator to os dsl bool-query MUST_NOT 2025-08-28 09:30:59 +02:00
fschade 3401f49a8c enhancement(search): implement kql bool to os dsl term-query 2025-08-28 09:30:59 +02:00
fschade d4183807dc enhancement(search): implement kql to os dsl range-query 2025-08-28 09:30:59 +02:00
fschade 48705c79f6 enhancement(search): implement os dsl range field 2025-08-28 09:30:59 +02:00
fschade 1c92f3db00 fix(search): implement support for versioned os index templates 2025-08-28 09:30:59 +02:00
fschade a7d4ff4872 enhancement(search): group opensearch related files together, housekeeping 2025-08-28 09:30:59 +02:00
fschade 5abfd1744e enhancement(search): implement cluster health checks 2025-08-28 09:30:59 +02:00
fschade 1236cedacc enhancement(search): implementation that the search ignores resources marked as deleted 2025-08-28 09:30:59 +02:00
fschade 2d325d70b8 enhancement(search): implement search engine match to pg-hit conversion 2025-08-28 09:30:59 +02:00
fschade f118e0d2c3 enhancement(search): implement kql to os dsl wildcard-query 2025-08-28 09:30:59 +02:00
fschade 2c316ea225 enhancement(search): implement kql string to os dsl match-phrase-query 2025-08-28 09:30:59 +02:00
fschade 4d5a5dde4b enhancement(search): implement non bool query compilation 2025-08-28 09:30:59 +02:00
fschade 098a220626 enhancement(search): implement kql to os dsl structure compilation and add basic tests 2025-08-28 09:30:59 +02:00
fschade 37d8b1d608 enhancement(search): implement engine search skeleton 2025-08-28 09:30:59 +02:00
fschade 492340f6f7 enhancement(search): implement engine docCount 2025-08-28 09:30:59 +02:00
fschade bd5dec7327 enhancement(search): implement engine restore 2025-08-28 09:30:59 +02:00
fschade 59b6788b28 enhancement(search): implement engine delete 2025-08-28 09:30:59 +02:00
fschade c18bfad222 enhancement(search): implement engine purge 2025-08-28 09:30:59 +02:00
fschade 4ad3865d52 enhancement(search): prepare opensearch integration 2025-08-28 09:30:59 +02:00
Jörn Friedrich Dreyer 2c3ee68f08 ack tag events
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-08-20 14:43:46 +02:00
André Duffeck 7c59e57d43 Make batch size configurable 2025-08-04 16:14:56 +02:00
André Duffeck 7a7d148dcf Fix tests 2025-08-04 15:39:27 +02:00
André Duffeck 3c8e2dacfd Improve indexing performance using batches 2025-08-04 15:15:54 +02:00
André Duffeck 1afc1331af Do not run the timout func if the work func has run 2025-08-01 10:03:56 +02:00
André Duffeck 4bafe2e611 Make sure to register prometheus collectors only once 2025-07-31 08:55:10 +02:00
André Duffeck b4cc781832 Do not try to log metrics when we failed to get the consumer info
Fixes #1285
2025-07-29 14:19:00 +02:00
André Duffeck 3bd7a61e46 Document the metrics exposed in search and postprocessing 2025-07-18 08:50:03 +02:00
André Duffeck dbd6a4db71 Fix tests 2025-07-17 16:00:11 +02:00
André Duffeck 791b4df173 Collect metrics about search and index durations 2025-07-17 15:23:57 +02:00
André Duffeck 273c0ed270 Expose nats metris for the postprocessing service 2025-07-17 15:23:57 +02:00