Commit Graph

282 Commits

Author SHA1 Message Date
WithoutPants 9a84726128 Fix xpath comment element parsing (#759) 2020-08-23 17:39:15 +10:00
WithoutPants 1fd3fcc6a8 Show and allow creation of unknown performers/tags/studios/movies from scraper dialog (#741)
* Fix toast container z-index
* Make scrape operations network only
* Add CollapseButton component
2020-08-22 18:12:39 +10:00
JoeSmithStarkers 85aa1d8790 Replace os.Rename with util.SafeMove to allow cross device moving to not fail. (#745)
Fixed annoyingly noisy transcoding progress log messages.
Fixed minor minor issue with directories than are named "blah.mp4" being detected as files to be scanned.
2020-08-21 17:57:07 +10:00
InfiniteTF ecf745162f Upgrade doublestar to v2.0.1 (#742) 2020-08-17 12:06:40 +10:00
JoeSmithStarkers ecc42e4e24 Preview generation fallback (#725)
* Added preview generation fallback feature.
When a preview generation fails (often for wmv/avi files), the new code tries with less stricted (no xerror) and more time consuming options (slow+fast seek).
Fix a minor issue when stash downloads ffmpeg/ffprobe, but doesn't re-detect their paths.
2020-08-17 09:21:58 +10:00
WithoutPants a39666467e Fix import performers/movies/studios with no image (#732) 2020-08-14 09:40:25 +10:00
woodgen e3ea3ea85e scraper/mapped: Add feetToCm post process. (#711)
This patch adds a feetToCm post process that converts imperial feet and
inches to centimeters.
2020-08-12 11:17:43 +10:00
WithoutPants e16118f551 Clear image (#722)
* Allow clearing of tag images
* Allow clearing of studio images
* Allow clearing of performer images
* Allow clearing of movie images
* Add filtering for missing images
2020-08-12 09:19:27 +10:00
woodgen 4045ddf3e9 Implement scraping movies by URL (#709)
* api/urlbuilders/movie: Auto format.

* graphql+pkg+ui: Implement scraping movies by URL.

This patch implements the missing required boilerplate for scraping
movies by URL, using performers and scenes as a reference.

Although this patch contains a big chunck of ground work for enabling
scraping movies by fragment, the feature would require additional
changes to be completely implemented and was not tested.

* graphql+pkg+ui: Scrape movie studio.

Extends and corrects the movie model for the ability to store and
dereference studio IDs with received studio string from the scraper.
This was done with Scenes as a reference. For simplicity the duplication
of having `ScrapedMovieStudio` and `ScrapedSceneStudio` was kept, which
should probably be refactored to be the same type in the model in the
future.

* ui/movies: Add movie scrape dialog.

Adds possibility to update existing movie entries with the URL scraper.

For this the MovieScrapeDialog.tsx was implemented with Performers and
Scenes as a reference. In addition DurationUtils needs to be called one
time for converting seconds from the model to the string that is
displayed in the component. This seemed the least intrusive to me as it
kept a ScrapeResult<string> type compatible with ScrapedInputGroupRow.
2020-08-10 15:34:15 +10:00
WithoutPants 7158e83b75 Add JSON scrape support (#717)
* Add support for scene fragment scrape in xpath
2020-08-10 14:21:50 +10:00
WithoutPants 470a2b5833 Fix sprite vtt panic (#718) 2020-08-10 09:20:04 +10:00
WithoutPants 634b892df1 Fix typo 2020-08-09 13:48:47 +10:00
WithoutPants 0ffefa6e16 Add plugin tasks (#651) 2020-08-08 12:05:35 +10:00
WithoutPants 0874852fa8 Improve oshash collision detection and logging (#713)
* Log colliding file when setting hash
* Check for existing using both hashes
2020-08-08 11:22:25 +10:00
WithoutPants 5992ff8706 Add oshash support (#667) 2020-08-06 11:21:14 +10:00
WithoutPants b166abfa7b Fix scraping error (#704) 2020-08-04 20:43:56 +10:00
bnkai 4373f9bf01 Add cdp support for xpath scrapers (#625)
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2020-08-04 10:42:40 +10:00
InfiniteTF 116f61b6bf Fix potential sprite generation segfault (#685) 2020-07-24 09:20:34 +10:00
WithoutPants a2341f0819 Allow customisation of preview generation (#673)
* Add generate-specific options
* Include no-cache in preview response
2020-07-23 12:51:35 +10:00
WithoutPants 37be146a9d Transcode stream refactor (#609)
* Remove forceMkv and forceHEVC
* Add HLS support and refactor
* Add new streaming endpoints
2020-07-23 11:56:08 +10:00
WithoutPants 2b9215702e Refactor xpath scraper code. Add fixed and map (#616)
* Refactor xpath scraper code
* Make post-process a list
* Add map post-process action
* Add fixed xpath values
* Refactor scrapers into cache
* Refactor into mapped config
* Trim test html
2020-07-21 14:06:25 +10:00
WithoutPants c104c6d075 Generate content for specific scenes (#672)
* Add UI dialog for scene(s)
* Move preview preset to config
2020-07-19 11:59:18 +10:00
bnkai ec2bcc7a74 Scan for files with ALLCAPS extensions (#650) 2020-07-11 17:22:36 +10:00
bnkai 56210cf456 Use referer on xpath getImage, apply printHTML to subscraper also (#661) 2020-07-10 08:42:06 +10:00
WithoutPants e9141b5dfc Fix tag query performance problems (#657)
* Fix sql tracing
* Disable query by marker count
* Disable unit test
2020-07-09 08:42:07 +10:00
WithoutPants 244ae54f3f Add grid view, image to tag (#641)
* Add grid view for tags
* Add tag page
* Import/export tags
* Add tag name uniqueness checks
* Fix styling on missing marker previews
* Add trace loglevel
* Add SQL trace
* Add filter options for tags
* Add tag sort by options
* Add tag page keyboard shortcuts
2020-07-07 10:35:43 +10:00
WithoutPants 455e16ece9 Support deleting multiple scenes (#630)
* Improve layout and add buttons
* Move functionality into ListFilter
* Make modal style dark
* Convert scene options into edit scenes dialog
* Add delete scenes dialog
* Clear selected ids on delete
* Refetch after update/delete
* Use DeleteScenesDialog in Scene page
* Show scene check boxes in small screens
* Change default multi-set mode to set
2020-06-23 10:40:11 +10:00
WithoutPants 7a74658a73 Move image blobs into separate tables (#618)
* Scene cover fallback to database
* Fix panic if studio not found
* Fix movie studio not being imported/exported
2020-06-23 09:19:19 +10:00
bnkai f8048dc27c Increase xpath redirects, use cookies (#624) 2020-06-22 12:18:02 +10:00
WithoutPants 77a5b1d814 Add custom served folders (#620) 2020-06-21 22:25:13 +10:00
WithoutPants d3ababf0a1 Gallery list improvement (#622)
* Add grid view to galleries
* Show scene in gallery card
* Add is missing scene gallery filter
* Don't store galleries with no images
2020-06-21 21:43:57 +10:00
bnkai 9d0522f62d Add "split" xpath in post-processing , newlines in replace support (#579) 2020-06-18 10:47:10 +10:00
bnkai a7ac02fb50 freeones fixes (#615) 2020-06-17 11:02:06 +10:00
bnkai f40e234748 Apply xpath parseDate after subScraper (#606) 2020-06-15 21:38:59 +10:00
WithoutPants 96e6e16507 Parent studios (#595)
* Refactor getMultiCriterionClause
Co-authored-by: Anon247 <61889302+Anon247@users.noreply.github.com>
2020-06-15 21:34:39 +10:00
WithoutPants d8ce137086 Reload scrapers button (#592)
* Add reload scraper option to performer details
* Add scraper reload to scene edit page
* Show scene scraper menu when no queryable scrapers
* Add 0.3 changelog
2020-06-10 13:43:17 +10:00
bnkai b89956de25 freeones scraper fixes/tweaking (#584) 2020-06-02 09:45:37 +10:00
bnkai dc5efb9e31 Unit testing for performers,studios,movies FindByName/s (#581) 2020-05-27 10:48:45 +10:00
InfiniteTF 4ec6d62e01 Selectable wall preview type (#510)
* Add optional image preview generation
* Add setting for video preview encoding preset
2020-05-27 09:33:49 +10:00
bnkai ccd75731b7 Change scrape matching (studio, movies, tag, performers) to case insensitive (#556)
* Change scrape matching (studio, movies, tag, performers) to case insensitive
* * fix collate order
* * make filename parser findbyname calls case insensitive
* * add unit testing for Tags GetFindbyName/s
2020-05-24 16:19:22 +10:00
InfiniteTF 32fce9ac6f Speed up tag count queries (#570)
* Speed up tag count queries
* Add test for marker CountByTagID

Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2020-05-24 16:18:02 +10:00
WithoutPants ec420df871 Add debug logging for xpath scraping (#555)
* Add debug logging for xpath scraping
* Add logging for processing scene members
2020-05-20 22:46:00 +10:00
WithoutPants 05488d59c3 Find scrapers in subdirectories (#554) 2020-05-19 08:44:33 +10:00
bnkai 0fc57ce1e0 Fix xpath comments text (#550) 2020-05-18 12:26:20 +10:00
WithoutPants 46746e6848 Fix redirect loops in login, migrate and setup pages (#549)
* Fix redirect loop when setup and migrate required
* Fix redirect loop between setup and login
2020-05-18 08:55:01 +10:00
WithoutPants 215c4e3bde Change builtin freeones scraper to community yml (#542) 2020-05-15 20:10:20 +10:00
WithoutPants 7fd9428cbb Fix stash directory with spaces (#541) 2020-05-15 09:12:17 +10:00
bnkai d74a303c00 Fix 0 undefined library size on empty db (#535) 2020-05-14 10:44:13 +10:00
WithoutPants fde02425ab Make auto tagger matching more flexible (#534) 2020-05-13 17:32:39 +10:00
WithoutPants 41c6d9e681 Revert preview generation change 2020-05-13 17:23:23 +10:00