45 Commits

Author SHA1 Message Date
Mike Gibson af85bf6588 Batch size flag 2024-01-08 11:18:00 +00:00
mgdigital b560874463 Cache tweaks (#99)
* Increase TTL and cache warmer intervals

* Don't delete cache entries prior to no-cache/warm queries
2024-01-07 12:03:44 +00:00
mgdigital 6358b27bc6 Search query string rework (#96)
- `search_string` field is removed
- text search vector for `content` and `torrent_contents` is generated in code with improvements in normalisation and tokenisation
- the slow `LIKE` query is removed from text search
- a query DSL is implemented which translates the input search string to a Postgres tsquery
- a CLI command (`reindex`) is provided for updating the tsv for pre-existing records
- the unused `tsv` and `search_string` fields are removed from the `torrents` table

See https://github.com/bitmagnet-io/bitmagnet/issues/89
2024-01-07 11:51:40 +00:00
mgdigital 5d74cf6c81 Lazy instantiation for services with side effects (#98) 2024-01-06 12:40:11 +00:00
dependabot[bot] c054e6b464 Bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#91)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-04 16:58:46 +00:00
mgdigital 32dc0bf6ec Remove asynqmon (#93) v0.4.1 2023-12-22 09:00:40 +00:00
mgdigital 9f87f4a429 Downgrade asynqmon (#92) v0.4.0 2023-12-21 17:39:02 +00:00
mgdigital f2dcbf98fc Reduce aggregations in web UI (#87)
Fix https://github.com/bitmagnet-io/bitmagnet/issues/85

- Accurate counts in the web UI will only be shown for the top-level filters; when deep-filtering, the counts will show a less than or equal sign: `≤`, indicating the number is a "maximum possible" rather than an accurate count; I'm still not sure about the UX aspect of showing the `≤` which could be confusing if you're not sure what it means, but I figured it's better than showing nothing at all - open to suggestions!
- A background process keeps the in-memory query cache warm for the top-level aggregations (by default every 10 minutes), allowing them to be served instantly
- The query cache TTL has been increased to 20 minutes; improved performance is preferable to seeing the most up-to-date information
- Pagination has been refactored to account for not knowing the total number of pages; internally the search engine will request 1 more item than it needs to know if there's a next page to advance to
- Some general refactoring of the web app
2023-12-21 16:08:38 +00:00
mgdigital c1e1d86095 Add gluetun configuration example (#83)
* Add gluetun configuration example and replace docker-compose.dev.yml with docker-compose.yml full featured example; update docsite
2023-11-29 22:02:29 +00:00
mgdigital f55d430303 Split webUI query (#82) v0.3.1 2023-11-29 06:42:11 +00:00
mgdigital 2092d1c60f Reduce frequency of crawler database queries (#79) v0.3.0 2023-11-28 21:20:22 +00:00
mgdigital 57710c4424 Add Discord link to web UI (#78) 2023-11-28 18:29:55 +00:00
mgdigital b199474a81 Add client suffix to DHT node ID (#77)
* Use DHT client ID with suffix

* Remove unwanted warn logs
2023-11-28 18:16:54 +00:00
mgdigital 85bf13bc6a Deletion (#75)
- Add a stable bloom filter, stored in the database, for blocked and deleted torrents
- Add GraphQL mutations for blocking and deleting torrents
- Add web UI for bulk actions (tagging and deleting)
- Some minor cosmetic web UI tweaks
- Move database operations to dao package
2023-11-28 17:54:49 +00:00
mgdigital ffe82f3c84 Add indexes (#74) 2023-11-24 12:20:38 +00:00
mgdigital 8224aded79 Add refresh button to web UI (#73) 2023-11-24 11:46:09 +00:00
dependabot[bot] f5e38856a9 Bump @babel/traverse from 7.22.17 to 7.23.2 in /webui (#41)
Bumps [@babel/traverse](https://github.com/babel/babel/tree/HEAD/packages/babel-traverse) from 7.22.17 to 7.23.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.23.2/packages/babel-traverse)

---
updated-dependencies:
- dependency-name: "@babel/traverse"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mgdigital <mgdigital@users.noreply.github.com>
2023-11-24 09:54:34 +00:00
mgdigital 41094ef9da Add attributes to magnet links (#72)
* Add attributes to magnet link
2023-11-24 07:49:45 +00:00
mgdigital 03f55da811 Torrent tagging (#60)
- Implement torrent tagging in API and UI (https://github.com/bitmagnet-io/bitmagnet/issues/50)
- Lay UX groundwork for actions on individual torrents (actions tab bar)
- Refactor GraphQL schema
2023-11-24 07:28:50 +00:00
mgdigital cffd5830c4 Fix link (#69) 2023-11-18 19:12:01 +00:00
mgdigital ec95f7cc3a Observability enhancements (#67) v0.2.0 2023-11-18 18:58:02 +00:00
mgdigital 630c0843e5 Respect savePieces config (#62) v0.1.1 2023-11-12 15:15:31 +00:00
mgdigital b19b224588 Fix config resolver bug (#61) 2023-11-12 14:57:12 +00:00
mgdigital 24f78274c7 Update docsite (#48)
* Update docsite
2023-11-06 21:31:43 +00:00
mgdigital 1f6bd96b83 Add checkers to reject crawled torrents based on criteria (#46) v0.1.0 2023-11-06 17:34:53 +00:00
mgdigital 070f38120b DHT responder: Implement BEP5 & BEP51 (#21)
- Address https://github.com/bitmagnet-io/bitmagnet/issues/11 with BEP5 & BEP51 implementations
- Reimplement DHT crawler around new routing table
- Fix https://github.com/bitmagnet-io/bitmagnet/issues/45
- Fix https://github.com/bitmagnet-io/bitmagnet/issues/42
- Fix https://github.com/bitmagnet-io/bitmagnet/issues/39
- Fix https://github.com/bitmagnet-io/bitmagnet/issues/37
- Make bootstrap nodes configurable: https://github.com/bitmagnet-io/bitmagnet/issues/27
- Add profiling tools to Docker image
2023-11-06 16:41:08 +00:00
dr4gn3l b9bd56381c when tmdb found and adult movie we set the type xxx (#44)
Co-authored-by: dragnel <dragnel@protonmail.com>
2023-10-28 14:21:53 +01:00
mgdigital 525b17d9d7 Update issue templates (#33)
* Update issue templates
v0.0.7
2023-10-09 19:00:18 +01:00
Niklas a132e63389 docker-compose is deprecated, use docker compose (#32)
Co-authored-by: mgdigital <mgdigital@users.noreply.github.com>
2023-10-09 18:59:16 +01:00
Niklas 5d2d94f502 Fix error message re: meta info from any pers (#34)
Co-authored-by: mgdigital <mgdigital@users.noreply.github.com>
2023-10-09 18:58:41 +01:00
mgdigital a47b748113 Add nil check (#31) 2023-10-09 18:54:49 +01:00
mgdigital 477e98ad01 Use torrent_contents.info_hash instead of torrent_contents.id in Torznab result (#30) 2023-10-09 18:24:27 +01:00
myyc 503b170af5 parsing uint from env variable (#26) v0.0.6 2023-10-08 17:10:53 +01:00
mgdigital 869bd0d05a Implement multiplatform builds (#18) (#20)
* Implement multiplatform builds (#18)

* Initial multiplatform build

* Change image name for pr

* Fix prettier errors

* Handle semver

---------

Co-authored-by: Francesco Bruno <f.bruno.dev@gmail.com>

* Avoid dirty version number

* Remove v prefix

* Add tags option

* Fix tags?

---------

Co-authored-by: Francesco Bruno <b.frenc@hotmail.it>
Co-authored-by: Francesco Bruno <f.bruno.dev@gmail.com>
2023-10-06 16:06:58 +01:00
mgdigital 2f75cea9a8 Update docsite (#13) (#16)
* Add screenshot
2023-10-05 22:11:51 +01:00
Mike Gibson ce8909dbdb Update docsite (#13) 2023-10-05 18:37:25 +01:00
Mike Gibson 37cf8f841f Fix importer (#12) 2023-10-05 16:44:57 +01:00
Mike Gibson 3bad4fd3e4 Add files_status field and dht_crawler.files_threshold configuration parameter (#9) 2023-10-05 15:17:16 +01:00
Mike Gibson 5ef55eb460 Move docker-compose comment to separate line (#8)
* Move comment to separate line
2023-10-05 12:13:32 +01:00
Mike Gibson 3e94e009a7 Fix: Add equal sign to keys options in docker-compose files (#7)
* Add equal sign to keys options
2023-10-05 12:03:57 +01:00
Mike Gibson a54fe26059 Add dht_crawler.save_files option; update docs (#5) 2023-10-05 08:37:18 +01:00
Mike Gibson 8275fcb9a2 Add bounds check to year conversion (#4)
* Add bounds check to year conversion
2023-10-04 22:27:03 +01:00
Mike Gibson d03f2bb44f Add codeql workflow (#3) 2023-10-04 21:23:32 +01:00
Mike Gibson e7327343f0 Initial commit (#1) 2023-10-04 21:04:26 +01:00
Mike Gibson 9fa3fe181f Add README 2023-10-04 19:03:14 +01:00