* fix: Correctly format listener address (https://github.com/TecharoHQ/anubis/issues/93)
Handle addresses that include a hostname, not just ports. If
the address starts with a colon, assume it's just a port and
prefix it with "http://localhost". Otherwise, prefix the
entire address with "http://". This ensures that the listener
URL is correctly formatted regardless of whether it includes
a hostname or just a port.
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* chore(docs): add changelog entry
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* cmd/anubis: add a debug option for benchmarking hashrate
Having the ability to benchmark different proof-of-work implementations
is useful for extending Anubis. This adds a flag `--debug-benchmark-js`
(and its associated environment variable `DEBUG_BENCHMARK_JS`) for
serving a tool to do so.
Internally, a there is a new policy action, "DEBUG_BENCHMARK", which
serves the benchmarking tool instead of a challenge. The flag then
replaces all bot rules with a special rule matching every request
to that action. The benchmark page makes heavy use of inline styles,
because currently all global styles are shared across all pages. This
could be fixed, but I wanted to avoid major changes to the templates.
* web/js: add signal for aborting an active proof-of-work algorithm
Both proof-of-work algorithms now take an optional `AbortSignal`, which
immediately terminates all workers and returns `false` if aborted before
the challenge is complete.
* web/js: add algorithm comparison to the benchmark page
"Compare:" is added to the benchmark page for testing the relative
performance between two algorithms. Since benchmark runs generally have
high variance, it may take a while for the averages to converge on a
stable difference.
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* Add periodic cleanup job for DecayMap
see https://github.com/TecharoHQ/anubis/issues/8
* Refactor: Improve DecayMap cleanup tests and add Len method
- Refactored DecayMap cleanup tests to use the new Len method
for more precise assertions.
- Added a Len method to DecayMap to retrieve the number of
entries.
- Simplified conditional checks in Get method.
* chore(changelog): add entry
* fix(tests): Use Impl.expire for decaymap cleanup
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Using TrimRight will remove all characters from `*dockerRepo` from right
to left that match a character contained on `"/"+filepath.Base(*dockerRepo)`
(the cutset) until it doesn't matches anymore.
So for example, if `dockerRepo` is `example.com/fijxu/anubis`, and
`"/"+filepath.Base(*dockerRepo)` is `/anubis`, it will remove
`u/anubis` and not just `/anubis` from `dockerRepo` because `u` is a character inside the
cutoff.
* Change how to make Anubis work without a reverse proxy
* Apply suggestions from code review
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Henri Vasserman <henv@hot.ee>
* add support for unix sockets.
* add env var docs
* lib: fix tests
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Henri Vasserman <henv@hot.ee>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* Refactor anubis to split business logic into a lib, and cmd to just be direct usage.
* Post-rebase fixes.
* Update changelog, remove unnecessary one.
* lib: refactor this
This is mostly based on my personal preferences for how Go code should
be laid out. I'm not sold on the package name "lib" (I'd call it anubis
but that would stutter), but people are probably gonna import it as
libanubis so it's likely fine.
Packages have been "flattened" to centralize implementation with area of
concern. This goes against the Java-esque style that many people like,
but I think this helps make things simple.
Most notably: the dnsbl client (which is a hack) is an internal package
until it's made more generic. Then it can be made external.
I also fixed the logic such that `go generate` works and rebased on
main.
* internal/test: run tests iff npx exists and DONT_USE_NETWORK is not set
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: install deps
Signed-off-by: Xe Iaso <me@xeiaso.net>
* .github/workflows: verbose go tests?
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: sleep 2
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: nix this test so CI works
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: warmup per browser?
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: disable for now :(
Signed-off-by: Xe Iaso <me@xeiaso.net>
* lib/anubis: do not apply bot rules if address check fails
Closes#83
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
The example/default bot policy document had a rule to allow RSS readers
through based on paths that end with ".rss", ".xml", ".atom", or
".json". Frameworks like Rails will treat these specially, meaning that
going to /things/12345-whateverhaha.json could bypass Anubis.
I checked the history of this rule and it was present in the original
example policy file in Xe/x. This rule is likely a mistake and it has
been removed. I think it was for making my blog still work with RSS
readers.
Thanks to Graham Sutherland for reporting this over email.
Signed-off-by: Xe Iaso <me@xeiaso.net>
hash.Write never returns error so removing it from
the results simplifies usage and eliminates dead error handling.
Signed-off-by: Alexander Yastrebov <yastrebov.alex@gmail.com>
* Added the possibility to define rules for remote addresses
* Added change in changelog
* Added check for X-Real-Ip and X-Forwarded-For when checking for remote address filtering
* cmd/anubis: refine IP filtering logic
* Optimize the configuration so that the IP trie is created once at
application start instead of dynamically being created every request.
* Document the changes in the changelog and docs site.
* Allow pure IP range filtering.
* Allow user agent based IP range filtering.
* Allow path based IP range filtering.
* Create --debug-x-real-ip-default flag for testing Anubis locally
without a HTTP load balancer.
---------
Co-authored-by: Xe Iaso <me@xeiaso.net>
Closes#30
Introduces the "challenge" field in bot rule definitions:
```json
{
"name": "generic-bot-catchall",
"user_agent_regex": "(?i:bot|crawler)",
"action": "CHALLENGE",
"challenge": {
"difficulty": 16,
"report_as": 4,
"algorithm": "slow"
}
}
```
This makes Anubis return a challenge page for every user agent with
"bot" or "crawler" in it (case-insensitively) with difficulty 16 using
the old "slow" algorithm but reporting in the client as difficulty 4.
This is useful when you want to make certain clients in particular
suffer.
Additional validation and testing logic has been added to make sure
that users do not define "impossible" challenge settings.
If no algorithm is specified, Anubis defaults to the "fast" algorithm.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* feat: allow binding to unix domain sockets
this is useful when the user does not want to expose more tcp ports than
needed. also simplifes configuration in some situation, like with nixos
modules as the socket paths can be automatically configured.
docs updated with additional configuration flags.
Signed-off-by: Cassie Cheung <me@soopy.moe>
* feat: graceful shutdown and cleanup on signal
this is needed to clean up left-over unix sockets, else on the next boot
listener panics with `address already in use`.
Co-authored-by: cat <cat@gensokyo.uk>
Signed-off-by: Cassie Cheung <me@soopy.moe>
* feat: support unix socket upstream targets
adds support for proxying unix socket upstreams, essentially allowing
anubis to run without listening on tcp sockets at all*.
*for metrics, neither prometheus and victoriametrics supports scraping
from unix sockets. if metrics are desired, tcp sockets are still needed.
Co-authored-by: cat <cat@gensokyo.uk>
Signed-off-by: Cassie Cheung <me@soopy.moe>
* docs: add changelog entry
---------
Signed-off-by: Cassie Cheung <me@soopy.moe>
Co-authored-by: cat <cat@gensokyo.uk>
* fix: no duplicate work when exceeding that 1xxx number
* run go generate and update CHANGELOG
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* cmd/anubis: drastically optimize proof of work
Closes#12Closes#17
This drastically optimizes the proof of work check by removing the
stringify call at every iteration. Additionally, this optimizes the
checks by running them in parallel for as many threads as the browser
has available (according to navigator.hardwareConcurrency).
This also changes the redirect lag to 250 milliseconds instead of 2000
milliseconds in order to be perceptually faster. This is below the
reaction time threshold of many people, so this will make the post-check
success phase perceptually instant.
Testing on an iPhone 7 Plus has shown that this can clear a difficulty 4
check in 3.4 seconds.
This actually optimizes the check so much it may be a logistical concern
for operators.
* cmd/anubis/js: fix happy cachebuster logic
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
