z0x/anubis
Code Activity

cmd/anubis: explain why users are seeing Anubis (#55)

Browse source 
* cmd/anubis: explain why users are seeing Anubis

Closes #25
Closes #38

Also includes the beginnings of a "user guides" section in the docs for
user-facing documentation.

* Update docs/docs/user/known-broken-extensions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Xe Iaso <me@xeiaso.net>

* cmd/anubis: fix indentation in index.templ

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Xe Iaso 2025-03-21 14:21:59 -04:00 • committed by GitHub
parent d3e509517c
commit e7b9b17b92
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 47 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

7
cmd/anubis/index.templ
View file

@ -186,6 +186,13 @@ templ index() {
<div></div>
<div></div>
</div>
<details>
<summary>Why am I seeing this?</summary>
<p>You are seeing this because the administrator of this website has set up <a href="https://github.com/TecharoHQ/anubis">Anubis</a> to protect the server against the scourge of <a href="https://thelibre.news/foss-infrastructure-is-under-attack-by-ai-companies/">AI companies aggressively scraping websites</a>. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.</p>
<p>Anubis is a compromise. Anubis uses a <a href="https://anubis.techaro.lol/docs/design/why-proof-of-work">Proof-of-Work</a> scheme in the vein of <a href="https://en.wikipedia.org/wiki/Hashcash">Hashcash</a>, a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.</p>
<p>Ultimately, this is a hack whose real purpose is to give a "good enough" placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of work page doesn't need to be presented to users that are much more likely to be legitimate.</p>
<p>Please note that Anubis requires the use of modern JavaScript features that plugins like <a href="https://jshelter.org/">JShelter</a> will disable. Please disable JShelter or other such plugins for this domain.</p>
</details>
<noscript>
<p>
Sadly, you must enable JavaScript to get past this challenge. This is required because AI companies have changed

6
cmd/anubis/index_templ.go
View file

@ -159,7 +159,7 @@ func index() templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 10, "\"></script><div id=\"spinner\" class=\"lds-roller\"><div></div><div></div><div></div><div></div><div></div><div></div><div></div><div></div></div><noscript><p>Sadly, you must enable JavaScript to get past this challenge. This is required because AI companies have changed the social contract around how website hosting works. A no-JS solution is a work-in-progress.</p></noscript><div id=\"testarea\"></div></div>")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 10, "\"></script><div id=\"spinner\" class=\"lds-roller\"><div></div><div></div><div></div><div></div><div></div><div></div><div></div><div></div></div><details><summary>Why am I seeing this?</summary><p>You are seeing this because the administrator of this website has set up <a href=\"https://github.com/TecharoHQ/anubis\">Anubis</a> to protect the server against the scourge of <a href=\"https://thelibre.news/foss-infrastructure-is-under-attack-by-ai-companies/\">AI companies aggressively scraping websites</a>. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.</p><p>Anubis is a compromise. Anubis uses a <a href=\"https://anubis.techaro.lol/docs/design/why-proof-of-work\">Proof-of-Work</a> scheme in the vein of <a href=\"https://en.wikipedia.org/wiki/Hashcash\">Hashcash</a>, a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.</p><p>Ultimately, this is a hack whose real purpose is to give a \"good enough\" placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of work page doesn't need to be presented to users that are much more likely to be legitimate.</p><p>Please note that Anubis requires the use of modern JavaScript features that plugins like <a href=\"https://jshelter.org/\">JShelter</a> will disable. Please disable JShelter or other such plugins for this domain.</p></details><noscript><p>Sadly, you must enable JavaScript to get past this challenge. This is required because AI companies have changed the social contract around how website hosting works. A no-JS solution is a work-in-progress.</p></noscript><div id=\"testarea\"></div></div>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
@ -195,7 +195,7 @@ func errorPage(message string) templ.Component {
var templ_7745c5c3_Var10 string
templ_7745c5c3_Var10, templ_7745c5c3_Err = templ.JoinStringErrs("/.within.website/x/cmd/anubis/static/img/sad.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 204, Col: 90}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 211, Col: 90}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var10))
if templ_7745c5c3_Err != nil {
@ -208,7 +208,7 @@ func errorPage(message string) templ.Component {
var templ_7745c5c3_Var11 string
templ_7745c5c3_Var11, templ_7745c5c3_Err = templ.JoinStringErrs(message)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 206, Col: 14}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 213, Col: 14}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var11))
if templ_7745c5c3_Err != nil {

1
docs/docs/CHANGELOG.md
View file

@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
- Add explanation of what Anubis is doing to the challenge page [#25](https://github.com/TecharoHQ/anubis/issues/25)
- Administrators can now define artificially hard challenges using the "slow" algorithm:
```json

8
docs/docs/user/_category_.json Normal file
View file

@ -0,0 +1,8 @@
{
"label": "User guides",
"position": 60,
"link": {
"type": "generated-index",
"description": "Information for users on sites that use Anubis."
}
}

19
docs/docs/user/known-broken-extensions.md Normal file
View file

@ -0,0 +1,19 @@
---
title: List of known browser extensions that can break Anubis
---
This page contains a list of all of the browser extensions that are known to break Anubis' functionality and their associated GitHub issues, along with instructions on how to work around the issue.
## [JShelter](https://jshelter.org/)
| Extension | JShelter |
| :----------- | :-------------------------------------------- |
| Website | [jshelter.org](https://jshelter.org/) |
| GitHub issue | https://github.com/TecharoHQ/anubis/issues/25 |
Workaround steps:
1. Open JShelter extension settings
2. Click on JS Shield details
3. Enter in the domain for a website protected by Anubis
4. Choose "Turn JavaScript Shield off"
5. Hit "Add to list"

9
docs/docs/user/why-see-challenge.md Normal file
View file

@ -0,0 +1,9 @@
---
title: Why is Anubis showing up on a website?
---
You are seeing Anubis because the administrator of that website has set up [Anubis](https://github.com/TecharoHQ/anubis) to protect the server against the scourge of [AI companies aggressively scraping websites](https://thelibre.news/foss-infrastructure-is-under-attack-by-ai-companies/). This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.
Anubis is a compromise. Anubis uses a [proof-of-work](/docs/design/why-proof-of-work) scheme in the vein of [Hashcash](https://en.wikipedia.org/wiki/Hashcash), a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.
Ultimately, this is a hack whose real purpose is to give a "good enough" placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of work page doesn't need to be presented to users that are much more likely to be legitimate.