z0x/anubis
Code Activity

initial import from /x/ monorepo

Browse source 
Signed-off-by: Xe Iaso <me@xeiaso.net>
This commit is contained in:
Xe Iaso 2025-03-17 19:33:07 -04:00
commit 9923878c5c
No known key found for this signature in database
61 changed files with 5615 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

62
.github/workflows/go.yml vendored Normal file
View file

@ -0,0 +1,62 @@
name: Go
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
permissions:
contents: read
actions: write
jobs:
build:
runs-on: alrest-techarohq
steps:
- uses: actions/checkout@v4
- name: build essential
run: |
sudo apt-get update
sudo apt-get install -y build-essential
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@master
- name: Setup Homebrew cellar cache
uses: actions/cache@v4
with:
path: |
/home/linuxbrew/.linuxbrew/Cellar
/home/linuxbrew/.linuxbrew/bin
/home/linuxbrew/.linuxbrew/etc
/home/linuxbrew/.linuxbrew/include
/home/linuxbrew/.linuxbrew/lib
/home/linuxbrew/.linuxbrew/opt
/home/linuxbrew/.linuxbrew/sbin
/home/linuxbrew/.linuxbrew/share
/home/linuxbrew/.linuxbrew/var
key: ${{ runner.os }}-go-homebrew-cellar-${{ hashFiles('go.sum') }}
restore-keys: |
${{ runner.os }}-go-homebrew-cellar-
- name: Install Brew dependencies
run: |
brew bundle
- name: Setup Golang caches
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-golang-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-golang-
- name: Build
run: go build ./...
- name: Test
run: go test ./...

2
.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
.env
*.rpm

3
Brewfile Normal file
View file

@ -0,0 +1,3 @@
# programming languages
brew "go@1.24"
brew "node"

19
LICENSE Normal file
View file

@ -0,0 +1,19 @@
Copyright (c) 2025 Xe Iaso <me@xeiaso.net>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

300
README.md Normal file
View file

@ -0,0 +1,300 @@
# Anubis
<center>
<img width=256 src="./cmd/anubis/static/img/happy.webp" alt="A smiling chibi dark-skinned anthro jackal with brown hair and tall ears looking victorious with a thumbs-up" />
</center>
![enbyware](https://pride-badges.pony.workers.dev/static/v1?label=enbyware&labelColor=%23555&stripeWidth=8&stripeColors=FCF434%2CFFFFFF%2C9C59D1%2C2C2C2C)
![GitHub Issues or Pull Requests by label](https://img.shields.io/github/issues/TecharoHQ/anubis)
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/TecharoHQ/anubis)
![language count](https://img.shields.io/github/languages/count/TecharoHQ/anubis)
![repo size](https://img.shields.io/github/repo-size/TecharoHQ/anubis)
Anubis [weighs the soul of your connection](https://en.wikipedia.org/wiki/Weighing_of_souls) using a sha256 proof-of-work challenge in order to protect upstream resources from scraper bots.
Installing and using this will likely result in your website not being indexed by some search engines. This is considered a feature of Anubis, not a bug.
This is a bit of a nuclear response, but AI scraper bots scraping so aggressively have forced my hand. I hate that I have to do this, but this is what we get for the modern Internet because bots don't conform to standards like robots.txt, even when they claim to.
In most cases, you should not need this and can probably get by using Cloudflare to protect a given origin. However, for circumstances where you can't or won't use Cloudflare, Anubis is there for you.
If you want to try this out, connect to [git.xeserv.us](https://git.xeserv.us).
## Support
If you run into any issues running Anubis, please [open an issue](https://github.com/Xe/x/issues/new?template=Blank+issue) and tag it with the Anubis tag. Please include all the information I would need to diagnose your issue.
For live chat, please join the [Patreon](https://patreon.com/cadey) and ask in the Patron discord in the channel `#anubis`.
## How Anubis works
Anubis uses a proof-of-work challenge to ensure that clients are using a modern browser and are able to calculate SHA-256 checksums. Anubis has a customizable difficulty for this proof-of-work challenge, but defaults to 5 leading zeroes.
```mermaid
---
title: Challenge generation and validation
---
flowchart TD
Backend("Backend")
Fail("Fail")
style PresentChallenge color:#FFFFFF, fill:#AA00FF, stroke:#AA00FF
style ValidateChallenge color:#FFFFFF, fill:#AA00FF, stroke:#AA00FF
style Backend color:#FFFFFF, stroke:#00C853, fill:#00C853
style Fail color:#FFFFFF, stroke:#FF2962, fill:#FF2962
subgraph Server
PresentChallenge("Present Challenge")
ValidateChallenge("Validate Challenge")
end
subgraph Client
Main("main.mjs")
Worker("Worker")
end
Main -- Request challenge --> PresentChallenge
PresentChallenge -- Return challenge & difficulty --> Main
Main -- Spawn worker --> Worker
Worker -- Successful challenge --> Main
Main -- Validate challenge --> ValidateChallenge
ValidateChallenge -- Return cookie --> Backend
ValidateChallenge -- If anything is wrong --> Fail
```
### Challenge presentation
Anubis decides to present a challenge using this logic:
- User-Agent contains `"Mozilla"`
- Request path is not in `/.well-known`, `/robots.txt`, or `/favicon.ico`
- Request path is not obviously an RSS feed (ends with `.rss`, `.xml`, or `.atom`)
This should ensure that git clients, RSS readers, and other low-harm clients can get through without issue, but high-risk clients such as browsers and AI scraper bots will get blocked.
```mermaid
---
title: Challenge presentation logic
---
flowchart LR
Request("Request")
Backend("Backend")
%%Fail("Fail")
PresentChallenge("Present
challenge")
HasMozilla{"Is browser
or scraper?"}
HasCookie{"Has cookie?"}
HasExpired{"Cookie expired?"}
HasSignature{"Has valid
signature?"}
RandomJitter{"Secondary
screening?"}
POWPass{"Proof of
work valid?"}
style PresentChallenge color:#FFFFFF, fill:#AA00FF, stroke:#AA00FF
style Backend color:#FFFFFF, stroke:#00C853, fill:#00C853
%%style Fail color:#FFFFFF, stroke:#FF2962, fill:#FF2962
Request --> HasMozilla
HasMozilla -- Yes --> HasCookie
HasMozilla -- No --> Backend
HasCookie -- Yes --> HasExpired
HasCookie -- No --> PresentChallenge
HasExpired -- Yes --> PresentChallenge
HasExpired -- No --> HasSignature
HasSignature -- Yes --> RandomJitter
HasSignature -- No --> PresentChallenge
RandomJitter -- Yes --> POWPass
RandomJitter -- No --> Backend
POWPass -- Yes --> Backend
PowPass -- No --> PresentChallenge
PresentChallenge -- Back again for another cycle --> Request
```
### Proof of passing challenges
When a client passes a challenge, Anubis sets an HTTP cookie named `"within.website-x-cmd-anubis-auth"` containing a signed [JWT](https://jwt.io/) (JSON Web Token). This JWT contains the following claims:
- `challenge`: The challenge string derived from user request metadata
- `nonce`: The nonce / iteration number used to generate the passing response
- `response`: The hash that passed Anubis' checks
- `iat`: When the token was issued
- `nbf`: One minute prior to when the token was issued
- `exp`: The token's expiry week after the token was issued
This ensures that the token has enough metadata to prove that the token is valid (due to the token's signature), but also so that the server can independently prove the token is valid. This cookie is allowed to be set without triggering an EU cookie banner notification; but depending on facts and circumstances, you may wish to disclose this to your users.
### Challenge format
Challenges are formed by taking some user request metadata and using that to generate a SHA-256 checksum. The following request headers are used:
- `Accept-Encoding`: The content encodings that the requestor supports, such as gzip.
- `Accept-Language`: The language that the requestor would prefer the server respond in, such as English.
- `X-Real-Ip`: The IP address of the requestor, as set by a reverse proxy server.
- `User-Agent`: The user agent string of the requestor.
- The current time in UTC rounded to the nearest week.
- The fingerprint (checksum) of Anubis' private ED25519 key.
This forms a fingerprint of the requestor using metadata that any requestor already is sending. It also uses time as an input, which is known to both the server and requestor due to the nature of linear timelines. Depending on facts and circumstances, you may wish to disclose this to your users.
### JWT signing
Anubis uses an ed25519 keypair to sign the JWTs issued when challenges are passed. Anubis will generate a new ed25519 keypair every time it starts. At this time, there is no way to share this keypair between instance of Anubis, but that will be addressed in future versions.
## Setting up Anubis
Anubis is meant to sit between your reverse proxy (such as Nginx or Caddy) and your target service. One instance of Anubis must be used per service you are protecting.
Anubis is shipped in the Docker image [`ghcr.io/xe/x/anubis:latest`](https://github.com/Xe/x/pkgs/container/x%2Fanubis). Other methods to install Anubis may exist, but the Docker image is currently the only supported method.
The Docker image runs Anubis as user ID 1000 and group ID 1000. If you are mounting external volumes into Anubis' container, please be sure they are owned by or writable to this user/group.
Anubis has very minimal system requirements. I suspect that 128Mi of ram may be sufficient for a large number of concurrent clients. Anubis may be a poor fit for apps that use WebSockets and maintain open connections, but I don't have enough real-world experience to know one way or another.
Anubis uses these environment variables for configuration:
| Environment Variable | Default value | Explanation |
| :------------------- | :------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `BIND` | `:8923` | The TCP port that Anubis listens on. |
| `DIFFICULTY` | `5` | The difficulty of the challenge, or the number of leading zeroes that must be in successful responses. |
| `METRICS_BIND` | `:9090` | The TCP port that Anubis serves Prometheus metrics on. |
| `POLICY_FNAME` | `/data/cfg/botPolicy.json` | The file containing [bot policy configuration](./docs/policies.md). See the bot policy documentation for more details. |
| `SERVE_ROBOTS_TXT` | `false` | If set `true`, Anubis will serve a default `robots.txt` file that disallows all known AI scrapers by name and then additionally disallows every scraper. This is useful if facts and circumstances make it difficult to change the underlying service to serve such a `robots.txt` file. |
| `TARGET` | `http://localhost:3923` | The URL of the service that Anubis should forward valid requests to. |
### Policies
Anubis has support for custom bot policies, matched by User-Agent string and request path. Check the [bot policy documentation](./docs/policies.md) for more information.
### Docker compose
Add Anubis to your compose file pointed at your service:
```yaml
services:
anubis-nginx:
image: ghcr.io/xe/x/anubis:latest
environment:
BIND: ":8080"
DIFFICULTY: "5"
METRICS_BIND: ":9090"
SERVE_ROBOTS_TXT: "true"
TARGET: "http://nginx"
ports:
- 8080:8080
nginx:
image: nginx
volumes:
- "./www:/usr/share/nginx/html"
```
### Kubernetes
This example makes the following assumptions:
- Your target service is listening on TCP port `5000`.
- Anubis will be listening on port `8080`.
Attach Anubis to your Deployment:
```yaml
containers:
# ...
- name: anubis
image: ghcr.io/xe/x/anubis:latest
imagePullPolicy: Always
env:
- name: "BIND"
value: ":8080"
- name: "DIFFICULTY"
value: "5"
- name: "METRICS_BIND"
value: ":9090"
- name: "SERVE_ROBOTS_TXT"
value: "true"
- name: "TARGET"
value: "http://localhost:5000"
resources:
limits:
cpu: 500m
memory: 128Mi
requests:
cpu: 250m
memory: 128Mi
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
```
Then add a Service entry for Anubis:
```diff
# ...
spec:
ports:
+ - protocol: TCP
+ port: 8080
+ targetPort: 8080
+ name: anubis
```
Then point your Ingress to the Anubis port:
```diff
rules:
- host: git.xeserv.us
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: git
port:
- name: http
+ name: anubis
```
## Known caveats
Anubis works with most programs without any issues as long as they're configured to trust `127.0.0.0/8` and `::1/128` as "valid proxy servers". Some combinations of reverse proxy and target application can have issues. This section documents them so that you can pattern-match and fix them.
### Caddy + Gitea/Forgejo
Gitea/Forgejo relies on the reverse proxy setting the `X-Real-Ip` header. Caddy does not do this out of the gate. Modify your Caddyfile like this:
```diff
ellenjoe.int.within.lgbt {
# ...
- reverse_proxy http://localhost:3000
+ reverse_proxy http://localhost:3000 {
+ header_up X-Real-Ip {remote_host}
+ }
# ...
}
```
Ensure that Gitea/Forgejo have `[security].REVERSE_PROXY_TRUSTED_PROXIES` set to the IP ranges that Anubis will appear from. Typically this is sufficient:
```ini
[security]
REVERSE_PROXY_TRUSTED_PROXIES = 127.0.0.0/8,::1/128
```
However if you are running Anubis in a separate Pod/Deployment in Kubernetes, you may have to adjust this to the IP range of the Pod space in your Container Networking Interface plugin:
```ini
[security]
REVERSE_PROXY_TRUSTED_PROXIES = 10.192.0.0/12
```

2
cmd/anubis/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
*.rpm
anubis

5
cmd/anubis/CHANGELOG.md Normal file
View file

@ -0,0 +1,5 @@
# CHANGELOG
## 2025-01-24
- Added support for custom bot policy documentation, allowing administrators to change how Anubis works to meet their needs.

70
cmd/anubis/botPolicies.json Normal file
View file

@ -0,0 +1,70 @@
{
"bots": [
{
"name": "amazonbot",
"user_agent_regex": "Amazonbot",
"action": "DENY"
},
{
"name": "googlebot",
"user_agent_regex": "\\+http\\:\\/\\/www\\.google\\.com/bot\\.html",
"action": "ALLOW"
},
{
"name": "bingbot",
"user_agent_regex": "\\+http\\:\\/\\/www\\.bing\\.com/bingbot\\.htm",
"action": "ALLOW"
},
{
"name": "qwantbot",
"user_agent_regex": "\\+https\\:\\/\\/help\\.qwant\\.com/bot/",
"action": "ALLOW"
},
{
"name": "us-artificial-intelligence-scraper",
"user_agent_regex": "\\+https\\:\\/\\/github\\.com\\/US-Artificial-Intelligence\\/scraper",
"action": "DENY"
},
{
"name": "well-known",
"path_regex": "^/.well-known/.*$",
"action": "ALLOW"
},
{
"name": "favicon",
"path_regex": "^/favicon.ico$",
"action": "ALLOW"
},
{
"name": "robots-txt",
"path_regex": "^/robots.txt$",
"action": "ALLOW"
},
{
"name": "rss-readers",
"path_regex": ".*\\.(rss|xml|atom|json)$",
"action": "ALLOW"
},
{
"name": "lightpanda",
"user_agent_regex": "^Lightpanda/.*$",
"action": "DENY"
},
{
"name": "headless-chrome",
"user_agent_regex": "HeadlessChrome",
"action": "DENY"
},
{
"name": "headless-chromium",
"user_agent_regex": "HeadlessChromium",
"action": "DENY"
},
{
"name": "generic-browser",
"user_agent_regex": "Mozilla",
"action": "CHALLENGE"
}
],
"dnsbl": true
}

87
cmd/anubis/decaymap.go Normal file
View file

@ -0,0 +1,87 @@
package main
import (
"sync"
"time"
)
func zilch[T any]() T {
var zero T
return zero
}
// DecayMap is a lazy key->value map. It's a wrapper around a map and a mutex. If values exceed their time-to-live, they are pruned at Get time.
type DecayMap[K comparable, V any] struct {
data map[K]decayMapEntry[V]
lock sync.RWMutex
}
type decayMapEntry[V any] struct {
Value V
expiry time.Time
}
// NewDecayMap creates a new DecayMap of key type K and value type V.
//
// Key types must be comparable to work with maps.
func NewDecayMap[K comparable, V any]() *DecayMap[K, V] {
return &DecayMap[K, V]{
data: make(map[K]decayMapEntry[V]),
}
}
// expire forcibly expires a key by setting its time-to-live one second in the past.
func (m *DecayMap[K, V]) expire(key K) bool {
m.lock.RLock()
val, ok := m.data[key]
m.lock.RUnlock()
if !ok {
return false
}
m.lock.Lock()
val.expiry = time.Now().Add(-1 * time.Second)
m.data[key] = val
m.lock.Unlock()
return true
}
// Get gets a value from the DecayMap by key.
//
// If a value has expired, forcibly delete it if it was not updated.
func (m *DecayMap[K, V]) Get(key K) (V, bool) {
m.lock.RLock()
value, ok := m.data[key]
m.lock.RUnlock()
if !ok {
return zilch[V](), false
}
if time.Now().After(value.expiry) {
m.lock.Lock()
// Since previously reading m.data[key], the value may have been updated.
// Delete the entry only if the expiry time is still the same.
if m.data[key].expiry == value.expiry {
delete(m.data, key)
}
m.lock.Unlock()
return zilch[V](), false
}
return value.Value, true
}
// Set sets a key value pair in the map.
func (m *DecayMap[K, V]) Set(key K, value V, ttl time.Duration) {
m.lock.Lock()
defer m.lock.Unlock()
m.data[key] = decayMapEntry[V]{
Value: value,
expiry: time.Now().Add(ttl),
}
}

31
cmd/anubis/decaymap_test.go Normal file
View file

@ -0,0 +1,31 @@
package main
import (
"testing"
"time"
)
func TestDecayMap(t *testing.T) {
dm := NewDecayMap[string, string]()
dm.Set("test", "hi", 5*time.Minute)
val, ok := dm.Get("test")
if !ok {
t.Error("somehow the test key was not set")
}
if val != "hi" {
t.Errorf("wanted value %q, got: %q", "hi", val)
}
ok = dm.expire("test")
if !ok {
t.Error("somehow could not force-expire the test key")
}
_, ok = dm.Get("test")
if ok {
t.Error("got value even though it was supposed to be expired")
}
}

159
cmd/anubis/index.templ Normal file
View file

@ -0,0 +1,159 @@
package main
import (
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/xess"
)
templ base(title string, body templ.Component) {
<!DOCTYPE html>
<html>
<head>
<title>{ title }</title>
<link rel="stylesheet" href={ xess.URL }/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<style>
body,
html {
height: 100%;
display: flex;
justify-content: center;
align-items: center;
width: 65ch;
margin-left: auto;
margin-right: auto;
}
.centered-div {
text-align: center;
}
.lds-roller,
.lds-roller div,
.lds-roller div:after {
box-sizing: border-box;
}
.lds-roller {
display: inline-block;
position: relative;
width: 80px;
height: 80px;
}
.lds-roller div {
animation: lds-roller 1.2s cubic-bezier(0.5, 0, 0.5, 1) infinite;
transform-origin: 40px 40px;
}
.lds-roller div:after {
content: " ";
display: block;
position: absolute;
width: 7.2px;
height: 7.2px;
border-radius: 50%;
background: currentColor;
margin: -3.6px 0 0 -3.6px;
}
.lds-roller div:nth-child(1) {
animation-delay: -0.036s;
}
.lds-roller div:nth-child(1):after {
top: 62.62742px;
left: 62.62742px;
}
.lds-roller div:nth-child(2) {
animation-delay: -0.072s;
}
.lds-roller div:nth-child(2):after {
top: 67.71281px;
left: 56px;
}
.lds-roller div:nth-child(3) {
animation-delay: -0.108s;
}
.lds-roller div:nth-child(3):after {
top: 70.90963px;
left: 48.28221px;
}
.lds-roller div:nth-child(4) {
animation-delay: -0.144s;
}
.lds-roller div:nth-child(4):after {
top: 72px;
left: 40px;
}
.lds-roller div:nth-child(5) {
animation-delay: -0.18s;
}
.lds-roller div:nth-child(5):after {
top: 70.90963px;
left: 31.71779px;
}
.lds-roller div:nth-child(6) {
animation-delay: -0.216s;
}
.lds-roller div:nth-child(6):after {
top: 67.71281px;
left: 24px;
}
.lds-roller div:nth-child(7) {
animation-delay: -0.252s;
}
.lds-roller div:nth-child(7):after {
top: 62.62742px;
left: 17.37258px;
}
.lds-roller div:nth-child(8) {
animation-delay: -0.288s;
}
.lds-roller div:nth-child(8):after {
top: 56px;
left: 12.28719px;
}
@keyframes lds-roller {
0% {
transform: rotate(0deg);
}
100% {
transform: rotate(360deg);
}
}
</style>
</head>
<body id="top">
<main>
<center>
<h1 id="title" class=".centered-div">{ title }</h1>
</center>
@body
<footer>
<center>
<p>Protected by <a href="https://xeiaso.net/blog/2025/anubis">Anubis</a> from <a href="https://within.website">Within</a>.</p>
</center>
</footer>
</main>
</body>
</html>
}
templ index() {
<div class="centered-div">
<img id="image" width="256" src={ "/.within.website/x/cmd/anubis/static/img/pensive.webp?cacheBuster=" + anubis.Version }/>
<img style="display:none;" width="256" src={ "/.within.website/x/cmd/anubis/static/img/happy.webp?cacheBuster=" + anubis.Version }/>
<p id="status">Loading...</p>
<script async type="module" src={ "/.within.website/x/cmd/anubis/static/js/main.mjs?cacheBuster=" + anubis.Version }></script>
<div id="spinner" class="lds-roller"><div></div><div></div><div></div><div></div><div></div><div></div><div></div><div></div></div>
<noscript>
<p>Sadly, you must enable JavaScript to get past this challenge. I would love to not have to do this, but god is dead and AI scrapers have destroyed the common good.</p>
</noscript>
<div id="testarea"></div>
</div>
}
templ errorPage(message string) {
<div class="centered-div">
<img id="image" width="256" src={ "/.within.website/x/cmd/anubis/static/img/sad.webp?cacheBuster=" + anubis.Version }/>
<p>{ message }.</p>
<button onClick="window.location.reload();">Try again</button>
<p><a href="/">Go home</a></p>
</div>
}

215
cmd/anubis/index_templ.go Normal file
View file

@ -0,0 +1,215 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.833
package main
//lint:file-ignore SA4006 This context is only used if a nested component is present.
import "github.com/a-h/templ"
import templruntime "github.com/a-h/templ/runtime"
import (
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/xess"
)
func base(title string, body templ.Component) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
return templ_7745c5c3_CtxErr
}
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
if !templ_7745c5c3_IsBuffer {
defer func() {
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
if templ_7745c5c3_Err == nil {
templ_7745c5c3_Err = templ_7745c5c3_BufErr
}
}()
}
ctx = templ.InitializeContext(ctx)
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
if templ_7745c5c3_Var1 == nil {
templ_7745c5c3_Var1 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<!doctype html><html><head><title>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var2 string
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(title)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 12, Col: 17}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</title><link rel=\"stylesheet\" href=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var3 string
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(xess.URL)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 13, Col: 41}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "\"><meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\"><style>\n body,\n html {\n height: 100%;\n display: flex;\n justify-content: center;\n align-items: center;\n width: 65ch;\n margin-left: auto;\n margin-right: auto;\n }\n\n .centered-div {\n text-align: center;\n }\n\n .lds-roller,\n .lds-roller div,\n .lds-roller div:after {\n box-sizing: border-box;\n }\n .lds-roller {\n display: inline-block;\n position: relative;\n width: 80px;\n height: 80px;\n }\n .lds-roller div {\n animation: lds-roller 1.2s cubic-bezier(0.5, 0, 0.5, 1) infinite;\n transform-origin: 40px 40px;\n }\n .lds-roller div:after {\n content: \" \";\n display: block;\n position: absolute;\n width: 7.2px;\n height: 7.2px;\n border-radius: 50%;\n background: currentColor;\n margin: -3.6px 0 0 -3.6px;\n }\n .lds-roller div:nth-child(1) {\n animation-delay: -0.036s;\n }\n .lds-roller div:nth-child(1):after {\n top: 62.62742px;\n left: 62.62742px;\n }\n .lds-roller div:nth-child(2) {\n animation-delay: -0.072s;\n }\n .lds-roller div:nth-child(2):after {\n top: 67.71281px;\n left: 56px;\n }\n .lds-roller div:nth-child(3) {\n animation-delay: -0.108s;\n }\n .lds-roller div:nth-child(3):after {\n top: 70.90963px;\n left: 48.28221px;\n }\n .lds-roller div:nth-child(4) {\n animation-delay: -0.144s;\n }\n .lds-roller div:nth-child(4):after {\n top: 72px;\n left: 40px;\n }\n .lds-roller div:nth-child(5) {\n animation-delay: -0.18s;\n }\n .lds-roller div:nth-child(5):after {\n top: 70.90963px;\n left: 31.71779px;\n }\n .lds-roller div:nth-child(6) {\n animation-delay: -0.216s;\n }\n .lds-roller div:nth-child(6):after {\n top: 67.71281px;\n left: 24px;\n }\n .lds-roller div:nth-child(7) {\n animation-delay: -0.252s;\n }\n .lds-roller div:nth-child(7):after {\n top: 62.62742px;\n left: 17.37258px;\n }\n .lds-roller div:nth-child(8) {\n animation-delay: -0.288s;\n }\n .lds-roller div:nth-child(8):after {\n top: 56px;\n left: 12.28719px;\n }\n @keyframes lds-roller {\n 0% {\n transform: rotate(0deg);\n }\n 100% {\n transform: rotate(360deg);\n }\n }\n </style></head><body id=\"top\"><main><center><h1 id=\"title\" class=\".centered-div\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var4 string
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(title)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 125, Col: 49}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "</h1></center>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = body.Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "<footer><center><p>Protected by <a href=\"https://xeiaso.net/blog/2025/anubis\">Anubis</a> from <a href=\"https://within.website\">Within</a>.</p></center></footer></main></body></html>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
return nil
})
}
func index() templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
return templ_7745c5c3_CtxErr
}
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
if !templ_7745c5c3_IsBuffer {
defer func() {
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
if templ_7745c5c3_Err == nil {
templ_7745c5c3_Err = templ_7745c5c3_BufErr
}
}()
}
ctx = templ.InitializeContext(ctx)
templ_7745c5c3_Var5 := templ.GetChildren(ctx)
if templ_7745c5c3_Var5 == nil {
templ_7745c5c3_Var5 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "<div class=\"centered-div\"><img id=\"image\" width=\"256\" src=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var6 string
templ_7745c5c3_Var6, templ_7745c5c3_Err = templ.JoinStringErrs("/.within.website/x/cmd/anubis/static/img/pensive.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 140, Col: 121}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var6))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 7, "\"> <img style=\"display:none;\" width=\"256\" src=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var7 string
templ_7745c5c3_Var7, templ_7745c5c3_Err = templ.JoinStringErrs("/.within.website/x/cmd/anubis/static/img/happy.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 141, Col: 130}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var7))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 8, "\"><p id=\"status\">Loading...</p><script async type=\"module\" src=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var8 string
templ_7745c5c3_Var8, templ_7745c5c3_Err = templ.JoinStringErrs("/.within.website/x/cmd/anubis/static/js/main.mjs?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 143, Col: 116}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var8))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 9, "\"></script><div id=\"spinner\" class=\"lds-roller\"><div></div><div></div><div></div><div></div><div></div><div></div><div></div><div></div></div><noscript><p>Sadly, you must enable JavaScript to get past this challenge. I would love to not have to do this, but god is dead and AI scrapers have destroyed the common good.</p></noscript><div id=\"testarea\"></div></div>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
return nil
})
}
func errorPage(message string) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
return templ_7745c5c3_CtxErr
}
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
if !templ_7745c5c3_IsBuffer {
defer func() {
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
if templ_7745c5c3_Err == nil {
templ_7745c5c3_Err = templ_7745c5c3_BufErr
}
}()
}
ctx = templ.InitializeContext(ctx)
templ_7745c5c3_Var9 := templ.GetChildren(ctx)
if templ_7745c5c3_Var9 == nil {
templ_7745c5c3_Var9 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 10, "<div class=\"centered-div\"><img id=\"image\" width=\"256\" src=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var10 string
templ_7745c5c3_Var10, templ_7745c5c3_Err = templ.JoinStringErrs("/.within.website/x/cmd/anubis/static/img/sad.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 154, Col: 117}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var10))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 11, "\"><p>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var11 string
templ_7745c5c3_Var11, templ_7745c5c3_Err = templ.JoinStringErrs(message)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `index.templ`, Line: 155, Col: 14}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var11))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 12, ".</p><button onClick=\"window.location.reload();\">Try again</button><p><a href=\"/\">Go home</a></p></div>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
return nil
})
}
var _ = templruntime.GeneratedTemplate

99
cmd/anubis/internal/config/config.go Normal file
View file

@ -0,0 +1,99 @@
package config
import (
"errors"
"fmt"
"regexp"
)
type Rule string
const (
RuleUnknown = ""
RuleAllow = "ALLOW"
RuleDeny = "DENY"
RuleChallenge = "CHALLENGE"
)
type Bot struct {
Name string `json:"name"`
UserAgentRegex *string `json:"user_agent_regex"`
PathRegex *string `json:"path_regex"`
Action Rule `json:"action"`
}
var (
ErrNoBotRulesDefined = errors.New("config: must define at least one (1) bot rule")
ErrBotMustHaveName = errors.New("config.Bot: must set name")
ErrBotMustHaveUserAgentOrPath = errors.New("config.Bot: must set either user_agent_regex, path_regex")
ErrBotMustHaveUserAgentOrPathNotBoth = errors.New("config.Bot: must set either user_agent_regex, path_regex, and not both")
ErrUnknownAction = errors.New("config.Bot: unknown action")
ErrInvalidUserAgentRegex = errors.New("config.Bot: invalid user agent regex")
ErrInvalidPathRegex = errors.New("config.Bot: invalid path regex")
)
func (b Bot) Valid() error {
var errs []error
if b.Name == "" {
errs = append(errs, ErrBotMustHaveName)
}
if b.UserAgentRegex == nil && b.PathRegex == nil {
errs = append(errs, ErrBotMustHaveUserAgentOrPath)
}
if b.UserAgentRegex != nil && b.PathRegex != nil {
errs = append(errs, ErrBotMustHaveUserAgentOrPathNotBoth)
}
if b.UserAgentRegex != nil {
if _, err := regexp.Compile(*b.UserAgentRegex); err != nil {
errs = append(errs, ErrInvalidUserAgentRegex, err)
}
}
if b.PathRegex != nil {
if _, err := regexp.Compile(*b.PathRegex); err != nil {
errs = append(errs, ErrInvalidPathRegex, err)
}
}
switch b.Action {
case RuleAllow, RuleChallenge, RuleDeny:
// okay
default:
errs = append(errs, fmt.Errorf("%w: %q", ErrUnknownAction, b.Action))
}
if len(errs) != 0 {
return fmt.Errorf("config: bot entry for %q is not valid:\n%w", b.Name, errors.Join(errs...))
}
return nil
}
type Config struct {
Bots []Bot `json:"bots"`
DNSBL bool `json:"dnsbl"`
}
func (c Config) Valid() error {
var errs []error
if len(c.Bots) == 0 {
errs = append(errs, ErrNoBotRulesDefined)
}
for _, b := range c.Bots {
if err := b.Valid(); err != nil {
errs = append(errs, err)
}
}
if len(errs) != 0 {
return fmt.Errorf("config is not valid:\n%w", errors.Join(errs...))
}
return nil
}

168
cmd/anubis/internal/config/config_test.go Normal file
View file

@ -0,0 +1,168 @@
package config
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"testing"
)
func p[V any](v V) *V { return &v }
func TestBotValid(t *testing.T) {
var tests = []struct {
name string
bot Bot
err error
}{
{
name: "simple user agent",
bot: Bot{
Name: "mozilla-ua",
Action: RuleChallenge,
UserAgentRegex: p("Mozilla"),
},
err: nil,
},
{
name: "simple path",
bot: Bot{
Name: "well-known-path",
Action: RuleAllow,
PathRegex: p("^/.well-known/.*$"),
},
err: nil,
},
{
name: "no rule name",
bot: Bot{
Action: RuleChallenge,
UserAgentRegex: p("Mozilla"),
},
err: ErrBotMustHaveName,
},
{
name: "no rule matcher",
bot: Bot{
Name: "broken-rule",
Action: RuleAllow,
},
err: ErrBotMustHaveUserAgentOrPath,
},
{
name: "both user-agent and path",
bot: Bot{
Name: "path-and-user-agent",
Action: RuleDeny,
UserAgentRegex: p("Mozilla"),
PathRegex: p("^/.secret-place/.*$"),
},
err: ErrBotMustHaveUserAgentOrPathNotBoth,
},
{
name: "unknown action",
bot: Bot{
Name: "Unknown action",
Action: RuleUnknown,
UserAgentRegex: p("Mozilla"),
},
err: ErrUnknownAction,
},
{
name: "invalid user agent regex",
bot: Bot{
Name: "mozilla-ua",
Action: RuleChallenge,
UserAgentRegex: p("a(b"),
},
err: ErrInvalidUserAgentRegex,
},
{
name: "invalid path regex",
bot: Bot{
Name: "mozilla-ua",
Action: RuleChallenge,
PathRegex: p("a(b"),
},
err: ErrInvalidPathRegex,
},
}
for _, cs := range tests {
cs := cs
t.Run(cs.name, func(t *testing.T) {
err := cs.bot.Valid()
if err == nil && cs.err == nil {
return
}
if err == nil && cs.err != nil {
t.Errorf("didn't get an error, but wanted: %v", cs.err)
}
if !errors.Is(err, cs.err) {
t.Logf("got wrong error from Valid()")
t.Logf("wanted: %v", cs.err)
t.Logf("got: %v", err)
t.Errorf("got invalid error from check")
}
})
}
}
func TestConfigValidKnownGood(t *testing.T) {
finfos, err := os.ReadDir("testdata/good")
if err != nil {
t.Fatal(err)
}
for _, st := range finfos {
st := st
t.Run(st.Name(), func(t *testing.T) {
fin, err := os.Open(filepath.Join("testdata", "good", st.Name()))
if err != nil {
t.Fatal(err)
}
defer fin.Close()
var c Config
if err := json.NewDecoder(fin).Decode(&c); err != nil {
t.Fatalf("can't decode file: %v", err)
}
if err := c.Valid(); err != nil {
t.Fatal(err)
}
})
}
}
func TestConfigValidBad(t *testing.T) {
finfos, err := os.ReadDir("testdata/bad")
if err != nil {
t.Fatal(err)
}
for _, st := range finfos {
st := st
t.Run(st.Name(), func(t *testing.T) {
fin, err := os.Open(filepath.Join("testdata", "bad", st.Name()))
if err != nil {
t.Fatal(err)
}
defer fin.Close()
var c Config
if err := json.NewDecoder(fin).Decode(&c); err != nil {
t.Fatalf("can't decode file: %v", err)
}
if err := c.Valid(); err == nil {
t.Fatal("validation should have failed but didn't somehow")
} else {
t.Log(err)
}
})
}
}

14
cmd/anubis/internal/config/testdata/bad/badregexes.json vendored Normal file
View file

@ -0,0 +1,14 @@
{
"bots": [
{
"name": "path-bad",
"path_regex": "a(b",
"action": "DENY"
},
{
"name": "user-agent-bad",
"user_agent_regex": "a(b",
"action": "DENY"
}
]
}

5
cmd/anubis/internal/config/testdata/bad/invalid.json vendored Normal file
View file

@ -0,0 +1,5 @@
{
"bots": [
{}
]
}

1
cmd/anubis/internal/config/testdata/bad/nobots.json vendored Normal file
View file

@ -0,0 +1 @@
{}

9
cmd/anubis/internal/config/testdata/good/challengemozilla.json vendored Normal file
View file

@ -0,0 +1,9 @@
{
"bots": [
{
"name": "generic-browser",
"user_agent_regex": "Mozilla",
"action": "CHALLENGE"
}
]
}

10
cmd/anubis/internal/config/testdata/good/everything_blocked.json vendored Normal file
View file

@ -0,0 +1,10 @@
{
"bots": [
{
"name": "everything",
"user_agent_regex": ".*",
"action": "DENY"
}
],
"dnsbl": false
}

95
cmd/anubis/internal/dnsbl/dnsbl.go Normal file
View file

@ -0,0 +1,95 @@
package dnsbl
import (
"errors"
"fmt"
"net"
"strings"
)
//go:generate go tool golang.org/x/tools/cmd/stringer -type=DroneBLResponse
type DroneBLResponse byte
const (
AllGood DroneBLResponse = 0
IRCDrone DroneBLResponse = 3
Bottler DroneBLResponse = 5
UnknownSpambotOrDrone DroneBLResponse = 6
DDOSDrone DroneBLResponse = 7
SOCKSProxy DroneBLResponse = 8
HTTPProxy DroneBLResponse = 9
ProxyChain DroneBLResponse = 10
OpenProxy DroneBLResponse = 11
OpenDNSResolver DroneBLResponse = 12
BruteForceAttackers DroneBLResponse = 13
OpenWingateProxy DroneBLResponse = 14
CompromisedRouter DroneBLResponse = 15
AutoRootingWorms DroneBLResponse = 16
AutoDetectedBotIP DroneBLResponse = 17
Unknown DroneBLResponse = 255
)
func Reverse(ip net.IP) string {
if ip.To4() != nil {
return reverse4(ip)
}
return reverse6(ip)
}
func reverse4(ip net.IP) string {
splitAddress := strings.Split(ip.String(), ".")
// swap first and last octet
splitAddress[0], splitAddress[3] = splitAddress[3], splitAddress[0]
// swap middle octets
splitAddress[1], splitAddress[2] = splitAddress[2], splitAddress[1]
return strings.Join(splitAddress, ".")
}
func reverse6(ip net.IP) string {
ipBytes := []byte(ip)
var sb strings.Builder
for i := len(ipBytes) - 1; i >= 0; i-- {
// Split the byte into two nibbles
highNibble := ipBytes[i] >> 4
lowNibble := ipBytes[i] & 0x0F
// Append the nibbles in reversed order
sb.WriteString(fmt.Sprintf("%x.%x.", lowNibble, highNibble))
}
return sb.String()[:len(sb.String())-1]
}
func Lookup(ipStr string) (DroneBLResponse, error) {
ip := net.ParseIP(ipStr)
if ip == nil {
return Unknown, errors.New("dnsbl: input is not an IP address")
}
revIP := Reverse(ip) + ".dnsbl.dronebl.org"
ips, err := net.LookupIP(revIP)
if err != nil {
var dnserr *net.DNSError
if errors.As(err, &dnserr) {
if dnserr.IsNotFound {
return AllGood, nil
}
}
return Unknown, err
}
if len(ips) != 0 {
for _, ip := range ips {
return DroneBLResponse(ip.To4()[3]), nil
}
}
return UnknownSpambotOrDrone, nil
}

55
cmd/anubis/internal/dnsbl/dnsbl_test.go Normal file
View file

@ -0,0 +1,55 @@
package dnsbl
import (
"fmt"
"net"
"testing"
)
func TestReverse4(t *testing.T) {
cases := []struct {
inp, out string
}{
{"1.2.3.4", "4.3.2.1"},
}
for _, cs := range cases {
t.Run(fmt.Sprintf("%s->%s", cs.inp, cs.out), func(t *testing.T) {
out := reverse4(net.ParseIP(cs.inp))
if out != cs.out {
t.Errorf("wanted %s\ngot: %s", cs.out, out)
}
})
}
}
func TestReverse6(t *testing.T) {
cases := []struct {
inp, out string
}{
{
inp: "1234:5678:9ABC:DEF0:1234:5678:9ABC:DEF0",
out: "0.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1.0.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1",
},
}
for _, cs := range cases {
t.Run(fmt.Sprintf("%s->%s", cs.inp, cs.out), func(t *testing.T) {
out := reverse6(net.ParseIP(cs.inp))
if out != cs.out {
t.Errorf("wanted %s, got: %s", cs.out, out)
}
})
}
}
func TestLookup(t *testing.T) {
resp, err := Lookup("27.65.243.194")
if err != nil {
t.Fatalf("it broked: %v", err)
}
t.Logf("response: %d", resp)
}

54
cmd/anubis/internal/dnsbl/droneblresponse_string.go Normal file
View file

@ -0,0 +1,54 @@
// Code generated by "stringer -type=DroneBLResponse"; DO NOT EDIT.
package dnsbl
import "strconv"
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[AllGood-0]
_ = x[IRCDrone-3]
_ = x[Bottler-5]
_ = x[UnknownSpambotOrDrone-6]
_ = x[DDOSDrone-7]
_ = x[SOCKSProxy-8]
_ = x[HTTPProxy-9]
_ = x[ProxyChain-10]
_ = x[OpenProxy-11]
_ = x[OpenDNSResolver-12]
_ = x[BruteForceAttackers-13]
_ = x[OpenWingateProxy-14]
_ = x[CompromisedRouter-15]
_ = x[AutoRootingWorms-16]
_ = x[AutoDetectedBotIP-17]
_ = x[Unknown-255]
}
const (
_DroneBLResponse_name_0 = "AllGood"
_DroneBLResponse_name_1 = "IRCDrone"
_DroneBLResponse_name_2 = "BottlerUnknownSpambotOrDroneDDOSDroneSOCKSProxyHTTPProxyProxyChainOpenProxyOpenDNSResolverBruteForceAttackersOpenWingateProxyCompromisedRouterAutoRootingWormsAutoDetectedBotIP"
_DroneBLResponse_name_3 = "Unknown"
)
var (
_DroneBLResponse_index_2 = [...]uint8{0, 7, 28, 37, 47, 56, 66, 75, 90, 109, 125, 142, 158, 175}
)
func (i DroneBLResponse) String() string {
switch {
case i == 0:
return _DroneBLResponse_name_0
case i == 3:
return _DroneBLResponse_name_1
case 5 <= i && i <= 17:
i -= 5
return _DroneBLResponse_name_2[_DroneBLResponse_index_2[i]:_DroneBLResponse_index_2[i+1]]
case i == 255:
return _DroneBLResponse_name_3
default:
return "DroneBLResponse(" + strconv.FormatInt(int64(i), 10) + ")"
}
}

71
cmd/anubis/js/main.mjs Normal file
View file

@ -0,0 +1,71 @@
import { process } from './proof-of-work.mjs';
import { testVideo } from './video.mjs';
// from Xeact
const u = (url = "", params = {}) => {
let result = new URL(url, window.location.href);
Object.entries(params).forEach((kv) => {
let [k, v] = kv;
result.searchParams.set(k, v);
});
return result.toString();
};
const imageURL = (mood) => {
return `/.within.website/x/cmd/anubis/static/img/${mood}.webp`;
};
(async () => {
const status = document.getElementById('status');
const image = document.getElementById('image');
const title = document.getElementById('title');
const spinner = document.getElementById('spinner');
// const testarea = document.getElementById('testarea');
// const videoWorks = await testVideo(testarea);
// console.log(`videoWorks: ${videoWorks}`);
// if (!videoWorks) {
// title.innerHTML = "Oh no!";
// status.innerHTML = "Checks failed. Please check your browser's settings and try again.";
// image.src = imageURL("sad");
// spinner.innerHTML = "";
// spinner.style.display = "none";
// return;
// }
status.innerHTML = 'Calculating...';
const { challenge, difficulty } = await fetch("/.within.website/x/cmd/anubis/api/make-challenge", { method: "POST" })
.then(r => {
if (!r.ok) {
throw new Error("Failed to fetch config");
}
return r.json();
})
.catch(err => {
title.innerHTML = "Oh no!";
status.innerHTML = `Failed to fetch config: ${err.message}`;
image.src = imageURL("sad");
spinner.innerHTML = "";
spinner.style.display = "none";
throw err;
});
status.innerHTML = `Calculating...<br/>Difficulty: ${difficulty}`;
const t0 = Date.now();
const { hash, nonce } = await process(challenge, difficulty);
const t1 = Date.now();
title.innerHTML = "Success!";
status.innerHTML = `Done! Took ${t1 - t0}ms, ${nonce} iterations`;
image.src = imageURL("happy");
spinner.innerHTML = "";
spinner.style.display = "none";
setTimeout(() => {
const redir = window.location.href;
window.location.href = u("/.within.website/x/cmd/anubis/api/pass-challenge", { response: hash, nonce, redir, elapsedTime: t1 - t0 });
}, 2000);
})();

62
cmd/anubis/js/proof-of-work.mjs Normal file
View file

@ -0,0 +1,62 @@
// https://dev.to/ratmd/simple-proof-of-work-in-javascript-3kgm
export function process(data, difficulty = 5) {
return new Promise((resolve, reject) => {
let webWorkerURL = URL.createObjectURL(new Blob([
'(', processTask(), ')()'
], { type: 'application/javascript' }));
let worker = new Worker(webWorkerURL);
worker.onmessage = (event) => {
worker.terminate();
resolve(event.data);
};
worker.onerror = (event) => {
worker.terminate();
reject();
};
worker.postMessage({
data,
difficulty
});
URL.revokeObjectURL(webWorkerURL);
});
}
function processTask() {
return function () {
const sha256 = (text) => {
const encoded = new TextEncoder().encode(text);
return crypto.subtle.digest("SHA-256", encoded.buffer).then((result) =>
Array.from(new Uint8Array(result))
.map((c) => c.toString(16).padStart(2, "0"))
.join(""),
);
};
addEventListener('message', async (event) => {
let data = event.data.data;
let difficulty = event.data.difficulty;
let hash;
let nonce = 0;
do {
hash = await sha256(data + nonce++);
} while (hash.substring(0, difficulty) !== Array(difficulty + 1).join('0'));
nonce -= 1; // last nonce was post-incremented
postMessage({
hash,
data,
difficulty,
nonce,
});
});
}.toString();
}

16
cmd/anubis/js/video.mjs Normal file
View file

@ -0,0 +1,16 @@
const videoElement = `<video id="videotest" width="0" height="0" src="/.within.website/x/cmd/anubis/static/testdata/black.mp4"></video>`;
export const testVideo = async (testarea) => {
testarea.innerHTML = videoElement;
return (await new Promise((resolve) => {
const video = document.getElementById('videotest');
video.oncanplay = () => {
testarea.style.display = "none";
resolve(true);
};
video.onerror = (ev) => {
testarea.style.display = "none";
resolve(false);
};
}));
};

574
cmd/anubis/main.go Normal file
View file

@ -0,0 +1,574 @@
package main
import (
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"crypto/subtle"
"embed"
"encoding/hex"
"encoding/json"
"flag"
"fmt"
"io"
"log"
"log/slog"
"math"
mrand "math/rand"
"net/http"
"net/http/httputil"
"net/url"
"os"
"strconv"
"strings"
"time"
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/cmd/anubis/internal/config"
"github.com/TecharoHQ/anubis/cmd/anubis/internal/dnsbl"
"github.com/TecharoHQ/anubis/internal"
"github.com/TecharoHQ/anubis/xess"
"github.com/a-h/templ"
"github.com/facebookgo/flagenv"
"github.com/golang-jwt/jwt/v5"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
var (
bind = flag.String("bind", ":8923", "TCP port to bind HTTP to")
challengeDifficulty = flag.Int("difficulty", 5, "difficulty of the challenge")
metricsBind = flag.String("metrics-bind", ":9090", "TCP port to bind metrics to")
robotsTxt = flag.Bool("serve-robots-txt", false, "serve a robots.txt file that disallows all robots")
policyFname = flag.String("policy-fname", "", "full path to anubis policy document (defaults to a sensible built-in policy)")
slogLevel = flag.String("slog-level", "INFO", "logging level (see https://pkg.go.dev/log/slog#hdr-Levels)")
target = flag.String("target", "http://localhost:3923", "target to reverse proxy to")
healthcheck = flag.Bool("healthcheck", false, "run a health check against Anubis")
//go:embed static botPolicies.json
static embed.FS
challengesIssued = promauto.NewCounter(prometheus.CounterOpts{
Name: "anubis_challenges_issued",
Help: "The total number of challenges issued",
})
challengesValidated = promauto.NewCounter(prometheus.CounterOpts{
Name: "anubis_challenges_validated",
Help: "The total number of challenges validated",
})
droneBLHits = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_dronebl_hits",
Help: "The total number of hits from DroneBL",
}, []string{"status"})
failedValidations = promauto.NewCounter(prometheus.CounterOpts{
Name: "anubis_failed_validations",
Help: "The total number of failed validations",
})
timeTaken = promauto.NewHistogram(prometheus.HistogramOpts{
Name: "anubis_time_taken",
Help: "The time taken for a browser to generate a response (milliseconds)",
Buckets: prometheus.ExponentialBucketsRange(1, math.Pow(2, 18), 19),
})
)
const (
cookieName = "within.website-x-cmd-anubis-auth"
staticPath = "/.within.website/x/cmd/anubis/"
)
//go:generate go tool github.com/a-h/templ/cmd/templ generate
//go:generate esbuild js/main.mjs --sourcemap --minify --bundle --outfile=static/js/main.mjs
//go:generate gzip -f -k static/js/main.mjs
//go:generate zstd -f -k --ultra -22 static/js/main.mjs
//go:generate brotli -fZk static/js/main.mjs
func doHealthCheck() error {
resp, err := http.Get("http://localhost" + *metricsBind + "/metrics")
if err != nil {
return fmt.Errorf("failed to fetch metrics: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected status code: %d", resp.StatusCode)
}
return nil
}
func main() {
flagenv.Parse()
flag.Parse()
internal.InitSlog(*slogLevel)
if *healthcheck {
if err := doHealthCheck(); err != nil {
log.Fatal(err)
}
return
}
s, err := New(*target, *policyFname)
if err != nil {
log.Fatal(err)
}
fmt.Println("Rule error IDs:")
for _, rule := range s.policy.Bots {
if rule.Action != config.RuleDeny {
continue
}
hash, err := rule.Hash()
if err != nil {
log.Fatalf("can't calculate checksum of rule %s: %v", rule.Name, err)
}
fmt.Printf("* %s: %s\n", rule.Name, hash)
}
fmt.Println()
mux := http.NewServeMux()
xess.Mount(mux)
mux.Handle(staticPath, internal.UnchangingCache(http.StripPrefix(staticPath, http.FileServerFS(static))))
// mux.HandleFunc("GET /.within.website/x/cmd/anubis/static/js/main.mjs", serveMainJSWithBestEncoding)
mux.HandleFunc("POST /.within.website/x/cmd/anubis/api/make-challenge", s.makeChallenge)
mux.HandleFunc("GET /.within.website/x/cmd/anubis/api/pass-challenge", s.passChallenge)
mux.HandleFunc("GET /.within.website/x/cmd/anubis/api/test-error", s.testError)
if *robotsTxt {
mux.HandleFunc("/robots.txt", func(w http.ResponseWriter, r *http.Request) {
http.ServeFileFS(w, r, static, "static/robots.txt")
})
mux.HandleFunc("/.well-known/robots.txt", func(w http.ResponseWriter, r *http.Request) {
http.ServeFileFS(w, r, static, "static/robots.txt")
})
}
if *metricsBind != "" {
go metricsServer()
}
mux.HandleFunc("/", s.maybeReverseProxy)
slog.Info("listening", "url", "http://localhost"+*bind, "difficulty", *challengeDifficulty, "serveRobotsTXT", *robotsTxt, "target", *target, "version", anubis.Version)
log.Fatal(http.ListenAndServe(*bind, mux))
}
func metricsServer() {
http.DefaultServeMux.Handle("/metrics", promhttp.Handler())
slog.Debug("listening for metrics", "url", "http://localhost"+*metricsBind)
log.Fatal(http.ListenAndServe(*metricsBind, nil))
}
func sha256sum(text string) (string, error) {
hash := sha256.New()
_, err := hash.Write([]byte(text))
if err != nil {
return "", err
}
return hex.EncodeToString(hash.Sum(nil)), nil
}
func (s *Server) challengeFor(r *http.Request) string {
fp := sha256.Sum256(s.priv.Seed())
data := fmt.Sprintf(
"Accept-Language=%s,X-Real-IP=%s,User-Agent=%s,WeekTime=%s,Fingerprint=%x,Difficulty=%d",
r.Header.Get("Accept-Language"),
r.Header.Get("X-Real-Ip"),
r.UserAgent(),
time.Now().UTC().Round(24*7*time.Hour).Format(time.RFC3339),
fp,
*challengeDifficulty,
)
result, _ := sha256sum(data)
return result
}
func New(target, policyFname string) (*Server, error) {
u, err := url.Parse(target)
if err != nil {
return nil, fmt.Errorf("failed to parse target URL: %w", err)
}
pub, priv, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
return nil, fmt.Errorf("failed to generate ed25519 key: %w", err)
}
rp := httputil.NewSingleHostReverseProxy(u)
var fin io.ReadCloser
if policyFname != "" {
fin, err = os.Open(policyFname)
if err != nil {
return nil, fmt.Errorf("can't parse policy file %s: %w", policyFname, err)
}
} else {
policyFname = "(static)/botPolicies.json"
fin, err = static.Open("botPolicies.json")
if err != nil {
return nil, fmt.Errorf("[unexpected] can't parse builtin policy file %s: %w", policyFname, err)
}
}
defer fin.Close()
policy, err := parseConfig(fin, policyFname)
if err != nil {
return nil, err // parseConfig sets a fancy error for us
}
return &Server{
rp: rp,
priv: priv,
pub: pub,
policy: policy,
dnsblCache: NewDecayMap[string, dnsbl.DroneBLResponse](),
}, nil
}
type Server struct {
rp *httputil.ReverseProxy
priv ed25519.PrivateKey
pub ed25519.PublicKey
policy *ParsedConfig
dnsblCache *DecayMap[string, dnsbl.DroneBLResponse]
}
func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request) {
cr, rule := s.check(r)
r.Header.Add("X-Anubis-Rule", cr.Name)
r.Header.Add("X-Anubis-Action", string(cr.Rule))
lg := slog.With(
"check_result", cr,
"user_agent", r.UserAgent(),
"accept_language", r.Header.Get("Accept-Language"),
"priority", r.Header.Get("Priority"),
"x-forwarded-for",
r.Header.Get("X-Forwarded-For"),
"x-real-ip", r.Header.Get("X-Real-Ip"),
)
policyApplications.WithLabelValues(cr.Name, string(cr.Rule)).Add(1)
ip := r.Header.Get("X-Real-Ip")
if s.policy.DNSBL && ip != "" {
resp, ok := s.dnsblCache.Get(ip)
if !ok {
lg.Debug("looking up ip in dnsbl")
resp, err := dnsbl.Lookup(ip)
if err != nil {
lg.Error("can't look up ip in dnsbl", "err", err)
}
s.dnsblCache.Set(ip, resp, 24*time.Hour)
droneBLHits.WithLabelValues(resp.String()).Inc()
}
if resp != dnsbl.AllGood {
lg.Info("DNSBL hit", "status", resp.String())
templ.Handler(base("Oh noes!", errorPage(fmt.Sprintf("DroneBL reported an entry: %s, see https://dronebl.org/lookup?ip=%s", resp.String(), ip))), templ.WithStatus(http.StatusOK)).ServeHTTP(w, r)
return
}
}
switch cr.Rule {
case config.RuleAllow:
lg.Debug("allowing traffic to origin (explicit)")
s.rp.ServeHTTP(w, r)
return
case config.RuleDeny:
clearCookie(w)
lg.Info("explicit deny")
if rule == nil {
lg.Error("rule is nil, cannot calculate checksum")
templ.Handler(base("Oh noes!", errorPage("Other internal server error (contact the admin)")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
hash, err := rule.Hash()
if err != nil {
lg.Error("can't calculate checksum of rule", "err", err)
templ.Handler(base("Oh noes!", errorPage("Other internal server error (contact the admin)")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
lg.Debug("rule hash", "hash", hash)
templ.Handler(base("Oh noes!", errorPage(fmt.Sprintf("Access Denied: error code %s", hash))), templ.WithStatus(http.StatusOK)).ServeHTTP(w, r)
return
case config.RuleChallenge:
lg.Debug("challenge requested")
default:
clearCookie(w)
templ.Handler(base("Oh noes!", errorPage("Other internal server error (contact the admin)")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
ckie, err := r.Cookie(cookieName)
if err != nil {
lg.Debug("cookie not found", "path", r.URL.Path)
clearCookie(w)
s.renderIndex(w, r)
return
}
if err := ckie.Valid(); err != nil {
lg.Debug("cookie is invalid", "err", err)
clearCookie(w)
s.renderIndex(w, r)
return
}
if time.Now().After(ckie.Expires) && !ckie.Expires.IsZero() {
lg.Debug("cookie expired", "path", r.URL.Path)
clearCookie(w)
s.renderIndex(w, r)
return
}
token, err := jwt.ParseWithClaims(ckie.Value, jwt.MapClaims{}, func(token *jwt.Token) (interface{}, error) {
return s.pub, nil
})
if !token.Valid {
lg.Debug("invalid token", "path", r.URL.Path)
clearCookie(w)
s.renderIndex(w, r)
return
}
claims := token.Claims.(jwt.MapClaims)
exp, ok := claims["exp"].(float64)
if !ok {
lg.Debug("exp is not int64", "ok", ok, "typeof(exp)", fmt.Sprintf("%T", exp))
clearCookie(w)
s.renderIndex(w, r)
return
}
if exp := time.Unix(int64(exp), 0); time.Now().After(exp) {
lg.Debug("token has expired", "exp", exp.Format(time.RFC3339))
clearCookie(w)
s.renderIndex(w, r)
return
}
if token.Valid && randomJitter() {
r.Header.Add("X-Anubis-Status", "PASS-BRIEF")
lg.Debug("cookie is not enrolled into secondary screening")
s.rp.ServeHTTP(w, r)
return
}
if claims["challenge"] != s.challengeFor(r) {
lg.Debug("invalid challenge", "path", r.URL.Path)
clearCookie(w)
s.renderIndex(w, r)
return
}
var nonce int
if v, ok := claims["nonce"].(float64); ok {
nonce = int(v)
}
calcString := fmt.Sprintf("%s%d", s.challengeFor(r), nonce)
calculated, err := sha256sum(calcString)
if err != nil {
lg.Error("failed to calculate sha256sum", "path", r.URL.Path, "err", err)
clearCookie(w)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
if subtle.ConstantTimeCompare([]byte(claims["response"].(string)), []byte(calculated)) != 1 {
lg.Debug("invalid response", "path", r.URL.Path)
failedValidations.Inc()
clearCookie(w)
s.renderIndex(w, r)
return
}
slog.Debug("all checks passed")
r.Header.Add("X-Anubis-Status", "PASS-FULL")
s.rp.ServeHTTP(w, r)
}
func (s *Server) renderIndex(w http.ResponseWriter, r *http.Request) {
templ.Handler(
base("Making sure you're not a bot!", index()),
).ServeHTTP(w, r)
}
func (s *Server) makeChallenge(w http.ResponseWriter, r *http.Request) {
challenge := s.challengeFor(r)
difficulty := *challengeDifficulty
lg := slog.With("user_agent", r.UserAgent(), "accept_language", r.Header.Get("Accept-Language"), "priority", r.Header.Get("Priority"), "x-forwarded-for", r.Header.Get("X-Forwarded-For"), "x-real-ip", r.Header.Get("X-Real-Ip"))
json.NewEncoder(w).Encode(struct {
Challenge string `json:"challenge"`
Difficulty int `json:"difficulty"`
}{
Challenge: challenge,
Difficulty: difficulty,
})
lg.Debug("made challenge", "challenge", challenge, "difficulty", difficulty)
challengesIssued.Inc()
}
func (s *Server) passChallenge(w http.ResponseWriter, r *http.Request) {
lg := slog.With("user_agent", r.UserAgent(), "accept_language", r.Header.Get("Accept-Language"), "priority", r.Header.Get("Priority"), "x-forwarded-for", r.Header.Get("X-Forwarded-For"), "x-real-ip", r.Header.Get("X-Real-Ip"))
nonceStr := r.FormValue("nonce")
if nonceStr == "" {
clearCookie(w)
lg.Debug("no nonce")
templ.Handler(base("Oh noes!", errorPage("missing nonce")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
elapsedTimeStr := r.FormValue("elapsedTime")
if elapsedTimeStr == "" {
clearCookie(w)
lg.Debug("no elapsedTime")
templ.Handler(base("Oh noes!", errorPage("missing elapsedTime")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
elapsedTime, err := strconv.ParseFloat(elapsedTimeStr, 64)
if err != nil {
clearCookie(w)
lg.Debug("elapsedTime doesn't parse", "err", err)
templ.Handler(base("Oh noes!", errorPage("invalid elapsedTime")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
lg.Info("challenge took", "elapsedTime", elapsedTime)
timeTaken.Observe(elapsedTime)
response := r.FormValue("response")
redir := r.FormValue("redir")
challenge := s.challengeFor(r)
nonce, err := strconv.Atoi(nonceStr)
if err != nil {
clearCookie(w)
lg.Debug("nonce doesn't parse", "err", err)
templ.Handler(base("Oh noes!", errorPage("invalid nonce")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
calcString := fmt.Sprintf("%s%d", challenge, nonce)
calculated, err := sha256sum(calcString)
if err != nil {
clearCookie(w)
lg.Debug("can't parse shasum", "err", err)
templ.Handler(base("Oh noes!", errorPage("failed to calculate sha256sum")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
if subtle.ConstantTimeCompare([]byte(response), []byte(calculated)) != 1 {
clearCookie(w)
lg.Debug("hash does not match", "got", response, "want", calculated)
templ.Handler(base("Oh noes!", errorPage("invalid response")), templ.WithStatus(http.StatusForbidden)).ServeHTTP(w, r)
failedValidations.Inc()
return
}
// compare the leading zeroes
if !strings.HasPrefix(response, strings.Repeat("0", *challengeDifficulty)) {
clearCookie(w)
lg.Debug("difficulty check failed", "response", response, "difficulty", *challengeDifficulty)
templ.Handler(base("Oh noes!", errorPage("invalid response")), templ.WithStatus(http.StatusForbidden)).ServeHTTP(w, r)
failedValidations.Inc()
return
}
// generate JWT cookie
token := jwt.NewWithClaims(jwt.SigningMethodEdDSA, jwt.MapClaims{
"challenge": challenge,
"nonce": nonce,
"response": response,
"iat": time.Now().Unix(),
"nbf": time.Now().Add(-1 * time.Minute).Unix(),
"exp": time.Now().Add(24 * 7 * time.Hour).Unix(),
})
tokenString, err := token.SignedString(s.priv)
if err != nil {
lg.Error("failed to sign JWT", "err", err)
clearCookie(w)
templ.Handler(base("Oh noes!", errorPage("failed to sign JWT")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
return
}
http.SetCookie(w, &http.Cookie{
Name: cookieName,
Value: tokenString,
Expires: time.Now().Add(24 * 7 * time.Hour),
SameSite: http.SameSiteLaxMode,
Path: "/",
})
challengesValidated.Inc()
lg.Debug("challenge passed, redirecting to app")
http.Redirect(w, r, redir, http.StatusFound)
}
func (s *Server) testError(w http.ResponseWriter, r *http.Request) {
err := r.FormValue("err")
templ.Handler(base("Oh noes!", errorPage(err)), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
}
func ohNoes(w http.ResponseWriter, r *http.Request, err error) {
slog.Error("super fatal error", "err", err)
templ.Handler(base("Oh noes!", errorPage("An internal server error happened")), templ.WithStatus(http.StatusInternalServerError)).ServeHTTP(w, r)
}
func clearCookie(w http.ResponseWriter) {
http.SetCookie(w, &http.Cookie{
Name: cookieName,
Value: "",
Expires: time.Now().Add(-1 * time.Hour),
MaxAge: -1,
SameSite: http.SameSiteLaxMode,
})
}
func randomJitter() bool {
return mrand.Intn(100) > 10
}
func serveMainJSWithBestEncoding(w http.ResponseWriter, r *http.Request) {
priorityList := []string{"zstd", "br", "gzip"}
enc2ext := map[string]string{
"zstd": "zst",
"br": "br",
"gzip": "gz",
}
for _, enc := range priorityList {
if strings.Contains(r.Header.Get("Accept-Encoding"), enc) {
w.Header().Set("Content-Type", "text/javascript")
w.Header().Set("Content-Encoding", enc)
http.ServeFileFS(w, r, static, "static/js/main.mjs."+enc2ext[enc])
return
}
}
w.Header().Set("Content-Type", "text/javascript")
http.ServeFileFS(w, r, static, "static/js/main.mjs")
}

146
cmd/anubis/policy.go Normal file
View file

@ -0,0 +1,146 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"regexp"
"github.com/TecharoHQ/anubis/cmd/anubis/internal/config"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)
var (
policyApplications = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_policy_results",
Help: "The results of each policy rule",
}, []string{"rule", "action"})
)
type ParsedConfig struct {
orig config.Config
Bots []Bot
DNSBL bool
}
type Bot struct {
Name string
UserAgent *regexp.Regexp
Path *regexp.Regexp
Action config.Rule `json:"action"`
}
func (b Bot) Hash() (string, error) {
var pathRex string
if b.Path != nil {
pathRex = b.Path.String()
}
var userAgentRex string
if b.UserAgent != nil {
userAgentRex = b.UserAgent.String()
}
return sha256sum(fmt.Sprintf("%s::%s::%s", b.Name, pathRex, userAgentRex))
}
func parseConfig(fin io.Reader, fname string) (*ParsedConfig, error) {
var c config.Config
if err := json.NewDecoder(fin).Decode(&c); err != nil {
return nil, fmt.Errorf("can't parse policy config JSON %s: %w", fname, err)
}
if err := c.Valid(); err != nil {
return nil, err
}
var err error
result := &ParsedConfig{
orig: c,
}
for _, b := range c.Bots {
if berr := b.Valid(); berr != nil {
err = errors.Join(err, berr)
continue
}
var botParseErr error
parsedBot := Bot{
Name: b.Name,
Action: b.Action,
}
if b.UserAgentRegex != nil {
userAgent, err := regexp.Compile(*b.UserAgentRegex)
if err != nil {
botParseErr = errors.Join(botParseErr, fmt.Errorf("while compiling user agent regexp: %w", err))
continue
} else {
parsedBot.UserAgent = userAgent
}
}
if b.PathRegex != nil {
path, err := regexp.Compile(*b.PathRegex)
if err != nil {
botParseErr = errors.Join(botParseErr, fmt.Errorf("while compiling path regexp: %w", err))
continue
} else {
parsedBot.Path = path
}
}
result.Bots = append(result.Bots, parsedBot)
}
if err != nil {
return nil, fmt.Errorf("errors validating policy config JSON %s: %w", fname, err)
}
result.DNSBL = c.DNSBL
return result, nil
}
type CheckResult struct {
Name string
Rule config.Rule
}
func (cr CheckResult) LogValue() slog.Value {
return slog.GroupValue(
slog.String("name", cr.Name),
slog.String("rule", string(cr.Rule)))
}
func cr(name string, rule config.Rule) CheckResult {
return CheckResult{
Name: name,
Rule: rule,
}
}
// Check evaluates the list of rules, and returns the result
func (s *Server) check(r *http.Request) (CheckResult, *Bot) {
for _, b := range s.policy.Bots {
if b.UserAgent != nil {
if b.UserAgent.MatchString(r.UserAgent()) {
return cr("bot/"+b.Name, b.Action), &b
}
}
if b.Path != nil {
if b.Path.MatchString(r.URL.Path) {
return cr("bot/"+b.Name, b.Action), &b
}
}
}
return cr("default/allow", config.RuleAllow), nil
}

65
cmd/anubis/policy_test.go Normal file
View file

@ -0,0 +1,65 @@
package main
import (
"os"
"path/filepath"
"testing"
)
func TestDefaultPolicyMustParse(t *testing.T) {
fin, err := static.Open("botPolicies.json")
if err != nil {
t.Fatal(err)
}
defer fin.Close()
if _, err := parseConfig(fin, "botPolicies.json"); err != nil {
t.Fatalf("can't parse config: %v", err)
}
}
func TestGoodConfigs(t *testing.T) {
finfos, err := os.ReadDir("internal/config/testdata/good")
if err != nil {
t.Fatal(err)
}
for _, st := range finfos {
st := st
t.Run(st.Name(), func(t *testing.T) {
fin, err := os.Open(filepath.Join("internal", "config", "testdata", "good", st.Name()))
if err != nil {
t.Fatal(err)
}
defer fin.Close()
if _, err := parseConfig(fin, fin.Name()); err != nil {
t.Fatal(err)
}
})
}
}
func TestBadConfigs(t *testing.T) {
finfos, err := os.ReadDir("internal/config/testdata/bad")
if err != nil {
t.Fatal(err)
}
for _, st := range finfos {
st := st
t.Run(st.Name(), func(t *testing.T) {
fin, err := os.Open(filepath.Join("internal", "config", "testdata", "bad", st.Name()))
if err != nil {
t.Fatal(err)
}
defer fin.Close()
if _, err := parseConfig(fin, fin.Name()); err == nil {
t.Fatal(err)
} else {
t.Log(err)
}
})
}
}

BIN
cmd/anubis/static/img/happy.webp Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 59 KiB

BIN
cmd/anubis/static/img/pensive.webp Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 48 KiB

BIN
cmd/anubis/static/img/sad.webp Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 50 KiB

2
cmd/anubis/static/js/main.mjs Normal file
View file

@ -0,0 +1,2 @@
(()=>{function l(n,s=5){return new Promise((i,e)=>{let o=URL.createObjectURL(new Blob(["(",w(),")()"],{type:"application/javascript"})),t=new Worker(o);t.onmessage=r=>{t.terminate(),i(r.data)},t.onerror=r=>{t.terminate(),e()},t.postMessage({data:n,difficulty:s}),URL.revokeObjectURL(o)})}function w(){return function(){let n=s=>{let i=new TextEncoder().encode(s);return crypto.subtle.digest("SHA-256",i.buffer).then(e=>Array.from(new Uint8Array(e)).map(o=>o.toString(16).padStart(2,"0")).join(""))};addEventListener("message",async s=>{let i=s.data.data,e=s.data.difficulty,o,t=0;do o=await n(i+t++);while(o.substring(0,e)!==Array(e+1).join("0"));t-=1,postMessage({hash:o,data:i,difficulty:e,nonce:t})})}.toString()}var h=(n="",s={})=>{let i=new URL(n,window.location.href);return Object.entries(s).forEach(e=>{let[o,t]=e;i.searchParams.set(o,t)}),i.toString()},m=n=>`/.within.website/x/cmd/anubis/static/img/${n}.webp`;(async()=>{let n=document.getElementById("status"),s=document.getElementById("image"),i=document.getElementById("title"),e=document.getElementById("spinner");n.innerHTML="Calculating...";let{challenge:o,difficulty:t}=await fetch("/.within.website/x/cmd/anubis/api/make-challenge",{method:"POST"}).then(a=>{if(!a.ok)throw new Error("Failed to fetch config");return a.json()}).catch(a=>{throw i.innerHTML="Oh no!",n.innerHTML=`Failed to fetch config: ${a.message}`,s.src=m("sad"),e.innerHTML="",e.style.display="none",a});n.innerHTML=`Calculating...<br/>Difficulty: ${t}`;let r=Date.now(),{hash:u,nonce:c}=await l(o,t),d=Date.now();i.innerHTML="Success!",n.innerHTML=`Done! Took ${d-r}ms, ${c} iterations`,s.src=m("happy"),e.innerHTML="",e.style.display="none",setTimeout(()=>{let a=window.location.href;window.location.href=h("/.within.website/x/cmd/anubis/api/pass-challenge",{response:u,nonce:c,redir:a,elapsedTime:d-r})},2e3)})();})();
//# sourceMappingURL=main.mjs.map

BIN
cmd/anubis/static/js/main.mjs.br Normal file
View file

Binary file not shown.

BIN
cmd/anubis/static/js/main.mjs.gz Normal file
View file

Binary file not shown.

7
cmd/anubis/static/js/main.mjs.map Normal file
View file

@ -0,0 +1,7 @@
{
"version": 3,
"sources": ["../../js/proof-of-work.mjs", "../../js/main.mjs"],
"sourcesContent": ["// https://dev.to/ratmd/simple-proof-of-work-in-javascript-3kgm\n\nexport function process(data, difficulty = 5) {\n return new Promise((resolve, reject) => {\n let webWorkerURL = URL.createObjectURL(new Blob([\n '(', processTask(), ')()'\n ], { type: 'application/javascript' }));\n\n let worker = new Worker(webWorkerURL);\n\n worker.onmessage = (event) => {\n worker.terminate();\n resolve(event.data);\n };\n\n worker.onerror = (event) => {\n worker.terminate();\n reject();\n };\n\n worker.postMessage({\n data,\n difficulty\n });\n\n URL.revokeObjectURL(webWorkerURL);\n });\n}\n\nfunction processTask() {\n return function () {\n const sha256 = (text) => {\n const encoded = new TextEncoder().encode(text);\n return crypto.subtle.digest(\"SHA-256\", encoded.buffer).then((result) =>\n Array.from(new Uint8Array(result))\n .map((c) => c.toString(16).padStart(2, \"0\"))\n .join(\"\"),\n );\n };\n\n addEventListener('message', async (event) => {\n let data = event.data.data;\n let difficulty = event.data.difficulty;\n\n let hash;\n let nonce = 0;\n do {\n hash = await sha256(data + nonce++);\n } while (hash.substring(0, difficulty) !== Array(difficulty + 1).join('0'));\n\n nonce -= 1; // last nonce was post-incremented\n\n postMessage({\n hash,\n data,\n difficulty,\n nonce,\n });\n });\n }.toString();\n}\n\n", "import { process } from './proof-of-work.mjs';\nimport { testVideo } from './video.mjs';\n\n// from Xeact\nconst u = (url = \"\", params = {}) => {\n let result = new URL(url, window.location.href);\n Object.entries(params).forEach((kv) => {\n let [k, v] = kv;\n result.searchParams.set(k, v);\n });\n return result.toString();\n};\n\nconst imageURL = (mood) => {\n return `/.within.website/x/cmd/anubis/static/img/${mood}.webp`;\n};\n\n(async () => {\n const status = document.getElementById('status');\n const image = document.getElementById('image');\n const title = document.getElementById('title');\n const spinner = document.getElementById('spinner');\n // const testarea = document.getElementById('testarea');\n\n // const videoWorks = await testVideo(testarea);\n // console.log(`videoWorks: ${videoWorks}`);\n\n // if (!videoWorks) {\n // title.innerHTML = \"Oh no!\";\n // status.innerHTML = \"Checks failed. Please check your browser's settings and try again.\";\n // image.src = imageURL(\"sad\");\n // spinner.innerHTML = \"\";\n // spinner.style.display = \"none\";\n // return;\n // }\n\n status.innerHTML = 'Calculating...';\n\n const { challenge, difficulty } = await fetch(\"/.within.website/x/cmd/anubis/api/make-challenge\", { method: \"POST\" })\n .then(r => {\n if (!r.ok) {\n throw new Error(\"Failed to fetch config\");\n }\n return r.json();\n })\n .catch(err => {\n title.innerHTML = \"Oh no!\";\n status.innerHTML = `Failed to fetch config: ${err.message}`;\n image.src = imageURL(\"sad\");\n spinner.innerHTML = \"\";\n spinner.style.display = \"none\";\n throw err;\n });\n\n status.innerHTML = `Calculating...<br/>Difficulty: ${difficulty}`;\n\n const t0 = Date.now();\n const { hash, nonce } = await process(challenge, difficulty);\n const t1 = Date.now();\n\n title.innerHTML = \"Success!\";\n status.innerHTML = `Done! Took ${t1 - t0}ms, ${nonce} iterations`;\n image.src = imageURL(\"happy\");\n spinner.innerHTML = \"\";\n spinner.style.display = \"none\";\n\n setTimeout(() => {\n const redir = window.location.href;\n window.location.href = u(\"/.within.website/x/cmd/anubis/api/pass-challenge\", { response: hash, nonce, redir, elapsedTime: t1 - t0 });\n }, 2000);\n})();"],
"mappings": "MAEO,SAASA,EAAQC,EAAMC,EAAa,EAAG,CAC5C,OAAO,IAAI,QAAQ,CAACC,EAASC,IAAW,CACtC,IAAIC,EAAe,IAAI,gBAAgB,IAAI,KAAK,CAC9C,IAAKC,EAAY,EAAG,KACtB,EAAG,CAAE,KAAM,wBAAyB,CAAC,CAAC,EAElCC,EAAS,IAAI,OAAOF,CAAY,EAEpCE,EAAO,UAAaC,GAAU,CAC5BD,EAAO,UAAU,EACjBJ,EAAQK,EAAM,IAAI,CACpB,EAEAD,EAAO,QAAWC,GAAU,CAC1BD,EAAO,UAAU,EACjBH,EAAO,CACT,EAEAG,EAAO,YAAY,CACjB,KAAAN,EACA,WAAAC,CACF,CAAC,EAED,IAAI,gBAAgBG,CAAY,CAClC,CAAC,CACH,CAEA,SAASC,GAAc,CACrB,OAAO,UAAY,CACjB,IAAMG,EAAUC,GAAS,CACvB,IAAMC,EAAU,IAAI,YAAY,EAAE,OAAOD,CAAI,EAC7C,OAAO,OAAO,OAAO,OAAO,UAAWC,EAAQ,MAAM,EAAE,KAAMC,GAC3D,MAAM,KAAK,IAAI,WAAWA,CAAM,CAAC,EAC9B,IAAKC,GAAMA,EAAE,SAAS,EAAE,EAAE,SAAS,EAAG,GAAG,CAAC,EAC1C,KAAK,EAAE,CACZ,CACF,EAEA,iBAAiB,UAAW,MAAOL,GAAU,CAC3C,IAAIP,EAAOO,EAAM,KAAK,KAClBN,EAAaM,EAAM,KAAK,WAExBM,EACAC,EAAQ,EACZ,GACED,EAAO,MAAML,EAAOR,EAAOc,GAAO,QAC3BD,EAAK,UAAU,EAAGZ,CAAU,IAAM,MAAMA,EAAa,CAAC,EAAE,KAAK,GAAG,GAEzEa,GAAS,EAET,YAAY,CACV,KAAAD,EACA,KAAAb,EACA,WAAAC,EACA,MAAAa,CACF,CAAC,CACH,CAAC,CACH,EAAE,SAAS,CACb,CCxDA,IAAMC,EAAI,CAACC,EAAM,GAAIC,EAAS,CAAC,IAAM,CACnC,IAAIC,EAAS,IAAI,IAAIF,EAAK,OAAO,SAAS,IAAI,EAC9C,cAAO,QAAQC,CAAM,EAAE,QAASE,GAAO,CACrC,GAAI,CAACC,EAAGC,CAAC,EAAIF,EACbD,EAAO,aAAa,IAAIE,EAAGC,CAAC,CAC9B,CAAC,EACMH,EAAO,SAAS,CACzB,EAEMI,EAAYC,GACT,4CAA4CA,CAAI,SAGxD,SAAY,CACX,IAAMC,EAAS,SAAS,eAAe,QAAQ,EACzCC,EAAQ,SAAS,eAAe,OAAO,EACvCC,EAAQ,SAAS,eAAe,OAAO,EACvCC,EAAU,SAAS,eAAe,SAAS,EAejDH,EAAO,UAAY,iBAEnB,GAAM,CAAE,UAAAI,EAAW,WAAAC,CAAW,EAAI,MAAM,MAAM,mDAAoD,CAAE,OAAQ,MAAO,CAAC,EACjH,KAAKC,GAAK,CACT,GAAI,CAACA,EAAE,GACL,MAAM,IAAI,MAAM,wBAAwB,EAE1C,OAAOA,EAAE,KAAK,CAChB,CAAC,EACA,MAAMC,GAAO,CACZ,MAAAL,EAAM,UAAY,SAClBF,EAAO,UAAY,2BAA2BO,EAAI,OAAO,GACzDN,EAAM,IAAMH,EAAS,KAAK,EAC1BK,EAAQ,UAAY,GACpBA,EAAQ,MAAM,QAAU,OAClBI,CACR,CAAC,EAEHP,EAAO,UAAY,kCAAkCK,CAAU,GAE/D,IAAMG,EAAK,KAAK,IAAI,EACd,CAAE,KAAAC,EAAM,MAAAC,CAAM,EAAI,MAAMC,EAAQP,EAAWC,CAAU,EACrDO,EAAK,KAAK,IAAI,EAEpBV,EAAM,UAAY,WAClBF,EAAO,UAAY,cAAcY,EAAKJ,CAAE,OAAOE,CAAK,cACpDT,EAAM,IAAMH,EAAS,OAAO,EAC5BK,EAAQ,UAAY,GACpBA,EAAQ,MAAM,QAAU,OAExB,WAAW,IAAM,CACf,IAAMU,EAAQ,OAAO,SAAS,KAC9B,OAAO,SAAS,KAAOtB,EAAE,mDAAoD,CAAE,SAAUkB,EAAM,MAAAC,EAAO,MAAAG,EAAO,YAAaD,EAAKJ,CAAG,CAAC,CACrI,EAAG,GAAI,CACT,GAAG",
"names": ["process", "data", "difficulty", "resolve", "reject", "webWorkerURL", "processTask", "worker", "event", "sha256", "text", "encoded", "result", "c", "hash", "nonce", "u", "url", "params", "result", "kv", "k", "v", "imageURL", "mood", "status", "image", "title", "spinner", "challenge", "difficulty", "r", "err", "t0", "hash", "nonce", "process", "t1", "redir"]
}

BIN
cmd/anubis/static/js/main.mjs.zst Normal file
View file

Binary file not shown.

47
cmd/anubis/static/robots.txt Normal file
View file

@ -0,0 +1,47 @@
User-agent: AI2Bot
User-agent: Ai2Bot-Dolma
User-agent: Amazonbot
User-agent: anthropic-ai
User-agent: Applebot
User-agent: Applebot-Extended
User-agent: Bytespider
User-agent: CCBot
User-agent: ChatGPT-User
User-agent: Claude-Web
User-agent: ClaudeBot
User-agent: cohere-ai
User-agent: cohere-training-data-crawler
User-agent: Diffbot
User-agent: DuckAssistBot
User-agent: FacebookBot
User-agent: FriendlyCrawler
User-agent: Google-Extended
User-agent: GoogleOther
User-agent: GoogleOther-Image
User-agent: GoogleOther-Video
User-agent: GPTBot
User-agent: iaskspider/2.0
User-agent: ICC-Crawler
User-agent: ImagesiftBot
User-agent: img2dataset
User-agent: ISSCyberRiskCrawler
User-agent: Kangaroo Bot
User-agent: Meta-ExternalAgent
User-agent: Meta-ExternalFetcher
User-agent: OAI-SearchBot
User-agent: omgili
User-agent: omgilibot
User-agent: PanguBot
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: Scrapy
User-agent: SemrushBot
User-agent: Sidetrade indexer bot
User-agent: Timpibot
User-agent: VelenPublicWebCrawler
User-agent: Webzio-Extended
User-agent: YouBot
Disallow: /
User-agent: *
Disallow: /

BIN
cmd/anubis/static/testdata/black.mp4 vendored Normal file
View file

Binary file not shown.

8
doc.go Normal file
View file

@ -0,0 +1,8 @@
// Package Anubis contains the version number of Anubis.
package anubis
// Version is the current version of Anubis.
//
// This variable is set at build time using the -X linker flag. If not set,
// it defaults to "devel".
var Version = "devel"

77
docs/policies.md Normal file
View file

@ -0,0 +1,77 @@
# Policies
Out of the box, Anubis is pretty heavy-handed. It will aggressively challenge everything that might be a browser (usually indicated by having `Mozilla` in its user agent). However, some bots are smart enough to get past the challenge. Some things that look like bots may actually be fine (IE: RSS readers). Some resources need to be visible no matter what. Some resources and remotes are fine to begin with.
Bot policies let you customize the rules that Anubis uses to allow, deny, or challenge incoming requests. Currently you can set policies by the following matches:
- Request path
- User agent string
Here's an example rule that denies [Amazonbot](https://developer.amazon.com/en/amazonbot):
```json
{
"name": "amazonbot",
"user_agent_regex": "Amazonbot",
"action": "DENY"
}
```
When this rule is evaluated, Anubis will check the `User-Agent` string of the request. If it contains `Amazonbot`, Anubis will send an error page to the user saying that access is denied, but in such a way that makes scrapers think they have correctly loaded the webpage.
Right now the only kinds of policies you can write are bot policies. Other forms of policies will be added in the future.
Here is a minimal policy file that will protect against most scraper bots:
```json
{
"bots": [
{
"name": "well-known",
"path_regex": "^/.well-known/.*$",
"action": "ALLOW"
},
{
"name": "favicon",
"path_regex": "^/favicon.ico$",
"action": "ALLOW"
},
{
"name": "robots-txt",
"path_regex": "^/robots.txt$",
"action": "ALLOW"
},
{
"name": "generic-browser",
"user_agent_regex": "Mozilla",
"action": "CHALLENGE"
}
]
}
```
This allows requests to [`/.well-known`](https://en.wikipedia.org/wiki/Well-known_URI), `/favicon.ico`, `/robots.txt`, and challenges any request that has the word `Mozilla` in its User-Agent string. The [default policy file](../botPolicies.json) is a bit more cohesive, but this should be more than enough for most users.
If no rules match the request, it is allowed through.
## Writing your own rules
There are three actions that can be returned from a rule:
| Action | Effects |
| :---------- | :-------------------------------------------------------------------------------- |
| `ALLOW` | Bypass all further checks and send the request to the backend. |
| `DENY` | Deny the request and send back an error message that scrapers think is a success. |
| `CHALLENGE` | Show a challenge page and/or validate that clients have passed a challenge. |
Name your rules in lower case using kebab-case. Rule names will be exposed in Prometheus metrics.
In case your service needs it for risk calculation reasons, Anubis exposes information about the rules that any requests match using a few headers:
| Header | Explanation | Example |
| :---------------- | :--------------------------------------------------- | :--------------- |
| `X-Anubis-Rule` | The name of the rule that was matched | `bot/lightpanda` |
| `X-Anubis-Action` | The action that Anubis took in response to that rule | `CHALLENGE` |
| `X-Anubis-Status` | The status and how strict Anubis was in its checks | `PASS-FULL` |
Policy rules are matched using [Go's standard library regular expressions package](https://pkg.go.dev/regexp). You can mess around with the syntax at [regex101.com](https://regex101.com), make sure to select the Golang option.

47
go.mod Normal file
View file

@ -0,0 +1,47 @@
module github.com/TecharoHQ/anubis
go 1.24.1
require (
github.com/a-h/templ v0.3.833
github.com/facebookgo/flagenv v0.0.0-20160425205200-fcd59fca7456
github.com/golang-jwt/jwt/v5 v5.2.1
github.com/prometheus/client_golang v1.21.1
)
require (
github.com/PuerkitoBio/goquery v1.10.1 // indirect
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e // indirect
github.com/andybalholm/brotli v1.1.0 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cli/browser v1.3.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 // indirect
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 // indirect
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/natefinch/atomic v1.0.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
golang.org/x/mod v0.24.0 // indirect
golang.org/x/net v0.37.0 // indirect
golang.org/x/sync v0.12.0 // indirect
golang.org/x/sys v0.31.0 // indirect
golang.org/x/tools v0.31.0 // indirect
google.golang.org/protobuf v1.36.4 // indirect
)
tool (
github.com/a-h/templ/cmd/templ
golang.org/x/tools/cmd/stringer
)

141
go.sum Normal file
View file

@ -0,0 +1,141 @@
github.com/PuerkitoBio/goquery v1.10.1 h1:Y8JGYUkXWTGRB6Ars3+j3kN0xg1YqqlwvdTV8WTFQcU=
github.com/PuerkitoBio/goquery v1.10.1/go.mod h1:IYiHrOMps66ag56LEH7QYDDupKXyo5A8qrjIx3ZtujY=
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e h1:HjVbSQHy+dnlS6C3XajZ69NYAb5jbGNfHanvm1+iYlo=
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e/go.mod h1:3mnrkvGpurZ4ZrTDbYU84xhwXW2TjTKShSwjRi2ihfQ=
github.com/a-h/templ v0.3.833 h1:L/KOk/0VvVTBegtE0fp2RJQiBm7/52Zxv5fqlEHiQUU=
github.com/a-h/templ v0.3.833/go.mod h1:cAu4AiZhtJfBjMY0HASlyzvkrtjnHWPeEsyGK2YYmfk=
github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cli/browser v1.3.0 h1:LejqCrpWr+1pRqmEPDGnTZOjsMe7sehifLynZJuqJpo=
github.com/cli/browser v1.3.0/go.mod h1:HH8s+fOAxjhQoBUAsKuPCbqUuxZDhQ2/aD+SzsEfBTk=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 h1:0JZ+dUmQeA8IIVUMzysrX4/AKuQwWhV2dYQuPZdvdSQ=
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51/go.mod h1:Yg+htXGokKKdzcwhuNDwVvN+uBxDGXJ7G/VN1d8fa64=
github.com/facebookgo/flagenv v0.0.0-20160425205200-fcd59fca7456 h1:CkmB2l68uhvRlwOTPrwnuitSxi/S3Cg4L5QYOcL9MBc=
github.com/facebookgo/flagenv v0.0.0-20160425205200-fcd59fca7456/go.mod h1:zFhibDvPDWmtk4dAQ05sRobtyoffEHygEt3wSNuAzz8=
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 h1:JWuenKqqX8nojtoVVWjGfOF9635RETekkoH6Cc9SX0A=
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052/go.mod h1:UbMTZqLaRiH3MsBH8va0n7s1pQYcu3uTb8G4tygF4Zg=
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870 h1:E2s37DuLxFhQDg5gKsWoLBOB0n+ZW8s599zru8FJ2/Y=
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870/go.mod h1:5tD+neXqOorC30/tWg0LCSkrqj/AR6gu8yY8/fpw1q0=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
github.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=
github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/natefinch/atomic v1.0.1 h1:ZPYKxkqQOx3KZ+RsbnP/YsgvxWQPGxjC0oBt2AhwV0A=
github.com/natefinch/atomic v1.0.1/go.mod h1:N/D/ELrljoqDyT3rZrsUmtsuzvHkeB/wWjHV22AZRbM=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.21.1 h1:DOvXXTqVzvkIewV/CDPFdejpMCGeMcbGCQ8YOmu+Ibk=
github.com/prometheus/client_golang v1.21.1/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg=
github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E=
github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY=
github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io=
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.24.0 h1:ZfthKaKaT4NrhGVZHO1/WDTwGES4De8KtWO0SIbNJMU=
golang.org/x/mod v0.24.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.37.0 h1:1zLorHbz+LYj7MQlSf1+2tPIIgibq2eL5xkrGk6f+2c=
golang.org/x/net v0.37.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw=
golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/tools v0.31.0 h1:0EedkvKDbh+qistFTd0Bcwe/YLh4vHwWEkiI0toFIBU=
golang.org/x/tools v0.31.0/go.mod h1:naFTU+Cev749tSJRXJlna0T3WxKvb1kWEx15xA4SdmQ=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.36.4 h1:6A3ZDJHn/eNqc1i+IdefRzy/9PokBTPvcqMySR7NNIM=
google.golang.org/protobuf v1.36.4/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

20
internal/headers.go Normal file
View file

@ -0,0 +1,20 @@
package internal
import (
"net/http"
"github.com/TecharoHQ/anubis"
)
// UnchangingCache sets the Cache-Control header to cache a response for 1 year if
// and only if the application is compiled in "release" mode by Docker.
func UnchangingCache(h http.Handler) http.Handler {
if anubis.Version == "devel" {
return h
}
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Cache-Control", "public, max-age=31536000")
h.ServeHTTP(w, r)
})
}

24
internal/slog.go Normal file
View file

@ -0,0 +1,24 @@
package internal
import (
"fmt"
"log/slog"
"os"
)
func InitSlog(level string) {
var programLevel slog.Level
if err := (&programLevel).UnmarshalText([]byte(level)); err != nil {
fmt.Fprintf(os.Stderr, "invalid log level %s: %v, using info\n", level, err)
programLevel = slog.LevelInfo
}
leveler := &slog.LevelVar{}
leveler.Set(programLevel)
h := slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{
AddSource: true,
Level: leveler,
})
slog.SetDefault(slog.New(h))
}

5
run/anubis.env.default Normal file
View file

@ -0,0 +1,5 @@
BIND=:8923
DIFFICULTY=3
METRICS_BIND=:9090
SERVE_ROBOTS_TXT=0
TARGET=http://localhost:3000

12
run/anubis@.service Normal file
View file

@ -0,0 +1,12 @@
[Unit]
Description="Anubis HTTP defense proxy (instance %i)"
[Service]
ExecStart=/usr/bin/anubis
Restart=always
RestartSec=30s
EnvironmentFile=/etc/anubis/anubis-%i.env
LimitNOFILE=infinity
[Install]
WantedBy=multi-user.target

2
var/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
*
!.gitignore

1
xess/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
node_modules

2411
xess/package-lock.json generated Normal file
View file

File diff suppressed because it is too large Load diff

20
xess/package.json Normal file
View file

@ -0,0 +1,20 @@
{
"name": "@xeserv/xess",
"version": "1.0.0",
"description": "Xe's CSS",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"build": "postcss xess.css -o xess.min.css"
},
"author": "",
"license": "ISC",
"devDependencies": {
"cssnano": "^7.0.6",
"cssnano-preset-advanced": "^7.0.6",
"postcss-cli": "^11.0.0",
"postcss-import": "^16.1.0",
"postcss-import-url": "^7.2.0",
"postcss-url": "^10.1.3"
}
}

8
xess/postcss.config.js Normal file
View file

@ -0,0 +1,8 @@
module.exports = {
plugins: [
require("cssnano")({
preset: "advanced",
}),
require("postcss-url")({ url: "inline" }),
],
};

BIN
xess/static/geist.woff2 Normal file
View file

Binary file not shown.

BIN
xess/static/iosevka-curly.woff2 Normal file
View file

Binary file not shown.

7
xess/static/podkova.css Normal file
View file

@ -0,0 +1,7 @@
@font-face {
font-family: "Podkova";
font-style: normal;
font-weight: 400 800;
font-display: swap;
src: url("podkova.woff2") format("woff2");
}

BIN
xess/static/podkova.woff2 Normal file
View file

Binary file not shown.

111
xess/xess.css Normal file
View file

@ -0,0 +1,111 @@
@font-face {
font-family: "Geist";
font-style: normal;
font-weight: 100 900;
font-display: swap;
src: url("./static/geist.woff2") format("woff2");
}
@font-face {
font-family: "Podkova";
font-style: normal;
font-weight: 400 800;
font-display: swap;
src: url("./static/podkova.woff2") format("woff2");
}
@font-face {
font-family: "Iosevka Curly";
font-style: monospace;
font-display: swap;
src: url("./static/iosevka-curly.woff2") format("woff2");
}
main {
font-family: Geist, sans-serif;
max-width: 50rem;
padding: 2rem;
margin: auto;
}
@media only screen and (max-device-width: 736px) {
main {
padding: 0;
}
}
::selection {
background: #d3869b;
}
body {
background: #1d2021;
color: #f9f5d7;
}
pre {
background-color: #3c3836;
padding: 1em;
border: 0;
font-family: Iosevka Curly Iaso, monospace;
}
a,
a:active,
a:visited {
color: #b16286;
background-color: #282828;
}
h1,
h2,
h3,
h4,
h5 {
margin-bottom: 0.1rem;
font-family: Podkova, serif;
}
blockquote {
border-left: 1px solid #bdae93;
margin: 0.5em 10px;
padding: 0.5em 10px;
}
footer {
text-align: center;
}
@media (prefers-color-scheme: light) {
body {
background: #f9f5d7;
color: #1d2021;
}
pre {
background-color: #ebdbb2;
padding: 1em;
border: 0;
}
a,
a:active,
a:visited {
color: #b16286;
background-color: #fbf1c7;
}
h1,
h2,
h3,
h4,
h5 {
margin-bottom: 0.1rem;
}
blockquote {
border-left: 1px solid #655c54;
margin: 0.5em 10px;
padding: 0.5em 10px;
}
}

38
xess/xess.go Normal file
View file

@ -0,0 +1,38 @@
// Package xess vendors a copy of Xess and makes it available at /.xess/xess.css
//
// This is intended to be used as a vendored package in other projects.
package xess
import (
"embed"
"net/http"
"path/filepath"
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/internal"
)
//go:generate go run github.com/a-h/templ/cmd/templ@latest generate
//go:generate npm ci
//go:generate npm run build
var (
//go:embed xess.min.css xess.css static
Static embed.FS
URL = "/.within.website/x/xess/xess.css"
)
func init() {
Mount(http.DefaultServeMux)
if anubis.Version != "devel" {
URL = filepath.Join(filepath.Dir(URL), "xess.min.css")
}
URL = URL + "?cachebuster=" + anubis.Version
}
func Mount(mux *http.ServeMux) {
mux.Handle("/.within.website/x/xess/", internal.UnchangingCache(http.StripPrefix("/.within.website/x/xess/", http.FileServerFS(Static))))
}

1
xess/xess.min.css vendored Normal file
View file

File diff suppressed because one or more lines are too long

41
xess/xess.templ Normal file
View file

@ -0,0 +1,41 @@
package xess
templ Base(title string, headArea, navBar, bodyArea, footer templ.Component) {
<!DOCTYPE html>
<html>
<head>
<title>{ title }</title>
<link rel="stylesheet" href={ URL }/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
if headArea != nil {
@headArea
}
</head>
<body id="top">
<main>
if navBar != nil {
<nav>
@navBar
</nav>
}
<h1>{ title }</h1>
@bodyArea
if footer != nil {
<footer>
@footer
</footer>
}
</main>
</body>
</html>
}
templ Simple(title string, body templ.Component) {
@Base(
title,
nil,
nil,
body,
nil,
)
}

164
xess/xess_templ.go Normal file
View file

@ -0,0 +1,164 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.833
package xess
//lint:file-ignore SA4006 This context is only used if a nested component is present.
import "github.com/a-h/templ"
import templruntime "github.com/a-h/templ/runtime"
func Base(title string, headArea, navBar, bodyArea, footer templ.Component) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
return templ_7745c5c3_CtxErr
}
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
if !templ_7745c5c3_IsBuffer {
defer func() {
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
if templ_7745c5c3_Err == nil {
templ_7745c5c3_Err = templ_7745c5c3_BufErr
}
}()
}
ctx = templ.InitializeContext(ctx)
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
if templ_7745c5c3_Var1 == nil {
templ_7745c5c3_Var1 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<!doctype html><html><head><title>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var2 string
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(title)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `xess.templ`, Line: 7, Col: 17}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</title><link rel=\"stylesheet\" href=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var3 string
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(URL)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `xess.templ`, Line: 8, Col: 36}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "\"><meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
if headArea != nil {
templ_7745c5c3_Err = headArea.Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "</head><body id=\"top\"><main>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
if navBar != nil {
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "<nav>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = navBar.Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "</nav>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 7, "<h1>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var4 string
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(title)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `xess.templ`, Line: 21, Col: 15}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 8, "</h1>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = bodyArea.Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
if footer != nil {
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 9, "<footer>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = footer.Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 10, "</footer>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 11, "</main></body></html>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
return nil
})
}
func Simple(title string, body templ.Component) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
return templ_7745c5c3_CtxErr
}
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
if !templ_7745c5c3_IsBuffer {
defer func() {
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
if templ_7745c5c3_Err == nil {
templ_7745c5c3_Err = templ_7745c5c3_BufErr
}
}()
}
ctx = templ.InitializeContext(ctx)
templ_7745c5c3_Var5 := templ.GetChildren(ctx)
if templ_7745c5c3_Var5 == nil {
templ_7745c5c3_Var5 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = Base(
title,
nil,
nil,
body,
nil,
).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
return nil
})
}
var _ = templruntime.GeneratedTemplate

22
yeetfile.js Normal file
View file

@ -0,0 +1,22 @@
go.install();
["amd64", "arm64"].forEach(goarch => rpm.build({
name: "anubis",
description: "Anubis weighs the souls of incoming HTTP requests and uses a sha256 proof-of-work challenge in order to protect upstream resources from scraper bots.",
homepage: "https://xeiaso.net/blog/2025/anubis",
license: "MIT",
goarch,
build: (out) => {
// install Anubis binary
go.build("-o", `${out}/usr/bin/anubis`);
// install systemd unit
yeet.run("mkdir", "-p", `${out}/usr/lib/systemd/system`);
yeet.run("cp", "run/anubis@.service", `${out}/usr/lib/systemd/system/anubis@.service`);
// install default config
yeet.run("mkdir", "-p", `${out}/etc/anubis`);
yeet.run("cp", "run/anubis.env.default", `${out}/etc/anubis/anubis-default.env`);
},
}));