Skip to content

Anubis proof-of-work for AI crawler protection

Require solving a hash puzzle before accessing content. Anubis sits in the background and weighs the risk of incoming requests. If it asks a client to complete a challenge, no user interaction is required. https://anubis.techaro.lol/

anubis

Anubis is a proof-of-work based anti-bot system that:

  • Requires computational work from the user/bot
  • Anubis sits in the background and weighs the risk of incoming requests. If it asks a client to complete a challenge, no user interaction is required. Invisible to humans using modern browsers
  • Visible cost for automated scrapers
  • Open-source and self-hostable

The way to integrate this is to break your configuration up into two parts: TLS termination and then HTTP routing. Consider this diagram:

---
title: Apache as tls terminator and HTTP router
---

flowchart LR
    T(User Traffic)
    subgraph Apache 2
        TCP(TCP 80/443)
        US(TCP 3001)
    end

    An(Anubis)
    B(Backend)

    T --> |TLS termination| TCP
    TCP --> |Traffic filtering| An
    An --> |Happy traffic| US
    US --> |whatever you're doing| B

Effectively you have one trip through Apache to do TLS termination, a detour through Anubis for traffic scrubbing, and then going to the backend directly. This final socket is what will do HTTP routing.

Configuration from (https://anubis.techaro.lol/docs/admin/environments/apache)

Assuming you are protecting anubistest.techaro.lol, you need the following server configuration blocks:

  1. A block on port 80 that forwards HTTP to HTTPS
  2. A block on port 443 that terminates TLS and forwards to Anubis
  3. A block on port 3001 that actually serves your websites
Bash
# Plain HTTP redirect to HTTPS
<VirtualHost *:80>
       ServerAdmin your@email.here
       ServerName anubistest.techaro.lol
       DocumentRoot /var/www/anubistest.techaro.lol
       ErrorLog /var/log/httpd/anubistest.techaro.lol_error.log
       CustomLog /var/log/httpd/anubistest.techaro.lol_access.log combined
       RewriteEngine on
       RewriteCond %{SERVER_NAME} =anubistest.techaro.lol
       RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

# HTTPS listener that forwards to Anubis
<IfModule mod_proxy.c>
<VirtualHost *:443>
       ServerAdmin your@email.here
       ServerName anubistest.techaro.lol
       DocumentRoot /var/www/anubistest.techaro.lol
       ErrorLog /var/log/httpd/anubistest.techaro.lol_error.log
       CustomLog /var/log/httpd/anubistest.techaro.lol_access.log combined

       SSLCertificateFile /etc/letsencrypt/live/anubistest.techaro.lol/fullchain.pem
       SSLCertificateKeyFile /etc/letsencrypt/live/anubistest.techaro.lol/privatekey.pem
       Include /etc/letsencrypt/options-ssl-apache.conf

       # These headers need to be set or else Anubis will
       # throw an "admin misconfiguration" error.
       RequestHeader set "X-Real-Ip" expr=%{REMOTE_ADDR}
       RequestHeader set X-Forwarded-Proto "https"
       RequestHeader set "X-Http-Version" "%{SERVER_PROTOCOL}s"

       ProxyPreserveHost On

       ProxyRequests Off
       ProxyVia Off

       # Replace 9000 with the port Anubis listens on
       ProxyPass / http://[::1]:9000/
       ProxyPassReverse / http://[::1]:9000/
</VirtualHost>
</IfModule>

# Actual website config
<VirtualHost *:3001>
       ServerAdmin your@email.here
       ServerName anubistest.techaro.lol
       DocumentRoot /var/www/anubistest.techaro.lol
       ErrorLog /var/log/httpd/anubistest.techaro.lol_error.log
       CustomLog /var/log/httpd/anubistest.techaro.lol_access.log combined

       # Pass the remote IP to the proxied application instead of 127.0.0.1
       # This requires mod_remoteip
       RemoteIPHeader X-Real-IP
       RemoteIPTrustedProxy 127.0.0.1/32
</VirtualHost>

Make sure to add a separate configuration file for the listener on port 3001:

Bash
sudo nano /etc/httpd/conf.d/listener-3001.conf
Nginx Configuration File
Listen 127.0.0.1:3001

In case you are running an IPv6-only system, use the following configuration instead:

Bash
nano /etc/httpd/conf.d/listener-3001.conf

In case you are running an IPv4-only system, use the following configuration instead:

Nginx Configuration File
Listen [::1]:3001

This can be repeated for multiple sites. Anubis does not care about the HTTP Host header and will happily cope with multiple websites via the same instance.

Then reload your Apache config and load your website. You should see Anubis protecting your apps!

Bash
sudo systemctl reload httpd.service
Copyright and License

Copyright (c) 2025 Xe Iaso me@xeiaso.net

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

```