diff --git a/content/network/IPAM.md b/content/network/IPAM.md index 2f11f56..c00562b 100644 --- a/content/network/IPAM.md +++ b/content/network/IPAM.md @@ -35,12 +35,15 @@ IP address tables |collector.dn42|172.20.129.4|fd42:4242:2601:ac12::1|Global Route Collector| |pingable.burble.dn42|172.20.129.5|fd42:4242:2601:ac05::1|Pingable IP Address| |nats.burble.dn42|172.20.129.6|fd42:4242:2601:ac06::1|nats.io Cluster| -|rproxy.burble.dn42|172.20.129.7|fd42:4242:2601:acf0::1|Distributed NGINX Reverse Proxy| +| |172.20.129.7| |_Unallocated_| |whois.burble.dn42|172.20.129.8|fd42:4242:2601:ac43::1|Whois service| |voip.burble.dn42|172.20.129.9|fd42:4242:2601:37:216:3eff:fe8f:6211|Asterisk VOIP Service| |shell.burble.dn42|172.20.129.10|fd42:4242:2601:ac22::1|Shell service| -|envoy.burble.dn42|172.20.129.11|fd42:4242:2601:ac81::1|Envoy load balancer & proxy| -| |_172.20.129.12-19_| |_Unallocated_| +| |172.20.129.11| |_Unallocated_| +|traefik.burble.dn42|172.20.129.12|fd42:4242:2601:ac82::1|Global traefik cluster| +|traefik-eu.burble.dn42|172.20.129.13|fd42:4242:2601:ac83::1|Europe traefik cluster| +|traefik-na.burble.dn42|172.20.129.14|fd42:4242:2601:ac84::1|North America traefik cluster| +| |_172.20.129.15-19_| |_Unallocated_| ||172.20.129.20/30|_n/a_|[Dialup Service](/retro/modem/) endpoints| | |_172.20.129.24-26_| |_Unallocated_| |shell.us-nyc2.burble.dn42|172.20.129.26|fd42:4242:2601:101d:216:3eff:fefc:722|us-nyc2 shell service| @@ -55,28 +58,28 @@ IP address tables |*unassigned* |172.20.129.164|fd42:4242:2601:3f::1|| |*unassigned* |172.20.129.165|fd42:4242:2601:3a::1|| |uk-lon2.burble.dn42 |172.20.129.166|fd42:4242:2601:2e::1|Private Node| -|dn42-ca-bhs2.burble.dn42|172.20.129.167|fd42:4242:2601:2d::1|| +|ca-bhs2.burble.dn42 |172.20.129.167|fd42:4242:2601:2d::1|*being decommissioned 2024*| |*unassigned* |172.20.129.168|fd42:4242:2601:34::1|| -|dn42-de-fra1.burble.dn42|172.20.129.169|fd42:4242:2601:31::1|| -|*unassigned* |172.20.129.170|fd42:4242:2601:2c::1|| -|*unassigned* |172.20.129.171|fd42:4242:2601:2b::1|| -|dn42-us-lax1.burble.dn42|172.20.129.172|fd42:4242:2601:2a::1|| +|de-fra1.burble.dn42 |172.20.129.169|fd42:4242:2601:31::1|| +|de-fra3.burble.dn42 |172.20.129.170|fd42:4242:2601:2c::1|| +|de-fra2.burble.d42 |172.20.129.171|fd42:4242:2601:2b::1|Private Node| +|us-lax1.burble.dn42 |172.20.129.172|fd42:4242:2601:2a::1|| |ch-zur2.burble.dn42 |172.20.129.173|fd42:4242:2601:27::1|Private Node| -|dn42-ch-zur1.burble.dn42|172.20.129.174|fd42:4242:2601:28::1|| -|dn42-us-nyc1.burble.dn42|172.20.129.175|fd42:4242:2601:29::1|| +|ch-zur1.burble.dn42 |172.20.129.174|fd42:4242:2601:28::1|| +|us-nyc1.burble.dn42 |172.20.129.175|fd42:4242:2601:29::1|| |us-nyc2.burble.dn42 |172.20.129.176|fd42:4242:2601:3d::1|Private Node| |*unassigned* |172.20.129.177|fd42:4242:2601:25::1|| |*unassigned* |172.20.129.178|fd42:4242:2601:24::1|| |*unassigned* |172.20.129.179|fd42:4242:2601:23::1|| -|fr-par2.burble.dn42 |172.20.129.180|fd42:4242:2601:38::1|Private Node| +|*unassigned* |172.20.129.180|fd42:4242:2601:38::1|| |*unassigned* |172.20.129.181|fd42:4242:2601:37::1|| |*unassigned* |172.20.129.182|fd42:4242:2601:3e::1|| |*unassigned* |172.20.129.183|fd42:4242:2601:3c::1|| |uk-lon3.burble.dn42 |172.20.129.184|fd42:4242:2601:30::1|Private Node| -|dn42-no-trd1.burble.dn42|172.20.129.185|fd42:4242:2601:39::1|| +|no-trd1.burble.dn42 |172.20.129.185|fd42:4242:2601:39::1|| |nl-ams1.burble.dn42 |172.20.129.186|fd42:4242:2601:32::1|Private Node| -|dn42-uk-lon1.burble.dn42|172.20.129.187|fd42:4242:2601:35::1|| -|dn42-fr-par1.burble.dn42|172.20.129.188|fd42:4242:2601:36::1|| +|uk-lon1.burble.dn42 |172.20.129.187|fd42:4242:2601:35::1|| +|fr-par1.burble.dn42 |172.20.129.188|fd42:4242:2601:36::1|| |fr-par3.burble.dn42 |172.20.129.189|fd42:4242:2601:26::1|Private Node| |*reserved* |172.20.129.190|fd42:4242:2601:20::1|Private Node| |*reserved* |172.20.129.191|fd42:4242:2601:20::1|Private Node| diff --git a/content/network/nodes.md b/content/network/nodes.md index 497fa13..09813be 100644 --- a/content/network/nodes.md +++ b/content/network/nodes.md @@ -72,6 +72,11 @@ weight: 20 ## North America ### dn42-ca-bhs2 + +{{}} +being decommissioned 2024 +{{}} + ||| |---|---| |**Location**|OVH (SoYouStart), Beauharnois, Canada| diff --git a/content/services/acme.md b/content/services/acme.md new file mode 100644 index 0000000..9d1a03d --- /dev/null +++ b/content/services/acme.md @@ -0,0 +1,167 @@ +--- +title: "ACME" +geekdocDescription: "ACME Service" +weight: 55 +--- +burble.dn42 provides an [ACME](https://en.wikipedia.org/wiki/Automatic_Certificate_Management_Environment) +service using an intermediate certificate issued by the +[dn42 certificate authority](https://dn42.dev/services/Certificate-Authority) and implemented using +a [HashiCorp Vault](https://vaultproject.io/) cluster to provide a highly available service. + +The following ACME challenge types are supported: + + - http-01 + - dns-01 + - tls-alpn-01 + +## dn42 endpoint + +- + +The dn42 endpoint serves certificates signed by an intermediate certificate issued by the +[dn42 certificate authority](https://dn42.dev/services/Certificate-Authority). + +{{}} +Note that certificates are issued with a validity period of **30 days**, which is +shorter than most clearnet ACME services. + +The recommended interval to check for expiry is 5 days. +{{}} + +## Staging endpoint + +- + +The staging endpoint can be used for testing and issues junk certificates. +The service uses an internal certificate authority that is specific to the staging service +and should not be trusted. + +The staging service issues short lived certificates with a validity period of a few days. + +## Certificate Transparency + +**TODO** A simpler process will be provided at a future stage, in the meantime the vault +API can be queried manually to list issued certificates. + +-- + +Vault provides an API for listing issued certificates, however the process for doing this +is somewhat complicated if you have not used vault before. The instructions below detail +how to interrogate the service using the vault CLI, however it is also possible to run +through the same process via the +[HTTP API](https://developer.hashicorp.com/vault/api-docs?product_intent=vault). + +```sh +# The API endpoint to list issued certificates is an authenticated +# endpoint that requires a vault token to access it. +# +# The burble.dn42 service includes an anonymous login that can be +# used to obtain a suitable token. + +# set the VAULT_ADDR environment variable to the ACME service + +$ export VAULT_ADDR="https://acme.burble.dn42" + +# you can also set VAULT_SKIP_VERIFY=1 if you do not have the +# dn42 certificate authority installed. + +# Issue an anonymous token and store it in the VAULT_TOKEN env variable + +$ export VAULT_TOKEN=$(vault write -field token auth/approle/login role_id=anonymous) + +# now the vault API can be accessed + + +# list issued certificates + +$ vault list dn42/certs + +Keys +---- +06:72:54:74:02:eb:68:da:62:76:14:92:b4:84:19:36:b1:d1:d0:5c +0c:bb:39:a0:0a:aa:9c:d9:06:e8:9e:87:ff:54:73:c4:a6:42:9c:f0 +13:91:4f:f7:3a:0b:ca:38:cd:c6:6e:7d:4d:fb:c5:7c:ed:b0:79:1b +39:5c:46:16:27:d8:f7:30:cc:64:1a:3c:6c:ff:c4:ac:f9:3c:3c:9c +4b:24:32:48:d0:64:55:3b:dd:b3:00:c6:33:2d:0f:3e:eb:d7:50:02 +4c:8f:ce:e6:18:7a:05:c1:a3:11:45:c9:3c:34:0f:50:e0:75:6d:fd +5a:03:a9:5b:07:60:d0:fb:25:28:4b:e9:93:a8:22:cd:78:d1:29:b2 +5d:26:b4:47:59:0c:0a:e9:88:b6:97:1d:2a:2b:e5:cb:d2:90:34:9e +65:c8:33:07:fc:9a:aa:fd:85:6b:fd:b4:de:29:71:e3:8e:6c:f2:11 +68:e1:a6:4a:e1:58:ee:71:c7:a6:12:48:e2:7a:c5:84:c1:7c:21:5e +75:cf:16:f9:06:71:ea:86:1c:51:95:89:c9:1d:ea:a1:eb:f5:6f:83 +76:91:6e:6a:23:14:00:7c:5f:c7:de:91:c4:40:73:d9:51:b4:f8:4d + +# view an invidual certificate + +$ vault read -field certificate "dn42/cert/76:91:6e:6a:23:14:00:7c:5f:c7:de:91:c4:40:73:d9:51:b4:f8:4d" + +-----BEGIN CERTIFICATE----- +MIIDTTCCAjWgAwIBAgIUdpFuaiMUAHxfx96RxEBz2VG0+E0wDQYJKoZIhvcNAQEL +BQAwVTELMAkGA1UEBhMCWEQxDTALBgNVBAoTBGRuNDIxFDASBgNVBAsTC2J1cmJs + +...snip... + +yait1CFFq4g9/bvsNfIsvN6EJ/BGXqqww6BzKt/ioSLj +-----END CERTIFICATE----- + +# human readable output using the step CLI (https://smallstep.com/) + +$ vault read -field certificate "dn42/cert/06:72:54:74:02:eb:68:da:62:76:14:92:b4:84:19:36:b1:d1:d0:5c" | step certificate inspect +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 36803586486229131299250018793512622456839458908 (0x672547402eb68da62761492b4841936b1d1d05c) + Signature Algorithm: SHA256-RSA + Issuer: C=XD,O=dn42,OU=burble.dn42,CN=burble.dn42 staging ACME + Validity + Not Before: Oct 2 18:21:36 2023 UTC + Not After : Nov 3 18:22:06 2023 UTC + Subject: CN=drone.git.dn42 + Subject Public Key Info: + Public Key Algorithm: RSA + Public-Key: (4096 bit) + Modulus: +...snip... + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Key Usage: critical + Digital Signature, Key Encipherment, Key Agreement + X509v3 Extended Key Usage: + Server Authentication + X509v3 Subject Key Identifier: + 01:4A:7E:02:F3:B7:78:03:66:F9:21:97:4B:31:34:7C:31:DE:BB:86 + X509v3 Authority Key Identifier: + keyid:94:D1:C3:60:C7:88:81:A6:8C:37:AE:40:42:22:48:6B:5F:36:8F:CC + Authority Information Access: + OCSP - URI:https://acme.burble.dn42/v1/dn42/ocsp + CA Issuers - URI:https://acme.burble.dn42/v1/dn42/ca + X509v3 Subject Alternative Name: + DNS:drone.git.dn42 + X509v3 CRL Distribution Points: + Full Name: + URI:https://acme.burble.dn42/v1/dn42/crl + Signature Algorithm: SHA256-RSA +...snip... +``` + +## Implementation + +The ACME implementation is provided by a 3-node [HashiCorp Vault](https://www.vaultproject.io/) +cluster behind the [burble.dn42 traefik load balancer](/services/internal/#traefik--traefik-eu--traefik-na). Together they provide a global, +high availability service. + +The cluster currently runs on the following nodes: + + - ch-zur2 + - de-fra1 + - fr-par1 + +At any time the cluster has one leader which processes all requests and replicates state to the +cluster members. The leader node automatically switches to one of the backup servers should +a failure occur. + +The traefik load balancer runs health checks against the vault servers and automatically redirects +users to the vault cluster leader. + +See the [vault HA reference architecture](https://developer.hashicorp.com/vault/tutorials/day-one-raft/raft-reference-architecture) for more details. + diff --git a/content/services/dn42.md b/content/services/dn42.md index 7126973..9bf9ac2 100644 --- a/content/services/dn42.md +++ b/content/services/dn42.md @@ -32,14 +32,19 @@ read-only. Please note that updates to the wiki may take several hours to sync with other mirrors. -The service is provided by regional mirrors fronted by an nginx proxy that is itself -anycasted across burble.dn42. The service is fully meshed and will continue to -operate as long as at least one proxy and mirror is available. +The wiki service is delivered using the [burble.dn42 nomad cluster](/services/internal/#nomadburbledn42). -Mirrors are located in the following locations: +## ACME -* dn42-de-fra1 -* dn42-ca-bhs2 + - + - + +burble.dn42 provides an [ACME](https://en.wikipedia.org/wiki/Automatic_Certificate_Management_Environment) +service using an intermediate certificate issued by the +[dn42 certificate authority](https://dn42.dev/services/Certificate-Authority) and implemented using +a [HashiCorp Vault](https://vaultproject.io/) cluster to provide a highly available service. + +More details can be found on the [ACME service](/services/acme/) page. ## Whois Service diff --git a/content/services/internal.md b/content/services/internal.md index c689d33..8007eb6 100644 --- a/content/services/internal.md +++ b/content/services/internal.md @@ -6,27 +6,15 @@ weight: 30 This page provides some documenation on other services used within burble.dn42 that are not directly available for public use. -## rproxy.burble.dn42 +## traefik / traefik-eu / traefik-na -Core nodes run an [nginx](https://nginx.com) container that acts as a reverse proxy -for services hosted in tier2. +burble.dn42 runs a global [traefik](https://traefik.io/traefik/) cluster which +acts as a reverse proxy and load balancer for burble.dn42 web services. -The reverse proxy is distributed to improve local response times and is -anycast as rproxy.burble.dn42. Most web services provided by burble.dn42 are -simply CNAMEs to the reverse proxy which then balances and forwards the -request to the actual service. - -As well as a reverse proxy, nginx also provides: - -- TLS termination -- A local page cache to act as a poor man's CDN -- Static content server - -## envoy.burble.dn42 - -[Envoy Proxy](https://www.envoyproxy.io/) is being introduced as a health -checking proxy and load balancer to replace anycasts and nginx for some -workloads. +The traefik instances are anycast globally (traefik.burble.dn42), but also +have regional load balancing groups for Europe (traefik-eu.burble.dn42) and +North America (traefik-na.burble.dn42). This regional split helps to direct +users to local servcices where possible. ## vault.burble.dn42 @@ -69,11 +57,23 @@ user token during the deployment process. This ensures that even if access was gained to the deployment server, secrets could still not be accessed without also having access to a live user token. +## nomad.burble.dn42 + +burble.dn42 runs a global [HashiCorp Nomad](https://www.nomadproject.io/) cluster +that is used primarily for web application workloads. Nomad integrates with +[containerd](https://containerd.io/), +[vault](/services/internal/#vaultburbledn42and) and +[traefik](/services/internal/#traefik--traefik-eu--traefik-na) +to provide resilient, globally available applications. + +The nomad configuration is publically available in the [burble.dn42 git](https://git.burble.dn42/burble.dn42/nomad) instance. + ## nats.burble.dn42 burble.dn42 operates a [nats.io](https://nats.io/) cluster as a distributed, -network wide, broadcast and RPC solution. The cluster uses Vault managed, -ephemeral TLS certs for authentication and encryption. +network wide, broadcast and RPC solution. + +The cluster uses decentralised JWT tokens for authentication. ## ci.burble.dn42 diff --git a/content/services/public.md b/content/services/public.md index ba34b79..70fa5d3 100644 --- a/content/services/public.md +++ b/content/services/public.md @@ -29,14 +29,6 @@ Functionality includes: - Changing your shell for the shell services - Viewing peering information -## Issue Log - -A public issue log is maintained on the [DN42 Registry](https://git.dn42.dev). - -- [Issue Log](https://git.dn42.dev/burble/burble.dn42/issues) - -Users are welcome to raise issues or enhancements via the log. - ## Diagnostic Services ### Looking Glass @@ -69,20 +61,6 @@ please be considerate and configure a reasonable test frequency. In all cases, do not set the ping frequency to be higher than once a second. {{}} -### Speed Test Service - -A speed test service is available in a few select locations.
-Note that the service is currently available over IPv6 only at this time. - -|Location|URL|Speed| -|:--|:--|:--| -| Paris, France | [https://speedtest.fr-par1.burble.dn42](https://speedtest.fr-par1.burble.dn42) | 1gbit down / 1gbit up | -| Beauharnois, Canada | [https://speedtest.ca-bhs2.burble.dn42](https://speedtest.ca-bhs2.burble.dn42) | 1gbit down / 500mbit up | - -{{}} -Remember this service is provided for your benefit, use responsibly. -{{}} - ## Network Status and Reporting ### Grafana Dashboards @@ -92,11 +70,6 @@ Remember this service is provided for your benefit, use responsibly. ### Uptime monitoring -- [https://uptime.burble.dn42/status/bdn42](https://uptime.burble.dn42/status/bdn42) (dn42) -- [https://uptime.burble.com/status/bdn42](https://uptime.burble.com/status/bdn42) (com) - -A self-hosted instance of [Uptime Kuma](https://github.com/louislam/uptime-kuma) provides the current status and alerts on many of the burble.dn42 services. - - [https://stats.uptimerobot.com/l2913c0R6](https://stats.uptimerobot.com/l2913c0R6) Major nodes are also monitored off-network by [UptimeRobot](https://uptimerobot.com/). @@ -123,14 +96,6 @@ The burble.dn42 shell service provides shell accounts for dn42 users who have a burble.dn42 password or SSH auth methods in the registry. See the [Shell Accounts](/services/shell/) page. -## Web SSH Client - -- [https://sshwifty.burble.dn42/](https://sshwifty.burble.dn42/) - -[sshwifty](https://github.com/nirui/sshwifty) provides a web based terminal for telnet and SSH sessions. - -Configuration includes presets for the burble.dn42 shell servers and collector.dn42. - ## S3 Compatible Object Store - [https://minio.burble.dn42](https://minio.burble.dn42) - Web interface @@ -259,3 +224,10 @@ Log in using your burble.dn42 username/password to lurk on [#dn42](https://wiki.dn42.us/services/IRC). (set a password using the [the burble.dn42 service portal](https://svc.burble.dn42/)) +## Invidious instance + +- + +burble.dn42 instance of [Invidious](https://invidious.io) the open source +alternative front-end to YouTube. +