Simon Marsh
239e3bbdd7
All checks were successful
continuous-integration/drone/push Build is passing
111 lines
3.6 KiB
Markdown
111 lines
3.6 KiB
Markdown
---
|
|
title: "Maintenance Log"
|
|
geekdocCollapseSection: true
|
|
geekdocDescription: "A log of changes to the burble.dn42 network"
|
|
---
|
|
A log of changes to the burble.dn42 network.
|
|
|
|
---
|
|
|
|
## 3rd April 2021
|
|
|
|
The collector is now using a TLS certificate issued by the
|
|
[DN42 ACME](https://acme.dn42) service. The collector is behind an anycasted
|
|
reverse proxy, so a normal ACME challenge will not work. Instead, the certificate
|
|
is managed using
|
|
[dnscontrol](https://stackexchange.github.io/dnscontrol/) to respond to
|
|
an ACME DNS challenge.
|
|
|
|
DNSSEC has been enabled on all edge nodes.
|
|
|
|
## 2nd April 2021
|
|
|
|
There was a major DNS outage today as a minor change took out the entire service.
|
|
|
|
What should have been a trivial config change actually upgraded the container
|
|
from Alpine 3.11 to Alpine 3.13 and caused a number of the DNS applications to
|
|
stop working due to incompatibilities.
|
|
|
|
The lack of working DNS meant it was more complicated to bootstrap the service
|
|
back again, leading to a long delay in restoring service.
|
|
|
|
## 27th March 2021
|
|
|
|
Fixed a bug in bird that was preventing MRT dumps from the collector working.
|
|
Hopefully the dumps can now be successfully parsed:
|
|
[https://mrt.collector.dn42](https://mrt.collector.dn42)
|
|
|
|
## 25th March 2021
|
|
|
|
Bird [2.0.8](https://gitlab.nic.cz/labs/bird/-/blob/master/NEWS) has been deployed
|
|
across the network. Please let me know if you see problems.
|
|
|
|
burble.dn42 uses a custom bird build that includes additional debugging. The
|
|
source code for the build is available on [git.burble.dn42](https://git.burble.dn42).
|
|
|
|
{{<hint warning>}}
|
|
**Advanced Notice**
|
|
- us-nyc1 will be decommissioned before 15/04/21
|
|
- us-chi1 will be decommissioned before 14/05/21
|
|
{{</hint>}}
|
|
|
|
## 23rd February 2021
|
|
|
|
Updated IPv6 address for hk-hkg1
|
|
|
|
## 10th January 2021
|
|
|
|
Upgraded the [looking glass](https://lg.burble.com/) to use
|
|
[bird-lg-go](https://github.com/xddxdd/bird-lg-go).
|
|
|
|
The main benefit of the go version is that it executes queries in
|
|
parallel, greatly improving response times with a large number
|
|
of nodes.
|
|
|
|
## 6th January 2021
|
|
|
|
hk-hkg1 is now open for IPv4 peering; see the [node information](https://dn42.burble.com/network/nodes/#dn42-hk-hgk1) for details.
|
|
|
|
IPv6 connectivity is expected ~February.
|
|
|
|
## 4th January 2021
|
|
|
|
Happy New Year DN42.
|
|
|
|
#### New Website
|
|
|
|
The new year brings a new website for burble.dn42 built using [Hugo](https://gohugo.io/) and statically
|
|
delivered from each core node for speed. As always, the source for the website is available in the
|
|
[gitea repo](https://git.burble.dn42/burble.dn42/www).
|
|
|
|
#### Anycast MTU
|
|
|
|
The MTU for anycast services has been reduced to 1280 after a problem was seen with IPv6 path MTU discovery.
|
|
|
|
The problem was due to an asymmetric path, where a request to the wiki went to one node but the return
|
|
path was via a different node. The other node also hosted a wiki instance, which meant that pmtud ICMP
|
|
messages on the return path were being picked up by the wrong node. To fix this, the MTU has been clamped
|
|
to the minimum allowable size of 1280.
|
|
|
|
Interestingly, Cloudflare also recognised the same type of issue and wrote up what they did in their [blog](https://blog.cloudflare.com/increasing-ipv6-mtu/).
|
|
|
|
The following services were impacted by the changes.
|
|
|
|
- DNS Services
|
|
- NGINX Reverse Proxy (and therefore also all websites, including the Wiki mirrors)
|
|
- WHOIS Service
|
|
|
|
|
|
#### New Nodes
|
|
|
|
es-mad1 in Madrid, Spain has already been delivered and is now open for peerings.
|
|
|
|
The new node in Hong Kong, hk-hkg1 has also been delivered and I'm now just waiting for IPv6 to be available
|
|
before it too will also be ready for peering.
|
|
|
|
|
|
---
|
|
|
|
#### Historical changes from previous years
|
|
|
|
{{<toc-tree>}} |