DN42&OneManISP – Confederation, Automation, IPAM and Overlay Network
This article does not go into implementation details in depth; it only records the internal network architecture design process and automation ideas. If you need reference code or configuration files, please see iYoRoy-Network/bird2-config: BIRD2 Configuration for iYoRoy Network (AS4242422024, AS205369). The repository contains a significant amount of AI-generated code. This article only borrows the logical layering concept of Underlay/Overlay (base layer/business layer), specifically referring to the separation of address roles, and does not involve tunneling encapsulation technologies such as VXLAN/GRE. TL;DR The entire internal network architecture was refactored. What was originally a horizontally split network based on “DN42 / IANA / WireGuard / BIRD config files” was transformed into a set of infrastructure organised by network intent. The final design can be summarised as: Use WireGuard to carry underlay transport between nodes; Use BGP Confederation instead of OSPF / full-mesh iBGP as the internal routing backbone; Separate node identity and service addresses with Underlay / Overlay; Automatically derive AS, loopback and link-local addresses from Tier / Region / Node ID / Token; Use BGP Large Community to express route origin, propagation scope and export policy; Use Ansible to compile high-level peer intent into WireGuard + BIRD + interface configurations and deploy them automatically. Background In some earlier articles from these two series, we successfully brought up a DN42 AS and an IANA AS, with the same infrastructure carrying traffic for both networks simultaneously. In a previous post, DN42&OneManISP - Troubleshooting OSPF Source Address in a Coexistence Environment - iYoRoy's Develop Diary, there was already some initial separation and isolation for the two networks. However, as the number of nodes grew, manually maintaining the WireGuard and BIRD configuration files for this whole thing became more and more complex. Moreover, public network nodes also needed to interconnect and provide mutual IP transit, so in the end I decided to do a thorough refactoring of this entire internal network. Analysis Problems with the Old Architecture Before the refactoring, DN42 routes between nodes were mutually propagated, using OSPF over WireGuard as the IGP, with full-mesh iBGP on top to carry full routing information between border routers. For IANA traffic, nodes were completely isolated from each other, with each node independently advertising its own /48 IANA IPv6 prefix. To sum up: WireGuard was responsible for tunnels between nodes; OSPF over WireGuard was responsible for the IGP; DN42 border routers ran full-mesh iBGP on top; IANA nodes basically each independently advertised their own public prefixes. This architecture worked well when there were few nodes, but problems started to become apparent as the number of nodes increased: WireGuard, OSPF, and BIRD configurations had to be maintained separately; Adding a new node required changes in multiple places, making it easy to miss something; Routing policies for DN42 and IANA influenced each other, yet the configuration was scattered; Cross-PoP scheduling of IANA prefixes was not natural; Some nodes were only transit nodes but were forced to have an overlay identity; Source address selection, route leaks, and exposure of internal addresses became increasingly difficult to control. Design Goals and BGP Confederation The main goals for this refactoring were: Adding nodes should be simple: ideally, only the node identity and peer relationships need to be described; DN42, IANA and the internal backbone should be able to share infrastructure while remaining policy-isolated from each other; IP addresses should be schedulable resources, not fixed identities of nodes; The internal topology should not leak to external BGP peers; Configurations should be generated from declarative data, not by handwriting lots of repetitive WireGuard/BIRD sessions; The deployment process should be automated. There were also several other, more detailed requirements: Some IANA nodes should be able to carry IP transit for other nodes, using an IANA PoP to advertise public IPs for other nodes; Currently, the IANA AS is maintained jointly by たのしい and myself; we need to coordinate, with some infrastructure needing mutual transit and route tuning; We only have one 44Net IPv4 /24 (sponsored by たのしい); if we want to advertise it, all nodes would need iBGP interconnection and an IGP, otherwise only a single PoP can use it; The IANA AS needs to accept downstreams; Route optimization is needed between IANA PoPs. The previous IANA IPv6 prefix assignment rule followed Continent 3 bits + Region 3 bits + Multi-node within region 2 bits, carving several /48s out of a complete /40. However, due to certain upstream limitations on the number of advertised prefixes, and because such fragmented prefixes are unfavourable for IPAM, we planned to eventually aggregate external advertisements only to the continent level, and then handle more detailed parts via internal iBGP routing. The internal network structure on たのしい's side is BGP Confederation, implemented with a manually simulated approach. Actually, after this analysis, defining the internal network architecture became quite straightforward: the most suitable solution was to switch the internal network to BGP Confederation as well, and then utilise the concept of "BGP as IGP" to advertise /32 and /128 routes within the confederation to handle internal routing. At the same time, because a confederation avoids the split-horizon characteristic of traditional iBGP, it doesn't require strict full-mesh, nor does it need separate iBGP instances, making it very decentralized xD. Furthermore, BGP Confederation can be easily merged with たのしい's infrastructure, requiring only that both sides treat the other's internal ASes as confederation ASes. This solution has another benefit, because the containment relationship of our infrastructure is actually like this: graph subgraph 4242423377 Infrastructure 3377_DN42_PoP[DN42 PoP] 3377_IANA_PoP[IANA PoP] end subgraph 4242422024 Infrastructure 2024_DN42_PoP[DN42 PoP] 2024_IANA_PoP[IANA PoP] end 3377_IANA_PoP <==Cross-Infra Confederation==> 2024_IANA_PoP 3377_DN42_PoP <--> DN42 <--> 2024_DN42_PoP 3377_IANA_PoP <--> IANA <--> 2024_IANA_PoP Our DN42 PoPs and transit are isolated, but the IANA PoPs and transit are interconnected. In the BGP Confederation scenario, we can use filters + BGP (Large) Community to isolate route origins while still exchanging IANA traffic engineering intents, achieving a single standard usable across both infrastructures. Overlay and IPAM In the old design, a node often simultaneously held two identities: It was a router in the network; It was also a service address holder reachable within DN42 / IANA. After the refactoring, I decided to separate these two identities: Underlay: the node's identity as a router, used for internal interconnection, next-hop, tunnels, and forwarding; Overlay: the DN42 / IANA address used when the node provides services to the outside world or accepts traffic. This way, a node can only participate in forwarding without having a DN42 IPv4 address; it can also temporarily announce a certain overlay /32 or /128 when needed. In practical application scenarios, my DN42 address space is not very abundant; I originally registered a /28, with only 16 usable addresses. Some nodes, like IEPL tunnel nodes or IX nodes, only perform forwarding roles, do not need to host services, and do not need to be connected from the outside. Therefore, theoretically they do not need to be assigned independent, reachable DN42/IANA addresses. Moreover, since we had already decided to switch the internal network to BGP Confederation, based on this configuration we could re-plan the originally horizontally split DN42 + IANA network structure into a vertical Underlay + Overlay division. DN42 and IANA addresses/traffic become the payload of the upper-layer Overlay Network, while the lower-layer Underlay Network serves as the infrastructure for underlying communication and forwarding between nodes. In this scenario, IP addresses become resources that can be easily scheduled within the internal network, rather than unique IDs bound to a node. The advantages are clear: Saves scarce address resources: transit nodes can have only an underlay identity, without needing DN42 IPv4 or public addresses; Anycast and address migration: Overlay addresses become resources schedulable via BGP, no longer tightly bound to a specific machine; Hides transit nodes: pure transit nodes do not need to expose globally reachable addresses, making it harder for outsiders to directly probe the internal topology; Easy new node onboarding: new nodes join the underlay first, and overlay addresses are assigned when they need to carry services; Elegant transit delivery: IANA PoPs can deliver customer prefixes or service addresses to other nodes via the internal confederation. Design In summary, the final design consisted of the following IPAM mechanism and automation workflow. Each node possesses some basic metadata: Tier: indicates the node's purpose, e.g., backbone, IX access, or intra-net node Region: the continent region where the node is located ID: the node's ID within that region Token: a random 16-bit string assigned to the node, used as a unique ID for the node under IPv6 For Tier and Region, several tables were created as data sources: Tier Desc 1 Backbone 2 IX 3 Backbone + IX Mixed 9 Node Region Desc 0 Reserved 1 Asia 2 Europe 3 Africa 4 North America 5 South America 6 Oceania 7 Antarctica Confederation AS: 4220240000 – 4220249999 Assignment rule: 422024{tier}{region}{node_id:02d} Where tier indicates the node's purpose, region indicates the area where the node is located, and the last two digits node_id distinguish different nodes within the same region. Underlay IPAM IPv4 Prefix: 100.64.0.0/16 Assignment rule: 100.64.{tier*10 + region}.{node_id}/32 IPv6 Prefix: fd18:3e15:61d0:ffff::/64 Assignment rule: fd18:3e15:61d0:ffff:{tier}:{region}::{node_token}/128 To simplify the automation workflow, I also intended to bring the link-local addresses between WireGuard tunnels under automated management, generating them using the aforementioned Tier, Region, and Token, with the automation scripts generating WireGuard configuration and automatically setting up the tunnel and confederation BGP session. The link-local assignment rule generally follows the normal IPv6 rule: fe80::2024:{tier}:{region}:{node_token}/64 Thus, each node only needs to be assigned its metadata to automatically generate the internal network confederation AS and addresses according to the rules above. Overlay IPAM The DN42 IPv4 part is assigned manually. This part has some historical legacy architecture, and changes need to be merged into the official registry, so I plan to assign it manually. The Underlay IPv6 is itself DN42-reachable; if other DN42 IPv6 addresses need to be allocated, they can simply be advertised on demand. For IANA IPv6, the current design format is to first calculate the continent /43 according to the initially proposed rule, e.g., for Asia: 2a14:7583:f220::/43 Then calculate the /46 based on the region, e.g., for Hong Kong region HKG: 2a14:7583:f224::/46 Next, bits from /46 to /112 are set to zero, and the last 16 bits are filled with the node Token to serve as the node's IANA IPv6 address. For example, one of my backbone nodes in Hong Kong: 2a14:7583:f224::7d89/128 Community Isolation Mechanism In this architecture, BGP Large Community effectively acts as the metadata and control information carried along with a route as it propagates between different modules and infrastructures. Based on this metadata, when we export routes to different external peers we can determine: Does it belong to DN42 or IANA? Is it locally originated, brought in by a downstream, learned from a peer, or learned from an upstream? Can it be exported to other ASes? Is it underlay-only? Does this route need AS prepending? As a concrete example, if the first identifier marks the route origin/destination as DN42 or IANA, all routes permitted to be sent to DN42 will be tagged with: (4242422024, 1, 1) Similarly, all routes destined for IANA will be tagged with: (205369, 1, 1) Within various protocols, this acts as a master switch, controlling whether export should occur. IANA upstream export filter: template bgp iana_upstream_v6 { ... ipv6 { ... import filter { if !iana_filter_default_check() then reject; # basic check remove_confederation_as(); # remove confederation as to prevent external peers from maliciously carrying internal as remove_private_community(); # remove private community to prevent external peers from maliciously manipulating internal routes iana_upstream_add_community(); # tag all routes from IANA with community permitting broadcast in IANA infrastructure accept; }; export filter { if !iana_filter_default_check() then reject; # basic check if !iana_upstream_check_community() then reject; # check community: is this a route destined for IANA / does it carry no-advertise/no-export etc. remove_confederation_as(); # remove confederation as remove_private_community(); # remove private community accept; }; ... }; ... } Where iana_filter_default_check() is used to check prefix length, ROA, whether it is a default route, and other miscellaneous content: function iana_filter_default_check(){ if net ~ [::/0] then return false; if net.len > 48 then return false; if bgp_large_community ~ [(IANA_OWNAS,1,1)] && is_self_iana_v6() then return true; if roa_check(iana_roa_v6, net, bgp_path.last) = ROA_INVALID then return false; return true; } remove_confederation_as(), as the name suggests, removes internal confederation ASes: function remove_confederation_as() { bgp_path.delete([4220240000..4220249999]); # 4242422024 Infrastructure bgp_path.delete([4233770000..4233779999]); # 4242423377 Infrastructure } remove_private_community(), as the name suggests, removes internal Communities. The current implementation is very rough and needs detailed refinement later, because some Communities should be open to downstreams, allowing them to use Communities to convey routing intents and perform optimizations, such as prepending AS path, which can be used for route tuning or traffic engineering: function remove_private_community(){ bgp_large_community.delete([(4242422024, *, *)]); bgp_large_community.delete([(205369, *, *)]); } iana_upstream_add_community(), tags all routes from upstream with the Community allowing circulation in the IANA infrastructure, as well as a route source identifier: function iana_upstream_add_community(){ bgp_large_community.add((205369,1,1)); bgp_large_community.add((205369,2,102)); } Where (205369,2,102) identifies the route as coming from an upstream. Route sources are divided into three types: Upstream, Peer, Downstream/Customer/Self (treated as customer). For these three different types of BGP sessions, the exported routes usually differ: To upstream: we need the upstream to advertise our own prefixes and our downstreams' prefixes, so all routes originating from downstream should be exported to upstream. To downstream: we need to provide network services, so we need to export all routes we know, i.e., routes from upstream, peer, downstream. To peer: peering connections are solely for both parties to access each other's networks, so we must not export each other's upstreams or peers, otherwise it becomes free transit/tunneling. Therefore, only routes from downstream are exported. Correspondingly in the current internal network, routes from peers are tagged (205369,2,101), and routes from downstreams are tagged (205369,2,100). iana_upstream_check_community() checks the route's Community, and based on the principles above, judges whether it should be advertised to upstream: function iana_upstream_check_community(){ if !(bgp_large_community ~ [(205369,1,1)]) then return false; # not permitted for broadcast in IANA infrastructure, reject if bgp_large_community ~ [(205369,65535,65282)] then return false; # no-advertise if bgp_large_community ~ [(205369,65535,65281)] then return false; # no-export if bgp_community ~ [(65535,65281)] then reject; # no-advertise if bgp_community ~ [(65535,65282)] then reject; # no-export if bgp_large_community ~ [(205369,2,0)] then return false; # from internal network, this community is used to identify routes from the internal network that should not be advertised externally, reject here if bgp_large_community ~ [(205369,2,101)] then return false; # from peer, reject advertisement if bgp_large_community ~ [(205369,2,102)] then return false; # from upstream, reject advertisement return true; } Similarly, a similar check mechanism applies to DN42. Based on this mechanism, isolating the DN42 and IANA networks becomes very straightforward. The above is just a general explanation; finer community design and various export policies could be further elaborated, but that would turn into another article on routing policy details. This part actually draws heavily from たのしい’s BGP Communities; after all, both infrastructures need to interoperate, and many community specifications are basically copied. Thanks to たのしい for providing the ideas~ Route Lifecycle Phase 1: Ingress / Import When all external routes enter the system, they first pass through the import filter of the corresponding domain. IANA has three types of sources: upstream; peer / IX; downstream. DN42 also has two main types of sources: regular transit / eBGP peer; IX / route server. In addition, the local machine also originates some routes: underlay loopback; DN42 overlay address; IANA own / anycast address. These routes are tagged with Large Communities upon entering the confederation. All subsequent filters must be adjusted based on this. Phase 2: Core / Intra Confederation After entering the BIRD RIB, routes propagate between nodes via the internal Intra BGP Confederation. Here, Intra Confederation is not just about handling connectivity, but more importantly, allowing internal routes to carry policy information as well. The previous OSPF was great at solving: Where is this loopback? How to reach this next-hop? But it could not convey AS information and policy intents, for example: This route was learned from an IANA peer and must not be exported to another peer; This route is DN42 underlay-only and must never be leaked to eBGP; This route is a downstream customer prefix and can be exported to upstream. All of these are strengths of BGP policy. So moving the internal backbone from OSPF / full-mesh iBGP to confederation-style BGP is essentially upgrading the internal control plane to a "policy-carryable control plane". Phase 3: Egress / Export When a route is about to leave the infrastructure, it passes through the export filter again. Here, based on communities, systemic prevention of route leaks and macro-control like AS prepending can be performed. For example: DN42 underlay routes carry an underlay-only community and thus are not exported to DN42 eBGP; Routes learned from an IANA upstream are not exported to another upstream; Routes learned from an IANA peer are not exported to another peer; Routes learned from a DN42 IX are not exported back to IX; NO_EXPORT / NO_ADVERTISE are respected; Before external export, internal confederation ASes and private communities are removed. In summary, it can be illustrated by this diagram: flowchart LR classDef ext fill:#eef7ff,stroke:#5b8def,stroke-width:1px; classDef local fill:#f5f5f5,stroke:#888,stroke-width:1px; classDef filter fill:#fff3d6,stroke:#d19a00,stroke-width:1px; classDef core fill:#eaf8ea,stroke:#3c9b43,stroke-width:1px; classDef export fill:#fdecec,stroke:#d45a5a,stroke-width:1px; subgraph SRC["Route Source"] direction TB IU["IANA upstream transit / full routes"] IP["IANA peer / IX"] ID["IANA downstream customer prefixes"] DT["DN42 transit / eBGP peer"] DX["DN42 IX / route server"] LU["Local underlay loopback 100.64.x.y / fd18:...:ffff"] LO["Local overlay address DN42 own / IANA own / anycast"] end subgraph INFRA["Bird2-Configuration Infrastructure"] direction LR subgraph IMPORT["Ingress / Import filters"] direction TB IUF["IANA upstream import tag: 205369:1:1 tag: 205369:2:102"] IPF["IANA peer import tag: 205369:1:1 tag: 205369:2:101"] IDF["IANA downstream import AS-SET / path check tag: 205369:2:100"] DTF["DN42 eBGP import prefix / ROA / self check tag: 4242422024:2:101/102"] DXF["DN42 IX import prefix / ROA check tag: 4242422024:2:101"] STF["Static origin underlay / overlay route tags"] end META["Large Community metadata layer domain + source + scope internal route API"] subgraph CORE["Internal control plane"] direction TB RIB["BIRD RIB"] CONFED["Intra BGP Confederation member AS: 422024xxxx policy: full / default / iana_full / no_iana"] WG["WireGuard underlay transport link-local next-hop fwmark policy routing"] UNDERLAY["net_underlay loopback + table 1142"] OVERLAY["net_overlay DN42 / IANA service addresses"] end subgraph EGRESS["Egress / Export filters"] direction TB DEXP["DN42 eBGP export reject underlay-only respect no-export/no-advertise"] DXEXP["DN42 IX export reject peer/transit learned routes"] IEXP["IANA upstream / peer export export local/downstream only reject peer/upstream learned"] DOWNEXP["IANA downstream export policy: default / own_only / reject"] KEXP["Kernel export krt_prefsrc by community"] CLEAN["cleanup before external export remove confed AS remove private communities"] end end subgraph DST["Route Destination"] direction TB OD["DN42 peers / transit"] OX["DN42 IX"] OI["IANA upstream / peer"] OC["IANA downstream"] KF["Linux kernel FIB actual packet forwarding"] end IU --> IUF IP --> IPF ID --> IDF DT --> DTF DX --> DXF LU --> STF LO --> STF IUF --> META IPF --> META IDF --> META DTF --> META DXF --> META STF --> META META --> RIB RIB <--> CONFED CONFED --- WG WG --- UNDERLAY RIB --- OVERLAY RIB --> DEXP RIB --> DXEXP RIB --> IEXP RIB --> DOWNEXP RIB --> KEXP DEXP --> CLEAN --> OD DXEXP --> CLEAN --> OX IEXP --> CLEAN --> OI DOWNEXP --> CLEAN --> OC KEXP --> KF class IU,IP,ID,DT,DX ext; class LU,LO local; class IUF,IPF,IDF,DTF,DXF,STF filter; class META,RIB,CONFED,WG,UNDERLAY,OVERLAY core; class DEXP,DXEXP,IEXP,DOWNEXP,KEXP,CLEAN export; class OD,OX,OI,OC,KF ext; Automation: Compiling Network Intent into Configuration All the above discussion on IPAM, Confederation AS, Underlay/Overlay, and BGP Communities would not truly solve the problem if it still ended up being maintained by hand-written configurations; it would just move the complexity from one form to another. Before this refactoring, adding a new node or peer often required simultaneous changes in many places: WireGuard configuration: interface name, listen port, peer public key, endpoint, allowed IPs BIRD configuration: BGP protocol name, neighbor address, neighbor interface, ASN, import/export filter IPAM-related configuration: loopback, link-local, router id, overlay address Deployment-related configuration: which nodes need rendering, which services need restarting Some special cases: nodes behind NAT, passive peers, special local AS, special filters Logically, these things describe the same thing: a connection belonging to a certain routing domain exists between two nodes. But in manual configuration, they end up scattered across WireGuard, BIRD, network interfaces, systemd/openrc, and other places. If just one field is missed or inconsistent, very strange problems can occur, such as WireGuard being up but BIRD neighbor pointing to the wrong interface; or the link-local being changed, but the peer session still having the old address. So in this refactoring, I didn't want Ansible to just be a tool that "fills YAML into Jinja2 templates". If we simply moved the original hand-written config into templates, we would essentially still be maintaining a pile of low-level configurations, just in a different file format. Since automation was introduced, the input should not be "I want to generate a configuration file that looks like this", but rather: Who is this node? Which tier / region does it belong to? Who does it interconnect with? Does this connection belong to DN42, IANA or Intra? What import/export policy should this connection use? Does it have special constraints like NAT, passive, endpoint override? As for the specific WireGuard interface wording, how the BIRD session is written, what the link-local address is – all these should, as much as possible, be generated by the automation system based on rules. Source of Truth: Node State, Not Config Files The input for this automation is mainly placed in Ansible's host_vars. The variables for each node are no longer just parameters needed for template rendering, but the description of the node's identity in the network. For example, a node will have at least: tier: node tier, e.g., backbone, IX, normal node region: region where the node is located node_id: node number within the same region node_token: a stable 16-bit token used to generate the IPv6 address suffix These fields are used for IPAM address management and identification information within the Confederation (as mentioned earlier, AS calculation and IP address derivation). The advantage of this approach is that when a new node joins, we just need to configure its basic metadata, and all its fundamental identities within the underlay are already determined. This significantly reduces subsequent maintenance effort. If link-local addresses were handwritten in every BIRD session, then any change to a node's token or address rule would require changes on all peers; but if the BIRD neighbor address is automatically derived from the peer node's metadata, it always stays consistent with the source of truth. Domain Intent: Declarative Peer Configuration The most important abstraction in this automation is Domain Intent, which treats the Session and its corresponding WireGuard tunnel binding as a single entity, then partitions them by the network type (DN42/IANA/Intra). Taking an Intra peer as an example, an internal connection fundamentally requires two types of configuration simultaneously: WireGuard interface, to provide the transport tunnel BIRD BGP session, to exchange routes over this tunnel If maintained manually, these two parts easily duplicate the same information: interface name peer node listen port endpoint neighbor interface BGP protocol name import/export policy So we abstract a regular Intra peer into a single peer intent. Simplified, it looks something like this: - node: tyo03-jp interface: intra_tyo03 wireguard: listen_port: 10234 passive: true bgp: protocol: intra_ibgp_tyo03 ipv4: import_policy: full export_policy: full ipv6: import_policy: full export_policy: full This declaration does not directly correspond to a single configuration file, but describes the peer intent: There is an internal connection named intra_tyo03 between the current node and tyo03-jp. This connection uses WireGuard for transport, and establishes an Intra BGP session over it, with BGP policy using full. Afterwards, Ansible automatically renders the corresponding WireGuard and BIRD configuration based on this. Thus, a peer relationship only needs to be described once, and the subsequent low-level configuration is generated automatically. For example, the BIRD neighbor's link-local address does not need to be handwritten; it can be automatically obtained from the peer node's tier / region / node_token. The WireGuard endpoint can also be automatically derived from the peer's ansible_host in the inventory and the peer's return peer listen_port. The biggest advantage of this is avoiding state inconsistency between multiple configuration layers. Automatic Endpoint Derivation and Passive Peers In my internal network architecture, not all nodes have perfectly symmetric public reachability. For example, some nodes / DN42 peers are behind NAT and can only initiate connections; some tunnels require a fixed endpoint as the listener, with the other end initiating. Therefore, the peer intent must support special cases like passive listening. If a peer is not passive, the automation can attempt to read ansible_host from the peer's inventory, combine it with the listen_port declared by the peer, and automatically generate the WireGuard endpoint: endpoint = peer_ansible_host + ":" + peer_return_listen_port If the peer is marked as passive, no endpoint is rendered, allowing the peer to actively initiate the connection. Essentially, the endpoint is determined by "how the peer is accessed" and "which port the peer listens on". These two pieces of information already exist in the inventory and peer intent, so there's no need to duplicate them. Of course, for extremely unusual links, the most basic manual override for the endpoint is retained. The goal of automation is not to eliminate all edge cases, but to make the normal cases require no special handling. Generic Scenarios & Exceptions This is essentially a fallback mechanism for special cases: Legacy tunnels Special internal peers carried over DN42 Nodes behind NAT Temporary workarounds Peers requiring strange BGP parameters Certain sessions needing special local AS Some routes only intended for the IANA domain, not the DN42 domain The final design kept: most regular Intra peers can generate WireGuard + BIRD via domain intent. DN42 peer WireGuard can also be generated from DN42 peer intents. However, some special BIRD sessions, such as those requiring separate local_as or special export policies, can still be explicitly declared using low-level configuration. Before final rendering, Ansible merges the generated configuration and legacy/manual overrides into the final configuration. The benefit is that automation covers 90% of the repetitive and error-prone parts, while still leaving room for the remaining 10% of special cases. Configuration Rendering The entire rendering process can be roughly understood as: flowchart TD A[host_vars: node metadata + domain intent] --> B[materialize domain intent] B --> C[effective_wireguard_interfaces] B --> D[effective_intra_ibgp_sessions] C --> E[render wg-quick configs] D --> F[render BIRD intra sessions] A --> G[render BIRD root / DN42 / IANA modules] A --> H[render underlay / overlay interfaces] E --> I[.rendered-wireguard] F --> J[.rendered] G --> J H --> K[.rendered-network] There are three types of final artifacts: .rendered/: BIRD configuration .rendered-wireguard/: WireGuard configuration .rendered-network/: underlay / overlay dummy interface configuration The BIRD configuration is further divided into several modules, written via Jinja2 templates; the actual overlay network separation is also handled here: root config: global defines, includes, kernel protocol DN42 module: DN42 filters, RPKI, eBGP peers, IX peers IANA module: upstream, downstream, peers, RPKI, static routes Intra module: confederation-style BGP sessions, static routes, internal filters Whether these modules are included is also decided automatically based on data as much as possible. For instance, if a node has no DN42 peers, there's no need to render and include ebgp.conf; no IX sessions, no need to include ix.conf. This reduces empty configurations and meaningless includes, and avoids some nodes failing template rendering due to missing related variables. Validation & Deployment The final deployment process also added some check and validation stages: render BIRD validate BIRD render WireGuard validate WireGuard deploy dummy interfaces deploy WireGuard deploy BIRD BIRD configuration is rendered and validated locally first to catch syntax errors as early as possible. WireGuard configuration is also rendered and validated first to avoid missing keys or generating obviously invalid wg-quick configs. Finally, after all validations pass, the underlying interfaces + WireGuard are deployed, and only then is BIRD uploaded and reloaded. Conclusion I'm exhausted; this article has drained me. Why is narrating this architecture in linear language so difficult? After sorting things out, most of the code was basically written by AI, and the results were decent. If you need a reference, see bird2-config/ansible at dev · iYoRoy-Network/bird2-config. The applicable scenarios for this solution are actually quite narrow; the DN42 + IANA BGP Player scene is really too niche xD. Moreover, a major reason this solution can function properly is that the IANA prefixes and DN42, along with the address ranges chosen for my own internal network, do not conflict at all. If there were address conflicts in the transit segments, one would probably still need to consider proper ISP isolation solutions like VRF/MPLS L3VPN. References: Configure BGP Confederation & Fake Confederation in Bird (Updated 2020-06-07) - Lan Tian @ Blog 浅谈 BGP 中的 Transit（中转）与 Peering（对等互联） | 网络蝙蝠侠部落
DN42&OneManISP - Troubleshooting OSPF Source Address in a Coexistence Environment
Backstory As mentioned in the previous post of this series, because the VRF solution was too isolating, the DNS service I deployed on the HKG node (172.20.234.225) became inaccessible from the DN42 network. Research indicated this could be achieved by setting up veth or NAT forwarding, but due to the scarcity of available documentation, I ultimately abandoned the VRF approach. Structure Analysis This time, I planned to place both DN42 and clearnet BGP routes into the system's main routing table, then separate them for export using filters to distinguish which should be exported. For clarity, I stored the configuration for the DN42 part and the clearnet part (hereinafter referred to as inet) separately, and then included them from the main configuration file. Also, since there should ideally only be one kernel configuration per routing table, I merged the DN42 and inet kernel parts, keeping only one instance. After multiple optimizations and revisions, my final directory structure is as follows: /etc/bird/ ├─envvars ├─bird.conf: Main Bird config file, defines basic info (ASN, IP, etc.), includes sub-configs below ├─kernel.conf: Kernel config, imports routes into the system routing table ├─dn42 | ├─defs.conf: DN42 function definitions, e.g., is_self_dn42_net() | ├─ibgp.conf: DN42 iBGP template | ├─rpki.conf: DN42 RPKI route validation | ├─ospf.conf: DN42 OSPF internal network | ├─static.conf: DN42 static routes | ├─ebgp.conf: DN42 Peer template | ├─ibgp | | └<ibgp configs>: DN42 iBGP configs for each node | ├─ospf | | └backbone.conf: OSPF area | ├─peers | | └<ibgp configs>: DN42 Peer configs for each node ├─inet | ├─peer.conf: Clearnet Peer | ├─ixp.conf: Clearnet IXP connection | ├─defs.conf: Clearnet function definitions, e.g., is_self_inet_v6() | ├─upstream.conf: Clearnet upstream | └static.conf: Clearnet static routes I separated the function definitions because I needed to reference them in the filters within kernel.conf, so I isolated them for early inclusion. After filling in the respective configurations and setting up the include relationships, I ran birdc configure and it started successfully. So, case closed... right? Problems occurred After running for a while, I suddenly found that I couldn't ping the HKG node from my internal devices, nor could I ping my other internal nodes from the HKG node. Strangely, external ASes could ping my other nodes or other external ASes through my HKG node, and my internal nodes could also ping other non-directly connected nodes (e.g., 226(NKG)->225(HKG)->229(LAX)) via the HKG node. Using ip route get <other internal node address> revealed: root@iYoRoyNetworkHKG:/etc/bird# ip route get 172.20.234.226 172.20.234.226 via 172.20.234.226 dev dn42_nkg src 23.149.120.51 uid 0 cache See the problem? The src address should have been the HKG node's own DN42 address (configured on the OSPF stub interface), but here it showed the HKG node's clearnet address instead. Attempting to read the route learned by Bird using birdc s r for 172.20.234.226: root@iYoRoyNetworkHKGBGP:/etc/bird/dn42/ospf# birdc s r for 172.20.234.226 BIRD 2.17.1 ready. Table master4: 172.20.234.226/32 unicast [dn42_ospf_iyoroynet_v4 00:30:29.307] * I (150/50) [172.20.234.226] via 172.20.234.226 on dn42_nkg onlink Looks seemingly normal...? Theoretically, although the DN42 source IP is different from the usual, DN42 rewrites krt_prefsrc when exporting to the kernel to inform the kernel of the correct source address, so this issue shouldn't occur: protocol kernel kernel_v4{ ipv4 { import none; export filter { if source = RTS_STATIC then reject; + if is_valid_dn42_network() then krt_prefsrc = DN42_OWNIP; accept; }; }; } protocol kernel kernel_v6 { ipv6 { import none; export filter { if source = RTS_STATIC then reject; + if is_valid_dn42_network_v6() then krt_prefsrc = DN42_OWNIPv6; accept; }; }; } Regarding krt_prefsrc, it stands for Kernel Route Preferred Source. This attribute doesn't manipulate the route directly but instead attaches a piece of metadata to it. This metadata directly instructs the Linux kernel to prioritize the specified IP address as the source address for packets sent via this route. I was stuck on this for a long time. The Solution Finally, during an unintentional attempt, I added the krt_prefsrc rewrite to the OSPF import configuration as well: protocol ospf v3 dn42_ospf_iyoroynet_v4 { router id DN42_OWNIP; ipv4 { - import where is_self_dn42_net() && source != RTS_BGP; + import filter { + if is_self_dn42_net() && source != RTS_BGP then { + krt_prefsrc=DN42_OWNIP; + accept; + } + reject; + }; export where is_self_dn42_net() && source != RTS_BGP; }; include "ospf/*"; }; protocol ospf v3 dn42_ospf_iyoroynet_v6 { router id DN42_OWNIP; ipv6 { - import where is_self_dn42_net_v6() && source != RTS_BGP; + import filter { + if is_self_dn42_net_v6() && source != RTS_BGP then { + krt_prefsrc=DN42_OWNIPv6; + accept; + } + reject; + }; export where is_self_dn42_net_v6() && source != RTS_BGP; }; include "ospf/*"; }; After running this, the src address became correct, and mutual pinging worked. Configuration files for reference: KaguraiYoRoy/Bird2-Configuration
Foreword I am a novice in BGP. This article may contain imprecise content/naive understandings/elementary mistakes. I kindly ask the experts to be lenient. If you find any issues, you are welcome to contact me via email, and I will correct them as soon as possible. If you find this unacceptable, it is recommended to close this article now. What are BGP Communities? TL;DR: BGP Communities "tag" routes, allowing others to use these tags for route optimization. This concept might be unfamiliar to newcomers (like me), who might not initially understand its purpose. Simply put, BGP Communities are a mechanism for tagging routes, similar to adding labels. They allow network administrators to attach one or more "tags" (i.e., community values) to routes propagated via BGP. These tags themselves do not alter the route's path attributes (like AS_PATH, LOCAL_PREF, MED, etc.), but they provide a signaling mechanism to indicate what policy or processing should be applied to that route by other routers within the same AS or in downstream peer ASes. BGP Communities can be used to: Simplify Policy Configuration: Routers inside the network or in downstream ASes only need to configure policies based on community values (like setting LOCAL_PREF, adding NO_EXPORT, applying route-maps, etc.), without needing to know the specific prefix details. This makes policies more centralized, easier to manage, and less prone to errors. Convey Policy Intent to Downstream ASes: An AS can attach community values to routes it advertises to its downstream customer or peer ASes. These communities convey requirements or suggestions on how these routes should be handled, such as route optimization based on geographic location, latency, or bandwidth. Coordinate Policies within an AS: Inside a large AS, when using IBGP full-mesh or route reflectors, edge routers (receiving EBGP routes or redistributing routes) can tag routes with community values. Core routers or route reflectors within the AS can recognize these communities and apply corresponding internal policies (like setting LOCAL_PREF, MED, deciding whether to advertise to certain IBGP peers, adding other communities, etc.), without needing complex prefix-based policies on every internal router. DN42 has its own set of Communities specifications. For details, please refer to: BGP-communities - DN42 Wiki Configuration The general idea and approach in this article are largely based on Xe_iu's method, focusing on adding BGP Communities for geographic information and performing route optimization. Generally, this is sufficient. (Another reason is that I haven't fully grasped the others yet) Note: We should ONLY add geographic information-related BGP Communities to routes originating from our own AS. We should not add such entries to routes received from neighbors. Adding our own regional Communities to a neighbor's routes constitutes forging the route origin, potentially leading to route hijacking. Downstream networks might misjudge the traffic path, routing traffic that should be direct through your network, increasing latency and consuming your network's bandwidth. (Large Communities are an exception, as they have a verification mechanism to prevent this, but that's beyond the scope of this article). The idea is clear. When exporting routes, we first need to verify if it's our own route. If it is, we add the communities tag to the route. The sample configuration provided by DN42 already includes two functions, is_self_net() and is_self_net_v6(), to check if a route is our own. Therefore, writing the configuration part is straightforward. Adding Communities to Routes First, we need to define the geographic region information for the current node at the beginning of the node configuration file. Please check BGP-communities - DN42 Wiki for details: define DN42_REGION = 52; # 52 represents East Asia define DN42_COUNTRY= 1344; # 1344 represents Hong Kong Please modify these values according to your node's actual geographic location. Then, modify the export filter in the dnpeers template: }; export filter { - if is_valid_network() && source ~ [RTS_STATIC, RTS_BGP] then accept; + if is_valid_network() && source ~ [RTS_STATIC, RTS_BGP] then{ + if (is_self_net()) then { # Check if it's our own route + bgp_community.add((64511, DN42_REGION)); # Add continent-level region info + bgp_community.add((64511, DN42_COUNTRY)); # Add country/region info + } + accept; + } reject; }; import limit 1000 action block; During export, it checks if it's our own route. If it is, it sets the bgp_community according to the defined DN42_REGION and DN42_COUNTRY. Here, 64511 is the reserved public AS number identifier specifically for geographic tags (Region/Country); you can just copy it. Apply the same method to the IPv6 export rules, but replace is_self_net() with is_self_net_v6(). Route Optimization Based on Communities Here we need to introduce another concept: local_pref (Local Preference). It is used within an AS to indicate the priority of a route. Its default value is 100, and a higher value indicates higher priority. Furthermore, in BGP route selection logic, local_pref has the highest priority, even higher than AS_PATH length. This means that by setting local_pref, we can adjust the priority of routes to achieve route optimization. Combining this with the BGP Communities mentioned above, we can set the corresponding local_pref based on Communities for optimization. Also, since BGP.local_pref is propagated within the AS, we need to modify the import logic for both eBGP and iBGP routes. My logic for handling BGP.local_pref here references (basically copies) Xe_iu's approach: For routes from the same region, priority +10 For routes from the same country, priority +5 additionally For routes received via direct peering with us, priority +20 additionally Create a function to calculate the priority: function ebgp_calculate_priority() { int priority = 100; # Base priority # Same region detection (+10) if bgp_community ~ [(64511, DN42_REGION)] then priority = priority + 10; # Same country detection (+5) if bgp_community ~ [(64511, DN42_COUNTRY)] then priority = priority + 5; # Direct eBGP neighbor detection (+20) if bgp_path.len = 1 then priority = priority + 20; return priority; } Then, in the import filter within the dnpeers template, set bgp_local_pref to the value calculated by the function: template bgp dnpeers { local as OWNAS; path metric 1; ipv4 { import filter { if is_valid_network() && !is_self_net() then { if (roa_check(dn42_roa, net, bgp_path.last) != ROA_VALID) then { print "[dn42] ROA check failed for ", net, " ASN ", bgp_path.last; reject; } + bgp_local_pref = ebgp_calculate_priority(); accept; } reject; }; Apply the same method for IPv6. After running birdc configure, we should be able to see that our routes have been tagged with Communities labels at our neighbors: (Screenshot source:https://lg.milu.moe/route_all/hk/172.20.234.224) Special thanks to Nuro Trace and Xe_iu. They helped deepen my understanding of BGP Communities and provided much assistance. Reference Articles: [DN42] bird2的配置文件 – Xe_iu's Blog | Xe_iu的杂物间 [DN42] 谈一谈如何配置 BGP community – Xe_iu's Blog | Xe_iu的杂物间 BGP-communities - DN42 Wiki
DN42 - Ep.3 Registering a Domain and Setting Up Authoritative DNS in DN42
Foreword I am a novice in BGP. This article may contain imprecise content/naive understandings/elementary mistakes. If you find any issues, you are welcome to contact me via email, and I will correct them as soon as possible. If you find this unacceptable, it is recommended to close this article now. Assumption: You have already joined DN42, can normally send and receive routing tables, and can access IPs within DN42. Article Update Log {timeline} {timeline-item color="#50BFFF"} August 3, 2025: First edition published. {/timeline-item} {timeline-item color="#4F9E28"} March 15, 2026: Update typos. Thanks to @Auride. {/timeline-item} {/timeline} Motivation While debugging the network, I noticed that pinging or tracerouting others' DN42 IPs could display the reverse-resolved domain names, making it clear which nodes the route passed through, rather than just looking at IPs (as shown in the figure below). It's very intuitive, letting others see at a glance if you've taken a detour. Therefore, I decided to register my own DN42 domain and set up an authoritative DNS service. After reading Lantian's article, I saw he used a PowerDNS + MySQL master-slave synchronization solution. However, my server has limited performance (only 1 core, 1GB RAM), so I plan to use KnotDNS as the DNS server, utilizing the standard zone transfer protocol (AXFR/IXFR) for master-slave synchronization. Preparations {alert type="warning"} The domain names and IPs mentioned in this and subsequent chapters are my own. Please replace them with your own during actual deployment; values enclosed in angle brackets need to be changed according to your requirements. {/alert} I chose the domain: yori.dn42, and plan to deploy DNS servers on three machines: 172.20.234.225, fd18:3e15:61d0::1, ns1.yori.dn42 172.20.234.227, fd18:3e15:61d0::3, ns2.yori.dn42 172.20.234.229, fd18:3e15:61d0::5, ns3.yori.dn42 Among them, ns1.yori.dn42 will be the master node, and ns2, ns3 will be the slave nodes. Installing KnotDNS If port 53 on the system is occupied by a process like systemd-resolvd, disable it first: systemctl stop systemd-resolved systemctl disable systemd-resolved unlink /etc/resolv.conf echo "nameserver 8.8.8.8" > /etc/resolv.conf I am using Debian 12, so I'll use APT for installation: apt install knot knot-dnsutils -y Set KnotDNS to start automatically: systemctl enable knot Configuring KnotDNS Creating a Key First, create a key for synchronization: keymgr -t key_knsupdate Copy the output: # hmac-sha256:key_knsupdate:<your secret> key: - id: key_knsupdate algorithm: hmac-sha256 secret: <your secret> Editing the Configuration File Master Node Edit /etc/knot/knot.conf and fill in the following content: server: rundir: "/run/knot" user: knot:knot automatic-acl: on listen: [ <listen_address1>@53, <listen_address2>@53, ... ] log: - target: syslog any: info database: storage: "/var/lib/knot" ### Paste the Key generated in the previous step here # hmac-sha256:key_knsupdate:<your secret> key: - id: key_knsupdate algorithm: hmac-sha256 secret: <your secret> remote: - id: <DNS_Node_1_ID> address: <DNS_Node_1_IP>@53 - id: <DNS_Node_2_ID> address: <DNS_Node_2_IP>@53 - id: <DNS_Node_3_ID> address: <DNS_Node_3_IP>@53 acl: - id: acl_slave key: key_knsupdate action: transfer - id: acl_master key: key_knsupdate action: notify - id: acl_knsupdate key: key_knsupdate action: update template: - id: default storage: "/var/lib/knot" file: "%s.zone" zone: - domain: <DN42 Domain> notify: [ <Slave_Node_1_ID>, <Slave_Node_2_ID> ] acl: [ acl_slave, acl_knsupdate ] - domain: <IPv4 Reverse Lookup Domain> notify: [ <Slave_Node_1_ID>, <Slave_Node_2_ID> ] acl: [ acl_slave, acl_knsupdate ] - domain: <IPv6 Reverse Lookup Domain> notify: [ <Slave_Node_1_ID>, <Slave_Node_2_ID> ] acl: [ acl_slave, acl_knsupdate ] The listen addresses should include the machine's DN42 IPv4 and DN42 IPv6 addresses. For local debugging, you can add internal IPs like 127.0.0.1 and ::1. The Slave Node IDs are the IDs set in the remote section for the servers you designated as slave nodes. The address in remote can be an internal address, DN42 IPv4, or DN42 IPv6, used only for master-slave synchronization. If using an internal address, add it to the listen list. The template section sets the Zone file storage location to /var/lib/knot. The IPv4 Reverse Lookup Domain should follow the format specified in RFC 2317 based on your allocated IPv4 block. For example, my IPv4 block is 172.20.234.224/28, so my IPv4 reverse lookup domain should be 224/28.234.20.172.in-addr.arpa. This treats the last octet 224/28 as a whole, reverses the order of the remaining parts, and appends .in-addr.arpa. The IPv6 Reverse Lookup Domain should follow the format specified in RFC 3152 based on your allocated IPv6 block. For example, my IPv6 block is fd18:3e15:61d0::/48, so my IPv6 reverse lookup domain should be 0.d.1.6.5.1.e.3.8.1.d.f.ip6.arpa. This involves reversing the order of the nibbles in the network prefix (excluding the /48) and appending .ip6.arpa. Pad with zeros if necessary. {collapse} {collapse-item label="Example"} server: rundir: "/run/knot" user: knot:knot automatic-acl: on listen: [ 172.20.234.225@53, fd18:3e15:61d0::1@53, localhost@53, 127.0.0.1@53 ] log: - target: syslog any: info database: storage: "/var/lib/knot" # hmac-sha256:key_knsupdate:<key> key: - id: key_knsupdate algorithm: hmac-sha256 secret: <key> remote: - id: 225 # Master Node address: 172.20.234.225@53 - id: 227 # Slave Node address: 172.20.234.227@53 - id: 229 # Slave Node address: 172.20.234.229@53 acl: - id: acl_slave key: key_knsupdate action: transfer - id: acl_master key: key_knsupdate action: notify - id: acl_knsupdate key: key_knsupdate action: update template: - id: default storage: "/var/lib/knot" file: "%s.zone" zone: - domain: yori.dn42 notify: [ 227, 229 ] acl: [ acl_slave, acl_knsupdate ] - domain: 224/28.234.20.172.in-addr.arpa notify: [ 227, 229 ] acl: [ acl_slave, acl_knsupdate ] - domain: 0.d.1.6.5.1.e.3.8.1.d.f.ip6.arpa notify: [ 227, 229 ] acl: [ acl_slave, acl_knsupdate ] {/collapse-item} {/collapse} Slave Nodes The configuration for slave nodes is largely similar to the master node. Just change the listen addresses to the slave node's addresses and modify the zone section configuration as follows: --- a/knot.conf +++ b/knot.conf zone: - domain: <DN42 Domain> - notify: [ <Slave_Node_1_ID>, <Slave_Node_2_ID> ] - acl: [ acl_slave, acl_knsupdate ] + master: <Master_Node_ID> + zonefile-load: whole + acl: acl_master - domain: <IPv4 Reverse Lookup Domain> - notify: [ <Slave_Node_1_ID>, <Slave_Node_2_ID> ] - acl: [ acl_slave, acl_knsupdate ] + master: <Master_Node_ID> + zonefile-load: whole + acl: acl_master - domain: <IPv6 Reverse Lookup Domain> - notify: [ <Slave_Node_1_ID>, <Slave_Node_2_ID> ] - acl: [ acl_slave, acl_knsupdate ] + master: <Master_Node_ID> + zonefile-load: whole + acl: acl_master The Master Node ID is the ID set in the remote section for the server you designated as the master node. {collapse} {collapse-item label="Examp;e"} server: rundir: "/run/knot" user: knot:knot automatic-acl: on listen: [ 172.20.234.227@53, fd18:3e15:61d0::3@53, localhost@53, 127.0.0.1@53 ] log: - target: syslog any: info database: storage: "/var/lib/knot" # hmac-sha256:key_knsupdate:<key> key: - id: key_knsupdate algorithm: hmac-sha256 secret: <key> remote: - id: 225 address: 172.20.234.225@53 - id: 227 address: 172.20.234.227@53 - id: 229 address: 172.20.234.229@53 acl: - id: acl_slave key: key_knsupdate action: transfer - id: acl_master key: key_knsupdate action: notify - id: acl_knsupdate key: key_knsupdate action: update template: - id: default storage: "/var/lib/knot" file: "%s.zone" zone: - domain: yori.dn42 master: 225 zonefile-load: whole acl: acl_master - domain: 224/28.234.20.172.in-addr.arpa master: 225 zonefile-load: whole acl: acl_master - domain: 0.d.1.6.5.1.e.3.8.1.d.f.ip6.arpa master: 225 zonefile-load: whole acl: acl_master {/collapse-item} {/collapse} After writing the configuration file, run the following command to restart KnotDNS: systemctl restart knot Editing Zone Files All configurations in this section are done on the primary DNS server. For record values (not hostnames) that require domain names, unless otherwise specified, please follow the RFC 1034 specification and use FQDN format. DN42 Domain Navigate to /var/lib/knot and create a file named <dn42_domain>.zone. SOA Record The first record of the zone must be the SOA record. The SOA record is the Start of Authority record, containing basic information about the domain, such as the primary NS server address. Fill in the following content: @ <TTL> SOA <Primary_NS_Server_Address> <Contact_Email> <Serial_Number> <Refresh_Time> <Retry_Time> <Expire_Time> <Minimum_TTL> @ represents the current domain itself, do not change it. TTL: The TTL (Time To Live) value for this SOA record. Primary NS Server Address: The address of the primary authoritative NS server for this domain. This can be a resolution within the domain. For example, my primary NS server is 172.20.234.225, and I plan to use ns1.yori.dn42. pointing to this address, so here I can fill in ns1.yori.dn42.. Contact Email: The email address, with @ replaced by .. For example, my email is i@iyoroy.cn, so here I can fill in i.iyoroy.cn. Serial Number: A 10-digit number following RFC 1912, representing the version of the zone file. Other DNS servers will refetch the records if they detect an increase in the serial number when querying the SOA. It's commonly encoded using the date + a sequence number, so this value should be incremented after each modification. Refresh Time:: The interval for AXFR slave nodes to pull the zone. Retry Time: The retry interval for AXFR slave nodes after a failed pull. Expire Time: The maximum time an AXFR slave node can continue serving with the last successfully pulled records after a failure, after which it stops responding. Minimum TTL: The minimum TTL value for the entire domain, the minimum refresh time for all records. Records won't be refreshed before at least this much time has passed. {collapse} {collapse-item label="Examp;e"} ; SOA @ 3600 SOA ns1.yori.dn42. i.iyoroy.cn. 2025072705 60 60 1800 60 {/collapse-item} {/collapse} NS Records @ <TTL> NS <NS_Server_1> @ <TTL> NS <NS_Server_2> @ <TTL> NS <NS_Server_3> Fill this in according to your actual situation; add as many records as you have servers. {collapse} {collapse-item label="Example"} ; NS @ 3600 NS ns1.yori.dn42. @ 3600 NS ns2.yori.dn42. @ 3600 NS ns3.yori.dn42. {/collapse-item} {/collapse} A, AAAA, CNAME, etc. Records Fill them in according to the following format: <Hostname> <TTL> <Type> <Record_Value> If your NS server values point to hosts within your own DN42 domain, be sure to add A or AAAA resolution records for them. {collapse} {collapse-item label="Example"} ; A ns1 600 A 172.20.234.225 ns2 600 A 172.20.234.227 ns3 600 A 172.20.234.229 hkg-cn.node 600 A 172.20.234.225 nkg-cn.node 600 A 172.20.234.226 tyo-jp.node 600 A 172.20.234.227 hfe-cn.node 600 A 172.20.234.228 lax-us.node 600 A 172.20.234.229 ; AAAA ns1 600 AAAA fd18:3e15:61d0::1 ns2 600 AAAA fd18:3e15:61d0::3 ns3 600 AAAA fd18:3e15:61d0::5 hkg-cn.node 600 AAAA fd18:3e15:61d0::1 nkg-cn.node 600 AAAA fd18:3e15:61d0::2 tyo-jp.node 600 AAAA fd18:3e15:61d0::3 hfe-cn.node 600 AAAA fd18:3e15:61d0::4 lax-us.node 600 AAAA fd18:3e15:61d0::5 {/collapse-item} {collapse-item label="Complete Example"} /var/lib/knot/yori.dn42.zone ; SOA @ 3600 SOA ns1.yori.dn42. i.iyoroy.cn. 2025072705 60 60 1800 60 ; NS @ 3600 NS ns1.yori.dn42. @ 3600 NS ns2.yori.dn42. @ 3600 NS ns3.yori.dn42. ; A ns1 600 A 172.20.234.225 ns2 600 A 172.20.234.227 ns3 600 A 172.20.234.229 hkg-cn.node 600 A 172.20.234.225 nkg-cn.node 600 A 172.20.234.226 tyo-jp.node 600 A 172.20.234.227 hfe-cn.node 600 A 172.20.234.228 lax-us.node 600 A 172.20.234.229 ; AAAA ns1 600 AAAA fd18:3e15:61d0::1 ns2 600 AAAA fd18:3e15:61d0::3 ns3 600 AAAA fd18:3e15:61d0::5 hkg-cn.node 600 AAAA fd18:3e15:61d0::1 nkg-cn.node 600 AAAA fd18:3e15:61d0::2 tyo-jp.node 600 AAAA fd18:3e15:61d0::3 hfe-cn.node 600 AAAA fd18:3e15:61d0::4 lax-us.node 600 AAAA fd18:3e15:61d0::5 {/collapse-item} {/collapse} IPv4 Reverse Lookup Domain Create a file in /var/lib/knot named <IPv4_Reverse_Lookup_Domain>.zone, replacing / with _. For example, my IPv4 block is 172.20.234.224/28, and my IPv4 reverse lookup domain is 224/28.234.20.172.in-addr.arpa, so the filename here would be 224_28.234.20.172.in-addr.arpa.zone. Fill in the resolution records: ; SOA @ <TTL> SOA <Primary_NS_Server_Address> <Contact_Email> <Serial_Number> <Refresh_Time> <Retry_Time> <Expire_Time> <Minimum_TTL> ; NS @ <TTL> NS <NS_Server_1> @ <TTL> NS <NS_Server_2> @ <TTL> NS <NS_Server_3> ; PTR <Last_IPv4_Octet> <TTL> PTR <Reverse_DNS_Value> <Last_IPv4_Octet> <TTL> PTR <Reverse_DNS_Value> <Last_IPv4_Octet> <TTL> PTR <Reverse_DNS_Value> ... The SOA and NS records are the same as above. Last IPv4 Octet: The last octet of the DN42 IPv4 address you assigned to the device. For example, my HK node is assigned 172.20.234.225, so here I would put 225. {collapse} {collapse-item label="Example"} 224_28.234.20.172.in-addr.arpa.zone ; SOA @ 3600 SOA ns1.yori.dn42. i.iyoroy.cn. 2025072802 60 60 1800 60 ; NS @ 3600 NS ns1.yori.dn42. @ 3600 NS ns2.yori.dn42. @ 3600 NS ns3.yori.dn42. ; PTR 225 600 PTR hkg-cn.node.yori.dn42. 226 600 PTR nkg-cn.node.yori.dn42. 227 600 PTR tyo-jp.node.yori.dn42. 228 600 PTR hfe-cn.node.yori.dn42. 229 600 PTR lax-us.node.yori.dn42. {/collapse-item} {/collapse} You might wonder why the CIDR mask is needed, which differs from the common Clearnet format of reversed octets (e.g., 234.20.172.in-addr.arpa). Also, if you test locally, you might find that reverse lookups for your own IP addresses fail directly. The reason lies in DN42's distributed registry mechanism: a single zone file cannot cover all reverse query entry points for your address block (i.e., the .in-addr.arpa name for each specific IP address). To solve this, after your PR is merged, the official DN42 DNS will add CNAME redirects for your address block on its authoritative servers, pointing individual IP PTR queries to your CIDR-formatted zone, as shown below: ~$ dig PTR 225.234.20.172.in-addr.arpa +short 225.224/28.234.20.172.in-addr.arpa. # <-- CNAME added by Registry (redirect) hkg-cn.node.yori.dn42. # <-- Final PTR record returned by your DNS When an external resolver queries the reverse record for a specific IP (e.g., 172.20.234.225) (querying 225.234.20.172.in-addr.arpa), the official DN42 DNS returns a CNAME record pointing it to the specific record under the CIDR zone name (225.224/28.234.20.172.in-addr.arpa). Ultimately, the PTR record is provided by your configured authoritative DNS server. IPv6 Reverse Lookup Domain Create a file in /var/lib/knot named <IPv6_Reverse_Lookup_Domain>.zone. For example, my IPv6 block is fd18:3e15:61d0::/48, and my IPv6 reverse lookup domain is 0.d.1.6.5.1.e.3.8.1.d.f.ip6.arpa, so the filename here would be 0.d.1.6.5.1.e.3.8.1.d.f.ip6.arpa.zone. Fill in the resolution records: ; SOA @ <TTL> SOA <Primary_NS_Server_Address> <Contact_Email> <Serial_Number> <Refresh_Time> <Retry_Time> <Expire_Time> <Minimum_TTL> ; NS @ <TTL> NS <NS_Server_1> @ <TTL> NS <NS_Server_2> @ <TTL> NS <NS_Server_3> ; PTR <Reversed_Last_20_Nibbles> <TTL> PTR <Reverse_DNS_Value> <Reversed_Last_20_Nibbles> <TTL> PTR <Reverse_DNS_Value> <Reversed_Last_20_Nibbles> <TTL> PTR <Reverse_DNS_Value> ... Handle SOA and NS records as above. For the PTR hostname, you need to take the last 80 bits of the host's IPv6 address (after removing the /48 prefix), expand them into 20 hexadecimal characters, and reverse the order of these characters, separating them with dots. For example, my Hong Kong node's IPv6 is fd18:3e15:61d0::1. Expanded, this is fd18:3e15:61d0:0000:0000:0000:0000:0001. The hostname here would be 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0. {collapse} {collapse-item label="Example"} 0.d.1.6.5.1.e.3.8.1.d.f.ip6.arpa.zone ; SOA @ 3600 SOA ns1.yori.dn42. i.iyoroy.cn. 2025072802 60 60 1800 60 ; NS @ 3600 NS ns1.yori.dn42. @ 3600 NS ns2.yori.dn42. @ 3600 NS ns3.yori.dn42. ; PTR 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 600 PTR hkg-cn.node.yori.dn42. 2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 600 PTR nkg-cn.node.yori.dn42. 3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 600 PTR tyo-jp.node.yori.dn42. 4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 600 PTR hfe-cn.node.yori.dn42. 5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 600 PTR lax-us.node.yori.dn42. {/collapse-item} {/collapse} Verifying the Setup After saving everything, run knot reload on each DNS server. If all goes well, you should see the slave nodes synchronizing the zone files from the master node. You can then use dig or nslookup specifying the server to query the resolution records. Registration Domain Clone the DN42 Registry, navigate to data/dns, create a new file named <your_desired_domain>, and fill in the following content: domain: <your_desired_domain> admin-c: <Admin_NIC_Handle> tech-c: <Tech_NIC_Handle> mnt-by: <Maintainer> nserver: <NS1_Server_Domain> <NS1_Server_IP> nserver: <NS2_Server_Domain> <NS2_Server_IP> nserver: <NS3_Server_Domain> <NS3_Server_IP> ... source: DN42 Refer to DN42 - Ep.1 Joining the DN42 Network for admin-c, tech-c, and mnt-by. {collapse} {collapse-item label="Example"} data/dns/yori.dn42 domain: yori.dn42 admin-c: IYOROY-DN42 tech-c: IYOROY-DN42 mnt-by: IYOROY-MNT nserver: ns1.yori.dn42 172.20.234.225 nserver: ns1.yori.dn42 fd18:3e15:61d0::1 nserver: ns2.yori.dn42 172.20.234.227 nserver: ns2.yori.dn42 fd18:3e15:61d0::3 nserver: ns3.yori.dn42 172.20.234.229 nserver: ns3.yori.dn42 fd18:3e15:61d0::5 source: DN42 {/collapse-item} {/collapse} IPv4 Reverse Lookup Domain Navigate to data/inetnum, find the file for your registered address block, and add nserver fields pointing to your own DNS servers: nserver: <your_DNS_server_address> nserver: <your_DNS_server_address> ... ... {collapse} {collapse-item label="Example"} diff --git a/data/inetnum/172.20.234.224_28 b/data/inetnum/172.20.234.224_28 index 50c800945..5ad60e23d 100644 --- a/data/inetnum/172.20.234.224_28 +++ b/data/inetnum/172.20.234.224_28 @@ -8,3 +8,6 @@ tech-c: IYOROY-DN42 mnt-by: IYOROY-MNT status: ASSIGNED source: DN42 +nserver: ns1.yori.dn42 +nserver: ns2.yori.dn42 +nserver: ns3.yori.dn42 {/collapse-item} {/collapse} IPv6 Reverse Lookup Domain Navigate to data/inet6num, find the file for your registered address block, and add nserver fields pointing to your own DNS servers: nserver: <your_DNS_server_address> nserver: <your_DNS_server_address> ... {collapse} {collapse-item label="Example"} diff --git a/data/inet6num/fd18:3e15:61d0::_48 b/data/inet6num/fd18:3e15:61d0::_48 index 53f0de06d..1ae067b00 100644 --- a/data/inet6num/fd18:3e15:61d0::_48 +++ b/data/inet6num/fd18:3e15:61d0::_48 @@ -8,3 +8,6 @@ tech-c: IYOROY-DN42 mnt-by: IYOROY-MNT status: ASSIGNED source: DN42 +nserver: ns1.yori.dn42 +nserver: ns2.yori.dn42 +nserver: ns3.yori.dn42 {/collapse-item} {/collapse} Submit a PR and Wait for Merge After filling everything out, push your changes and submit a Pull Request. Because anyone in DN42 can run recursive DNS, it might take up to a week for the DNS configuration to fully propagate, although I found that the public DNS (172.20.0.53) could query my records within half a day after merging. Special thanks to たのしい for clarifying the differences between IPv4 reverse lookup in DN42 and the public internet. Reference Articles: https://www.haiyun.me/archives/1398.html https://www.jianshu.com/p/7d69ec2976c7 https://www.potat0.cc/posts/20220726/Register_DN42_Domain/ https://bbs.csdn.net/topics/393775423 https://blog.snorlax.blue/knot-reverse-dns-kickstart/ http://www.kkdlabs.jp/dns/automatic-dnssec-signing-by-knot-dns/ https://lantian.pub/article/modify-website/register-own-domain-in-dn42.lantian/ https://datatracker.ietf.org/doc/html/rfc2317 https://datatracker.ietf.org/doc/html/rfc3152 https://datatracker.ietf.org/doc/html/rfc1912#section-2.2
DN42 - Ep.2 Building Internal Network with OSPF and Enabling iBGP
Foreword I am a novice in BGP. This article may contain imprecise content/naive understandings/elementary mistakes. I kindly ask the experts to be lenient. If you find any issues, you are welcome to contact me via email, and I will correct them as soon as possible. If you find this unacceptable, it is recommended to close this article now. Article Update Log {timeline} {timeline-item color="#50BFFF"} July 22, 2025: First edition published, using VXLAN over WireGuard tunnel. {/timeline-item} {timeline-item color="#50BFFF"} July 25, 2025: Updated tunneling solution, using type ptp; to support OSPF traffic via WireGuard (Special thanks to Nuro Trance for the guidance!). {/timeline-item} {timeline-item color="#50BFFF"} August 8, 2025: Added explanation and configuration for iBGP. {/timeline-item} {timeline-item color="#4F9E28"} August 27, 2025: Updated node topology diagram. {/timeline-item} {/timeline} Why Do We Need Internal Routing? As the number of nodes increases, we need a proper way to handle internal routing within our AS (Autonomous System). BGP only handles routing to different ASes, which causes a problem: if nodes A and B are both peering with external networks, a request from node A may have its response routed to node B, even though they are part of the same AS. Without internal routing, node A will not receive the reply. To solve this, we need to ensure that all devices within our AS can communicate with each other. The common solutions are: Using network tools like ZeroTier: Simple to set up, just install the client on each node for P2P connectivity. Using P2P tools like WireGuard to manually create $\frac{n(n-1)}{2}$ tunnels, which works like the first solution but becomes cumbersome as nodes grow. Using WireGuard to establish $\frac{n(n-1)}{2}$ tunnels, then using an internal routing protocol like OSPF or Babel to manage the routing. This is more flexible and easier to scale, but it can be risky and could break the DN42 network due to misconfigurations. Thus, I decided to take the risk. Node Topology graph LR A[HKG 172.20.234.225 fd18:3e15:61d0::1] B[NKG 172.20.234.226 fd18:3e15:61d0::2] C[TYO 172.20.234.227 fd18:3e15:61d0::3] D[FRA 172.20.234.228 fd18:3e15:61d0::4] E[LAX 172.20.234.229 fd18:3e15:61d0::5] B <--> A C <--> A A <--> E A <--> D C <--> D C <--> E D <--> E Update Bird2 to v2.16 or Above To use IPv6 Link-Local addresses to transmit IPv4 OSPF data, Bird v2.16 or later is required. Here are the steps to update: sudo apt update && sudo apt -y install apt-transport-https ca-certificates wget lsb-release sudo wget -O /usr/share/keyrings/cznic-labs-pkg.gpg https://pkg.labs.nic.cz/gpg echo "deb [signed-by=/usr/share/keyrings/cznic-labs-pkg.gpg] https://pkg.labs.nic.cz/bird2 $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/cznic-labs-bird2.list sudo apt update && sudo apt install bird2 -y Tunnel Configuration [Interface] PrivateKey = <Local WireGuard Private Key> ListenPort = <Listen Port> Table = off Address = <IPv6 LLA>/64 PostUp = sysctl -w net.ipv6.conf.%i.autoconf=0 [Peer] PublicKey = <Peer Public Key> Endpoint = <Peer Public Endpoint> AllowedIPs = 10.0.0.0/8, 172.20.0.0/14, 172.31.0.0/16, fd00::/8, fe00::/8, ff02::5 ff02::5is the OSPFv3 router-specific link-local multicast address and should be included in AllowedIPs. If you're using Bird versions earlier than v2.16, you'll need to add an IPv4 address for the tunnel as well. See the example below: {collapse} {collapse-item label="WireGuard Configuration Example with IPv4"} [Interface] PrivateKey = <Local WireGuard Private Key> ListenPort = <Listen Port> Table = off Address = <IPv6 LLA>/64 PostUp = ip addr add 100.64.0.225/32 peer 100.64.0.226/32 dev %i PostUp = sysctl -w net.ipv6.conf.%i.autoconf=0 [Peer] PublicKey = <Peer Public Key> Endpoint = <Peer Public Endpoint> AllowedIPs = 10.0.0.0/8, 172.20.0.0/14, 100.64.0.0/16, 172.31.0.0/16, fd00::/8, fe00::/8, ff02::5 Please replace 100.64.0.225 and 100.64.0.226 with your local and peer IPv4 addresses, and remember to add AllowedIPs. {/collapse-item} {/collapse} Enable OSPF You should have already configured basic Bird settings as described in the previous article. Create a new file called ospf.conf under /etc/bird and add the following: protocol ospf v3 <name> { ipv4 { import where is_self_net() && source != RTS_BGP; export where is_self_net() && source != RTS_BGP; }; include "/etc/bird/ospf/*"; }; protocol ospf v3 <name> { ipv6 { import where is_self_net_v6() && source != RTS_BGP; export where is_self_net_v6() && source != RTS_BGP; }; include "/etc/bird/ospf/*"; }; Theoretically, OSPF v2 should be used for handling IPv4, but since we need to communicate IPv4 using IPv6 Link-Local addresses, we are using OSPF v3 for IPv4 in this case as well. The filter rules ensure that only routes within the local network segment are allowed to propagate through OSPF, and routes from external BGP protocols are filtered out. Never use import all; export all; indiscriminately, as this could lead to route hijacking and affect the entire DN42 network. OSPF should only handle internal network routes. {collapse} {collapse-item label="Example"} /etc/bird/ospf.conf protocol ospf v3 dn42_iyoroynet_ospf { ipv4 { import where is_self_net() && source != RTS_BGP; export where is_self_net() && source != RTS_BGP; }; include "/etc/bird/ospf/*"; }; protocol ospf v3 dn42_iyoroynet_ospf6 { ipv6 { import where is_self_net_v6() && source != RTS_BGP; export where is_self_net_v6() && source != RTS_BGP; }; include "/etc/bird/ospf/*"; }; {/collapse-item} {/collapse} Next, create the /etc/bird/ospf folder and then create an area configuration file (e.g., /etc/bird/ospf/backbone.conf) with the following content: area 0.0.0.0 { interface "<DN42 dummy interface>" { stub; }; interface "<wg0 interface>" { cost 80; # Modify according to your network situation type ptp; }; interface "<wg1 interface>" { cost 100; # Modify according to your network situation type ptp; }; # Continue for other interfaces }; The 0.0.0.0 area represents the backbone network. he dummy interface here refers to the DN42 virtual interface mentioned in the previous article The cost value is typically used for cost calculation but in DN42's case, where bandwidth is less critical but latency is more important, you can directly assign the latency value. OSPF will automatically choose the route with the lowest cost (sum of the cost values). {collapse} {collapse-item label="Example"} /etc/bird/ospf/backbone.conf area 0.0.0.0 { interface "dn42" { stub; }; interface "dn42_hkg" { cost 80; type ptp; }; interface "dn42_hfe" { cost 150; type ptp; }; interface "dn42_lax"{ cost 100; type ptp; }; }; {/collapse-item} {/collapse} Finally, open /etc/bird/bird.conf and add the following to include the OSPF configuration file at the end: include "ospf.conf"; Run birdc configure, and then birdc show protocols should show the OSPF status as Running. If not, check the configuration steps for errors. At this point, you should be able to ping between two non-directly connected machines: Enable iBGP Before establishing multiple peer connections, each of your nodes must first have complete knowledge of the internal AS topology. This involves configuring another key component: internal BGP (iBGP). Necessity of iBGP iBGP ensures that all routers within the AS have complete knowledge of external destination routes. It ensures that: Internal routers can select the best exit path. Traffic is correctly routed to the boundary routers responsible for specific external networks. Even if there are multiple boundary routers connected to the same external network, internal routers can choose the best exit based on policies. Compared to using a default route pointing to the border router within the AS, iBGP provides precise external route information, allowing internal routers to make more intelligent forwarding decisions. Disadvantages and Solutions To prevent uncontrolled propagation of routing information within the AS, which could cause loops, an iBGP router will not readvertise routes learned from one iBGP neighbor to other iBGP neighbors. This necessitates that traditional iBGP requires a full mesh of iBGP neighbor relationships between all iBGP-running routers within the same AS. (You still need to establish $\frac{n(n+1)}{2}$ connections , there's no way around it. But configuring iBGP is still easier than configuring tunnels after OSPF is set up ). Solutions include: Using a Route Reflector (RR): An RR router manages all routing information within the entire AS. The disadvantage is that if the RR router fails, the entire network can be paralyzed (which is not very Decentralized). Using BGP Confederation: This involves virtually dividing the routers within the AS into sub-ASes, treating the connections between routers as eBGP, and finally stripping the internal AS path information when advertising routes externally. I haven't tried the latter two solutions. Here are some potentially useful reference articles. This article focuses on the configuration of iBGP. DN42 Experimental Network: Intro and Registration (Updated 2022-12) - Lan Tian @ Blog Configure BGP Confederation & Fake Confederation in Bird (Updated 2020-06-07) - Lan Tian @ Blog Writing the iBGP Configuration File Create a new file ibgp.conf in /etc/bird and fill it with the following content: template bgp ibgpeers { local as OWNAS; ipv4 { import where source = RTS_BGP && is_valid_network() && !is_self_net(); export where source = RTS_BGP && is_valid_network() && !is_self_net(); next hop self; extended next hop; }; ipv6 { import where source = RTS_BGP && is_valid_network_v6() && !is_self_net_v6(); export where source = RTS_BGP && is_valid_network_v6() && !is_self_net_v6(); next hop self; }; }; include "ibgp/*"; The import and export filters ensure that iBGP only processes routes learned via the BGP protocol and filters out IGP routes to prevent loops. next hop self is required. It instructs BIRD to rewrite the next hop to the border router's own IP address (instead of the original external next hop) when exporting routes to iBGP neighbors. This is because internal routers cannot directly access the external neighbor's address; without rewriting, the address would be considered unreachable. After rewriting, internal routers only need to send traffic to the border router via IGP routing, and the border router handles the final external forwarding. Because I want to use IPv6 addresses to establish MP-BGP and route IPv4 over IPv6, extended next hop is enabled for IPv4. Next, create the /etc/bird/ibgp directory. Inside, create an iBGP Peer configuration file for each node: protocol bgp 'dn42_ibgp_<Node Name>' from ibgpeers{ neighbor <Corresponding Node's IPv6 ULA Address> as OWNAS; }; {collapse} {collapse-item label="Example"} /etc/bird/ibgp/hkg.conf: protocol bgp 'dn42_ibgp_HKG' from ibgpeers{ neighbor fd18:3e15:61d0::1 as OWNAS; }; {/collapse-item} {/collapse} Note: Each node needs to establish (n-1) iBGP connections, ensuring connectivity with all other machines within the AS. This is why ULA addresses are used. Using ULA addresses ensures that even if the WireGuard connection between two nodes goes down, iBGP can still establish connections via the internal routing established by OSPF. Otherwise, it could lead to the collapse of the entire internal network. Finally, add the inclusion of ibgp.conf in /etc/bird/bird.conf: include "ibgp.conf"; And run birdc configure to apply the configuration. References: BIRD 与 BGP 的新手开场 - 海上的宫殿 萌新入坑 DN42 之 —— 基于 tailscale + vxlan + OSPF 的组网 – 米露小窝 使用 Bird2 配置 WireGuard + OSPF 实现网络的高可用 | bs' realm DN42 实验网络介绍及注册教程（2022-12 更新） - Lan Tian @ Blog 如何引爆 DN42 网络（2023-05-12 更新） - Lan Tian @ Blog Bird 配置 BGP Confederation，及模拟 Confederation（2020-06-07 更新） - Lan Tian @ Blog 深入解析OSPF路径开销、优先级和计时器 - 51CTO New release 2.16 | BIRD Internet Routing Daemon 第一章·第二节 如何在 Linux 上安装最新版本的 BIRD？ | BIRD 中文文档 [DN42] 使用 OSPF ptp 搭建内网与IBGP配置 – Xe_iu's Blog | Xe_iu的杂物间 [译] dn42 多服务器环境中的 iBGP 与 IGP 配置 | liuzhen932 的小窝
Foreword I am a novice in BGP. This article may contain imprecise content/naive understandings/elementary mistakes. I kindly ask the experts to be lenient. If you find any issues, you are welcome to contact me via email, and I will correct them as soon as possible. If you find this unacceptable, it is recommended to close this article now. Welcome to peer with me! For details, please visit: iYoRoy DN42 Network I wanted to study BGP, but renting an ASN and IP block is too expensive, and I was afraid of taking down half the internet due to a configuration error. So, I decided to look into DN42, a virtual network. DN42 is a large decentralized network that uses the BGP protocol for routing, making its structure very similar to today's internet. This makes it suitable for learning networking technologies like BGP. In DN42, everyone plays the role of an ISP (Internet Service Provider), peering with other users to join and participate in building the entire DN42 network. DN42 operates on 172.20.0.0/14 and fd00::/8, both of which are private address ranges, so it won't affect the clearnet. Registration You need to know basic git commands, GPG, and Linux. It's also best to sign your commits with GPG. If you have basic Git knowledge, you can refer to my commit: Add AS4242422024 · d1f9046ecb - registry - dn42 git Fork and Clone the DN42 Registry Git Repository Register an account on dn42 git and Fork the dn42/registry repository. Clone your forked repository and enter it. Register a Contact Create a file named <Nickname>-DN42 under data/person and fill in the following content: person: <Nickname> e-mail: <Email Address> pgp-fingerprint: <GPG Key Fingerprint> nic-hdl: <NIC Handle> mnt-by: <Maintainer> source: DN42 person: Nickname. e-mail: Email address. pgp-fingerprint: GPG key fingerprint, used for some authentication services. nic-hdl: NIC handle, points to the file itself, just use the current filename. mnt-by: Maintained by, points to the maintainer information from the Register Maintainer section below. source: Source, keep it as DN42. www: Optional, your website URL. {collapse} {collapse-item label="Example"} data/person/IYOROY-DN42 @ dn42/registry@master person: Kagura iYoRoy e-mail: i@iyoroy.cn www: https://www.iyoroy.cn pgp-fingerprint: 3ECCFFDEC2CB4CB8DA8089BE9AF2F2E03CE8FD67 nic-hdl: IYOROY-DN42 mnt-by: IYOROY-MNT source: DN42 {/collapse-item} {/collapse} Register a Maintainer Create a file data/mntner/<Nickname>-MNT and fill in the following content: mntner: <Nickname>-MNT admin-c: <Contact> tech-c: <Contact> auth: <Authentication Method> mnt-by: <Maintainer> source: DN42 mntner: Maintainer name, generally the same as the filename. admin-c: Administrator contact information, points to a file in the person folder, use the filename from the Register Contact section above. tech-c: Technical contact information, points to a file in the person folder, use the filename from the Register Contact section above. auth: Authentication method, supports pgp or ssh-key. mnt-by: Maintained by, generally points to the filename itself. source: Source, keep it as DN42. {collapse} {collapse-item label="Example"} data/mntner/IYOROY-MNT @ dn42/registry@master mntner: IYOROY-MNT admin-c: IYOROY-DN42 tech-c: IYOROY-DN42 auth: pgp-fingerprint 3ECCFFDEC2CB4CB8DA8089BE9AF2F2E03CE8FD67 mnt-by: IYOROY-MNT source: DN42 {/collapse-item} {/collapse} Register an ASN On the clearnet, 4200000000 - 4294967294 is the reserved range for ASNs. DN42 uses 4242420000 - 4242429999, and currently, 4242420000 - 4242423999 is open for registration.The official recommendation is not to manually find an available ASN but to use Burble's DN42 Free ASN Explorer to select an available ASN for registration. After choosing your desired ASN, create a file <ASN, including AS letters> in data/aut-num and fill in the following content: aut-num: <ASN> as-name: <Autonomous System Name> descr: <Autonomous System Description> admin-c: <Administrator NIC Handle> tech-c: <Technical Contact NIC Handle> mnt-by: <Maintainer> source: DN42 aut-num: Your chosen ASN, including the AS prefix. It should be in the format AS424242xxxx. as-name: Autonomous System Name. descr: Autonomous System Description, can contain spaces. admin-c: Administrator contact information, points to a file in the person folder, use the filename from the Register Contact section above. tech-c: Technical contact information, points to a file in the person folder, use the filename from the Register Contact section above. mnt-by: Maintained by, points to the filename from the Create Maintainer section above. source: Source, keep it as DN42. {collapse} {collapse-item label="示例"} data/aut-num/AS4242422024 @ dn42/registry@master aut-num: AS4242422024 as-name: IYOROYNET-AS-DN42 descr: iYoRoy DN42 Network admin-c: IYOROY-DN42 tech-c: IYOROY-DN42 mnt-by: IYOROY-MNT source: DN42 {/collapse-item} {/collapse} Register an IPv4 Block and Add Route You can skip this section if you don't want to register IPv4 The official provides a tool for finding available address blocks: DN42 Free IPv4 Explorer. It is also not recommended to manually specify but to use the available blocks found through the search tool. DN42 also faces IPv4 address shortages, so please register according to your needs. Generally, a /27 block containing 30 usable IPv4 addresses is sufficient. For smaller networks, you can apply for /28 (14 usable addresses) and /29 (6 usable addresses). The largest block that can be directly applied for is /26, which is 62 addresses. Applying for /25, /24 requires submitting an application and waiting for review. For details, refer to: DN42 Experimental Network: Intro and Registration (Updated 2022-12) - Lan Tian @ Blog After selecting your IPv4 block, create a file in data/inetnum named after the CIDR format of the IPv4 block, using _ instead of / (e.g., if the applied IPv4 block is 172.20.234.224/28, the filename is 172.20.234.224_28), and fill in the content: inetnum: <First IPv4 Address in Block> - <Last IPv4 Address in Block> cidr: <IPv4 Block in CIDR Format> netname: <IPv4 Block Name> descr: <IPv4 Block Description> country: <Country Code for IPv4 Block> admin-c: <Administrator NIC Handle> tech-c: <Technical Contact NIC Handle> mnt-by: <Maintainer> status: ASSIGNED source: DN42 inetnum: IPv4 address range, from the first address to the last address, connected with -. cidr: IPv4 block in CIDR format, do not replace / with _. netname: IPv4 block name, fill in as desired, no special requirements. descr: IPv4 block description, can contain spaces. country: Two-character country code from ISO 3166, for China, fill in CN. admin-c: Administrator contact information, points to a file in the person folder, use the filename from the Register Contact section above. tech-c: Technical contact information, points to a file in the person folder, use the filename from the Register Contact section above. mnt-by: Maintained by, points to the filename from the Create Maintainer section above. status: Status, keep it as ASSIGNED. source: Source, keep it as DN42. {collapse} {collapse-item label="Example"} data/inetnum/172.20.234.224_28 @ dn42/registry@master inetnum: 172.20.234.224 - 172.20.234.239 cidr: 172.20.234.224/28 netname: IYOROYNET-DN42-V4 descr: iYoRoy DN42 Network IPv4 Block country: CN admin-c: IYOROY-DN42 tech-c: IYOROY-DN42 mnt-by: IYOROY-MNT status: ASSIGNED source: DN42 {/collapse-item} {/collapse} Next, under data/route, create a file also named after the IPv4 block's CIDR format, using _ instead of /, and fill in the content: route: <IPv4 Block in CIDR Format> origin: <ASN> max-length: <IPv4 Prefix Length> mnt-by: <Maintainer> source: DN42 route: IPv4 block, fill in CIDR format, do not convert /. origin: ASN, needs to include AS prefix. max-lenth: Prefix Length, i.e., the number after / in the CIDR format. mnt-by: Maintained by, points to the filename from the Create Maintainer section above. source: Source, keep it as DN42. {collapse} {collapse-item label="Example"} data/route/172.20.234.224_28 @ dn42/registry@master route: 172.20.234.224/28 origin: AS4242422024 max-length: 28 mnt-by: IYOROY-MNT source: DN42 {/collapse-item} {/collapse} Register an IPv6 Block and Add Route You can skip this section if you don't want to register IPv6 Similarly, use the official tool to find available IPv6: DN42 Free IPv6 Explorer 。 After selecting the address block, create a file under data/inet6num named after the IPv6 block's CIDR format, using _ instead of / (e.g., if the IPv6 block is fd18:3e15:61d0::/48, the filename is fd18:3e15:61d0::_48), and fill in the content: inet6num: <First IPv6 Address in Block> - <Last IPv6 Address in Block> cidr: <IPv6 Block in CIDR Format> netname: <IPv6 Block Name> descr: <IPv6 Block Description> country: <Country Code for IPv6 Block> admin-c: <Administrator NIC Handle> tech-c: <Technical Contact NIC Handle> mnt-by: <Maintainer> status: ASSIGNED source: DN42 inet6num: IPv6 address range, from the first address to the last address, cannot use :: shorthand, connected with -. For example, if my block is fd18:3e15:61d0::/48, I need to fill in fd18:3e15:61d0:0000:0000:0000:0000:0000 - fd18:3e15:61d0:ffff:ffff:ffff:ffff:ffff. cidr: IPv6 block in CIDR format, do not replace / with _. netname: IPv6 block name, fill in as desired, no special requirements descr: IPv6 block description, can contain spaces country: Two-character country code from ISO 3166, for China, fill in CN. admin-c: Administrator contact information, points to a file in the person folder, use the filename from the Register Contact section above. tech-c: Technical contact information, points to a file in the person folder, use the filename from the Register Contact section above. mnt-by: Maintained by, points to the filename from the Create Maintainer section above. status: Status, keep it as ASSIGNED. source: Source, keep it as DN42. {collapse} {collapse-item label="Example"} data/inet6num/fd18:3e15:61d0::_48 @ dn42/registry@master inet6num: fd18:3e15:61d0:0000:0000:0000:0000:0000 - fd18:3e15:61d0:ffff:ffff:ffff:ffff:ffff cidr: fd18:3e15:61d0::/48 netname: IYOROYNET-DN42-V6 descr: iYoRoy DN42 Network IPv6 Block country: CN admin-c: IYOROY-DN42 tech-c: IYOROY-DN42 mnt-by: IYOROY-MNT status: ASSIGNED source: DN42 {/collapse-item} {/collapse} Next, under data/route6, create a file also named after the IPv6 block's CIDR format, using _ instead of /, and fill in the content: route6: <IPv6 Block in CIDR Format> origin: <ASN> max-length: <IPv6 Prefix Length> mnt-by: <Maintainer> source: DN42 route6: IPv6 block, fill in CIDR format, do not convert /. origin: ASN, needs to include AS prefix. max-lenth: Prefix Length, i.e., the number after / in the CIDR format. mnt-by: Maintained by, points to the filename from the Create Maintainer section above. source: Source, keep it as DN42. {collapse} {collapse-item label="Example"} data/route6/fd18:3e15:61d0::_48 @ dn42/registry@master route6: fd18:3e15:61d0::/48 origin: AS4242422024 max-length: 48 mnt-by: IYOROY-MNT source: DN42 {/collapse-item} {/collapse} Create and Push Commit, Submit PR It's best to sign your commits with GPG and upload your public key to dn42 git. After filling in the above information, return to the root directory of the repository and run the script ./fmt-my-stuff <Maintainer> to automatically format the configuration files. Note: Here <Maintainer> is the value of mntner from the Register Maintainer chapter. Then follow the normal git commit process: git add data/, git commit -S -m "<commit-msg>" to create the commit, git push origin master to push. Then, go to the dn42 git web interface, open your forked repository, and submit a Pull Request. If there are configuration issues, the automated review bot and administrators will inform you. Please patiently modify according to the requirements and merge into the original commit, do not create a new commit. It is recommended to use git commit --amend after git add. After modification, force push directly (git push origin master --force). The working language of the DN42 Registry is English. Please use English for the entire process to avoid unnecessary trouble. Configuring Bird First, install bird. Using Ubuntu 22.04 as an example: apt install bird2 -y If you also want to install WireGuard for later use, add the wireguard and wireguard-tools packages: apt install bird2 wireguard wireguard-tools -y Create Configuration Files bird.conf Modify /etc/bird/bird.conf： define OWNAS = <AS Number>; define OWNIP = <DN42 IPv4 Address>; define OWNIPv6 = <DN42 IPv6 Address>; define OWNNET = <DN42 IPv4 Block, CIDR Format>; define OWNNETv6 = <DN42 IPv6 Block, CIDR Format>; define OWNNETSET = [ <DN42 IPv4 Block, CIDR Format>+ ]; define OWNNETSETv6 = [ <DN42 IPv6 Block, CIDR Format>+ ]; router id OWNIP; protocol device { scan time 10; } function is_self_net() { return net ~ OWNNETSET; } function is_self_net_v6() { return net ~ OWNNETSETv6; } function is_valid_network() { return net ~ [ 172.20.0.0/14{21,29}, # dn42 172.20.0.0/24{28,32}, # dn42 Anycast 172.21.0.0/24{28,32}, # dn42 Anycast 172.22.0.0/24{28,32}, # dn42 Anycast 172.23.0.0/24{28,32}, # dn42 Anycast 172.31.0.0/16+, # ChaosVPN 10.100.0.0/14+, # ChaosVPN 10.127.0.0/16{16,32}, # neonetwork 10.0.0.0/8{15,24} # Freifunk.net ]; } function is_valid_network_v6() { return net ~ [ fd00::/8{44,64} # ULA address space as per RFC 4193 ]; } protocol kernel { scan time 20; ipv6 { import none; export filter { if source = RTS_STATIC then reject; krt_prefsrc = OWNIPv6; accept; }; }; }; protocol kernel { scan time 20; ipv4 { import none; export filter { if source = RTS_STATIC then reject; krt_prefsrc = OWNIP; accept; }; }; } protocol static { route OWNNET reject; ipv4 { import all; export none; }; } protocol static { route OWNNETv6 reject; ipv6 { import all; export none; }; } include "rpki.conf"; include "ebgp.conf"; Here, the OWNIP and OWNIPV6 parameters refer to the DN42 IPv4 and IPv6 addresses assigned to this machine, not the address of the block. For example, if I obtained the address segment 172.20.234.224/28 and assigned 172.20.234.225 to machine A, then OWNIP is 172.20.234.225. The same applies to IPv6. rpki.conf You might have seen other tutorials using cron jobs + reloading Bird to update ROAs. Configuring RPKI here is another way to configure ROAs. Create /etc/bird/rpki.conf and fill in the following content: roa4 table dn42_roa; roa6 table dn42_roa_v6; protocol rpki dn42_rpki_akix { roa4 { table dn42_roa; }; roa6 { table dn42_roa_v6; }; remote "<rpki server address>" port 8082; # change it refresh 30; retry 5; expire 600; } For the RPKI server, you can use the service provided by my friend's AkaereIX: rpki.akae.re. {collapse} {collapse-item label="Example"} roa4 table dn42_roa; roa6 table dn42_roa_v6; protocol rpki dn42_rpki_akix { roa4 { table dn42_roa; }; roa6 { table dn42_roa_v6; }; remote "rpki.akae.re" port 8082; refresh 30; retry 5; expire 600; } {/collapse-item} {/collapse} ebgp.conf Create /etc/bird/ebgp.conf and fill in the following content: template bgp dnpeers { local as OWNAS; path metric 1; ipv4 { import filter { if is_valid_network() && !is_self_net() then { if (roa_check(dn42_roa, net, bgp_path.last) != ROA_VALID) then { print "[dn42] ROA check failed for ", net, " ASN ", bgp_path.last; reject; } accept; } reject; }; export filter { if is_valid_network() && source ~ [RTS_STATIC, RTS_BGP] then accept; reject; }; import limit 1000 action block; }; ipv6 { import filter { if is_valid_network_v6() && !is_self_net_v6() then { if (roa_check(dn42_roa_v6, net, bgp_path.last) != ROA_VALID) then { print "[dn42] ROA check failed for ", net, " ASN ", bgp_path.last; reject; } accept; } reject; }; export filter { if is_valid_network_v6() && source ~ [RTS_STATIC, RTS_BGP] then accept; reject; }; import limit 1000 action block; }; } include "peers/*"; This configuration defines a template for peering within DN42 and references all configuration files under /etc/bird/peers. Later, when we peer, we only need to create configuration files here according to the template and write them. Create the /etc/bird/peers folder for subsequent peering. System Configuration Kernel Because each node may act as a router for other nodes, Linux kernel packet forwarding needs to be enabled. Also, because WireGuard is used, which doesn't forward through physical network cards, the rp_filter strict mode needs to be disabled. echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf echo "net.ipv6.conf.default.forwarding=1" >> /etc/sysctl.conf echo "net.ipv6.conf.all.forwarding=1" >> /etc/sysctl.conf echo "net.ipv4.conf.default.rp_filter=0" >> /etc/sysctl.conf echo "net.ipv4.conf.all.rp_filter=0" >> /etc/sysctl.conf sysctl -p DN42 Virtual Network Interface Because MP-BGP typically uses LLA addresses to establish BGP sessions, we need to set up a dummy network interface in the system and bind the DN42 IP we assigned to this machine. Run the following commands: ip link add dn42 type dummy ip addr add <DN42 IPv4 assigned to this machine> dev dn42 ip addr add <DN42 IPv6 assigned to this machine> dev dn42 If persistent configuration is needed, you can refer to the following three methods: {tabs} {tabs-pane label="systemd-networkd"} tee /etc/systemd/network/10-dn42.netdev > /dev/null <<EOF [NetDev] Name=dn42 Kind=dummy EOF tee /etc/systemd/network/20-dn42.network > /dev/null <<EOF [Match] Name=dn42 [Network] Address=<DN42 IPv4 assigned to this machine>/32 Address=<DN42 IPv6 assigned to this machine>/128 EOF systemctl enable systemd-networkd systemctl restart systemd-networkd {/tabs-pane} {tabs-pane label="netplan"} tee /etc/netplan/99-dn42.yaml > /dev/null <<EOF network: version: 2 renderer: networkd ethernets: dn42: match: name: dn42 addresses: - <DN42 IPv4 assigned to this machine>/32 - "<DN42 IPv6 assigned to this machine>/128" accept-ra: no EOF netplan apply {/tabs-pane} {tabs-pane label="/etc/network/interfaces"} tee -a /etc/network/interfaces > /dev/null <<EOF auto dn42 iface dn42 inet static address <DN42 IPv4 assigned to this machine> netmask 255.255.255.255 iface dn42 inet6 static address <DN42 IPv6 assigned to this machine>/128 EOF ifup dn42 {/tabs-pane} {/tabs} Peering with Other Members A user-friendly option for beginners is Potat0's self-service peering service. For details, refer to his DN42 Network page. Follow the instructions to find his Automatic Peering Bot, register your account with the bot by providing your DN42 ASN, the email recorded in the Maintainer during registration above (this is why it's best to fill in the email during registration, as many verification services use it), and receive the verification code with that email. Then, follow the bot's requirements to establish the peer. Install WireGuard apt install wireguard wireguard-tools -y Note: Modern package managers usually automatically install wireguard-tools when installing wireguard to use the wg-quick command. Generate Key Pair wg genkey | tee privatekey | wg pubkey > publickey Note down the contents of privatekey and publickey, representing the private and public keys respectively. Establishing a Peer Generally, there are two modes for establishing a peer: using MP-BGP (Multiprotocol BGP) and not using MP-BGP. Personally, I find MP-BGP more common because its configuration is relatively simple. The configurations are slightly different, so please refer according to your needs. Using MP-BGP Exchanging Information with the Peer You need to provide the other party with the following information, and you also need to know the other party's corresponding information: Public Key Public IP Address (non-DN42 address) Public WireGuard Port, usually listening on the last 5 digits of the other party's ASN DN42 ASN LLA (Link-Local Address), generally fe80::<last 4 digits of peer's ASN> Whether ENH (Extended Next Hop) is supported. Note: If using v6 to exchange routes without enabling ENH, v4 routes cannot be exchanged. Sometimes the public IP address and public WG port are combined and called the Endpoint. Configuring WireGuard Create a configuration file under /etc/wireguard. The name is arbitrary, but my naming convention is dn42-<last 4 digits of peer's ASN>.conf, where dn42-xxxx is the tunnel name. Fill the file with the following content: [Interface] PrivateKey = <Private key generated in the "Generate Key Pair" section above> ListenPort = <Public WireGuard Listen Port> Table = off Address = <Your LLA Address>/64 PostUp = sysctl -w net.ipv6.conf.%i.autoconf=0 [Peer] PublicKey = <Peer's Public Key> Endpoint = <Peer's Endpoint, i.e., IP:Port> AllowedIPs = 10.0.0.0/8, 172.20.0.0/14, 172.31.0.0/16, fd00::/8, fe00::/8 After editing and saving, run: wg-quick up <Tunnel Name> This will start the WG tunnel. Use wg show <Tunnel Name> to check the connection status. Running wg directly shows the status of all tunnels. If you want the tunnel to start automatically on boot, run: systemctl enable wg-quick@<Tunnel Name> Configuring the Bird2 Peer Section Create a new file <Name>.conf under /etc/bird/peers and fill in the following content: protocol bgp <BGP Session Name> from dnpeers{ neighbor <Peer's LLA Address> % '<WireGuard Tunnel Name>' as <Peer's ASN, without AS prefix>; }; If Extended Next Hop is used (as negotiated during information exchange above), configure: protocol bgp <BGP Session Name> from dnpeers{ neighbor <Peer's LLA Address> % '<WireGuard Tunnel Name>' as <Peer's ASN, without AS prefix>; ipv4{ extended next hop; }; }; Applying Configuration Run birdc configure (or birdc c, equivalent) to reload the Bird configuration, then run birdc show protocols (or birdc s p, equivalent) to view the currently established BGP connections. root@hkg2-202501092021514df2f0:~# birdc show protocols BIRD 2.0.12 ready. Name Proto Table State Since Info device1 Device --- up 07:24:14.255 static1 Static dn42_roa up 07:24:14.255 static2 Static dn42_roa_v6 up 07:24:14.255 kernel1 Kernel master6 up 07:24:14.255 kernel2 Kernel master4 up 07:24:14.255 static3 Static master4 up 07:24:14.255 static4 Static master6 up 07:24:14.255 dn42-0298 BGP --- up 08:03:13.347 Established dn42-1816 BGP --- up 07:53:05.028 Established Not Using MP-BGP Exchanging Information with the Peer You need to provide the other party with the following information, and you also need to know the other party's corresponding information: Public Key Public IP Address (non-DN42 address) Public WireGuard Port, usually listening on the last 5 digits of the other party's ASN DN42 ASN DN42 IP, if exchanging IPv4 routes, an IPv4 address is needed; if exchanging IPv6 routes, an IPv6 address is needed. Sometimes the public IP address and public WG port are combined and called the Endpoint. Configuring WireGuard Create a configuration file under /etc/wireguard. The name is arbitrary, but my naming convention is dn42-<last 4 digits of peer's ASN>.conf, where dn42-xxxx is the tunnel name. Fill the file with the following content: [Interface] PrivateKey = <Private key generated in the "Generate Key Pair" section above> ListenPort = <Public WireGuard Listen Port> Table = off PostUp = ip addr add <LLA>/64 dev %i PostUp = ip addr add <Local DN42 IPv6> dev %i PostUp = ip addr add <Local DN42 IPv4> peer <Peer's DN42 IPv4> dev %i PostUp = sysctl -w net.ipv6.conf.%i.autoconf=0 [Peer] PublicKey = <Peer's Public Key> Endpoint = <Peer's Endpoint, i.e., IP:Port> AllowedIPs = 10.0.0.0/8, 172.20.0.0/14, 172.31.0.0/16, fd00::/8, fe00::/8 After editing and saving, run: wg-quick up <Tunnel Name> This will start the WG tunnel. Use wg show <Tunnel Name> to check the connection status. Running wg directly shows the status of all tunnels. If you want the tunnel to start automatically on boot, run: systemctl enable wg-quick@<Tunnel Name> Configuring the Bird2 Peer Section Create a new file <Name>.conf under /etc/bird/peers and fill in the following content: protocol bgp <v4 BGP Session Name> from dnpeers{ neighbor <Peer's DN42 IPv4 Address> as <Peer's ASN, without AS prefix>; direct; ipv6{ import none; export none; }; }; protocol bgp <v6 BGP Session Name> from dnpeers{ neighbor <Peer's DN42 IPv6 Address> % '<WireGuard Tunnel Name>' as <Peer's ASN, without AS prefix>; direct; ipv4{ import none; export none; }; }; Applying Configuration Run birdc configure (or birdc c, equivalent) to reload the Bird configuration, then run birdc show protocols (or birdc s p, equivalent) to view the currently established BGP connections. At this point, we have successfully joined the DN42 network. To make the network more stable, we can peer with more users, establishing multiple BGP lines to prevent disconnection if some nodes fail. You can join the DN42 Unofficial Telegram Group for more information. If BGP is Established but you cannot Ping internal DN42 IPs, check if the IP set in bird.conf matches the IP assigned to the dummy interface or the local IP in WireGuard. Configuring DNS DN42 has its own public DNS with the anycast address 172.20.0.53, which can also resolve normal internet domains. To access internal domains ending with .dn42, you need to put this address at the top of resolv.conf: nameserver 172.20.0.53 nameserver 223.5.5.5 # Below are normal DNS server configurations After this, you can resolve DN42 internal domain names. Also, commands like ping, traceroute, and mtr will show rDNS resolution results when querying internal IPs. Welcome to peer with me! For details, please visit: iYoRoy DN42 Network References: https://blog.baoshuo.ren/post/dn42-network/ https://dn42.dev/howto/Bird2#example-configuration https://dn42.eu/howto/wireguard https://blog.udon.eu.org/archives/dbf21067.html https://blog.byteloid.one/2025/06/02/babeld-over-wireguard/ https://blog.wcysite.com/2021/%E8%B8%A9%E5%9D%91DN42-p2-peer/ https://blog.chs.pub/p/23-14-joindn42/ https://www.cnblogs.com/FengZeng666/p/15583434.html