Telling ZeroTier not to use Peering LANs for traffic

So, I use ZeroTier as my Layer 2 Mesh network between VMs and Physical hosts, and this morning I got a message telling me I was sending non allowed traffic over the DE-CIX VLAN… This was news to me, but after some digging, and help from Marek over at Vserver.Site, i managed to find out it was ZeroTier traffic causing the problem. ZeroTier in its nature will try and use all NICs on a machine to find an internet connection… but if you are on a peering VLAN, you do not want this to happen…

So, after more digging, I found the “Local Configuration File” in ZeroTier. This includes some features I did not know about, including how to place Subnets and Interfaces into the “Black List” so they wont be used by ZeroTier.

I wont reproduce the config file here, but i will give some tips, and explain what i have done. First, on Debian and Ubuntu, the file lives in /var/lib/zerotier-one/local.conf. If the file does not exist, you can create it. Its a standard JSON file. What i ended up adding was an IP range for each IX the box was connected to and marked it as black listed. Under settings, i also set the interfacePrefixBlacklist with a list of the interfaces connected to the IXes. an example is below:

This is the example for my machine connected to DE-CIX. ens19, 20 and 21 are connected to the peering VLANs on Munich, Dusseldorf and Hamburg. Given I have stopped sending bad IPv6 traffic, my guess then is the interfacePrefixBlacklist is only required, but I added the IPs also, just in case… I should probably include V6 here too…

When you make those changes, a restart of the ZeroTier service is required. And that is that. After Merak did some checks, and he was happy it was fixed, I was allowed back on the IX… Marek When you make those changes, a restart of the ZeroTier service is required. And that is that. After Merak did some checks, and he was happy it was fixed, I was allowed back on the IX…

Reverse DNS with Route53 and DNSControl

I was looking for a nice and somewhat simple way of managing my Reverse DNS settings, and I think i found it. I use DNSControl for managing all my DNS records for all (30+) domains, so i decided to see if I could get it to work for my Reverse (PTR) records too… and it works!

First, DNSControl is a project from the StackOverflow Guys and Gals, that allows easy enough managing of your DNS records. There is a bit of a learning curve, but once your over that, your laughing!

Second, Route53 is Amazon’s DNS service, as part of AWS. You can use it for AWS internal specific stuff, DNS resolution for your public domains, or, in my case, a reverse DNS for my IP block. I also use it for a private DNS for my ZeroTier network too, but that’s a separate matter…

So, to get them both hooked up, first you need DNSControl configured. They have extensive documentation on the matter, so check that out, as i wont be going into too much detail on that part.

After creating a record in Route53 on the AWS portal, my credentials file with details for Route53, creating a “noreg” record, etc, i then created the reverse domain for my IP range:

REV('') is a custom block that will automagically create the correct record, which in my case would be you could leave that in there, but if you have a few, it may be prone to “fat fingering” the records…

Next, you have all your PTR records. each one has the last part of the IP, along with the name of it. Once you have all your records, it is then a matter of pushing to Route53 using DNSControl:

./dnscontrol-Darwin preview --domains

will show you what changes have been made, assuming you have the config correct.

./dnscontrol-Darwin push --domains

will do the actual pushing.

It is then just a matter of making sure you have the correct records pointing at the correct DNS servers, as per the RIPE (or other provider’s) documentation. And that is that. A quick “dig” on the command line shows your work:

That should be it! happy days! and in theory, since the config file is “just” Javascript, you could pull it from a DB, a IPAM such as Netbox, or some other place. Thats my next challenge…

Building Tinc 1.1 (pre) on Ubuntu 18.04

So, a while back, I used Tinc for as a mesh VPN network. Tinc, for the uninitiated, is as follows:

tinc is a Virtual Private Network (VPN) daemon that uses tunnelling and encryption to create a secure private network between hosts on the Internet

Its been a while, but i noticed that they did have a version 1.1 preview, and did some tests with it previouly, and seen some decient perf with it. But, due to a mix of laziness, testing and more laziness, i ended up on ZeroTier for my internal peering network…

Now, however, i am starting to look at other options. This post will explain how to build Tinc 1.1 preview on Ubuntu 18.04. I will look into other options later on…

First, this will not be a configuration of Tinc… I am only building it.

You need to install some Apt Packages for the build to run:

sudo apt update

sudo apt install build-essential libncurses5-dev libreadline6-dev libzlcore-dev zlib1g-dev liblzo2-dev libssl-dev

next, download the source and extract it:


tar -zxvf tinc-1.1pre17.tar.gz

cd tinc-1.1pre17

next, configure and build.


make -j5

I use -j5 to use all 4 cores on my machine + 1 (its something i read years ago and i keep doing this). Just change that number as required.

Now your done. you can do a sudo make install to install in the required folders, or just run it from where its build.

Bird BGP Daemon tips and tricks

Some of these are more than likely known to people, but i though i would add them here. Partially for you and partially for me!

Useful Links

Command Line stuff

  • birdc show route for shows all routes known for that ip address
  • birdc show route protocol INTERNAL_LON1 shows all routes for a given Peer.
  • birdc show route where bgp_path ~ [= * 15133 * =] show routes where AS15133 is in the path list.
  • birdc show route where bgp_path.last = 15133 show routes where AS15133 is the LAST path.


Quick Overview of the network

AS204994 spans 5 countries (Ireland, UK, Germany, Amsterdam and USA), with 5 servers from 4 providers (3 from Vultr, 1 from and 1 from Packet.Net) and some on-prem machines (mix of VMs and Physical routers and machines). this may change over time, so check the homepage

Upstream providers are listed on the homepage also, but in house I use Virgin Media Business broadband and from there tunnel back to the upstream servers using ZeroTier. I also connect to EVIX over ZeroTier and to both LocIX and KleyReX with the VM in

Once the connection is made, I use Bird BGP to announce and receive routes from each of the providers. This is done as follows:

  • I have a /24 block of IPs announced though Vultr, Packet, VServer.Site and also though each of the IXs I’m connected to. Currently I am only using V4 to try and get this sorted first, then will add my /48 V6 space to the mix.
  • the /24 is split into a /28 for “internal peering” (ZeroTier network), a /28 for Any cast stuff (websites hosted across multiple sites, etc), a /27 for the house and multiple /28s, one for each of the VMs.
  • Any upstream VM that gets a full (or even partial) route feed announces all those routes (bar some filtered routes) back to Dublin. It also announces the /28 it has back to all other (internal) VMs. Only the /24 is announced to the upstream.
  • If a provider does not give any routes back, we leave it as is. It will still announce its internal /28 back to the rest of the network.
  • All servers also run a copy of NGinx which then hosts any of the anycast websites ( currently being the main one). That site is “hidden” behind CloudFlare for their security and bandwidth/caching features.
  • In house, all routes are managed by a single Ubuntu VM running on Hyper-V. It has a main internet connection with Virgin Media, and then connects internally to the rest of the network. Its set to allow internal routing, and the /27 for the house is connected to an internal VLAN.
  • I have a Ubiquiti EdgeRouter POE connected to that VLAN and it gets multiple of those IPs. Workstations, phones, etc, the connect to it and use it as a connection to the internet.

So far, this works about 90% of the time. This is not currently the production network: Netflix, Amazon Prime Video, etc, wont work, so all media devices go direct to the internet over a different VLAN (Luckily Virgin Media Business gave me a /29 to use). There is some work that will need to be done before it can go full production. Ideally, if I could announce my V4 and V6 though Virgin Media, that would make life easier, but we will see if they can sort it out.

So, Any questions? You can mail me at Tiernan [at] as204994 [dot] net.