Frequently asked questions

If your question is not here, email helpdesk@ev-ix.org and we will answer it, and probably add it to this page.

General

How do we contact EVIX?

Email helpdesk@ev-ix.org, which opens a ticket in our helpdesk. You can also join #evix-support on Discord for quicker, less formal help.

Please whitelist the address in your mail client. We have ended up on blacklists a few times, and the automated notices about your session going down come from the same place.

That address is also the right one for anything not covered here: an idea, a complaint, an offer of help, or a question we have not thought of. Note that our GitHub repository is a dormant snapshot, so issues opened there are not seen.

What does it cost?

Nothing, and there is no plan to charge. All of our infrastructure is donated, so our recurring costs are small, and the whole point is the lowest possible barrier to entry: several of us could not afford the fees other exchanges charge, including the ones aimed at hobbyists.

We also prefer members not to be customers. We are volunteers with day jobs and studies, and a fee would imply a service level we cannot promise. Donations are very welcome and carry no such obligation.

What are the rules?

The exchange is a fabric shared by everyone. Do not do anything that might overload it, misdirect traffic, or otherwise harm other members. Send traffic only to destinations advertised to you by BGP: no default routes pointed at the fabric, and no static routes pointed at other members. A member showing willful disregard for others will be disconnected.

Keep your connection up, allow ICMP echo, and maintain a session to at least one route server. Members who stop peering with the route servers are de-peered automatically. If you would rather not peer with them, tell us and we will exempt you from monitoring, but then we cannot send you automated notices when your connection breaks. Blocking ping is technically allowed and a bad idea: it is our main debugging tool and will cause you grief.

We also do not tolerate harassment of any kind, including in support tickets, in private messages to other members, and in network names. That includes derogatory statements about women and minorities. EVIX is open to everyone. We will warn you before acting, and we reserve the right to remove anyone who breaks these rules.

Not a rule, but a request: list yourself on PeeringDB. Visibility there brings more networks in, which benefits everyone, and we read your PeeringDB record to set your prefix limit.

What happens if my session goes down?

If one or more of your route-server sessions is down, we email you at most once a week. If all of your sessions come back up, the counter resets. Warnings are per-IP, so with two addresses you get separate emails for each.

After three weekly warnings we stop emailing. If all of your sessions are down and the longest-down one passes four weeks, we send a final email and remove the connection. So in the normal case you get three warnings and then a removal notice.

Note the edge case: if only some of your sessions were down, you stop getting warnings after three weeks but you are in a precarious position, because if the rest then go down you are removed immediately.

Removed and want to come back? Email us rather than reapplying, and we can give you your old addresses back instead of allocating new ones.

Something is not working

Work through these in order, they catch most cases:

  1. Check the example configuration for your platform.
  2. Check the looking glass, remembering that each route server has a separate IPv4 and IPv6 view. About a quarter of the sessions that are down show a reason; the rest show nothing, which usually means the session never established.
  3. Set your MTU to 1400. If you can ping the route servers but the BGP session will not establish, this is almost always the cause. See the MTU question.
  4. Check that your side is the one connecting. Our route servers are passive: they never open a session themselves and wait for yours. If your daemon is also passive, or only listens, neither side connects and it looks idle at both ends.
  5. Check both address families. IPv4 and IPv6 are separate sessions even over one tunnel, so IPv6 up and IPv4 down usually means only one of them was configured.
  6. Restart your BGP daemon. A surprising share of problems end here.
  7. Check whether you have hit your prefix limit. If you have, the session is latched down until it is re-enabled.

Still stuck? Email helpdesk@ev-ix.org. Include as much as you can: your configuration, the BGP daemon's log, the output of a ping and a traceroute to the route server, and what you have already tried. It genuinely shortens the exchange, and if you are on a platform we do not know well, we are learning alongside you.

I cannot ping the route servers at all

If ping fails outright, and ip neigh show dev <your EVIX interface> shows the route servers as FAILED or not at all, then your frames are not reaching the fabric. No amount of BGP configuration will help until that is fixed, because the two routers cannot even resolve each other's addresses.

In order of how often it turns out to be the cause:

  1. The addresses are on the wrong interface. By far the most common one. On an iFog VM in particular, EVIX needs its own dedicated interface: FogIXP is iFog's own exchange on a different VLAN, and configuring our addresses there fails silently. Ask iFog to add an EVIX interface to your VM and move the addresses onto it.
  2. The wrong prefix length. Configure the whole /24 and /64, not a /31 or a /127. Every member is on one flat segment, so your router needs the full prefix on-link to resolve anyone.
  3. A firewall in the way. Check that the interface is not caught by a default-deny policy, and on a bridged setup that bridged traffic is not being filtered.
  4. For GRETAP, protocol 47 is blocked or NATed. GRE has no port numbers and cannot be translated; use VXLAN if there is NAT in the path.

If none of that is it, email helpdesk@ev-ix.org with your interface configuration and the output of that ip neigh command. We can see from our side whether your MAC address is reaching the fabric at all, which settles quickly whose side the problem is on.

What is the process for getting connected?

Everything you need is on the get connected page: what to have ready before you apply, and the form itself. Pick your site from the locations list first. A volunteer replies with your peering LAN addresses and tunnel details, though it may take a while, especially during the school year.

If you have a VPS with one of our hosting partners you do not need a tunnel at all; see what technology connects peers.

Do you have a looking glass?

Yes, at lg.evix.org. It lists every session on both route servers, as four views: each route server has a separate one for IPv4 and for IPv6, which is also the quickest way to spot that one family is up and the other is not.

Sessions that are not established are grouped separately, and roughly a quarter of them carry a reason from the route server. When there is one it is worth reading: Hold timer expired, Automatic shutdown: Route limit exceeded (your prefix limit), Received: Administrative shutdown (your side closed it), Received: Bad peer AS (an ASN mismatch) and Socket: Connection reset by peer all say something specific. The rest show nothing, usually because the session never established at all, and then the troubleshooting checklist is the better starting point.

Where is EVIX?

Being virtual, it has no single location. Tunnel endpoints are in Fremont, Amsterdam, Zurich, Toronto and New Zealand. Frankfurt and Vancouver are local-only extensions with no tunnel endpoint. The locations page has the details for each.

Physical cross-connects are possible at Hurricane Electric's Fremont 2 and at Kolo Data Centers DC1 in Dronten, the Netherlands (formerly Serverius). A VLAN or layer-2 transit hand-off to our Amsterdam, Zurich or Frankfurt sites also works. If your servers are in Europe, ask us and we will look into a direct connection.

Our volunteers are spread across North America and Europe. Maybe we will do a meetup some day.

Can I complain, and have others?

Yes, please do: email helpdesk@ev-ix.org. If someone complains about us to you, send them our way as well. Criticism reaching us directly is far more useful than criticism we only hear about second-hand.

There is a recurring debate in the hobbyist networking community about whether virtual exchanges should exist at all, and we have had our share of it. We think there are good arguments for what we do, and we are genuinely happy to discuss them with anyone who disagrees. Our reasoning is on the about page.

How do I leave the exchange?

Just email helpdesk@ev-ix.org and say so. No notice period, no hard feelings, and it helps us keep the peers list and the monitoring honest. Tell us if it was something we could have done better, too.

If you simply stop, the automated process described above notices, emails you a few times and removes the connection after four weeks.

Either way your database record stays: removing a connection only marks it as no longer provisioned. So if you come back later, email us rather than filling in the form again and we can usually give you the same addresses back.

Technical

What technology connects peers?

There are two kinds of connection: local and remote.

Local peers have a VPS or a port in a facility we are present in and get a cross-connect onto the EVIX switch at no cost. That covers VPSs from Free Range Cloud (Fremont, Vancouver), iFog (Zurich, Frankfurt) and Xenyth Cloud (Toronto), as well as physical cross-connects and carrier hand-offs at Fremont and Amsterdam. Physical cross-connects are 1G copper today; we can support fibre at 1G and 10G, though 10G would gain you very little here.

Remote peers build a layer-2 tunnel to the nearest site. Supported types are VXLAN, GRETAP, EoIP, OpenVPN and ZeroTier. Plain GRE is not supported, because it only carries layer-3 packets. GRETAP needs a clean public IPv4 endpoint with no NAT in the path, since GRE has no port numbers and cannot be translated; use VXLAN if that is a problem.

Either way, you run BGP to the route servers and to any member you arrange a direct session with.

Which operating systems are supported?

Anything that can do one of our tunnel types and BGP will most likely work. What we have actually tested, and can help you with:

  • BIRD 2 on Debian and Ubuntu Server. This is what we run ourselves and what we can help with best.
  • MikroTik RouterOS over EoIP. The tunnel side is the same on versions 6 and 7, but the BGP example we have on file uses the version 6 /routing bgp peer syntax; on version 7 you will need the /routing bgp connection form.
  • VyOS, over OpenVPN. Note the caveat in the example configuration: the config we have on file is for the old 1.1.8 release, so newer VyOS will need adapting.
  • FRR and Quagga work, though our example config for them is thin.

BIRD 1 is end of life and we no longer run it; if you are still on it, the BIRD 2 example is the one to follow. If you get another platform working, please send us the configuration and we will add it.

What are the subnets and route servers?

The peering LAN is 206.81.104.0/24, acquired directly from ARIN, and 2602:fed2:fff:ffff::/64, donated by 10VPN Hosting.

Route servers
SiteASNIPv4IPv6
FremontAS137933206.81.104.12602:fed2:fff:ffff::1
AmsterdamAS209762206.81.104.2532602:fed2:fff:ffff::253

Both accept sessions from every location. Peering with at least one is required; peering with both is recommended, so a maintenance window on one does not take you off the exchange.

IPv4 is scarce here, so we allocate it only to networks that actually announce IPv4 space. An IPv6-only connection is the default and is perfectly normal; you can always ask for IPv4 later.

What is the MTU?

Configure 1400 on your EVIX interface. That is the number to use and the single most common cause of a broken session.

The reason: the fabric between our sites is itself built from tunnels, and the underlay paths between some sites only carry 1500 bytes. Encapsulation costs roughly 50 bytes, so the fabric runs at about 1450 internally. If you configure 1500, small packets pass and large ones are silently dropped. That has a characteristic symptom: ping to the route server works, but the BGP session never establishes, because the OPEN exchange is small and the first UPDATE is not.

1400 leaves comfortable headroom and is what every working peer uses. If ping works but BGP does not, check this before anything else.

Can I get the peer list automatically?

Two ways:

  • participants.json is the standard IXP-JSON export, understood by a number of tools already.
  • evix.json is our internal representation. More faithful to how we actually model peers, less useful to standard tools.

Both are regenerated regularly and need no authentication. The peers page is built from the same data.

What other services are there?

Two route servers (AS137933 and AS209762), a looking glass, per-site traffic graphs on the locations page, and the IXP-JSON export. There is no route collector, no NTP and no DNS service from us.

If you run something useful on the fabric, such as NTP, DNS or a mirror, tell us and we will list it here.

Filtering and policy

Do you validate RPKI?

Yes, and we enforce it. Both route servers validate route origins against RPKI data from a local validator, and a route that comes back invalid is rejected, not merely tagged. Valid and unknown routes are accepted and tagged so you can tell them apart.

This means a route with an ROA that does not match its origin will not be distributed by our route servers, no matter who sends it. If a prefix of yours is not showing up, check your ROAs first.

What else do the route servers filter?

The route servers are configured with ARouteServer, and reject a route if any of the following applies:

  • The origin ASN is not in your registered AS-SET, or the prefix is not covered by your IRR objects.
  • The route is RPKI-invalid.
  • The prefix is a bogon, is in our global blacklist, or is outside IPv6 global unicast space.
  • The prefix is shorter or longer than we accept: IPv4 must be between /8 and /24, IPv6 between /12 and /48.
  • The next hop is not yours.
  • The AS_PATH is longer than 32 hops, begins with the wrong ASN, or contains a reserved or transit-free ASN.

Every rejection is tagged with a 64599:<cause> community explaining why, which the looking glass shows.

Is there a prefix limit on my session?

Yes. Each session has a maximum prefix count taken from your PeeringDB record (info_prefixes4 and info_prefixes6). Without a PeeringDB record you get the general default of 75 prefixes per address family, which is deliberately low, so listing yourself and keeping those two fields realistic is very much in your interest.

On the looking glass this shows up as Automatic shutdown: Route limit exceeded. That wording matters: a reason starting with Automatic shutdown came from our side, while one starting with Received: came from yours.

When you exceed the limit the route server disables the session rather than accepting the excess. It does not flap: it stays down until it is re-enabled, which our automation does periodically. If you keep hitting the limit, fix your PeeringDB numbers or your export filter, and email us if you need the limit raised.

Which BGP communities do you support?

Quite a lot. Everything below works as a standard community, as a large community with the route server's own ASN as the first element, and as an extended community. In the tables, rs_as means the ASN of the route server you are talking to: 137933 in Fremont, 209762 in Amsterdam. RFC 1997 well-known communities are honoured too.

Informational: where a route entered the fabric

Set by us on every route, so you can see which site a prefix came in at.

StandardLargeSite
65517:1rs_as:65517:1Fremont
65517:2rs_as:65517:2Amsterdam
65517:3rs_as:65517:3New Zealand
65517:4rs_as:65517:4Vancouver
65517:5rs_as:65517:5Zurich
65517:6rs_as:65517:6Frankfurt
65517:7rs_as:65517:7Toronto

Informational: how a route was validated

StandardMeaning
64512:31Prefix validated by an RPKI ROA
64512:11Prefix present in the client's AS-SET
64512:10Prefix not present in the client's AS-SET
64512:21Origin ASN present in the client's AS-SET
64512:20Origin ASN not present in the client's AS-SET
64512:41Route validated via a whitelist entry
64599:<n>Reject cause, on routes we dropped

Control: you set these, we act on them

Use <peer> for the ASN of the member you want the action to apply to, or the "to any" form to apply it to everyone.

EffectStandardLarge
Do not announce to anyone0:65534rs_as:0:0
Do not announce to one peer0:<peer>rs_as:0:<peer>
Announce only to one peer65534:<peer>rs_as:1:<peer>
Prepend once to everyone65501:65534rs_as:101:0
Prepend twice to everyone65502:65534rs_as:102:0
Prepend three times to everyone65503:65534rs_as:103:0
Prepend once to one peer65511:<peer>rs_as:101:<peer>
Prepend twice to one peer65512:<peer>rs_as:102:<peer>
Prepend three times to one peer65513:<peer>rs_as:103:<peer>
Add NO_EXPORT towards one peer65281:<peer>rs_as:65281:<peer>
Add NO_ADVERTISE towards one peer65282:<peer>rs_as:65282:<peer>

Standard communities can only carry a 16-bit peer ASN. If the member you are targeting has a 32-bit ASN, you must use the large-community form.

What is the AS-SET for, and what if I do not have one?

We build your import filter from it. An AS-SET is an object you register in your RIR's routing registry listing the ASNs whose prefixes you may announce, which is normally your own plus any downstreams. Give us the object name when you apply, in the form AS-EXAMPLE or, if it is ambiguous, qualified as RIPE::AS-EXAMPLE.

Without one, we can only accept prefixes that your own ASN is registered as the origin of. That is perfectly workable if you announce nothing but your own space, which is true of most members here, so there is no need to invent one.

You do need one as soon as you have a downstream, otherwise their prefixes will be rejected with origin ASN not in allowed as-sets. Register the object with your RIR and email us the name; we regenerate the filters from it.

Other

Do you have a PeeringDB page?

Yes: peeringdb.com/ix/2274. Please list yourself there as a participant once you are connected.

Do you have social media or a mailing list?

Day-to-day discussion happens on Discord, which is also where our volunteers coordinate. There is a LinkedIn page that we rarely use.

There is no mailing list. Operational notices that affect your session are sent directly by email to your registered contact address, so keep that address current and make sure helpdesk@ev-ix.org is not filtered.

I want to volunteer

Please do. All coordination happens on our Discord, so start there and say hello in the EVIX channel, or email helpdesk@ev-ix.org.

Help is welcome on operations, automation, documentation and answering tickets. Our public GitHub snapshot gives a reasonable picture of how the system fits together, though it stopped updating in April 2022 and is well behind what we run now. Ask us for the current picture and we will walk you through it.

I want to sell you something

We are volunteers paying for most things out of pocket and have no budget, so we almost certainly cannot buy your product. Donations are a different matter and very welcome. Also, an internet exchange does not need transit for its own ASNs, so please do not offer us that.

Does this website have a dark mode?

Yes. It follows your operating system's preference by default, and the sun-and-moon button in the header overrides it if you want the other one. Your choice is remembered in your browser and never leaves it.

The palette: dark blue #091540, periwinkle #7692ff, pale blue #abd2fa, and an accent of #e60073. Links use a darker blue in light mode and a lighter one in dark mode so they actually meet contrast requirements. The fonts are Fira Sans and Fira Code.

I am an employer. What skills do your volunteers have?

Our volunteers pick up teamwork, user support, dev-ops, software development, network operations and leadership experience, on a system with real users and real consequences. Email us to hear about a specific person or about what we do.