RFC 3378 EtherIP with FreeBSD

I have written this guide to share with the World how to use RFC 3378 to bridge two networks at the Ethernet level across a routed IP network – especially as it appears this will be the only guide on the Internet covering how to do this!

Background

I work for the national health service department of a local government with more than 35 sites covering health and social care services.

Most of these sites are on-island where direct communications (provided by the local telecoms company) exist as WAN links between our main site and these satellite sites. One site is on another island and our connection to it runs through the central government WAN infrastructure and therefore we have no direct control over the intermediate links. Simply put: we route IP out of our permiter router and it arrives at our remote site’s router into their network. What happens in between is a series of hops and transitions from fibre-optic, copper and radio links.

Seperate to this, a wireless deployment had already been completed throughout the main hospital site and utilised VLANs to separate the staff (private) and patient (public) SSIDs at the access points into seperate firewalls.

Requirement

The hospital in the neighbouring island needed to be updated to include the same wireless solution as implemented locally. The provisioning of cabling, power injectors and wireless access points would not be a problem. A cabler had deployed new Ethernet around the building. The budget for access points and PoE adaptors approved and items purchased.

Problem

The problem was getting the private and patient wireless channels back to the central site. We could create new subnets but these would converge at the router and traverse the various hops on the way back to our central network. However this would mean the “dirty” data of the patient wireless would be traversing alongside the “clean” data of our network and posed a massive risk. We had already eliminated these risks by implementing VLAN seperation on the central network, so if only there was a way to pass VLANs across an IP routed network. Unfortunately routers block so called “Layer 2” traffic, and so VLANs cannot pass through to another network.

Additionally we needed to avoid requiring the assistance from central government network services since they had much tighter change control and greater level of beauroucracies… but less said about that!

Solution

An Internet standard had emerged recently, released under the code RFC 3378, and better known as EtherIP that provides the perfect solution to our requirements – that of bridging two Ethernet networks together spanning an IP router link.

Thankfully EtherIP is available in most open source operating systems, including FreeBSD of which I have plenty of knowledge in administrating.

Implementation

You will need two computers, one for each side of the spanned link and both will need two network cards. One of the network cards connects to the local network as a normal computer would, with an IP address to suit the subnet it is in. The other network card is connected to a trunk port of a switch (in my case, a Cisco) and is what captures the network traffic at the Ethernet level.

I did a basic installation of FreeBSD; there isn’t even any need for the ports to be downloaded.

Computer 1 (on the local site) had an Intel EtherExpress card fxp0 with IP address 10.1.1.20 and em0 being used for the Ethernet port.

Computer 2 (on the remtoe site) had a Broadcom card bge0 with IP address 10.15.1.20 and em0 being used for the Ethernet port.

FreeBSD has a virtual network interface called “gif” that creates an Etherhet tunnel between two end-points. A gif0 network interface is created on both computers and bridges them together across a routed IP link. Another virtual network device is created called “bridge” that can span network traffic between similiar network interfaces, typically Ethernet ones. The bridge0 network device is configured to combine the gif0 and em0 network interfaces together. This allows Ethernet data passing into em0 to get passed onto the gif0 and (after passing through the routed IP link) onto the remote computer’s gif0 and further onto the em0 network interface of that computer and finally into the network there.

This might sound more complicated than it really is! A textual diagram would look like this:

Network –(Trunk Port)– em0 — gif0 — fxp0 — router – routed IP link – router – bge0 – gif0 – em0 –(Trunk Port)– Network

Configuration

On the Cisco switch configure an interface to trunk mode:

interface FastEthernet0/24
description Link: EtherIP
switchport trunk pruning vlan none
switchport mode trunk
end

This is the port the em0 (on my computer systems) will connect to. The same was done on the remote switch as well for the respective em0 of the other computer.

If you are using a non-Cisco device you will need to enable trunking using the vendor specific methods.

On computer 1 configure FreeBSD as follows within the /etc/rc.conf file:

# — sysinstall generated deltas — # Mon Jan 31 12:09:47 2011
# Created: Mon Jan 31 12:09:47 2011
# Enable network daemons for user convenience.
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/rc.conf.
defaultrouter=”10.1.1.1″
hostname=”etherip01.mydomain.com”
ifconfig_fxp0=”inet 10.1.1.20 netmask 255.255.0.0″

# EtherIP Configuration
cloned_interfaces=”gif0 bridge0″
ifconfig_gif0=”mtu 1500 tunnel 10.1.1.20 10.15.1.20 up”
ifconfig_bridge0=”addm em0 addm gif0 up”
ifconfig_em0=”up”
################

keymap=”uk.iso”
sshd_enable=”YES”

On computer 2:

# — sysinstall generated deltas — # Thu Feb 3 11:20:45 2011
# Created: Thu Feb 3 11:20:45 2011
# Enable network daemons for user convenience.
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/rc.conf.
defaultrouter=”10.15.1.1″
hostname=”etherip02.mydomain.com”
ifconfig_bge0=”inet 10.15.1.20 netmask 255.255.0.0″

# EtherIP Configuration
cloned_interfaces=”gif0 bridge0″
ifconfig_gif0=”mtu 1500 tunnel 10.15.1.20 10.1.1.20 up”
ifconfig_bridge0=”addm em0 addm gif0 up”
ifconfig_em0=”up”
################

keymap=”uk.iso”
sshd_enable=”YES”

Note that the gif0 tunnel IP addresses are reversed between computer 1 and computer 2. These IP addresses are those of the local network interface. The FreeBSD computers in my case were not acting as perimeter routers but they could just as easily be with a third network card. I did not need this added complexity as IP routing was already available within the health and central government networks.

Results

The switch at the remote site was configured to be a VTP Client and emediately joined the central network VLAN domain (a Cisco notion, not Microsoft related) and issuing the “show vlan”” command revealed all the central network VLANs had propagated across. In addition issuing the “show cdp neighbors” command revealed that both central and remote switches could see each other as though they were connected directly to each other.

Hopefully this guide can be of use to all those that have drained Google of searches for this information (I know, I tried too!). Necessity really is the Mother of just figuring it out for yourself… and now, you don’t need to!

I can be contacted in various ways detailed on my Contact page.