Netlab L3 Switches: Brocade FCX -- Play with VLANs, BGP, OSPF, VRFs, VRRP, sFlow etc.

These switches are wonderful high-value playthings for the home-/space-lab with extensive
documentation and a CLI based interface (web is available but crappy). They
support real ASIC hardware backed L3 routing together with dynamic routing
protocols (BGP/OSPF) for both IPv6 and IPv4, ARP/NDP security with DHCPv4
snooping, IP High-Availability with VRRP and more.

For the most part they only do 1Gb/s Ethernet but they have a couple of
funky CX4 (SFF-8470 plugs same as Infiniband) interfaces that can be used
either for stacking at 16Gb/s (proprietary) or 10Gb/s (10GBase-CX4)
Ethernet via relatively obscure but cheaply available PCIe NICs
(we have the NICs but are still missing the CX4 cables ATM).

Docs

Access

Note: Right now these are used as infra in the temporarium, password in Vault

The switches are reachable over SSH password is in
vaultwarden. Unfortunately the SSH implementation is rather outdated so
some hax need to be applied with recent openssh clients:

$ ssh fcx1.asozial.it-syndikat.org -l root -o KexAlgorithms=+diffie-hellman-group1-sha1 -o PreferredAuthentications=password -o HostKeyAlgorithms=+ssh-rsa

Quick Command Reference

Logging in/out and saving config:

SSH@fcx1> en
# can do read-only stuff like ping at this auth level
SSH@fcx1# conf t
# enable read-write config stuff
SSH@fcx1(config)#

# do stuff ...

SSH@fcx1(config)# write mem
Write startup-config done.
Flash Memory Write (8192 bytes per dot) .
Copy Done.
SSH@fcx1(config)# exit
SSH@fcx1# exit
SSH@fcx1> exit
(SSH disconnects)

Show full config file:

SSH@fcx1> show running-config
SSH@fcx1> show configuration    # aka. startup-config in cisco land

Show interfaces:

SSH@fcx1>show inter br

Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/1/1      Down    None    None None  None  Yes 1    0   0024.38c1.1f00
1/1/2      Down    None    None None  None  Yes 1    0   0024.38c1.1f00
[...]

Show VLANs:

SSH@fcx1> show vlan
PORT-VLAN 1, Name LAN, Priority level0, Spanning tree Off
 DualMode Ports: (U1/M1)   1   2   3   4   5   6   7   8   9  10  11  12

PORT-VLAN 5, Name MODEM, Priority level0, Spanning tree Off
   Tagged Ports: (U1/M1)  26  45  47  48
 DualMode Ports: (U1/M1)  46
 
 [...]

# Just one VLAN
SSH@fcx1> show vlan 1

Add port to VLAN:

SSH@fcx1(config)# vlan 5
SSH@fcx1(config-vlan-1)# tagged e 1/1/1

# If port should be untagged continue with enabling dual-mode
SSH@fcx1(config)# inter e 1/1/1
SSH@fcx1(config)# dual-mode 1

WARNING: Brocade FW is finicky about switching to/from dual-mode on an
untagged port. Never configure untagged ports in vlan command. Always
conigure port as tagged first and then enable dual-mode.

If this warning is not heeded changing tagging config requires downtime.