SMART SFP - Mini Linux System on a Stick (Literally)
Today I’m taking a look at the Plumspace Smart SFP - a dual core ARM Cortex A53 stuffed inside a 1310nm SFP optical transceiver, which can deal with packet flows at line rate*. Quite a neat little thing, with plenty of use cases.
Plumspace Smart SFPs: Link
Contents⌗
- Video
- Bridging
- Bridging - Ntopng
- Routing
- Routing - NAT44
- Routing - NAT46 (CLAT)
- Wireguard
- OpenSSL Bench
Video⌗
Bridging⌗
Kernel defaults for br0, configured using /etc/network/interfaces:
auto gbe0
allow-hotplug gbe0
iface gbe0 inet manual
auto gbe1
allow-hotplug gbe1
iface gbe1 inet manual
auto br0
allow-hotplug br0
iface br0 inet6 auto
bridge_ports gbe0 gbe1
I also installed avahi-daemon which I’m using to discover it (smart-sfp.local). You can use a static IP if you’d rather.
Iperf3⌗
Connecting to host 172.27.15.180, port 5201
[ 5] local 172.27.15.181 port 44144 connected to 172.27.15.180 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 85.8 MBytes 719 Mbits/sec 443 2.56 MBytes
[ 5] 1.00-2.00 sec 85.5 MBytes 717 Mbits/sec 252 693 KBytes
[ 5] 2.00-3.00 sec 111 MBytes 934 Mbits/sec 27 785 KBytes
[ 5] 3.00-4.00 sec 112 MBytes 935 Mbits/sec 152 870 KBytes
[ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 100 684 KBytes
[ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 802 KBytes
[ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 27 899 KBytes
[ 5] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 28 713 KBytes
[ 5] 8.00-9.00 sec 112 MBytes 936 Mbits/sec 6 820 KBytes
[ 5] 9.00-10.00 sec 113 MBytes 945 Mbits/sec 0 922 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.04 GBytes 895 Mbits/sec 1035 sender
[ 5] 0.00-10.01 sec 1.04 GBytes 892 Mbits/sec
Connections Per Second (Cyperf)⌗
----------------------------------------------------------
Test Result Summary
----------------------------------------------------------
Test duration 41 seconds
----------------------------------------------------------
Connections
----------------------------------------------------------
Total succeeded 283790
Total failed 0
Average rate 6921.000000 connections per second
Average latency 564 µs
Bridging Ntopng⌗
Disabled segment offloading, running ntopng installed via apt packages
Throughput Test (Cyperf)⌗
-------------------------------------------------------
Test Result Summary
-------------------------------------------------------
Test duration 27 seconds
-------------------------------------------------------
Throughput
-------------------------------------------------------
Average TX & RX 546.294224 Mb/s
Average TX 543.728808 Mb/s
Average RX 2.565416 Mb/s
-------------------------------------------------------
TCP Data
-------------------------------------------------------
Total transferred 1.713713 GB
Total sent 1.713713 GB
Total received 0.000000 GB
Connections Per Second (Cyperf)⌗
----------------------------------------------------------
Test Result Summary
----------------------------------------------------------
Test duration 32 seconds
----------------------------------------------------------
Connections
----------------------------------------------------------
Total succeeded 193386
Total failed 0
Average rate 6043.000000 connections per second
Average latency 650 µs
Routing⌗
Configured using /etc/network/interfaces (and using IPv4 for cyperf):
auto gbe0
allow-hotplug gbe0
iface gbe0 inet static
address 192.0.0.1/29
auto gbe1
allow-hotplug gbe1
iface gbe1 inet dhcp
iface gbe1 inet6 auto
This will add a default route via dhcp. Added return route on test server: ip route add 192.0.0.0/29 via 172.27.15.182
IPerf⌗
Connecting to host 172.27.15.180, port 5201
[ 5] local 192.0.0.5 port 59314 connected to 172.27.15.180 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 106 MBytes 886 Mbits/sec 15 393 KBytes
[ 5] 1.00-2.00 sec 112 MBytes 944 Mbits/sec 11 561 KBytes
[ 5] 2.00-3.00 sec 112 MBytes 941 Mbits/sec 7 686 KBytes
[ 5] 3.00-4.00 sec 112 MBytes 940 Mbits/sec 50 796 KBytes
[ 5] 4.00-5.00 sec 113 MBytes 946 Mbits/sec 6 884 KBytes
[ 5] 5.00-6.00 sec 113 MBytes 945 Mbits/sec 9 973 KBytes
[ 5] 6.00-7.00 sec 87.9 MBytes 737 Mbits/sec 240 734 KBytes
[ 5] 7.00-8.00 sec 112 MBytes 935 Mbits/sec 0 843 KBytes
[ 5] 8.00-9.00 sec 112 MBytes 942 Mbits/sec 4 707 KBytes
[ 5] 9.00-10.00 sec 112 MBytes 935 Mbits/sec 0 819 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.07 GBytes 915 Mbits/sec 342 sender
[ 5] 0.00-10.02 sec 1.07 GBytes 913 Mbits/sec receiver
Throughput (Cyperf)⌗
-------------------------------------------------------
Test Result Summary
-------------------------------------------------------
Test duration 30 seconds
-------------------------------------------------------
Throughput
-------------------------------------------------------
Average TX & RX 905.054472 Mb/s
Average TX 891.920008 Mb/s
Average RX 13.134464 Mb/s
-------------------------------------------------------
TCP Data
-------------------------------------------------------
Total transferred 2.900737 GB
Total sent 2.900737 GB
Total received 0.000000 GB
CPS (Cyperf)⌗
----------------------------------------------------------
Test Result Summary
----------------------------------------------------------
Test duration 40 seconds
----------------------------------------------------------
Connections
----------------------------------------------------------
Total succeeded 271844
Total failed 0
Average rate 6796.000000 connections per second
Average latency 579 µs
Routing NAT44⌗
Configured using /etc/network/interfaces (again using IPv4 for cyperf):
auto gbe0
allow-hotplug gbe0
iface gbe0 inet static
address 192.0.0.1/29
auto gbe1
allow-hotplug gbe1
iface gbe1 inet dhcp
iface gbe1 inet6 auto
and using iptables (not the nf_tables version due to kernel limitations): iptables-legacy -t nat -A POSTROUTING -o gbe1 -j MASQUERADE
IPerf⌗
Connecting to host 172.27.15.180, port 5201
[ 5] local 192.0.0.5 port 58318 connected to 172.27.15.180 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 103 MBytes 866 Mbits/sec 59 400 KBytes
[ 5] 1.00-2.00 sec 112 MBytes 943 Mbits/sec 6 554 KBytes
[ 5] 2.00-3.00 sec 112 MBytes 942 Mbits/sec 94 680 KBytes
[ 5] 3.00-4.00 sec 112 MBytes 936 Mbits/sec 27 796 KBytes
[ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 3 894 KBytes
[ 5] 5.00-6.00 sec 112 MBytes 938 Mbits/sec 41 741 KBytes
[ 5] 6.00-7.00 sec 112 MBytes 938 Mbits/sec 0 851 KBytes
[ 5] 7.00-8.00 sec 112 MBytes 943 Mbits/sec 2 943 KBytes
[ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 49 737 KBytes
[ 5] 9.00-10.00 sec 112 MBytes 935 Mbits/sec 1 843 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.09 GBytes 932 Mbits/sec 282 sender
[ 5] 0.00-10.01 sec 1.08 GBytes 930 Mbits/sec receiver
Throughput (Cyperf)⌗
-------------------------------------------------------
Test Result Summary
-------------------------------------------------------
Test duration 51 seconds
-------------------------------------------------------
Throughput
-------------------------------------------------------
Average TX & RX 860.770592 Mb/s
Average TX 851.627024 Mb/s
Average RX 9.143568 Mb/s
-------------------------------------------------------
TCP Data
-------------------------------------------------------
Total transferred 4.853985 GB
Total sent 4.853985 GB
Total received 0.000000 GB
CPS (Cyperf)⌗
---------------------------------------------------------
Test Result Summary
---------------------------------------------------------
Test duration 33 seconds
---------------------------------------------------------
Connections
---------------------------------------------------------
Total succeeded 8187
Total failed 48
Average rate 248.000000 connections per second
Average latency 671 µs
Routing NAT46⌗
Configured using /etc/network/interfaces:
auto gbe0
allow-hotplug gbe0
iface gbe0 inet static
address 192.0.0.1/29
auto gbe1
allow-hotplug gbe1
iface gbe1 inet6 auto
IPv4-island inside, IPv6-only outside ‘inside’ is 192.0.0.0/29 (DSLite range) mapped to 2001:db8::6460/125, leading to this Tayga config:
tun-device nat64
ipv4-addr 192.0.0.0
ipv6-addr 2601:40e:8100:ac30::6468
prefix 64:ff9b::/96
wkpf-strict no
log drop reject icmp self dyn
#Map IPv4-island to a seqential range of v6s on LAN
map 192.0.0.0/29 2601:40e:8100:ac30::6460/125
#test system on IPv6, mapped to a random IPv4 so we can iperf to it
map 10.0.0.1/32 2001:db8::be24:11ff:feeb:716e/128
And the commands I ran to set up routing and such (didn’t bother trying to make this persistent):
#Enable IP forwarding of IPv6 and IPv4
echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
echo 1 > /proc/sys/net/ipv4/conf/all/forwarding
#Enable Accept RA (=2 due to forwarding) on gbe1 for SLAAC address assignment
echo 2 > /proc/sys/net/ipv6/conf/gbe1/accept_ra
#Proxy NDP the CLAT addresses
echo 1 > /proc/sys/net/ipv6/conf/gbe1/proxy_ndp
#Proxy ND addresses from /29
ip neigh add proxy 2601:40e:8100:ac30::6460 dev gbe1
ip neigh add proxy 2601:40e:8100:ac30::6461 dev gbe1
ip neigh add proxy 2601:40e:8100:ac30::6462 dev gbe1
ip neigh add proxy 2601:40e:8100:ac30::6463 dev gbe1
ip neigh add proxy 2601:40e:8100:ac30::6464 dev gbe1
ip neigh add proxy 2601:40e:8100:ac30::6465 dev gbe1
ip neigh add proxy 2601:40e:8100:ac30::6466 dev gbe1
ip neigh add proxy 2601:40e:8100:ac30::6467 dev gbe1
#plus one for Tayga itself
ip neigh add proxy 2601:40e:8100:ac30::6468 dev gbe1
#Start tayga (using default.conf and included systemd unit)
systemctl enable --now tayga@default
#Bring up iface and route to it
#no need for IPs on this iface
ip link set dev nat64 up
ip route add default dev nat64 mtu 1480
ip route add 2601:40e:8100:ac30::6464/124 dev nat64
IPerf⌗
Connecting to host 10.0.0.1, port 5201
[ 5] local 192.0.0.5 port 43146 connected to 10.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 39.9 MBytes 334 Mbits/sec 5 771 KBytes
[ 5] 1.00-2.00 sec 43.1 MBytes 362 Mbits/sec 0 866 KBytes
[ 5] 2.00-3.00 sec 43.4 MBytes 364 Mbits/sec 0 937 KBytes
[ 5] 3.00-4.00 sec 43.1 MBytes 362 Mbits/sec 0 987 KBytes
[ 5] 4.00-5.00 sec 42.9 MBytes 360 Mbits/sec 0 1022 KBytes
[ 5] 5.00-6.00 sec 41.9 MBytes 351 Mbits/sec 8 766 KBytes
[ 5] 6.00-7.00 sec 43.5 MBytes 365 Mbits/sec 0 813 KBytes
[ 5] 7.00-8.00 sec 43.2 MBytes 363 Mbits/sec 0 844 KBytes
[ 5] 8.00-9.00 sec 42.0 MBytes 352 Mbits/sec 0 867 KBytes
[ 5] 9.00-10.00 sec 42.1 MBytes 353 Mbits/sec 0 905 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 425 MBytes 357 Mbits/sec 13 sender
[ 5] 0.00-10.01 sec 424 MBytes 355 Mbits/sec receiver
Wireguard VPN⌗
I did this using network namespaces instead of multiple table. You can choose to either have the WAN or LAN on the ‘default’ namespace, where normal processes on Linux will bind to. I chose WAN in this case, since I am relying on ssh + avahi mdns on my normal network.
Wireguard will always send/receive packets from the network namespace it is started in, even if you move the adapter to a new netns! So start the adapter from the WAN-side (or in my case, use wireguard-go launched from the default netns), THEN move the adapter into the netns and bring it up.
So, steps:
# Create netns named 'user', add inteface gbe0 (downstream 'LAN') to netns
ip netns add user
ip link set dev gbe0 netns user
# Placed Mullvad's conf generator in /etc/wg/wg0.conf
# Removed Address and DNS lines, which are for wg-quick
# will need those values separately
#
# Launch wireguard-go
wireguard-go wg0
# Configure it using wg
wg setconf wg0 /etc/wg/wg0.conf
# Move wg tun adapter to netns
ip link set dev wg0 netns user
Config within the netns (ip netns exec user bash)
# enable forwarding within the netns (it has its own settings!)
echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
echo 1 > /proc/sys/net/ipv4/conf/all/forwarding
# bring up wg
ip link set up wg0
# These came from Mullvad's conf generator
ip addr add 10.67.156.35/32 dev wg0
ip addr add fc00:bbbb:bbbb:bb01::4:9c22/128 dev wg
# add default routes
ip route add default dev wg0
ip -6 route add default dev wg0
# add masquerade rules
iptables-legacy -t nat -A POSTROUTING -o wg0 -j MASQUERADE
ip6tables-legacy -t nat -A POSTROUTING -o wg0 -j MASQUERADE
# add IPs to gbe0 in our netns (it loses config when moved)
ip addr add 192.0.0.1/29 dev gbe0
ip addr add 2001:db8:6464::1/64 dev gbe0
Config for dnsmasq for this setup
#static dns resolver conf
no-resolv
#server which came from Mullvad
server=10.64.0.1
#iface to listen on
interface=gbe0
#using the dslite range since I already had configured this for 464xlat
dhcp-range=192.0.0.2,192.0.0.6,1h
#using doc prefix for nat66
dhcp-range=2001:db8:6969::, ra-stateless
I launched dnsmasq within the netns so it will serve DHCP on interface gbe0.
OpenSSL bench⌗
OpenSSL bench on this hw:
#Speed test AES-128-GCM
root@smart-sfp:~# openssl speed -evp aes-128-gcm
Doing AES-128-GCM for 3s on 16 size blocks: 753963 AES-128-GCM's in 2.99s
Doing AES-128-GCM for 3s on 64 size blocks: 733150 AES-128-GCM's in 3.00s
Doing AES-128-GCM for 3s on 256 size blocks: 660259 AES-128-GCM's in 2.99s
Doing AES-128-GCM for 3s on 1024 size blocks: 469975 AES-128-GCM's in 2.99s
Doing AES-128-GCM for 3s on 8192 size blocks: 141423 AES-128-GCM's in 2.99s
Doing AES-128-GCM for 3s on 16384 size blocks: 78532 AES-128-GCM's in 2.99s
version: 3.0.17
built on: Tue Aug 5 07:09:41 2025 UTC
options: bn(64,64)
compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -fzero-call-used-regs=used-gpr -DOPENSSL_TLS_SECURITY_LEVEL=2 -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/reproducible-path/openssl-3.0.17=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
CPUINFO: OPENSSL_armcap=0xbd
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
AES-128-GCM 4034.58k 15640.53k 56530.54k 160954.65k 387470.64k 430323.84k
#Speed test AES-256-GCM
root@smart-sfp:~# openssl speed -evp aes-256-gcm
Doing AES-256-GCM for 3s on 16 size blocks: 735862 AES-256-GCM's in 2.99s
Doing AES-256-GCM for 3s on 64 size blocks: 712262 AES-256-GCM's in 2.99s
Doing AES-256-GCM for 3s on 256 size blocks: 632462 AES-256-GCM's in 2.99s
Doing AES-256-GCM for 3s on 1024 size blocks: 417102 AES-256-GCM's in 3.00s
Doing AES-256-GCM for 3s on 8192 size blocks: 125089 AES-256-GCM's in 2.99s
Doing AES-256-GCM for 3s on 16384 size blocks: 69521 AES-256-GCM's in 2.99s
version: 3.0.17
built on: Tue Aug 5 07:09:41 2025 UTC
options: bn(64,64)
compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -fzero-call-used-regs=used-gpr -DOPENSSL_TLS_SECURITY_LEVEL=2 -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/reproducible-path/openssl-3.0.17=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
CPUINFO: OPENSSL_armcap=0xbd
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
AES-256-GCM 3937.72k 15245.74k 54150.59k 142370.82k 342718.76k 380947.18k
#Speed test ChaCha20-Poly1305
root@smart-sfp:~# openssl speed -evp chacha20-poly1305
Doing ChaCha20-Poly1305 for 3s on 16 size blocks: 5122281 ChaCha20-Poly1305's in 2.99s
Doing ChaCha20-Poly1305 for 3s on 64 size blocks: 2859045 ChaCha20-Poly1305's in 2.99s
Doing ChaCha20-Poly1305 for 3s on 256 size blocks: 1448219 ChaCha20-Poly1305's in 2.99s
Doing ChaCha20-Poly1305 for 3s on 1024 size blocks: 457043 ChaCha20-Poly1305's in 3.00s
Doing ChaCha20-Poly1305 for 3s on 8192 size blocks: 60887 ChaCha20-Poly1305's in 2.99s
Doing ChaCha20-Poly1305 for 3s on 16384 size blocks: 30481 ChaCha20-Poly1305's in 2.99s
version: 3.0.17
built on: Tue Aug 5 07:09:41 2025 UTC
options: bn(64,64)
compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -fzero-call-used-regs=used-gpr -DOPENSSL_TLS_SECURITY_LEVEL=2 -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/reproducible-path/openssl-3.0.17=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
CPUINFO: OPENSSL_armcap=0xbd
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
ChaCha20-Poly1305 27410.20k 61196.95k 123994.67k 156004.01k 166818.16k 167023.65k
Final results for your comparison:
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
AES-128-GCM 4034.58k 15640.53k 56530.54k 160954.65k 387470.64k 430323.84k
AES-256-GCM 3937.72k 15245.74k 54150.59k 142370.82k 342718.76k 380947.18k
ChaCha20-Poly1305 27410.20k 61196.95k 123994.67k 156004.01k 166818.16k 167023.65k
read other posts