SmartNIC Device Installation
Some motherboards have certain PCI Express slots connected directly to the processor and others connected to the chipset; for optimal performance, install the Cisco Nexus SmartNIC (formerly ExaNIC) in a slot directly connected to the processor. If installing in a dual-processor server with only one processor installed, take special care to install in the correct slot(s), otherwise the NIC may not show up at all.
Install SFP+ modules into the SmartNIC as required. There are no limitations on the type or manufacturer of SFP+ modules that can be used.
It is not recommended to connect BNC cables to the SMA ports using BNC-SMA adaptors. These adaptors exert extreme force on the SMA connectors, which can cause them to detach from the card. This damage is irreparable. An example of a BNC cable that should not be used.
For PPS/GPS distribution, we recommend using lightweight SMA to SMA coaxial cabling. An example of a SMA cable that should be used.
If termination of a BNC coaxial cable is required, we recommend using a BNC-SMA pigtail cable, and anchoring the cable so that force is not transferred to the SmartNIC. An example of a BNC-SMA pigtail cable that can be used (with appropriate strain relief).
Damage caused by using inappropriate adaptors, or by excessive force due to cabling is not covered by the SmartNIC warranty.
There are a number of different ways of installing the Cisco Nexus SmartNIC (formerly ExaNIC) software: using the pre-built packages provided by Cisco, building your own packages, or installing from source.
A mailing list is available to be notified when updates to the SmartNIC software are released. Please feel free to add yourself here
Pre-built packages are available for some Linux distributions. The easiest way to access these is to add the Exablaze repository to your RPM or Deb sources, or you can manually download the desired packages directly from the repository.
The yum repository containing .rpm packages for Redhat/CentOS is located at:
The apt repository containing .deb packages for Debian/Ubuntu is located at:
In both cases there is a README.txt file which explains how to add the repository, what distributions are currently supported and what packages are available.
Build your own packages
You can also build your own packages using the packaging provided by Cisco.
To build your own .rpm packages, download the latest source rpm from
https://exablaze.com/downloads/yum/redhat/el6/SRPMS/ and run
rpmbuild --rebuild exanic-*.src.rpm . Packages will be output to
To build your own .deb packages, download and unpack the source from the Exablaze repository as per the first step below, then run dpkg-buildpackage from within the source directory. Packages will be output to the parent directory.
Install from source
If binary packages are not available for your distribution, the Cisco Nexus SmartNIC (formerly ExaNIC) software is also easy to build and install from source. To start, download the ExaNIC source package from the Exablaze repository and unpack it.
$ tar xzf exanic-1.7.0.tar.gz $ cd exanic-1.7.0
Running the following commands should build and install the driver, library and utilities:
$ make $ sudo make install
By default, the library will be installed to
/usr/local/lib/libexanic.a, include files to
/usr/local/include/exanic and utilities to
(You can add a
PREFIX= argument to the
make install command to use a
prefix other than
/usr/local.) Additionally the driver (
will be installed into
If the driver fails to build, ensure that you have the kernel headers
package for your kernel installed (this is typically named
kernel-devel [RedHat/CentOS] or
Driver Installation Verification
Assuming installation completed successfully, load the driver and verify
exanic-config utility works:
$ sudo modprobe exanic $ sudo exanic-config exanic0
If either of these commands fail, run
dmesg and check for errors.
exanic-config output will show the Linux interfaces corresponding
to each port of the SmartNIC. You can verify that packets are being
received by bringing up an interface and running
tcpdump. For example:
$ sudo ifconfig eth7 up $ sudo tcpdump -n -i eth7
Some systems may present the
following error in the
modprobe: ERROR: could not insert 'exanic': Required key not available
To resolve this issue, change the BIOS from Windows to "Other OS". This should alter the "Secure boot" setup of the host and the module can now be loaded.
If the SmartNIC is to be used in kernel bypass mode, it may be useful to
set permissions on the SmartNIC device files (
/dev/exanic0, etc, and the
/dev/exasock device file used by ExaNIC Sockets) so that they can be
accessed by certain non-root applications. Permissions can be set with
chmod as normal, however this will not persist across
reboots. A persistent configuration can be achieved by appropriate
configuration of the
udev daemon. For example, to force the device
files to be accessible by a group called
exanic, create a
/etc/udev/rules.d/exanic.rules file as follows:
KERNEL=="exanic*", GROUP="exanic", MODE="0660" KERNEL=="exasock", GROUP="exanic", MODE="0660"
In the current software release, the SmartNIC device files also expose the device control registers so only trusted users should be given access to the SmartNIC in this way.
If non-root users require utilities such as
exanic-config, this can be
accomplished by setting the appropriate Linux capability on the relevant
setcap cap_net_admin+ep $(which exanic-config)
Would allow non-root users to enable/disable interfaces via
The firmware version currently running on the SmartNIC card is shown under
'Firmware date' in
exanic-config. If there is a newer version
available on the website, you can update your firmware as follows.
exanic-fwupdate utility can be used to update the firmware image stored on the SmartNIC. This utility can use the compressed .fw.gz file format that firmware images are released as, or uncompressed .fw files.
$ exanic-fwupdate exanic_x40_20171019.fw Querying target device...done (0.0s) Loading and verifying update...done (0.2s) Erasing............
Note that there are different firmware files for each type of SmartNIC. The utility will prevent you from installing firmware incompatible with your hardware.
If there are multiple SmartNICs installed into the system, the utility will require you to specify which NIC you are updating:
$ exanic-fwupdate -d exanic1 exanic_x40_20171019.fw
After the SmartNIC has been updated, the FPGA needs to be reloaded for the new firmware image to be active. This can be done via a host reboot.
Instead of rebooting the host,
exanic-fwupdate can "hot reload" the FPGA by passing a flag to the utility either during the update process or afterwards as shown below. This will cause the FPGA to reload it's firmware from flash. Note that the SmartNIC firmware image running on the card at the time of the update process must support the hot reload feature.
exanic-fwupdate will output a message if hot reload is available, otherwise the host will need to be rebooted.
$ exanic-fwupdate -r Reloading card.....done (2.8s) The new firmware will take effect immediately.
The SmartNIC stores two firmware images onboard. The first is the main "production" image that will be booted normally. The second image is the "recovery" image that can be booted if the main image is corrupted.
The recovery image can be booted into by holding down the recovery button on power up of the NIC. On the Cisco Nexus SmartNIC K35-S (formerly X10), the red LED will illuminate to indicate that the recovery program is being initiated. On the Cisco Nexus SmartNIC K35-Q (formerly X40), Cisco Nexus SmartNIC K3P-Q (formerly X100) and Cisco Nexus SmartNIC+ V5P cards, the link lights will flash in an alternating amber colored pattern. Recovery mode is only intended to allow a new firmware image to be loaded onto the device and should not be used for standard operation.
Additional steps are required in order to recover the Cisco Nexus SmartNIC+ V9P. This is described in detail on the SmartNIC V9P-specifc documentation.
SmartNIC recovery button
If you installed from packages then you should use the appropriate yum or apt command (
sudo yum remove 'exanic' or
sudo apt-get remove 'exanic').
If you installed from source the following command should remove
everything that was installed by
$ sudo make uninstall
Also remove any files that were manually created while following these
/etc/udev/rules.d/exanic.rules and any scripts that run
The SmartNIC card should follow the following transitions during power on and boot up:
- When first powered all LEDs should flash on momentarily.
- With no SFPs/QSFP modules inserted, all LEDs should then switch off. If the LEDs remain on with no modules inserted, and the card does not appear in
lspci, attempt the firmware recovery process.
- Upon SFP/QSFP module insertion, the port LEDs should turn orange. If the LEDs do not light check that the connector is fully inserted, or try a different module.
- Once enabled from software, the port LEDs should turn green if link is detected. If LEDs remain orange, check that the port is enabled and has correct settings using the
- If the red LED near the PPS connector is lit, or the port LEDs flash with no modules inserted, this indicates that the firmware was corrupt and the card has entered recovery mode. A new firmware must be loaded with the
- If no LEDs light at any time, the card or slot may be damaged.
When using a server with only one of two possible processors installed, make sure the PCIe expansion slot used is one connected to the installed CPU, otherwise the NIC will not appear in
lspci. This is a common installation error in dual-processor servers such as Dell R630 and R730.
This page was last updated on Dec-10-2020.