Understand the architecture of OFP, the purpose of the project, main features and components
The intent of this project is to enable accelerated routing/forwarding for IPv4 and IPv6, tunneling and termination for a variety of protocols. Unsupported functionality is provided by the host OS networking stack or slowpath. However, in the future we expect to have OFP working independently of a slowpath in a bare metal environment.
OpenFastPath functionality is provided as a library to Fast Path applications that use ODP run to completion execution model and framework. DPDK is in this initial release supported through the ODP-DPDK layer. Native DPDK support is being evaluated.
On UDP and TCP level, the stack uses an optimized callback based zero-copy socket API which enables the usage of the complete packet, including metadata, in user space. This is done without the copy operation typically used by the traditional BSD sockets. Termination of protocols with BSD socket interface for legacy applications is also supported.
UDP, TCP, ICMP code was ported from libuinet which is a well known user space FreeBSD port. The remaining software is ported from reliable open source projects or code developed in house by the OFP team.
A Linux system built with OpenFastPath support consists of a number main building blocks.
The green boxes represents OFP, light blue user application code, black hardware, dark blue ODP and finally orange represents Linux host system:
One core (typically #0) is required for Linux system calls, mainly for CLI and route copy and for communication with Linux kernel using TUN/TAP interface. Additional Linux cores might be needed for slowpath if there are a lot of slowpath traffic. Other cores are allocated by ODP for fast path processing.
ODP by default takes control of all Ethernet which means that packet sent by Linux slow path are sent though the TAP interface to Ethernet interfaces.
User Conf Code is a management thread that is running on the Linux core. Usually it is the main thread of the application. It is used to start and configure ODP and OFP and shares same memory as the fastpath cores.
The block called user/default dispatcher implements the dispatcher functionality that reads packets through the ODP API’s. It’s been placed outside OFP to give the user control over which API to use to get packets form ODP. Depending on the underlying ODP implementation and HW, different methods can be selected.
OFP is a multithreaded multicore application so there is one instance of OFP that run across all data plane cores. However there is a separate independent dispatcher threads to allow different dispatchers on each core
On the cores allocated to fastpath processing, ODP starts only one thread where the dispatcher, OFP and the user application code runs. This is the case when using the hook or callback APIs. If legacy BSD socket APIs are used, those need to run on a separate core or cores not to interfere with the ODP run to completion thread.
Setup is done when the application starts. All the memory for the port configuration table, routing trees, AVL trees etc. is allocated. This ensures that any pointer to a data structure will never be invalid although data may not be correct. Memory is shared between processor cores.
State is kept in memory zones shared between packet processing cores and master core. Routing tables and port configurations are synchronized with slowpath through Netlink API and system commands.
MAIN FEATURES AND COMPONENTS
Main features are:
OFP is composed of the following components:
INGRESS AND EGRESS PACKET PROCESSING
The packet processing is handled through a series of self-contained processing functions which means that traffic can be inserted at various places in the packet processing flow. Through the use of ODP Classification API OFP performance can be improved by use of hardware to shorten the packet processing paths.
More hardware supported packet processing functions like packet validation, checksum and cryptographic transformations as well as optimized memory/buffers operations abstracted underneath ODP API will further improve overall packet processing rate.
Packet processing component receives data packets from Ethernet interfaces and Linux kernel. Ethernet packets are processed the following way:
The packet processing component also provides API for sending packets.
New zero-copy APIs optimized for single thread run-to-completion environments
Other OFP user application APIs
You can find the public API in the ./include/api/ folder at the GitHub project page.
COMMAND LINE INTERFACE
OFP contains a Telnet based CLI for configuration and statistics browsing. It can also read the commands from a file when the application starts. Basically CLI is used for debugging and enabling some routing features that are not available by copying routes from Linux kernel. Basic commands are:
IP and UDP implementations has been optimized for performance, TCP implementation is functional but not performance optimized.
If you want to read more about OFP, take a look at the project documentation in Github.