Introduction to eBPF - Revolutionizing Linux Kernel

Key takeaways

eBPF, which stands for Extended Berkeley Packet Filter, is a technology that allows Linux programs to run code in the kernel.
The run-time interpreter includes a Just-In-Time (JIT) compiler.
For traceability, active network control, and continuous integration, the capacity to run programs in a protected and sandboxes manner in the kernel is a highly useful resource.

#

Defining eBPF and How It Works #

eBPF, which stands for Extended Berkeley Packet Filter, is a technology that allows Linux programs to run code in the kernel. eBPF was previously also used to construct debugging, networking, profiling, monitoring, firewalls, and other types of programs.

The kernel has a privileged capacity to monitor and manage the whole system. Due to this, the operating system was always an excellent platform to incorporate accessibility, privacy, and networking features.

Some of the main functions we can achieve by using eBPF are:

Security
Monitoring
Observability
Tracing and Profiling
Networking

Around the same time, owing to its valuable function and a strong need for stability and safety, an operating system kernel is difficult to build. As a result, in comparison to capability deployed outside of the operating system, the pace of technological development at the operating system level has generally been slower.

This is where eBPF comes in and changes the game completely. Without modifying kernel source code or importing kernel modules, eBPF may execute sandboxes programs in the Linux kernel. The operating system subsequently ensures the same level of safety and efficiency as if the code had been originally built using a Just-In-Time (JIT) compiler and verification engine.

To understand in detail, the eBPF program works in the following way:

Firstly, a developer sends the bytecode of the eBPF program to the kernel. This includes a program type that specifies where the program should be linked, which in-kernel helper features the validator will allow being called. This is regardless of whether network packet data can be accessed directly, and what type of object should be passed as the program’s first argument.
On the bytecode, the kernel executes a verifier. The verifier performs numerous security checks here on bytecode to guarantee that the program ends properly and does not, in any way, include any paths that may cause the kernel to lock up.

It also replicates the eBPF program’s execution and verifies the remote machine’s status at each step to guarantee that the register and stack states are correct. Finally, the program type is used to limit the kernel function calls that may be made by the application.

JIT-compiles the bytecode into native code and saves it to the given place.
The program is performed and data is written to the ring buffer or the map once the particular event happens.
The user space can read the map or ring buffer to acquire the program result.

How Do Developers Write eBPF Programs? #

The eBPF Go library is a general eBPF library that separates the loading and maintenance of eBPF programs from the method of getting to the eBPF bytecode. eBPF applications are often written in a higher-level programming language and then compiled to eBPF bytecode using the clang/LLVM compiler.

All programs should still run through the eBPF validator even if a task is authorized to load an eBPF program. The program’s security is validated by the eBPF verifier. This implies, for example, that programs are verified to guarantee that they always terminate, e.g., an eBPF program may not ever freeze or loop indefinitely.

In many cases, eBPF is utilized via indirect means. For instance, eBPF is used through projects like Cilium, bcc, or bpftrace, which offer an abstraction in addition to eBPF. This permits users to offer additional intent-based rules that are subsequently executed using eBPF rather than writing programs outright.

Programs must be written directly if no higher-level abstraction exists. eBPF applications are likely to be loaded as bytecode by the Linux kernel. Although it is possible to write bytecode directly, the more frequent programming technique is to convert pseudo-C code into eBPF bytecode using a compiler suite such as LLVM.

eBPF Basic Concepts and Architecture #

Different techniques of monitoring and tracing user space programs are inferior to eBPF. eBPF applications may be connected to any kernel function and can examine and alter the function’s scope and parameters. eBPF delivers powerful statistical data structures that collect important data quickly and simply.

Let’s take a look at some of the basic concepts and architecture that make up an extended Berkeley Packet Filter or eBPF.

Predefined Hooks #

Event-driven eBPF programs are launched when the kernel or an application reaches a specified hook point. System calls, function entry/exit, kernel tracepoints, network events, and a variety of additional hooks are among the pre-defined hooks.

If a preset hook does not exist for a specific purpose, a kernel probe (probe) or a user probe (probe) can be used to join eBPF programs nearly anywhere either around the kernel or in the user applications.

Program Verification #

Following the detection of a hook, the eBPF system function may then be used to load the matching eBPF application into the Linux kernel. This generally entails the use of an eBPF library. Once a program is loaded into the kernel, it must be validated before it can be deemed safe to use.

Validation needs certain things to be in order. Only a qualified eBPF process could load the software (unless specified as something else). The software will not harm or destroy the computer. The software will almost always run till it is finished (not be stuck in an endless loop).

JIT Compilation or Just-in-time Compilation #

The run-time interpreter includes a Just-In-Time (JIT) compiler. Since only the methods that are required at run-time are compiled into machine code by the JIT Compiler, the JIT Compiler uses less memory. Also, because functions required simultaneously are most often on the same memory page, implementing the JIT compiler reduces page faults.

In order to improve program execution speed, the Just-in-Time (JIT) compilation stage converts the program’s generic bytecode into a device-specific command sequence. This allows eBPF programs to run as quickly as originally built kernel code or kernel module code.

eBPF Maps

The capacity to exchange acquired data and retain information is a necessary aspect of eBPF applications. eBPF applications can use the notion of eBPF maps to store and retrieve data in a variety of data structures for this objective. eBPF maps can be fetched through a function call from both eBPF programs and user space programs.

Here are some of the available map types to demonstrate the range of data structures:

Ring Buffer
Stack Trace
LPM (Longest Prefix match)
Hash tables, Arrays
LRU (Least Recently Used)

Helper Calls #

An eBPF application cannot invoke a kernel function at will. This is due to the fact that eBPF applications must retain functionality and prevent being tied to certain kernel versions. As a result, eBPF applications perform function calls using helper functions. Helper functions are kernel-provided APIs that can be quickly tweaked.

A helper can only have five parameters according to eBPF standards. Essentially, eBPF applications use the built auxiliary functions directly without the need for a different function interface. As a consequence, invoking helpers has no overhead, resulting in superior efficiency.

Some examples of helper calls are as follows:

eBPF map access
Get process/group context
Manipulate network packets and forwarding logic
Generate random numbers
Get current time & date

Function and Tail Calls #

These calls allow eBPF programs to be modularized. In a program, function calls allow functions to be defined and invoked. Other eBPF applications can be run via tail calls. They have the ability to modify the execution context as well.

With the idea of tail and function calls, eBPF applications may be assembled. Within an eBPF application, function calls allow you to define and invoke functions. Identical to how the execve() system call works for standard methods.

A General Idea of eBPF Use Cases #

Security

Integrating a packet and socket-level perspective of all activities involving networking with a bedrock of analyzing and comprehending all system calls provides for new and innovative methods to protect systems.

Although network-level filtering, system call filtering, and process backdrop tracing have traditionally been managed by separate systems, eBPF enables the real-time insights of all elements to be combined to build security frameworks that operate on more background with a higher degree of control.

Networking

eBPF is an excellent contender for all networking systems’ packet processing tasks because of its quality and programmability. eBPF’s programmability allows for the inclusion of new format parsers and the easy programming of any forwarding algorithm. This has the ability to meet changing demands without ever abandoning the Linux kernel’s context of packet processing. The JIT compiler’s efficiency comes close to that of naively assembled in-kernel code in terms of running performance.

Tracing and Profiling

The potential to connect eBPF programs to locate points, along with kernel and probe points of the user application, provides extraordinary insight into application and system behavior during operation. Multiple viewpoints may be integrated by offering access capabilities to both the application and system perspectives.

This, in turn, allows new and valuable insights to address system stability issues. Sophisticated numerical data structures allow for the efficient extraction of significant visibility information. This eliminates the need for large volumes of sample data, which is generally required by existing programs.

Observability and Monitoring

eBPF allows the collection and in-kernel aggregation of customizable metrics. In addition to this, it also allows the creation of visibility events. These events depend on a wide variety of conceivable sources, rather than just relying on static counters and gauges accessible by the operating system.

It simply collects the visibility data which is necessary and thereafter constructs histograms and similar types of data structures at the origin of the event. Rather than depending on sample export, this process increases the depth of the transparency that may be accomplished while also greatly diminishing the overall integrated framework.

The Need for eBPF and its Place in the Future #

eBPF is a new Linux kernel feature that increases observability, networking, and security. It avoids the need to modify kernel source code or install modules, allowing you to build a more robust architecture to manage your system without adding unnecessary complexity.

With eBPF, a simple update for reprogramming the behavior of the Linux kernel without needing modifications to kernel source code or the deployment of a kernel module is enabled. In many respects, this is comparable to how JavaScript and other scripting languages made it possible to adapt systems that were previously costly and difficult to update.

eBPF is a fantastic addition to the Linux kernel. For traceability, active network control, and continuous integration, the capacity to run programs in a protected and sandboxes manner in the kernel is a highly useful resource.

Related reading