Fprobe: Efficient Kernel Function Tracing in Linux

Fprobe is a function entry/exit probe mechanism in the Linux kernel, built on top of the ftrace framework. It allows developers to attach callbacks to function entry and exit points, similar to kprobes and kretprobes, but with improved performance for multiple functions through a single handler. This makes fprobe particularly useful for tracing and debugging kernel functions without the overhead associated with full ftrace features.

Key Features of Fprobe:

Performance: Provides faster instrumentation for multiple functions compared to traditional kprobes and kretprobes.

Structure: The fprobe structure includes fields for entry and exit handlers, allowing customized behavior when functions are entered or exited.

Registration: Fprobes can be registered using various methods, including function-name filters or direct address registration.

The usage of fprobe :

The fprobe is a wrapper of ftrace (+ kretprobe-like return callback) to attach callbacks to multiple function entry and exit. User needs to set up the struct fprobe and pass it to register_fprobe(). Typically, fprobe data structure is initialized with the entry_handler and/or exit_handler as below.

NOTE: callbacks refer to functions that are executed in response to specific events, particularly when a function is entered or exited

A typical fprobe setup might look like this:

struct fprobe fp = {
    .entry_handler = my_entry_callback,
    .exit_handler = my_exit_callback,
};
register_fprobe(&fp, "func*", "func2");

-----------------------------------------------------

Simple C program that demonstrates how to use fprobe in the Linux kernel. This program sets up an fprobe to monitor the entry and exit of a specific kernel function. For this example, we'll assume that you want to probe a function named "target_function".

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fprobe.h>
#include <linux/init.h>

// Function prototypes
int my_entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data);
void my_exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data);
void target_function(void); // Declare target_function prototype

// Define the fprobe structure
static struct fprobe fp = {
    .entry_handler = my_entry_callback,
    .exit_handler = my_exit_callback,
};

// The function we want to probe
void target_function(void) {
    printk(KERN_INFO "Inside target_function\n");
}

// Entry callback
int my_entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data) {
    printk(KERN_INFO "Entered target_function\n");
    return 0; // Return an int as expected
}

// Exit callback
void my_exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data) {
    printk(KERN_INFO "Exited target_function\n");
}

// Module initialization
static int __init my_module_init(void) {
    // Register the fprobe for the target function
    if (register_fprobe(&fp, "target_function", NULL)) {
        printk(KERN_ERR "Failed to register fprobe\n");
        return -1;
    }

    printk(KERN_INFO "Fprobe registered successfully\n");

    // Call the target function to see the callbacks in action
    target_function();

    return 0;
}

// Module cleanup
static void __exit my_module_exit(void) {
    // Unregister the fprobe
    unregister_fprobe(&fp);
    printk(KERN_INFO "Fprobe unregistered successfully\n");
}

module_init(my_module_init);
module_exit(my_module_exit);

MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Fprobe Example Module");
MODULE_AUTHOR("SACHIN P BAPPALIGE");

--------------------------------------------------------------------------

Compilation and Loading:

Step 1 : Save the code in a file named fprobe_example.c.

Step 2 : Create a Makefile as shown below

obj-m += fprobe_example.o

all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

--------------------------------------

# make
make -C /lib/modules/6.12.0-rc2+/build M=/root/fprobe modules
make[1]: Entering directory '/root/linux'
  CC [M]  /root/fprobe/fprobe_example.o
  MODPOST /root/fprobe/Module.symvers
  CC [M]  /root/fprobe/fprobe_example.mod.o
  CC [M]  /root/fprobe/.module-common.o
  LD [M]  /root/fprobe/fprobe_example.ko
  BTF [M] /root/fprobe/fprobe_example.ko
make[1]: Leaving directory '/root/linux'

# ls

Makefile  Module.symvers  fprobe_example.c  fprobe_example.ko  fprobe_example.mod  fprobe_example.mod.c  fprobe_example.mod.o  fprobe_example.o  modules.order

--------------------------------------------------

Step 3 : Insert module-->  insmod fprobe_example.ko

Step 4: Check kernel logs with dmesg to see output from the callbacks.

Step 5: Remove the module using  -->  rmmod fprobe_example

Here’s a C program that demonstrates how to use fprobe to monitor the do_fork function in the Linux kernel. This example sets up entry and exit callbacks to log when the do_fork function is called and when it returns.

-------------------------------------

#include <linux/module.h>

#include <linux/kernel.h>

#include <linux/fprobe.h>

#include <linux/init.h>

#include <linux/sched.h>

// Function prototypes for the entry and exit callbacks

int my_entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data);

void my_exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data);

// Define the fprobe structure

static struct fprobe fp = {

    .entry_handler = my_entry_callback,

    .exit_handler = my_exit_callback,

};

// Entry callback for do_fork

int my_entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data) {

    printk(KERN_INFO "do_fork called: PID = %d\n", current->pid);

    return 0; // Return an int as required

}

// Exit callback for do_fork

void my_exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long args, struct pt_regs *regs, void *data) {

    printk(KERN_INFO "do_fork returned: PID = %d\n", current->pid);

}

// Module initialization function

static int __init my_module_init(void) {

    // Register the fprobe for the do_fork function

    if (register_fprobe(&fp, "do_fork", NULL)) {

        printk(KERN_ERR "Failed to register fprobe for do_fork\n");

        return -1;

    }

    printk(KERN_INFO "Fprobe registered successfully for do_fork\n");

    return 0;

}

// Module cleanup function

static void __exit my_module_exit(void) {

    // Unregister the fprobe

    unregister_fprobe(&fp);

    printk(KERN_INFO "Fprobe unregistered successfully for do_fork\n");

}

module_init(my_module_init);

module_exit(my_module_exit);

MODULE_LICENSE("GPL");

MODULE_DESCRIPTION("Fprobe Example for do_fork");

MODULE_AUTHOR("SACHIN P BAPPALIGE");

------------------------------------------------------------------------------

NOTE: Fprobe and kprobe are both mechanisms used in the Linux kernel for tracing and debugging, but they have distinct characteristics and use cases.

Kprobe :

Purpose: Kprobes allow developers to insert probes at almost any kernel function entry or exit point. They enable dynamic instrumentation by setting breakpoints on specified functions.

Structure: Each kprobe consists of a handler that is invoked when the probe is hit, allowing for custom actions or logging.

Performance: Kprobes can introduce overhead, especially when multiple probes are registered, as they rely on software breakpoints (SWBP) which can be slower compared to other methods.

Recursion Handling: Kprobes maintain a per-CPU variable (`current_kprobe`) to manage recursion safely, which prevents re-entry into the same probe handler.

Developers commonly probe various Linux kernel functions to gather insights about system performance, debug issues, and monitor behavior. Here are some typical functions that are often probed:

Commonly Probed Kernel Functions : 

1)  do_fork: Used for process creation. Probing this function can help track process spawning and resource allocation.

2)  sys_read / sys_write: These functions handle system calls for reading from and writing to files. Probing them allows developers to monitor file I/O operations.

3)  schedule: Responsible for context switching between processes. Probing this function can provide insights into scheduling behavior and CPU usage.

4)  kmalloc / kfree: Functions for memory allocation and deallocation in the kernel. Probing these can help identify memory usage patterns and potential leaks.

5)  vfs_read / vfs_write:Virtual filesystem layer functions that manage file operations. Probing these functions helps in understanding file access patterns across different filesystems.

7)  tcp_sendmsg / tcp_recvmsg:Used for sending and receiving TCP messages. Probing these can assist in analyzing network performance and behavior.

8)  netif_receive_skb: This function processes incoming packets in the network stack. Probing it can help monitor network traffic handling.

9)  exit_notify: Handles the cleanup of a process upon exit. Probing this function can track process termination and resource cleanup.

10) __wake_up: Used to wake up processes waiting on a condition variable. Probing this can provide insights into synchronization and concurrency issues.

11) File System Operations: Functions like lookup, create, and unlink are also commonly probed to monitor filesystem interactions

In summary, while both kprobe and fprobe serve the purpose of tracing kernel functions, fprobe is tailored for efficiency and ease of use in scenarios requiring multiple function tracing. Kprobes offer broader capabilities but at the cost of performance overhead and complexity.

Reference :

1) https://docs.kernel.org/trace/fprobe.html