The Linux Test Project (LTP) is a collaborative initiative aimed at enhancing the reliability, robustness, and stability of the Linux kernel. This project was initiated by SGI, OSDL, and Bull, and is currently developed and maintained by a consortium of major technology companies, including SUSE, Red Hat, Fujitsu, IBM, Cisco, and Oracle. The primary objective of LTP is to provide a comprehensive suite of tests that serve the open-source community.
Goals and Objectives
LTP's primary objective is to provide a collection of automated tools to rigorously test the Linux kernel and associated system libraries. The test suites focus on various kernel components, including file systems, networking, memory management, and system calls. By using LTP, developers can gain valuable insights into potential bugs, stability issues, or areas for performance optimization.
Key Components of LTP:
- Test Suites: LTP offers several specialized test suites, including:
- Syscalls: Verifies the implementation and response of different system calls.
- File Systems: Tests various file systems for integrity, performance, and stability.
- Networking: Assesses network protocol handling, performance, and error management.
- Security: Evaluates the kernel’s security features, enforcing secure data handling and user permissions.
The main goals of LTP include:
- Testing Automation: Automating the testing process to improve the quality of the Linux kernel and its associated system libraries.
- Reliability Validation: Ensuring that the Linux kernel can handle various workloads without failure.
- Robustness Assessment: Testing the kernel's ability to recover from errors and handle unexpected situations gracefully.
Testing Suites : LTP consists of a collection of tools designed specifically for testing the Linux kernel and related features. These testing suites cover a wide range of scenarios to validate different aspects of kernel performance.
Key Features:
- Comprehensive Coverage: The tests are designed to cover various functionalities within the kernel.
- Stress Testing: Some tests are intended to stress specific components of the system, which can help identify potential issues before they affect production environments.
How LTP Works: The LTP testing workflow follows these primary steps:
1. Initialization: Set up the test environment and prerequisites, configuring the necessary resources.
2. Test Selection: Choose relevant tests based on the kernel area under evaluation—such as file systems, network protocols, or memory management.
3. Test Execution: Run the selected tests on the system to validate the kernel's behavior and performance.
4. Result Collection: Gather data from each test run, noting any failures, errors, or performance metrics.
5. Reporting: Summarize test results with detailed logs and reports for analysis.
Important Warning
It is crucial to note that LTP tests should not be run on production systems. Certain tests like "growfiles, doio, and iogen" are particularly aggressive as they stress the I/O capabilities of the system. These tests are designed to discover or even induce problems, making them unsuitable for environments where stability is critical.
Contribution and Community
LTP is an open-source project that encourages contributions from developers around the world. Users can report issues, submit pull requests, and participate in discussions regarding improvements and new features. The project maintains an active GitHub repository where users can access the latest code, documentation, and testing results.
How to Subscribe: Visit the Mailing List Subscription Page: Go to the LTP Mailing List Info Page to find details about the mailing list. Fill Out Subscription Form: You will need to provide your email address and follow the instructions to complete your subscription.
This is a mailing list for the Linux Test Project. The list is used for patches, bug reports as well as general discussion about LTP and Linux testing.
Sending Patches: Prepare Your Patch. Make sure your patch is ready for submission. It's advisable to test your patch using GitHub Actions to ensure it compiles cleanly across various architectures and distributions. Use git send-email: The preferred method for sending patches is using git send-email. This will format your patch correctly for the mailing list. Once prepared, send your patch to the mailing list (ltp@lists.linux.it).
To post a message to all the list members, send email to ltp@lists.linux.it
LWN.net is a subscriber-supported publication that focuses on providing in-depth coverage of the Linux and free software development communities. LWN relies on subscriptions to fund its operations, as traditional advertising models have proven ineffective for their needs. Subscribing helps ensure the continuity of their publication. https://lwn.net/Articles/708182/#t
Test case tutorial:
This is a step-by-step tutorial on writing a simple C LTP test, where topics of the LTP and Linux kernel testing will be introduced gradually using a concrete example. Please refer this link
https://linux-test-project.readthedocs.io/en/latest/developers/test_case_tutorial.html
The list of syscalls which are tested under testcases/kernel/syscalls:
- kernel syscalls: 365
- tested syscalls: 341
NOTE:
Cross-Compilation Process:
- Source Code Compilation: process begins with compiling source files (e.g., .c or .cpp) into object files (.o). This is done using a cross-compiler that targets the architecture of the device for which the code is intended
- Linking: After generating object files, a linker combines them into a single executable file, such as .ELF (or main.axf). This step may also involve using a linker script that defines memory regions and other configurations specific to the target device1.
- Conversion: Once the ELF(or .axf) file is created, it can be converted into other formats like .bin, which are suitable for uploading to the target hardware.
For the ppc64le (PowerPC 64-bit Little Endian) architecture ( the equivalent of the .axf** format used in ARM cross-compilation) is typically the ELF (Executable and Linkable Format). The ELF format is widely used for executable files, object code, shared libraries, and core dumps across various architectures, including ppc64le. When cross-compiling for ppc64le, the output binary files are usually in ELF format, which can be identified by the .elf extension or simply no extension at all.
Toolchain Setup: To cross-compile for ppc64le, you need to install a suitable toolchain. A toolchain is a set of programming tools that work together to facilitate the development, building, and deployment of software applications. This typically includes compilers, linkers, debuggers, and libraries that are executed in a sequence where the output of one tool serves as the input for the next. This architecture, developed by the AIM alliance (Apple, IBM, and Motorola), is based on the RISC (Reduced Instruction Set Computing) principles and supports both 32-bit and 64-bit processing.
Example : install gcc-powerpc64le-linux-gnu
Compiling and Linking: Use the cross-compiler to compile source files:
powerpc64le-linux-gnu-gcc -o output_binary source.c
This command generates an ELF executable named "output_binary".
file output_binary
The output should indicate that it is an ELF file for PowerPC architecture and the resulting ELF binaries can be transferred to a ppc64le system for execution. This format allows developers to create executables that are compatible with PowerPC systems, facilitating software development in embedded and server environments.
Tests setup:
The internal LTP library provides a set of features that permits to customize tests behavior by setting environment variables and using specific tests arguments.
Library environment variables:
Following environment variables are expected to be set by LTP users. Therefore, with some exceptions, they have LTP_ prefix. Environment variables with TST_ prefix are used inside LTP shell API and should not be set by users.
https://linux-test-project.readthedocs.io/en/latest/users/setup_tests.html
- git
- autoconf
- automake
- make
- gcc
- m4
- pkgconf / pkg-config
- libc headers
- linux headers
- git-email
git clone --recurse-submodules https://github.com/linux-test-project/ltp.git
cd ltp
make autotools
./configure
Running single tests:
cd testcases/kernel/syscalls/foo
make
PATH=$PATH:$PWD ./foo01
To compile all tests :
make
# install LTP inside /opt/ltp by default
make install
Running tests: To run all the test suites
cd /opt/ltp
# run syscalls testing suite
./kirk -f ltp -r syscalls
Note: Many test cases have to be executed as root.
Test suites (e.g. syscalls) are defined in the runtest directory. Each file contains a list of test cases in a simple format.
Each test case has its own executable or script that can directly executed:
testcases/bin/abort01
# some tests have arguments
testcases/bin/mesgq_nstest -m none
# vast majority of tests have a help
testcases/bin/ioctl01 -h
# Many require certain environment variables to be set
LTPROOT=/opt/ltp PATH="$PATH:$LTPROOT/testcases/bin" testcases/bin/wc01.sh
Most commonly, the PATH variable needs to be set and also LTPROOT, but there are a number of other variables which usually kirk sets for you.
How to send a patch:
Here are the detailed steps for cloning the Linux Test Project (LTP), adding tests, building, running test cases, and sending patches to the community
Pre-requisite : subscribe to mailing list as mentioned in above section.
1. Check Your Email ID and Username:
- Verify your Git configuration by running:
cat .gitconfig
2. Navigate to the Patch Directory:
- Change to the directory where your patch is located
cd LTP_vxyz_patch/
3. Clone the LTP Repository:
- Clone the LTP repository from GitHub:
git clone git@github.com:linux-test-project/ltp.git
4. Change Directory to LTP:
- Move into the cloned LTP directory
cd ltp
5. Build the Project:
- Execute the build script to compile the project:
./build.sh
6. Navigate to the Test Case Directory:
- Go to the specific test case directory you want to modify or add tests
cd testcases/kernel/mem/hugetlb/hugeshmget
7. Modify/Create a Test Case:
- Open the test case file in a text editor (e.g., `vi`) and write or modify your test case:
vi hugeshmget06.c
8. Compile and Check the Test Case:
- Build and check your test case for any errors:
make && make check
9. Run the Test Case:
- Execute your test case to verify its functionality:
./hugeshmget06
10. Run with Input Argument:
- Optionally, run your test case with an input argument for loop of count=5 (e.g., `-i 5`):
./hugeshmget06 -i 5
11. Check Git Status:
- Check which files have been modified or added in your Git repository
git status
12. Stage Your Changes:
- Add your modified files to the staging area for commit:
git add ../../../../../runtest/hugetlb hugeshmget06.c ../../.gitignore
13. Check Git Status Again:
- Verify that your changes are staged correctly:
git status
14. Commit Your Changes:
- Commit your changes with a sign-off message:
git commit -s
15. Create a Patch File:
- Generate a patch file for your commit:
git format-patch -1
16. Edit the Patch File Before Sending:
- Open the generated patch file and edit it as necessary (e.g., update version, email IDs):
vi 0001-Migrating-the-libhugetlbfs-testcases-shm-gettest.c-t.patch
17. Send the Patch via Email:
- Use `git send-email` to send your patch to the community mailing list and reviewers:
git send-email --to=ltp@lists.linux.it --cc=reviewer1@xyz.co --cc=reviewer2@abc.co ./0001-Migrating-the-libhugetlbfs-testcases-shm-gettest.c-t.patch
- Ensure that you fill in the subject line and other email details as prompted.
18. Respond to Review Comments via Email:
- After receiving feedback, reply to the email addressing any review comments and mention any new versions of patches based on those comments.
By following these steps, you can effectively contribute to the Linux Test Project by adding new tests, verifying their functionality, and submitting patches for review by the community.
Search your Patch in mailing list :
To search details on patch with search key for example - "libhugetlbfs" or "shmget-test" or by username "sachinpb"
Example 1 : https://lore.kernel.org/ltp/?q=libhugetlbfs
Example 2 : https://lore.kernel.org/ltp/?q=sachinpb
Example 3: https://lore.kernel.org/ltp/ZvqQxP9KVW6PqFOo@yuki.lan/
[LTP] [ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2024
Let us test the newly added testcase "hugeshmget06 A hugepage shm test"
Step 1 : Clone the LTP repo from https://github.com/linux-test-project/ltp
Step 2 : Confirm the testcase merged with master branch
Step 3 : Check the new testcase "hugeshmget06.c" existing in "/root/ltp/testcases/kernel/mem/hugetlb/hugeshmget"
Step 4 : Build test cases by running script ./build.sh as shown below:
Step 5 : Do make install and check the binaries at "/root/ltp-install/"
Step 6 : Execute the testcase ./hugeshmget06 as shown below
Step 7: Execute the above testcase in loop of any count . For example count=3
Conclusion
The Linux Test Project plays a vital role in ensuring that the Linux kernel remains reliable and robust for users across various industries. By providing a structured approach to testing, LTP helps developers identify potential issues early in the development process, ultimately contributing to a more stable operating environment for all Linux users.
For more information or to get involved with LTP, you can visit their [GitHub repository](https://github.com/linux-test-project/ltp).
In this blog, we've explored the Linux Test Project (LTP), covering how to add tests and submit patches to the mailing list. I hope you found this information useful and that it enhances your understanding and contributions to LTP. Best of luck on your journey with LTP!
Reference :
1) https://linux-test-project.readthedocs.io/en/latest/
2) https://linux-test-project.readthedocs.io/en/latest/index.html
3) https://jitwxs.cn/51dc9e04