From Chips to Cloud: The Invisible Pyramid Powering the AI Revolution

An AI query served from Azure, AWS, or Google Cloud starts its journey years earlier inside semiconductor factories across the Netherlands, United States, Japan, South Korea, Taiwan, and Singapore. The modern AI revolution depends on hundreds of companies working together across multiple layers of the technology stack, from lithography machines and wafer fabrication equipment to memory chips, advanced packaging technologies, AI accelerators, servers, and hyperscale cloud data centers.

The semiconductor industry resembles a pyramid where every layer depends on the layer below it. If one layer slows down, the entire AI ecosystem feels the impact.

---

The Semiconductor Supply Chain Pyramid

                    ┌─────────────────────┐
                    │   AI APPLICATIONS   │
                    │ ChatGPT, Copilot,   │
                    │ Claude, Gemini      │
                    └─────────▲───────────┘

                    ┌─────────────────────┐
                    │ CLOUD PROVIDERS     │
                    │ AWS Azure Google    │
                    │ Meta Oracle xAI     │
                    └─────────▲───────────┘

                    ┌─────────────────────┐
                    │ AI SERVERS          │
                    │ Dell HPE Lenovo     │
                    │ Supermicro IBM      │
                    └─────────▲───────────┘

                    ┌─────────────────────┐
                    │ GPU DESIGNERS       │
                    │ NVIDIA AMD          │
                    │ Broadcom Intel      │
                    └─────────▲───────────┘

                    ┌─────────────────────┐
                    │ PACKAGING & TEST    │
                    │ ASE Amkor TSMC      │
                    │ CoWoS BESI DISCO    │
                    └─────────▲───────────┘

                    ┌─────────────────────┐
                    │ MEMORY MAKERS       │
                    │ Samsung SK hynix    │
                    │ Micron Kioxia       │
                    └─────────▲───────────┘

                    ┌─────────────────────┐
                    │ FOUNDRIES           │
                    │ TSMC Samsung Intel  │
                    └─────────▲───────────┘

                    ┌─────────────────────┐
                    │ EQUIPMENT VENDORS   │
                    │ ASML AMAT LRCX KLA  │
                    └─────────────────────┘

---

Layer 1: Equipment Vendors - The Foundation of Everything

ASML (Netherlands)

ASML produces the world's most advanced lithography equipment.

Without ASML:

• No 5nm chips

• No 3nm chips

• No 2nm chips

• No advanced NVIDIA GPUs

The company's EUV (Extreme Ultraviolet) lithography machine is arguably the most complex machine ever built.

An EUV system contains:

• 100,000+ components

• Mirrors from Germany

• Lasers from the United States

• Precision mechanics from Europe

Cost per machine:

$200M to $400M+

Architect Insight:

Every advanced AI chip begins with ASML.

---

Applied Materials (USA)

Applied Materials helps build the transistor structures inside the chip.

Major technologies:

• Deposition

• Materials Engineering

• CMP

• Process integration

Think of ASML printing the chip while Applied Materials creates the materials that make the chip work.

---

Lam Research (USA)

Critical for:

• Etching

• 3D NAND fabrication

• HBM manufacturing

Without Lam Research:

• Modern NAND flash would not exist.

• High-density HBM stacks would be difficult to manufacture.

---

KLA (USA)

KLA ensures quality.

Responsibilities:

• Defect detection

• Wafer inspection

• Process control

• Yield optimization

Architect Perspective:

A defect discovered late may waste millions of dollars of production.

---

Tokyo Electron (Japan)

Provides:

• Coaters

• Developers

• Deposition tools

• Etch systems

Another critical supplier for advanced fabs.

---

Layer 2: Foundries - Where Chips Are Born

These companies convert designs into physical silicon.

TSMC (Taiwan)

Produces chips for:

• NVIDIA

• AMD

• Apple

• Broadcom

• Qualcomm

Critical technologies:

• 5nm

• 3nm

• 2nm

When NVIDIA announces a new GPU, TSMC manufactures it.

---

Samsung Foundry (South Korea)

Produces:

• Memory

• Logic chips

• Mobile SoCs

Strong presence in:

• HBM

• Advanced packaging

• Foundry services

---

Intel Foundry (USA)

Transitioning into a major foundry provider.

Focus areas:

• Intel 18A

• Advanced packaging

• US-based manufacturing

---

Layer 3: Memory Makers - Feeding the AI Beast

AI cannot run on compute power alone.

Memory has become the bottleneck.

Samsung

Produces:

• DRAM

• NAND

• HBM

Used in:

• AI accelerators

• Enterprise storage

• HPC systems

---

SK hynix

Current leader in:

HBM3
HBM3E

Supplying memory for:

• NVIDIA H100

• NVIDIA B100/B200

• Future AI accelerators

Many analysts consider HBM the most critical component in AI hardware today.

---

Micron

Competes in:

• HBM

• DDR5

• Enterprise memory

Strategic supplier to hyperscalers and HPC vendors.

---

Layer 4: Packaging & Advanced Integration

This is where modern AI systems are fundamentally different.

Twenty years ago:

One Chip = One Package

Today:

GPU + HBM + Interposer
+ Chiplets + Advanced Packaging

---

TSMC CoWoS

CoWoS enables:

• GPU integration

• HBM integration

• Extremely high bandwidth

Without CoWoS:

• Modern NVIDIA accelerators cannot exist.

---

ASE & Amkor

Provide:

• Packaging

• Assembly

• Final testing

They convert bare silicon into production-ready products.

---

BESI

Provides advanced die bonding.

Critical for:

• Chiplet architectures

• High-bandwidth memory assembly

---

Layer 5: Chip Designers

These companies create the architecture.

---

NVIDIA

Known for:

• H100

• H200

• B100

• B200

• NVLink

AI training leadership.

---

AMD

Known for:

• MI300

• MI350

• Infinity Fabric

Strong AI and HPC strategy.

---

Broadcom

Essential for:

• AI networking

• Custom AI silicon

Increasingly important in hyperscale infrastructure.

---

Intel

Provides:

• CPUs

• Gaudi accelerators

• Networking technology

---

Layer 6: AI Infrastructure Builders

These companies assemble complete systems.

Dell

Builds:

• AI clusters

• Enterprise AI systems

• Storage infrastructure

---

HPE

Major supplier of:

• HPC supercomputers

• AI factories

• Exascale systems

---

IBM

Known for:

• Power Systems

• Storage

• AI integration

• Enterprise AI deployments

---

Supermicro

Rapidly growing provider of:

• GPU servers

• AI racks

• Liquid-cooled infrastructures

---

Layer 7: Cloud Providers and AI Factories

The final consumers of the semiconductor pyramid.

Microsoft Azure

Deploys:

• NVIDIA clusters

• AI supercomputers

• Copilot infrastructure

---

AWS

Largest cloud consumer of GPUs.

Developing:

• Trainium

• Inferentia

---

Google

Develops TPUs while also deploying NVIDIA infrastructure.

---

Meta

Consumes enormous amounts of:

• GPUs

• HBM memory

• Networking equipment

for foundation models and AI training.

---

How Dependencies Flow

A simplified dependency chain looks like:

ASML
  ↓
TSMC / Samsung
  ↓
HBM (SK hynix, Samsung, Micron)
  ↓
CoWoS Packaging
  ↓
NVIDIA / AMD GPUs
  ↓
Dell / HPE / IBM Servers
  ↓
Azure / AWS / Google
  ↓
ChatGPT / Copilot / AI Services

Notice how every layer depends on the previous one.

If ASML shipments slow:

Foundry production slows
          ↓
GPU supply drops
          ↓
Cloud expansion slows
          ↓
AI deployment slows

This is why architects look beyond individual products and study the entire ecosystem.

====

NOTE:

Without ASML lithography systems:

• TSMC cannot build advanced nodes.
• Samsung cannot fabricate HBM and advanced logic.
• NVIDIA cannot obtain cutting-edge GPUs.
• Cloud providers cannot deploy AI clusters.

This dependency chain is what makes the semiconductor ecosystem one of the most interconnected industries in the world.

---

What Data Center Architects Care About

Modern AI architects evaluate:

Silicon Layer

• Process node (3nm, 2nm)

• Yield rates

• Supply availability

Memory Layer

• HBM capacity

• Memory bandwidth

• HBM supply constraints

Packaging Layer

• CoWoS availability

• Chiplet integration

AI Accelerator Layer

• GPU performance

• NVLink architecture

• Power consumption

Infrastructure Layer

• InfiniBand

• Ethernet fabrics

• Storage throughput

Cloud Layer

• Scalability

• TCO

• Energy efficiency

---

Conclusion

The AI revolution is not powered by a single company. It is powered by an interconnected semiconductor supply chain spanning continents and thousands of specialized technologies. ASML enables advanced manufacturing, Applied Materials and Lam Research enable fabrication, TSMC and Samsung produce silicon, SK hynix and Micron provide memory, ASE and Amkor package devices, NVIDIA and AMD design accelerators, and cloud providers transform all of that innovation into services consumed by billions of users.

Designing the AI Factory: How NVLink, InfiniBand, and PCIe Shape Modern HPC and AI Infrastructure

Artificial Intelligence (AI) and High-Performance Computing (HPC) have entered an era where data movement is just as critical as compute performance. Modern AI models often span multiple GPUs, servers, and even entire data center clusters, making interconnect technology a key architectural decision. Three technologies play a central role in this ecosystem: PCIe, NVLink, and InfiniBand.

While all three are responsible for moving data, they operate at different layers of the infrastructure. PCIe provides the foundational connectivity inside a server, NVLink enables high-speed communication between GPUs within a node, and InfiniBand extends that communication across multiple servers in a cluster. Understanding the strengths and architectural roles of each interconnect is essential for designing scalable, efficient, and cost-effective AI and HPC systems. 

Most people focus on:

• GPU FLOPS

• GPU Memory

• GPU Count

Architects focus on:

• Communication topology

• Data movement efficiency

• Memory access hierarchy

• Scale-out vs Scale-up designs

• Network congestion

• Power efficiency

• Cost per trained model

The diagram should explain that AI infrastructure has evolved into a hierarchy.

Application Layer
      |
AI Frameworks
(PyTorch, TensorFlow)
      |
GPU Compute Layer
(H100, B100, MI300)
      |
GPU Fabric Layer
(NVLink/NVSwitch)
      |
Server Fabric Layer
(PCIe Gen5/Gen6)
      |
Cluster Fabric Layer
(InfiniBand/Ethernet)
      |
Storage Fabric
(GPFS/Lustre/Ceph)

Modern AI performance depends on every layer.

---

Section 1: The New AI Data Center Problem

Introduce the challenge.

Ten years ago:

CPU-Centric Computing

Today:

GPU-Centric Computing

Tomorrow:

GPU Fabric-Centric Computing

For training LLMs:

• GPT

• Llama

• Granite

• DeepSeek

• Mistral

The model no longer fits in one GPU.

The model may span:

• 8 GPUs

• 64 GPUs

• 512 GPUs

• Thousands of GPUs

Communication becomes as important as computation.

---

Section 2: AI Infrastructure Architecture

Show a typical AI node.

             CPU
              |
        PCIe Gen5/6
              |
    --------------------
    |    |    |    |
   GPU  GPU  GPU  GPU
     \   |   |  /
      NVSwitch

Architectural message:

PCIe connects everything.

NVLink accelerates GPU communication.

---

Section 3: Why PCIe Alone Is No Longer Enough

Example:

A 70B parameter model.

During training:

GPU → GPU
Gradient Exchange
Tensor Sharing
Activation Movement
Checkpoint Sync

PCIe becomes the bottleneck.

Architects ask:

Can GPUs exchange data faster than through the CPU root complex?

This is where NVLink enters.

---

Section 4: NVLink Creates a GPU Memory Fabric

This is a very important architectural concept.

Instead of saying:

NVLink is faster.

Say:

NVLink transforms individual GPUs into a collective memory-compute fabric.

Illustration:

GPU1 <--> GPU2
 |          |
 |          |
GPU3 <--> GPU4

Benefits:

• Faster tensor transfer

• Reduced synchronization delay

• Efficient model parallelism

• Lower communication overhead

Architect Insight:

NVLink is not just a link, it is a distributed GPU memory architecture.

---

Section 5: Why AI Factories Need InfiniBand

Single server scaling eventually hits limits.

When training a trillion-parameter model:

Server 1
Server 2
Server 3
Server 4
...

Now communication must cross servers.

Requirements:

• Microsecond latency

• RDMA

• Congestion control

• Collective communication acceleration

This is InfiniBand's domain.

Architecture:

NODE A
  |
GPU Mesh
  |
InfiniBand
  |
NODE B
  |
GPU Mesh

Key architectural point:

NVLink = Scale-Up

InfiniBand = Scale-Out

---

Section 6: Scale-Up vs Scale-Out

This section will attract architects.

Design Goal	Preferred Technology
8 GPU Server	NVLink
16 GPU Server	NVLink + NVSwitch
Multi-Rack Cluster	InfiniBand
Enterprise AI	PCIe + Ethernet
Exascale HPC	NVLink + InfiniBand

Architects always evaluate:

• How large can one node become?

• How many nodes are required?

• What is the communication pattern?

---

Section 7: Emerging AI Data Center Trends

This will make your blog more future-looking.

NVIDIA AI Factories

Components:

• Blackwell GPUs

• NVLink Switch

• Quantum InfiniBand

• Spectrum-X Ethernet

Goal:

Create a unified GPU fabric.

---

AMD Instinct Ecosystem

Using:

• Infinity Fabric

• ROCm

• Ultra Ethernet Consortium standards

---

CXL Integration

Upcoming architectures may use:

• CXL Memory Pools

• Shared accelerator memory

• Memory disaggregation

Future hierarchy:

HBM
 ↓
NVLink Fabric
 ↓
CXL Memory Pool
 ↓
Storage

This is where AI infrastructure is moving.

---

Section 8: Cost Architecture Perspective

Architects care about TCO.

A useful discussion:

Architecture	Cost	Complexity	Scalability
PCIe Only	Low	Low	Limited
NVLink	Medium	Medium	High
InfiniBand	High	High	Very High
NVLink + InfiniBand	Highest	Highest	AI Factory Scale

The objective is not maximum bandwidth.

The objective is:

Performance per Dollar and Training Time per Dollar

---

Section 9: Design Recommendations

Enterprise AI

• PCIe Gen5

• 4 to 8 GPUs

• Ethernet

Suitable for:

• RAG

• Fine-tuning

• Enterprise inference

---

AI Research Labs

• NVLink

• 8 to 16 GPU systems

Suitable for:

• LLM training

• Foundation model experimentation

---

National Labs / Hyperscalers

• NVLink

• NVSwitch

• InfiniBand NDR/XDR

Suitable for:

• Trillion-parameter models

• Exascale AI

• Scientific HPC

---

Conclusion

A powerful ending would be:

In modern AI systems, GPUs are no longer isolated compute devices. They are nodes within a hierarchy of interconnected fabrics. PCIe provides system connectivity, NVLink builds high-speed GPU memory fabrics, and InfiniBand extends that fabric across thousands of servers. Architects designing next-generation AI factories must think beyond GPU specifications and focus on the data movement architecture that ultimately determines scalability, efficiency, and time-to-insight. 

Hybrid Computing: Why the Future of HPC Is No Longer CPU vs GPU, But CPU + GPU + QPU + DPU

For decades, CPUs were the center of computing. Then GPUs revolutionized AI, simulation, and scientific computing. Today, HPC systems are evolving even further, integrating specialized processors such as Quantum Processing Units (QPUs), Data Processing Units (DPUs), Tensor Processing Units (TPUs), Neural Processing Units (NPUs), and Language Processing Units (LPUs).

This model is known as Hybrid Computing, where different processors collaborate, each handling the tasks they perform best.

---

What Is Hybrid Computing in HPC?

Hybrid computing refers to combining multiple types of processors within the same system to accelerate complex workloads efficiently.

Instead of forcing a CPU to perform every task, modern systems distribute work intelligently across specialized accelerators:

Processor	Primary Role
CPU	Control, orchestration, scheduling, preprocessing
GPU	Massively parallel computation for AI, simulation, and analytics
QPU	Quantum algorithms for optimization and quantum simulations
DPU	Networking, storage, security, and infrastructure offload
TPU	Tensor operations optimized for AI training and inference
NPU	Energy-efficient AI inference at the edge
LPU	Ultra-low latency Large Language Model inference

The result is a system where every processor focuses on its strengths rather than competing for the same workload.

---

The Evolution of HPC

Traditional HPC architecture looked like this:

Application
     |
    CPU
     |
  Memory

Modern HPC evolved to:

Application
     |
 CPU + GPU

The emerging architecture now looks more like:

CPU + GPU + DPU + QPU + Specialized Accelerators

This transformation is driven by growing demands from:

• Large Language Models (LLMs)

• Scientific simulations

• Climate modeling

• Drug discovery

• Financial risk analysis

• Genomics

• Material science

• Digital twins

Each of these domains benefits from a different type of accelerator.

---

Hybrid Quantum-Classical Computing

One of the most exciting developments in HPC is the integration of Quantum Processing Units (QPUs).

A QPU is a specialized processor designed to perform quantum computations using qubits rather than classical bits.

However, quantum systems are not designed to replace classical HPC systems. Instead, they complement them.

A hybrid quantum-classical workflow typically operates as follows:

Problem Definition
         |
         v
 CPU/GPU Cluster
(Prepare Data)
         |
         v
 Quantum Algorithm
      on QPU
         |
         v
 Results Returned
         |
         v
 CPU/GPU Post Processing

The classical HPC environment manages:

• Workflow orchestration

• Data preparation

• Job scheduling

• Result verification

• Large-scale analytics

The QPU accelerates the specific portion of the problem where quantum mechanics offers potential advantages.

---

Where QPUs Can Help

Quantum computing excels at exploring massive solution spaces simultaneously.

Potential applications include:

Combinatorial Optimization

Finding optimal solutions among millions of possibilities.

Examples:

• Airline scheduling

• Supply chain optimization

• Manufacturing planning

• Vehicle routing

Quantum Chemistry

Simulating molecular interactions that are extremely difficult for classical computers.

Examples:

• Drug discovery

• Material science

• Catalyst design

Financial Modeling

Risk analysis requiring exploration of huge probability spaces.

Examples:

• Portfolio optimization

• Derivative pricing

• Market simulations

---

Why Hybrid Systems Matter

Despite the excitement around quantum computing, today's quantum processors face limitations:

• Limited qubit counts

• Noise and error rates

• Short coherence times

• Scalability challenges

As a result, quantum computers are currently most effective when integrated alongside classical HPC systems.

Hybrid architectures provide:

• The scalability of classical computing

• The computational advantages of quantum algorithms

• Practical deployment in real-world workloads

This is why vendors such as IBM, NVIDIA, Microsoft, Google, and Intel are investing heavily in hybrid quantum-classical infrastructure.

---

How CPUs, GPUs, DPUs, and QPUs Work Together

Building a powerful AI model is only part of the challenge. Equally important is choosing the processor architecture that best matches the workload. While CPUs and GPUs receive most of the attention, modern AI platforms increasingly rely on a diverse set of processors, each optimized for specific tasks.

• CPU serves as the control center, handling orchestration, scheduling, operating systems, and application management.

• GPU delivers massive parallel processing capabilities for AI training, inference, and scientific computing.

• TPU is specialized for tensor operations and large-scale machine learning workloads.

• NPU brings power-efficient AI inference to edge devices such as laptops, smartphones, and IoT systems.

• LPU is optimized for ultra-low-latency Large Language Model (LLM) inference and real-time AI interactions.

• DPU accelerates networking, storage, security, and infrastructure services, allowing CPUs and GPUs to focus on computation.

  
  

Consider a modern AI-driven scientific computing workflow.

CPU

The CPU acts as the coordinator.

Responsibilities include:

• Managing jobs

• Scheduling resources

• Workflow orchestration

• Data preprocessing

Think of the CPU as the project manager of the system.

---

GPU

The GPU performs large-scale parallel computation.

Examples:

• AI training

• Deep learning

• Matrix operations

• HPC simulations

Think of the GPU as a team of thousands of workers performing calculations simultaneously.

---

DPU

The DPU manages infrastructure tasks.

Examples:

• RDMA networking

• Security isolation

• Firewall functions

• Encryption

• Storage services

• NVMe-over-Fabrics

Think of the DPU as the operations and security team that keeps the data center running efficiently.

By offloading these responsibilities, GPUs spend more time computing and less time waiting for data.

---

QPU

The QPU tackles highly specialized quantum algorithms.

Examples:

• Quantum chemistry

• Optimization

• Quantum machine learning

Think of the QPU as a specialist consultant brought in to solve specific problems that classical systems struggle with.

---

Real-World Hybrid Workflow

Imagine a pharmaceutical company searching for a promising drug molecule.

Step 1: Data Preparation

CPU nodes collect and prepare molecular datasets.

Step 2: AI Screening

GPUs process millions of candidate molecules using AI models.

Step 3: Quantum Simulation

A QPU evaluates a small set of promising molecular interactions using quantum algorithms.

Step 4: Result Analysis

CPU and GPU resources analyze outcomes and rank candidates.

The entire workflow uses multiple processors working together, each contributing its unique strengths.

---

The Rise of Specialized Accelerators

The future of computing will not be built using only CPUs and GPUs.

New accelerators are emerging rapidly.

TPU (Tensor Processing Unit)

Designed specifically for tensor operations used in AI.

Commonly associated with Google Cloud AI workloads.

NPU (Neural Processing Unit)

Focused on efficient AI inference.

Commonly found in:

• Laptops

• Smartphones

• Edge devices

LPU (Language Processing Unit)

Optimized for low-latency generative AI and LLM inference.

Designed to reduce response times for conversational AI systems.

DPU (Data Processing Unit)

Handles infrastructure services such as:

• Networking

• Storage

• Security

• Telemetry

Allowing CPUs and GPUs to focus on applications and computation.

---

An Architect's Perspective

The key question is no longer:

Which processor should I choose?

The better question is:

Which processor should perform each task?

A successful architecture balances:

• Compute performance

• Memory capacity

• Network bandwidth

• Latency requirements

• Security requirements

• Power consumption

• Scalability targets

Modern HPC systems increasingly resemble an orchestra rather than a single instrument.

Each processor plays a different role, but together they create a far more powerful system.

A common misconception is that these processors compete with one another. In reality, they complement each other by accelerating different layers of the AI stack. Modern AI and HPC systems often combine several of these processors within the same architecture to maximize efficiency and performance.

The key architectural questions are not "Which processor is fastest?" but rather:

• What type of workload am I running?

• Is the workload training or inference?

• How much parallelism is required?

• What latency targets must be achieved?

• What are the power, memory, networking, and infrastructure constraints?

Choosing the appropriate processor architecture can significantly improve performance, efficiency, scalability, and cost effectiveness, while selecting the wrong one can become an expensive infrastructure mistake.

---

Final Thoughts

The future of HPC is not about replacing CPUs with GPUs or replacing GPUs with QPUs.

It is about combining the right processors for the right tasks.

CPUs orchestrate, GPUs accelerate computation, DPUs manage infrastructure, and QPUs solve specialized quantum problems. Together, they form a hybrid computing platform capable of tackling problems that would be impractical for any single processor architecture.

The future of AI infrastructure is not built around a single processor. It is built around heterogeneous computing, where CPUs, GPUs, TPUs, NPUs, LPUs, DPUs, and even QPUs work together, each accelerating the tasks they were designed to perform best.

The Fast Lane to AI and HPC: Direct GPU Communication with Storage and Network

Introduction

For years, architects have focused on adding more CPU cores, faster GPUs, and larger memory capacities. However, in modern AI training, HPC workloads, and large-scale analytics, the biggest bottleneck is often not computation.

It is data movement.

A modern NVIDIA H100 or Blackwell GPU can process massive amounts of data, but if every byte must travel through CPU memory before reaching the GPU, the CPU becomes a traffic controller rather than a compute engine.

This is where GPUDirect RDMA and GPUDirect Storage (GDS) change the game.

The core idea is simple:

Let the GPU communicate directly with storage and network devices without making unnecessary stops in CPU memory.

---

Traditional Data Path

Before GPUDirect, data traveled through several stages:

NVMe Storage
      |
      v
System Memory
      |
      v
CPU
      |
      v
GPU Memory

For network communication:

Remote Node
     |
     v
NIC
     |
     v
CPU Memory
     |
     v
CPU
     |
     v
GPU Memory

Every transfer requires:

• Additional memory copies

• CPU interrupts

• Context switches

• Higher latency

• Lower throughput

The CPU becomes a bottleneck.

---

Understanding the Diagram

The diagram shows two hosts connected through:

• RoCE (RDMA over Converged Ethernet)

• InfiniBand

Each host contains:

• CPU

• System Memory

• NVIDIA GPU

• NVMe Storage

• RDMA-capable NIC

The interesting part is that data bypasses the CPU.

---

GPUDirect Storage Explained

Look at the storage path in the diagram.

Instead of:

NVMe
 -> CPU Memory
 -> GPU Memory

GPUDirect Storage allows:

NVMe
 -> GPU Memory

directly through DMA.

What happens?

The NVMe drive performs a DMA transfer directly into GPU memory.

The CPU simply initiates the transfer.

The actual data never flows through CPU memory.

Benefits

• Lower latency

• Higher storage throughput

• Reduced CPU utilization

• Fewer memory copies

This is especially useful for:

• AI training datasets

• GenAI model training

• Scientific simulations

• Video analytics

• Climate modeling

---

GPUDirect RDMA Explained

Now look at the bottom of the diagram.

A traditional network transfer follows:

GPU
 -> CPU Memory
 -> NIC
 -> Network
 -> NIC
 -> CPU Memory
 -> GPU

Several copies occur during transmission.

With GPUDirect RDMA:

GPU Memory
      |
      v
Remote GPU Memory

through the RDMA-capable NIC.

The CPU sets up the transaction but is not involved in the actual data movement.

---

Why RDMA Matters

RDMA stands for:

Remote Direct Memory Access

It allows one system to read or write memory on another system without involving the remote CPU.

In the diagram:

GPU Memory (Host 1)
         |
      RDMA
         |
GPU Memory (Host 2)

This creates an extremely low-latency communication channel.

---

Why AI Clusters Need This

Consider a large language model training job.

Thousands of GPUs exchange:

• Gradients

• Weights

• Activations

• Checkpoints

every second.

Without RDMA:

GPU -> CPU -> NIC

With RDMA:

GPU -> NIC

The difference becomes enormous when scaled across:

• 512 GPUs

• 2048 GPUs

• 10000+ GPUs

This is why modern AI supercomputers use:

• NVIDIA Quantum InfiniBand

• NVIDIA Spectrum Ethernet

• Mellanox ConnectX NICs

• RoCE v2

along with GPUDirect RDMA.

---

DMA: The Unsung Hero

The diagram highlights DMA.

DMA means:

Direct Memory Access

DMA allows hardware devices to move data without CPU intervention.

Without DMA:

CPU copies every byte

With DMA:

Device transfers data directly

GPUDirect uses DMA extensively to create direct data paths.

---

Real-World Example

Imagine training a 70-billion-parameter LLM.

Dataset size:

50 TB

Traditional path:

NVMe
 -> RAM
 -> CPU
 -> GPU

Millions of memory copies occur.

Using GPUDirect Storage:

NVMe
 -> GPU Memory

The CPU is freed to manage orchestration rather than moving data.

This improves:

• GPU utilization

• Training time

• System efficiency

---

Architectural Benefits

1. Lower Latency

Fewer software layers.

Fewer memory copies.

Less waiting.

---

2. Reduced CPU Overhead

CPU cycles become available for:

• Scheduling

• Orchestration

• Monitoring

• Control-plane tasks

instead of moving data.

---

3. Better GPU Utilization

GPUs spend more time computing.

Less time waiting for data.

---

4. Higher Bandwidth

RDMA networks can deliver:

• 200 Gbps

• 400 Gbps

• 800 Gbps

with very low CPU involvement.

---

5. Improved Scalability

As cluster size grows, CPU bottlenecks become increasingly expensive.

GPUDirect allows architectures to scale more efficiently.

---

Why Architects Should Care

Many architects focus on:

• CPU speed

• Core count

• GPU count

The real performance challenge often lies elsewhere:

Data movement costs more than computation.

A GPU capable of 100+ TFLOPS is useless if it waits for data.

The purpose of GPUDirect is not to make the GPU faster.

The purpose is to ensure the GPU is never idle.

---

Evolution in system architecture.

Traditional systems were designed around the CPU being the center of all communication.

Modern AI and HPC systems are shifting to a new model:

• Storage talks directly to GPU memory.

• Network talks directly to GPU memory.

• CPU becomes the coordinator, not the data mover.

This architectural shift is one of the key reasons why modern AI supercomputers achieve unprecedented scalability and efficiency.

Understanding the Building Blocks of Persistent Memory in Linux : PMEM DAX Folios

  

Introduction

When debugging DAX-related issues, engineers often encounter terms like:

• PMEM

• devdax

• fsdax

• folio

• compound page

• page order

• MAX_FOLIO_ORDER

• vmemmap_shift

This blog explains how these concepts fit together and why a seemingly simple operation such as creating a DAX namespace can trigger complex memory-management behavior inside the kernel.

---

What is Persistent Memory (PMEM)?

Persistent Memory (PMEM) is storage that retains data across power cycles while being directly addressable like memory.

Traditionally:

Technology	Fast	Persistent
DRAM	✅ Yes	❌ No
SSD/HDD	❌ No	✅ Yes
PMEM	✅ Yes	✅ Yes

PMEM combines characteristics of both memory and storage.

Examples include:

• Intel Optane Persistent Memory

• NVDIMM

• Future CXL-attached persistent memory devices

  
  
  
  

---

Where is PMEM Located?

Think of PMEM as being physically attached to the memory subsystem rather than accessed through traditional storage controllers.

Traditional storage path:

Application
      ↓
Page Cache
      ↓
SSD / NVMe

----------------------

PMEM path:

Application
      ↓
CPU Memory Bus
      ↓
Persistent Memory

On Power systems and x86 systems, PMEM is exposed to Linux as a memory region that can be mapped directly into applications.

You can view PMEM devices using:

ndctl list

or

daxctl list

---

What is DAX?

DAX stands for Direct Access.

DAX allows applications to bypass the page cache and directly access persistent memory.

Without DAX:

Application
      ↓
Page Cache
      ↓
Storage Device

With DAX:

Application
      ↓
Persistent Memory

This removes:

• Extra memory copies

• Page cache overhead

• Additional latency

Benefits:

• Faster databases

• Lower latency

• Better HPC performance

• Reduced memory consumption

---

Types of DAX

1. Filesystem DAX (fsdax)

Uses a filesystem such as:

• XFS

• ext4

Example:

PMEM
 └── XFS (DAX enabled)
      └── File

Applications access files directly from persistent memory.

---

2. Device DAX (devdax)

Provides direct raw access to the PMEM device.

Example:

/dev/dax0.0

Applications mmap() the region directly.

No filesystem is involved.

Creation example:

ndctl create-namespace \
    --mode=devdax

---

What is a Folio?

A folio is a Linux kernel memory object used to represent one or more physically contiguous pages.

Before folios:

struct page

represented every individual page.

Now:

struct folio

can represent:

• 1 page

• 2 pages

• 512 pages

• 16384 pages

as a single unit.

Benefits:

• Less memory-management overhead

• Better performance

• Improved page-cache scalability

---

Understanding Page Order

Linux measures compound page size using order.

Formula:

Size = PAGE_SIZE × 2^order

For ppc64le:

PAGE_SIZE = 64 KB

Examples:

Order	Size
0	64 KB
1	128 KB
2	256 KB
4	1 MB
5	2 MB
10	64 MB
14	1 GB

For example:

64 KB × 2^14
= 1 GB

Therefore:

Order 14 = 1 GB object

---

What is a Compound Page?

A compound page is a collection of many base pages treated as a single memory object.

Example:

Compound Page
     │
 ├── Page 0
 ├── Page 1
 ├── Page 2
 ├── ...
 └── Page N

Instead of handling thousands of pages individually, Linux manages them as a single object.

This improves:

• TLB efficiency

• Memory management

• Mapping performance

---

What is a DAX Folio?

A DAX folio is simply a folio backing DAX memory.

Normal memory:

RAM
 ↓
Folio

DAX memory:

Persistent Memory
      ↓
ZONE_DEVICE
      ↓
DAX Folio

A DAX folio represents PMEM pages that can be directly mapped into applications.

---

How a 1 GB DAX Folio Gets Created

Suppose we create:

ndctl create-namespace \
    --mode=devdax \
    -a 1073741824

Notice:

1073741824 = 1 GB

The kernel interprets this alignment requirement and calculates:

PAGE_SHIFT = 16

because:

64 KB pages

and

1 GB = 2^30

Therefore:

Order = 30 - 16
       = 14

Linux now attempts to create:

Order-14 folio

which represents:

16384 pages

or:

1 GB DAX folio

---

Why Does MAX_FOLIO_ORDER Matter?

The kernel limits how large a folio can be.

Inside memory management code:

folio_set_order(folio, order);

Linux verifies:

VM_WARN_ON_ONCE(order > MAX_FOLIO_ORDER);

This prevents creation of excessively large folios.

If:

Order = 14

but

MAX_FOLIO_ORDER < 14

then Linux generates:

VM_WARN_ON_ONCE()

indicating the folio exceeds supported limits.

---

Real Example

User creates:

ndctl create-namespace \
    --mode=devdax \
    -a 1073741824

Kernel receives:

1 GB aligned PMEM range

Kernel computes:

Order 14 folio

During initialization:

memremap_pages()
    ↓
memmap_init_zone_device()
    ↓
memmap_init_compound()
    ↓
prep_compound_head()
    ↓
folio_set_order()

Kernel checks:

order > MAX_FOLIO_ORDER

and generates a warning if the architecture does not support such a large folio.

---

Commands Useful for DAX Investigation

Commands Useful for DAX Investigation

List PMEM devices

ndctl list

Show namespaces

ndctl list -N

Show DAX devices

daxctl list

View PMEM regions

ndctl list -R

Check memory block size

getconf PAGE_SIZE

Check DAX device

ls -l /dev/dax*

View DAX mappings

dmesg | grep dax

View compound page warnings

dmesg | grep -i folio

---

Key Takeaways

• PMEM is persistent memory that combines storage persistence with memory-like access.

• DAX allows direct access to PMEM without page cache involvement.

• devdax exposes PMEM as a raw memory device.

• Folios are Linux memory objects representing one or more pages.

• Order defines folio size as PAGE_SIZE × 2^order.

• On ppc64le with 64 KB pages, a 1 GB DAX alignment translates to order 14.

• The kernel creates a DAX folio to represent that PMEM region.

• If the calculated order exceeds MAX_FOLIO_ORDER, Linux triggers a warning during DAX initialization.

Final One-Line Summary DAX device

ls -l /dev/dax*

View DAX mappings

dmesg | grep dax

View compound page warnings

dmesg | grep -i folio

---

Key Takeaways

• PMEM is persistent memory that combines storage persistence with memory-like access.

• DAX allows direct access to PMEM without page cache involvement.

• devdax exposes PMEM as a raw memory device.

• Folios are Linux memory objects representing one or more pages.

• Order defines folio size as PAGE_SIZE × 2^order.

• On ppc64le with 64 KB pages, a 1 GB DAX alignment translates to order 14.

• The kernel creates a DAX folio to represent that PMEM region.

• If the calculated order exceeds MAX_FOLIO_ORDER, Linux triggers a warning during DAX initialization.

DAX enables direct access to persistent memory, and a DAX folio is the kernel's representation of that PMEM region; large DAX alignments can create large compound folios whose order may exceed architecture-specific folio limits.

CUDA Meets DOCA : One Trains the AI, The Other Keeps It Running

Introduction

When people talk about AI infrastructure, the conversation almost always starts with GPUs. Terms like LLM training, inference acceleration, CUDA cores, and Tensor Cores have become common across the industry. As a result, many engineers are familiar with CUDA, NVIDIA's software platform that unlocks the power of GPU computing.

However, there is another NVIDIA technology quietly becoming essential in modern AI factories, hyperscale data centers, and HPC environments: DOCA.

While CUDA accelerates the computation performed by GPUs, DOCA accelerates the networking, storage, security, and infrastructure services that keep thousands of GPUs efficiently connected and utilized.

To understand the future of AI infrastructure, it's important to understand both.

---

What is CUDA?

CUDA (Compute Unified Device Architecture) is NVIDIA's parallel computing platform and programming model that allows software developers to leverage GPUs for general-purpose computing.

Before CUDA, graphics processors were primarily used for rendering images. CUDA transformed GPUs into powerful computing engines capable of executing thousands of parallel threads.

Key CUDA Use Cases

• AI and Deep Learning Training

• Large Language Model (LLM) Training

• Model Inference

• Scientific Simulations

• Computational Physics

• Weather Forecasting

• Financial Modeling

• High Performance Computing (HPC)

Frameworks such as:

• PyTorch

• TensorFlow

• RAPIDS

• cuDNN

• NCCL

all rely heavily on CUDA underneath.

Think of CUDA as:

The software layer that tells GPUs how to perform massive parallel computations.

Without CUDA, modern AI training at scale would be dramatically slower and more difficult.

---

What is DOCA?

DOCA (Data Center Infrastructure-on-a-Chip Architecture) is NVIDIA's software framework for programming BlueField DPUs (Data Processing Units).

As AI clusters continue growing from tens of GPUs to thousands of GPUs, infrastructure challenges become significant:

• Network congestion

• Storage bottlenecks

• Security overhead

• Packet processing

• Telemetry collection

• Multi-tenant isolation

Traditionally these tasks consumed valuable CPU resources.

DOCA allows these services to be offloaded to the DPU, freeing CPU cores for application workloads while improving overall data center efficiency.

Key DOCA Use Cases

• RoCE and RDMA acceleration

• Network packet processing

• Zero-trust security

• Firewall services

• Storage virtualization

• Data path acceleration

• Network telemetry

• Service isolation

• SmartNIC applications

Think of DOCA as:

The software layer that tells DPUs how to accelerate infrastructure services.

---

Understanding the Difference

A simple way to understand the distinction is by looking at where the work happens.

Technology	Hardware	Focus Area
CUDA	GPU	Compute acceleration
DOCA	DPU	Infrastructure acceleration
CPU	Processor	General purpose processing

CUDA Accelerates

• Matrix operations

• Neural network training

• Deep learning inference

• Scientific computation

DOCA Accelerates

• Network traffic

• Storage access

• Data movement

• Security processing

• Infrastructure management

  

---

The AI Factory Analogy

Imagine a modern AI factory with 8,000 GPUs training a trillion-parameter model.

CUDA's Job

CUDA helps every GPU perform billions of mathematical operations per second.

It powers:

• Tensor operations

• Backpropagation

• Gradient updates

• Model inference

In short:

CUDA helps the AI think.

DOCA's Job

DOCA ensures data reaches GPUs efficiently and securely.

It handles:

• GPU-to-GPU communication

• RDMA transfers

• Storage connectivity

• Flow monitoring

• Security enforcement

In short:

DOCA keeps the AI factory running smoothly.

---

Why DOCA Matters More Than Ever

In early HPC clusters, compute was the primary bottleneck.

Today, many AI environments struggle more with:

• Data movement

• Network latency

• Storage throughput

• Security overhead

When thousands of GPUs communicate simultaneously, inefficient infrastructure can leave expensive accelerators sitting idle.

For example:

A cluster with 1,024 GPUs may deliver only a fraction of its potential performance if networking becomes saturated.

This is where DOCA and BlueField DPUs become critical.

They move infrastructure functions away from CPUs and closer to the network, reducing latency and improving scalability.

---

CUDA + DOCA: Better Together

Modern AI systems need both technologies.

CUDA Provides

1. GPU acceleration
2. AI training performance
3. AI inference performance
4. Scientific computing acceleration

DOCA Provides

1. Networking acceleration
2. Storage acceleration
3. Security offload
4. Infrastructure isolation
5. Telemetry and monitoring

Together they create a complete AI platform:

Application Layer
       │
AI Frameworks (PyTorch, TensorFlow)
       │
CUDA → GPU Acceleration
       │
High-Speed Fabrics (RoCE, InfiniBand)
       │
DOCA → DPU Infrastructure Acceleration
       │
Storage, Network, Security Services

---

Why Infrastructure Engineers Should Care

For years, infrastructure teams focused on servers, networks, and storage while AI teams focused on GPUs.

That separation is rapidly disappearing.

Modern AI deployments require engineers who understand:

• GPUs

• DPUs

• RDMA

• RoCE

• InfiniBand

• Storage fabrics

• Cluster networking

Understanding CUDA alone explains how AI workloads run.

Understanding DOCA explains how AI infrastructure scales.

-----------

Restaurant Example

Imagine a restaurant:

• GPU = Expert Chef

  • Cooks food very fast

  • Focuses only on cooking

• CPU = Manager

  • Handles orders

  • Coordinates staff

  • Manages operations

• DPU = Support Staff

  • Takes orders

  • Handles billing

  • Manages security

  • Delivers food

Without support staff, the chef keeps getting interrupted.

Chef cooking
↓
Customer arrives
↓
Chef answers phone
↓
Chef processes payment
↓
Chef serves food
↓
Back to cooking

The chef becomes less productive.

A DPU does the same thing for a server.

---

In a Server

Traditionally the CPU does:

• Networking

• Security

• Firewall

• Encryption

• Storage I/O

• Virtual networking

• Kubernetes networking

• Monitoring

and also runs applications.

CPU
├─ Application
├─ Networking
├─ Encryption
├─ Storage
├─ Security
└─ Monitoring

CPU becomes overloaded.

---

What a DPU Does

A DPU takes infrastructure tasks away from the CPU.

CPU
└─ Runs applications

GPU
└─ Runs AI / HPC

DPU
├─ Networking
├─ Security
├─ Storage
├─ Telemetry
└─ Virtualization

Now each processor focuses on its specialty.

---

Networking Example

Suppose your AI cluster has:

1000 GPUs

During training, GPUs exchange gradients every few milliseconds.

Without DPU:

GPU
 ↓
CPU
 ↓
NIC
 ↓
Network

CPU must handle:

• Packet creation

• Packet checking

• Interrupts

• Network queues

Result:

CPU utilization = High
Latency = Higher
GPU waiting = More

---

With BlueField DPU

GPU
 ↓
DPU
 ↓
Network

The DPU handles:

• RDMA

• RoCE

• Packet processing

• Network scheduling

CPU stays free.

Benefits:

• Lower latency

• Faster AI training

• Better GPU utilization

---

Security Example

Consider a cloud provider.

You rent a server.

You should never be able to access another customer's data.

The DPU acts like a security guard.

Without DPU:

CPU runs applications
CPU runs firewall
CPU runs security
CPU runs encryption

If CPU is compromised, security may also be compromised.

---

With DPU

Application
    ↓
   CPU

Security
Firewall
Isolation
Encryption
    ↓
   DPU

Even if somebody compromises the operating system:

Linux OS hacked

the DPU can still enforce:

• Network isolation

• Firewall rules

• Encryption

• Tenant separation

This concept is known as:

Infrastructure Isolation

---

Real AI Cluster Example

Imagine an NVIDIA DGX SuperPOD.

AI training traffic:

100s of GB/s

Without DPU:

GPU → CPU → Network

With DPU:

GPU → DPU → Network

The CPU no longer becomes the bottleneck.

This is why hyperscalers use DPUs.

Examples:

• NVIDIA BlueField

• AWS Nitro

• Microsoft Azure SmartNIC

• Google Infrastructure Processing Unit (IPU)

All are variations of the same concept.

Suppose a 500 TB training dataset is stored in NVMe storage.

Traditional flow:

NVMe
 ↓
CPU
 ↓
GPU

CPU copies data.

With DPU + GPUDirect Storage:

NVMe
 ↓
DPU
 ↓
GPU Memory

Benefits:

• Less copying

• Lower latency

• Reduced CPU overhead

• Faster training startup

---

Simple Analogy

Think of a modern AI server as:

GPU = Scientist
CPU = Project Manager
DPU = Operations Team

You do not want your scientist:

• Checking badges

• Managing security

• Dispatching trucks

• Moving files

You want the scientist doing science.

• GPU should do computation

• CPU should coordinate

• DPU should run infrastructure

---

A DPU is to a data center what a dedicated operations and security team is to a company: it offloads networking, storage, and security services from CPUs so GPUs can spend more time computing and less time waiting for data.

What is GPU and GPU Memory: 

Component	Human Analogy	Purpose
GPU	Brain	Performs calculations
GPU Memory (HBM/GDDR)	Short-term memory	Stores data that the GPU is currently working on

Simple Example

Imagine you're solving a large Sudoku puzzle.

CPU World

• CPU = You

• RAM = Table where papers are placed

GPU World

• GPU = 10,000 assistants solving parts of the puzzle simultaneously

• GPU Memory = The desk space available directly beside those assistants

The assistants (GPU) need a place to keep the numbers they're processing.

That place is GPU Memory.

---

Physical View

A modern NVIDIA H100 looks roughly like this:

+-----------------------------------+
|                                   |
|  HBM Memory   GPU   HBM Memory    |
|                                   |
|  HBM Memory       HBM Memory      |
|                                   |
+-----------------------------------+

The large center chip is:

GPU

Surrounded by:

HBM Memory Chips

connected through extremely fast links.

---

What Does GPU Do?

The GPU performs:

• AI training

• AI inference

• Matrix multiplication

• Scientific computing

• Image rendering

• HPC simulations

Example:

2 + 2 = 4

CPU may perform a few calculations quickly.

GPU performs:

Millions or billions

of such calculations simultaneously.

---

What Does GPU Memory Do?

GPU memory stores:

• AI model weights

• Training data

• Activations

• Tensors

• Intermediate results

For example, during LLM training:

Llama
GPT
Granite

model parameters reside in GPU memory.

The GPU continuously reads and updates them.

---

Why Not Use System RAM?

System RAM is much slower.

Example:

Memory Type	Bandwidth
DDR5 System RAM	~50-100 GB/s
HBM3 GPU Memory	~3-5 TB/s

That's about:

3000-5000 GB/s

for HBM.

A modern GPU can consume data much faster than CPU RAM can provide.

---

Data Flow Example

Without GPUDirect:

NVMe
  ↓
System RAM
  ↓
CPU
  ↓
GPU Memory
  ↓
GPU

With GPUDirect Storage:

NVMe
  ↓
GPU Memory
  ↓
GPU

Notice:

The GPU still computes.

The GPU Memory still stores data.

GPUDirect simply moves data into GPU Memory more efficiently.

---

Real NVIDIA H100 Example

An NVIDIA H100 contains roughly:

GPU Compute Engine
+
80 GB HBM3 Memory

The GPU and memory are separate hardware blocks but packaged together.

Think of:

GPU = Processor
HBM = Local Memory

just like:

CPU = Processor
RAM = System Memory

---

Architect View

In the diagram you were studying:

GPU Memory ←→ GPU

The GPU Memory holds:

• AI model parameters

• Training batches

• Input tensors

The GPU performs:

• Matrix operations

• Tensor operations

• AI computations

The GPU is the worker.

The GPU Memory is the workspace.

---

Conclusion

The future of AI is not just about faster GPUs. It is equally about building efficient infrastructure capable of feeding thousands of GPUs with data, securely and at scale.

CUDA accelerates the compute.

DOCA accelerates the infrastructure.

As AI factories continue to grow, engineers who understand both technologies will be better equipped to design, optimize, and troubleshoot next-generation AI, HPC, and cloud environments.

Database Selection Guide

 Database Landscape Overview

Modern infrastructure platforms — especially large-scale Linux test automation systems — require careful selection of the right database technology. Choosing incorrectly leads to scalability bottlenecks, operational complexity, and poor query performance. This guide maps each database category to its ideal use case, special features, and trade-offs.

Who This Is ForDatabase architects, platform leads, and customer-specific infrastructure teams evaluating database choices for storing skill files, Avocado/op-test metadata, Bugzilla defects, Jira tasks, CI/CD execution history, and AI-generated testcase mappings.

Master Database Classification Table

Database	Category	SQL/NoSQL	Primary Strength	Best For
MariaDB	RDBMS	SQL	Mature relational engine, drop-in MySQL replacement	Small/medium deployments, traditional workloads
PostgreSQL	RDBMS	SQL	Advanced SQL, extensibility, JSONB support	Complex queries, open-source enterprise
EDB Postgres	Enterprise PostgreSQL	SQL	Oracle compatibility + enterprise support	Oracle migrations, enterprise SQL compliance
CockroachDB	Distributed SQL (NewSQL)	NewSQL	Automatic sharding, multi-region, cloud-native	Distributed SQL with resilience & geo-routing
YugabyteDB	Distributed SQL (NewSQL)	NewSQL	PostgreSQL-compatible distributed SQL	PostgreSQL at large/distributed scale
MongoDB	Document DB	NoSQL	Flexible schema, rich document queries	Semi-structured data, rapid iteration
Aerospike	Key-Value DB	NoSQL	Sub-millisecond latency, high throughput	Real-time analytics, ad-tech, fraud detection
Cassandra	Wide Column DB	NoSQL	Write-heavy workloads, massive scale	Time-series, event logging, IoT
Redis	In-Memory Key-Value	NoSQL	Ultra-fast in-memory store, pub/sub	Caching, session management, fast cache
GridGain	In-Memory Data Grid	Hybrid	Distributed in-memory compute + SQL over cache	In-memory cache layer, distributed computing

2. Database Taxonomy — Architecture Diagram

DATABASE CLASSIFICATION TREE

3. Traditional SQL Databases

Traditional SQL databases provide ACID-compliant relational data management with mature ecosystems, familiar query languages, and well-understood operational patterns. Best suited for structured data with defined schemas.

MariaDB

MariaDB — Classic RDBMS

Drop-in MySQL replacement with strong community support. Ideal for small to medium deployments. Familiar SQL interface, mature replication, and low operational overhead. Best choice when simplicity matters.

PostgreSQL

PostgreSQL — Advanced Open Source SQL

Highly extensible with JSONB, full-text search, window functions, CTEs, and advanced indexing. Strong choice for complex query workloads and data-rich platforms. The de-facto open-source enterprise RDBMS.

EDB Postgres

EDB Postgres — Enterprise PostgreSQL

Commercial distribution of PostgreSQL adding Oracle compatibility, 24×7 vendor support, enterprise HA/DR tools, security enhancements, and compliance certifications. The bridge for Oracle-to-PostgreSQL migrations.

EDB vs PostgreSQL — Feature Comparison

Feature	PostgreSQL	EDB Postgres
Open Source	✓ Yes	Core is PostgreSQL
SQL Support	✓ Yes	✓ Yes
Oracle Compatibility	~ Limited	✓ Strong
Enterprise Support	Community only	✓ Commercial SLA
HA / DR Tools	Community tools (Patroni, pgBackRest)	Enterprise tools (EDB Failover Manager)
Security Enhancements	Standard	Advanced audit, encryption
Cost	✓ Free	Subscription

When to Choose EDBChoose EDB when you need Oracle migration assistance, enterprise-grade support contracts, compliance certifications (SOC 2, HIPAA), or when your team lacks the internal expertise to operate community PostgreSQL at scale.

4. Distributed SQL (NewSQL) Databases

NewSQL databases combine the scalability characteristics of NoSQL systems with full ACID transactional SQL semantics. They are designed for cloud-native, multi-region, horizontally scaled deployments where traditional RDBMS hits vertical scaling limits.

CockroachDB

CockroachDB — Cloud-Native Distributed SQL

Combines NoSQL-style horizontal scaling with full SQL, ACID transactions, automatic sharding, and multi-region active-active deployment. Inspired by Google Spanner. Resilient to node and zone failures with no manual intervention required.

YugabyteDB

YugabyteDB — PostgreSQL-Compatible Distributed SQL

Highest PostgreSQL wire-protocol compatibility among distributed SQL engines. Supports both YSQL (PostgreSQL API) and YCQL (Cassandra API). Ideal when migrating an existing PostgreSQL application to distributed scale without rewriting queries.

CockroachDB — Key Characteristics

Feature	Support	Details
Horizontal Scaling	✓	Add nodes without downtime; automatic rebalancing
Distributed Architecture	✓	Shared-nothing; each node is a full peer
Multi-Region Deployment	✓	Geo-partitioning, follower reads, global tables
Automatic Sharding	✓	Range-based sharding; no manual partitioning
SQL Interface	✓	PostgreSQL-compatible SQL dialect
ACID Transactions	✓	Serializable isolation by default
Joins	✓	Full JOIN support including distributed joins
Schemas & Tables	✓	Standard relational schemas

CockroachDB Is NOT NoSQLDespite having horizontal scaling and distributed architecture (characteristics often associated with NoSQL), CockroachDB is a Distributed SQL (NewSQL) database. It provides full SQL, ACID transactions, JOINs, and relational schemas — making it fundamentally different from MongoDB or Cassandra.

EDB vs CockroachDB vs YugabyteDB — Decision Matrix

Feature	EDB Postgres	CockroachDB	YugabyteDB
PostgreSQL Compatible	✓ Yes	~ Partial	✓ Very High
Distributed by Default	✗ No	✓ Yes	✓ Yes
Oracle Migration	✓ Excellent	✗ Limited	✗ Limited
Multi-Region Native	~ Via extensions	✓ Native	✓ Native
Traditional Enterprise DB	✓ Yes	✗ No	✗ No
Horizontal Scaling	~ Limited	✓ Excellent	✓ Excellent
ACID Transactions	✓ Yes	✓ Yes	✓ Yes

5. NoSQL Databases

NoSQL databases sacrifice strict relational schema constraints in favour of flexible data models, horizontal scalability, and tunable consistency. Each NoSQL sub-type is optimised for a specific access pattern.

MongoDB

MongoDB — Document Database

Stores data as JSON-like BSON documents. Flexible schema allows rapid iteration. Rich query language with aggregation pipelines. Best for semi-structured data where schema evolves frequently.

Aerospike

Aerospike — High-Performance Key-Value

Designed for sub-millisecond latency at massive throughput. Hybrid memory architecture (RAM + SSD). Data model: Namespace → Set → Key → Bins (JSON-like). Perfect for real-time analytics and fraud detection.

Cassandra

Cassandra — Wide-Column Store

Optimised for write-heavy, time-series, and high-availability workloads. Masterless ring topology ensures no single point of failure. Tunable consistency (AP-focused). Best for large-scale event logging.

Redis

Redis — In-Memory Key-Value Store

Microsecond read/write latency. Supports strings, hashes, lists, sets, sorted sets, streams, and pub/sub. Primarily used as a fast cache, session store, or execution state bus. Persistence is optional.

Aerospike — Deep Dive

Aerospike is a high-performance distributed NoSQL database with a unique hybrid memory architecture that keeps indexes in RAM while storing data on NVMe SSDs — delivering sub-millisecond latency without requiring all data to fit in RAM.

Attribute	Detail
Data Model	Namespace → Set → Key → Bins (similar to JSON fields)
Latency	Sub-millisecond (often <1 ms) for reads and writes
Throughput	Millions of operations per second per node
Scaling	Horizontal (add nodes, auto-rebalance)
Consistency	Strong consistency available (SC mode)
SQL Support	Limited — via connectors and secondary indexes
Ideal Use Cases	Fraud detection, ad-tech, recommendation engines, session storage, real-time analytics, execution state tracking

Aerospike Data Model vs Traditional SQL

AEROSPIKE RECORD MODEL VS MARIADB/POSTGRESQL ROW

AEROSPIKE — NAMESPACE / SET / KEY / BINS MODEL

6. In-Memory Data Grid — GridGain

GridGain (built on Apache Ignite) occupies a unique category — it is primarily an In-Memory Data Grid (IMDG) and distributed computing platform, not a traditional database. It sits between the application and the underlying database as a high-speed distributed cache and compute layer.

GridGain / Apache Ignite

GridGain — In-Memory Data Grid

Keeps "hot" data in distributed RAM across a cluster for sub-millisecond access. Supports ANSI SQL queries, JDBC/ODBC, ACID transactions, and key-value APIs. Often deployed as a caching layer in front of MariaDB, PostgreSQL, Oracle, or NoSQL stores.

Feature	GridGain (Apache Ignite)
Architecture	Distributed In-Memory Data Grid + Compute Grid
SQL Support	✓ ANSI SQL, JDBC/ODBC
ACID Transactions	✓ Supported
Key-Value API	✓ Supported
Data Location	Primarily RAM; optional persistent store to disk
Typical Deployment	Between application tier and backend database (cache layer)
Integration	MariaDB, PostgreSQL, Oracle, NoSQL stores as backing store
Use Cases	Distributed caching, in-memory compute, hot data acceleration

GridGain is NOT a replacement for MariaDB or CockroachDBGridGain accelerates access to frequently-read data by keeping it in distributed RAM. It synchronises with an underlying persistent database. Use it as a compute/cache tier, not as your system of record.

7. Layered Database Architecture for Linux Test Platforms

RECOMMENDED LAYERED DATABASE ARCHITECTURE — LINUX TEST INFRASTRUCTURE

 

8. Linux Test Platform — Database Selection by Use Case

For a large-scale Linux Test Infrastructure platform storing skill files, Avocado test metadata, op-test metadata, Bugzilla defects, Jira tasks, CI/CD execution history, and AI-generated testcase mappings — the following mapping applies:

Data Type	Recommended DB	Category	Reason
Skill Files	PostgreSQL / EDB	SQL	Structured metadata, versioning, relational queries
Avocado Test Metadata	PostgreSQL / MariaDB	SQL	Schema is known; relational joins across testcases
Op-test Metadata	MariaDB / PostgreSQL	SQL	Small/medium scale; traditional RDBMS sufficient
Bugzilla Defects	EDB / PostgreSQL	SQL	Relational data with complex queries and audit trail
Jira Tasks	EDB / PostgreSQL	SQL	Structured project data; reporting and analytics
CI/CD Execution History	CockroachDB / YugabyteDB	NewSQL	High volume writes; distributed across regions/sites
AI-Generated Testcase Mappings	YugabyteDB / CockroachDB	NewSQL	PostgreSQL-compatible + scales as AI data grows
Execution State / Real-time Cache	Aerospike / Redis	NoSQL	Sub-millisecond access; cache running test states
Hot Data Acceleration	GridGain	IMDG	In-memory compute layer for frequently-queried data

9. Architect Decision Guide

Need Oracle replacement?

→

EDB Postgres

Need enterprise open-source SQL?

→

PostgreSQL / EDB

Need small/medium traditional SQL?

→

MariaDB

Need cloud-native distributed SQL?

→

CockroachDB

Need PostgreSQL-compatible at scale?

→

YugabyteDB

Need sub-millisecond real-time data?

→

Aerospike

Need fast cache / session state?

→

Redis

Need distributed in-memory compute?

→

GridGain (Apache Ignite)

Need massive write-heavy time-series?

→

Cassandra

10. Full Feature Comparison Matrix

Feature	MariaDB	PostgreSQL	EDB	CockroachDB	YugabyteDB	MongoDB	Aerospike	Cassandra	Redis	GridGain
SQL Support	✓	✓	✓	✓	✓	✗	~	~	✗	✓
ACID Transactions	✓	✓	✓	✓	✓	~	~	✗	✗	✓
Horizontal Scaling	✗	✗	~	✓	✓	✓	✓	✓	✓	✓
Multi-Region	✗	✗	~	✓	✓	✓	✓	✓	~	✓
Sub-ms Latency	✗	✗	✗	✗	✗	✗	✓	✗	✓	✓
Schema Flexibility	✗	~	~	✗	~	✓	✓	✓	✓	~
In-Memory Primary	✗	✗	✗	✗	✗	✗	~	✗	✓	✓
Oracle Compat.	✗	~	✓	✗	✗	✗	✗	✗	✗	✗
Open Source	✓	✓	Core OSS	BSL/CCL	✓	✓	✓	✓	✓	✓

✓ = Full support  |  ~ = Partial / via extension  |  ✗ = Not supported natively

11. Quick Mental Model — Memory Trick for Teams

Database	One-Line Mental Model
MariaDB	Traditional SQL — reliable workhorse, simple operations
PostgreSQL	Advanced SQL — the most capable open-source RDBMS
EDB Postgres	PostgreSQL + Oracle migration + enterprise support contract
CockroachDB	Distributed SQL — cockroach survives anything; so does your database
YugabyteDB	PostgreSQL that scales horizontally across data centres
MongoDB	Document store — flexible JSON, no rigid schema needed
Aerospike	Speed demon — sub-millisecond, for real-time decisions
Cassandra	Write firehose — absorbs millions of events per second
Redis	RAM-first — fastest possible cache and state store
GridGain	Turbocharger — in-memory layer sitting in front of your real DB

12. Architect Selection Checklist

• Define whether your workload is OLTP (transactional), OLAP (analytical), or mixed — this determines SQL vs NoSQL vs IMDG
• Assess current scale (rows/records count, QPS) and projected growth over 3 years
• Determine if multi-region or geo-distributed deployment is required (forces NewSQL or NoSQL choice)
• Identify if sub-millisecond latency is required (forces Aerospike or Redis into the stack)
• Evaluate whether Oracle compatibility or Oracle migration path is needed (forces EDB Postgres)
• Check if PostgreSQL wire-protocol compatibility is required for existing tooling (YugabyteDB > CockroachDB)
• Assess team SQL expertise — NoSQL requires different operational skills than RDBMS
• Review licensing: CockroachDB uses BSL/CCL; confirm compliance with your legal team
• Plan for dual-layer architecture: persistent DB (MariaDB/CockroachDB/EDB) + cache layer (Redis/Aerospike)
• Evaluate GridGain only if in-memory distributed compute acceleration is required — not as a primary store
• For a small/medium test platform: start with PostgreSQL/MariaDB for metadata, add Aerospike/Redis for execution state
• For large-scale distributed test platforms: upgrade persistent tier to CockroachDB or YugabyteDB as volume grows

13. Database Architecture Block Diagrams

Each diagram below illustrates the internal architecture and data flow of a specific database — from the client application down to persistent storage. Use these as a quick visual reference when evaluating options with your team.

13.1 MariaDB — Traditional Relational Architecture

MARIADB — CLIENT → PARSER → STORAGE ENGINE → DISK

13.2 PostgreSQL — Advanced RDBMS Architecture

POSTGRESQL — POSTMASTER → BACKEND PROCESS → SHARED BUFFERS → WAL → DATA FILES

13.3 EDB Postgres — Enterprise PostgreSQL Architecture

EDB POSTGRES — POSTGRESQL CORE + ENTERPRISE ADD-ONS + ORACLE COMPATIBILITY LAYER

13.4 CockroachDB — Distributed SQL Architecture

COCKROACHDB — SQL LAYER → KV LAYER → RAFT CONSENSUS → DISTRIBUTED STORAGE NODES

13.5 YugabyteDB — Distributed PostgreSQL Architecture

YUGABYTEDB — YSQL/YCQL API → DOCDB STORAGE → RAFT REPLICATION

13.6 MongoDB — Document Database Architecture

MONGODB — DRIVER → MONGOS → CONFIG SERVERS → SHARDS → WIREDTIGER STORAGE

13.7 Aerospike — High-Performance Key-Value Architecture

AEROSPIKE — CLIENT → SMART CLIENT LAYER → CLUSTER NODES → HYBRID MEMORY (RAM + NVME)

13.8 Apache Cassandra — Wide-Column Store Architecture

CASSANDRA — DRIVER → COORDINATOR NODE → TOKEN RING → SSTABLE STORAGE

13.9 Redis — In-Memory Key-Value Store Architecture

REDIS — SINGLE-THREADED EVENT LOOP → IN-MEMORY DATA STRUCTURES → OPTIONAL PERSISTENCE

13.10 GridGain / Apache Ignite — In-Memory Data Grid Architecture

GRIDGAIN — APPLICATION → SQL/KEY-VALUE API → DISTRIBUTED MEMORY GRID → BACKEND DB (OPTIONAL PERSISTENCE)

14. References & Further Reading

Resource	URL
MariaDB Documentation	mariadb.com/kb/en/
PostgreSQL Documentation	postgresql.org/docs/
EDB Postgres Documentation	enterprisedb.com/docs/
CockroachDB Documentation	cockroachlabs.com/docs/
YugabyteDB Documentation	docs.yugabyte.com/
MongoDB Documentation	mongodb.com/docs/
Aerospike Documentation	docs.aerospike.com/
Apache Cassandra Documentation	cassandra.apache.org/doc/
Redis Documentation	redis.io/docs/
GridGain / Apache Ignite Documentation	ignite.apache.org/docs/

Made with IBM Bob