Database Landscape Overview
Modern infrastructure platforms — especially large-scale Linux test automation systems — require careful selection of the right database technology. Choosing incorrectly leads to scalability bottlenecks, operational complexity, and poor query performance. This guide maps each database category to its ideal use case, special features, and trade-offs.
Master Database Classification Table
| Database | Category | SQL/NoSQL | Primary Strength | Best For |
|---|---|---|---|---|
| MariaDB | RDBMS | SQL | Mature relational engine, drop-in MySQL replacement | Small/medium deployments, traditional workloads |
| PostgreSQL | RDBMS | SQL | Advanced SQL, extensibility, JSONB support | Complex queries, open-source enterprise |
| EDB Postgres | Enterprise PostgreSQL | SQL | Oracle compatibility + enterprise support | Oracle migrations, enterprise SQL compliance |
| CockroachDB | Distributed SQL (NewSQL) | NewSQL | Automatic sharding, multi-region, cloud-native | Distributed SQL with resilience & geo-routing |
| YugabyteDB | Distributed SQL (NewSQL) | NewSQL | PostgreSQL-compatible distributed SQL | PostgreSQL at large/distributed scale |
| MongoDB | Document DB | NoSQL | Flexible schema, rich document queries | Semi-structured data, rapid iteration |
| Aerospike | Key-Value DB | NoSQL | Sub-millisecond latency, high throughput | Real-time analytics, ad-tech, fraud detection |
| Cassandra | Wide Column DB | NoSQL | Write-heavy workloads, massive scale | Time-series, event logging, IoT |
| Redis | In-Memory Key-Value | NoSQL | Ultra-fast in-memory store, pub/sub | Caching, session management, fast cache |
| GridGain | In-Memory Data Grid | Hybrid | Distributed in-memory compute + SQL over cache | In-memory cache layer, distributed computing |
2. Database Taxonomy — Architecture Diagram
3. Traditional SQL Databases
Traditional SQL databases provide ACID-compliant relational data management with mature ecosystems, familiar query languages, and well-understood operational patterns. Best suited for structured data with defined schemas.
MariaDB — Classic RDBMS
Drop-in MySQL replacement with strong community support. Ideal for small to medium deployments. Familiar SQL interface, mature replication, and low operational overhead. Best choice when simplicity matters.
PostgreSQL — Advanced Open Source SQL
Highly extensible with JSONB, full-text search, window functions, CTEs, and advanced indexing. Strong choice for complex query workloads and data-rich platforms. The de-facto open-source enterprise RDBMS.
EDB Postgres — Enterprise PostgreSQL
Commercial distribution of PostgreSQL adding Oracle compatibility, 24×7 vendor support, enterprise HA/DR tools, security enhancements, and compliance certifications. The bridge for Oracle-to-PostgreSQL migrations.
EDB vs PostgreSQL — Feature Comparison
| Feature | PostgreSQL | EDB Postgres |
|---|---|---|
| Open Source | ✓ Yes | Core is PostgreSQL |
| SQL Support | ✓ Yes | ✓ Yes |
| Oracle Compatibility | ~ Limited | ✓ Strong |
| Enterprise Support | Community only | ✓ Commercial SLA |
| HA / DR Tools | Community tools (Patroni, pgBackRest) | Enterprise tools (EDB Failover Manager) |
| Security Enhancements | Standard | Advanced audit, encryption |
| Cost | ✓ Free | Subscription |
4. Distributed SQL (NewSQL) Databases
NewSQL databases combine the scalability characteristics of NoSQL systems with full ACID transactional SQL semantics. They are designed for cloud-native, multi-region, horizontally scaled deployments where traditional RDBMS hits vertical scaling limits.
CockroachDB — Cloud-Native Distributed SQL
Combines NoSQL-style horizontal scaling with full SQL, ACID transactions, automatic sharding, and multi-region active-active deployment. Inspired by Google Spanner. Resilient to node and zone failures with no manual intervention required.
YugabyteDB — PostgreSQL-Compatible Distributed SQL
Highest PostgreSQL wire-protocol compatibility among distributed SQL engines. Supports both YSQL (PostgreSQL API) and YCQL (Cassandra API). Ideal when migrating an existing PostgreSQL application to distributed scale without rewriting queries.
CockroachDB — Key Characteristics
| Feature | Support | Details |
|---|---|---|
| Horizontal Scaling | ✓ | Add nodes without downtime; automatic rebalancing |
| Distributed Architecture | ✓ | Shared-nothing; each node is a full peer |
| Multi-Region Deployment | ✓ | Geo-partitioning, follower reads, global tables |
| Automatic Sharding | ✓ | Range-based sharding; no manual partitioning |
| SQL Interface | ✓ | PostgreSQL-compatible SQL dialect |
| ACID Transactions | ✓ | Serializable isolation by default |
| Joins | ✓ | Full JOIN support including distributed joins |
| Schemas & Tables | ✓ | Standard relational schemas |
EDB vs CockroachDB vs YugabyteDB — Decision Matrix
| Feature | EDB Postgres | CockroachDB | YugabyteDB |
|---|---|---|---|
| PostgreSQL Compatible | ✓ Yes | ~ Partial | ✓ Very High |
| Distributed by Default | ✗ No | ✓ Yes | ✓ Yes |
| Oracle Migration | ✓ Excellent | ✗ Limited | ✗ Limited |
| Multi-Region Native | ~ Via extensions | ✓ Native | ✓ Native |
| Traditional Enterprise DB | ✓ Yes | ✗ No | ✗ No |
| Horizontal Scaling | ~ Limited | ✓ Excellent | ✓ Excellent |
| ACID Transactions | ✓ Yes | ✓ Yes | ✓ Yes |
5. NoSQL Databases
NoSQL databases sacrifice strict relational schema constraints in favour of flexible data models, horizontal scalability, and tunable consistency. Each NoSQL sub-type is optimised for a specific access pattern.
MongoDB — Document Database
Stores data as JSON-like BSON documents. Flexible schema allows rapid iteration. Rich query language with aggregation pipelines. Best for semi-structured data where schema evolves frequently.
Aerospike — High-Performance Key-Value
Designed for sub-millisecond latency at massive throughput. Hybrid memory architecture (RAM + SSD). Data model: Namespace → Set → Key → Bins (JSON-like). Perfect for real-time analytics and fraud detection.
Cassandra — Wide-Column Store
Optimised for write-heavy, time-series, and high-availability workloads. Masterless ring topology ensures no single point of failure. Tunable consistency (AP-focused). Best for large-scale event logging.
Redis — In-Memory Key-Value Store
Microsecond read/write latency. Supports strings, hashes, lists, sets, sorted sets, streams, and pub/sub. Primarily used as a fast cache, session store, or execution state bus. Persistence is optional.
Aerospike — Deep Dive
Aerospike is a high-performance distributed NoSQL database with a unique hybrid memory architecture that keeps indexes in RAM while storing data on NVMe SSDs — delivering sub-millisecond latency without requiring all data to fit in RAM.
| Attribute | Detail |
|---|---|
| Data Model | Namespace → Set → Key → Bins (similar to JSON fields) |
| Latency | Sub-millisecond (often <1 ms) for reads and writes |
| Throughput | Millions of operations per second per node |
| Scaling | Horizontal (add nodes, auto-rebalance) |
| Consistency | Strong consistency available (SC mode) |
| SQL Support | Limited — via connectors and secondary indexes |
| Ideal Use Cases | Fraud detection, ad-tech, recommendation engines, session storage, real-time analytics, execution state tracking |
Aerospike Data Model vs Traditional SQL
6. In-Memory Data Grid — GridGain
GridGain (built on Apache Ignite) occupies a unique category — it is primarily an In-Memory Data Grid (IMDG) and distributed computing platform, not a traditional database. It sits between the application and the underlying database as a high-speed distributed cache and compute layer.
GridGain — In-Memory Data Grid
Keeps "hot" data in distributed RAM across a cluster for sub-millisecond access. Supports ANSI SQL queries, JDBC/ODBC, ACID transactions, and key-value APIs. Often deployed as a caching layer in front of MariaDB, PostgreSQL, Oracle, or NoSQL stores.
| Feature | GridGain (Apache Ignite) |
|---|---|
| Architecture | Distributed In-Memory Data Grid + Compute Grid |
| SQL Support | ✓ ANSI SQL, JDBC/ODBC |
| ACID Transactions | ✓ Supported |
| Key-Value API | ✓ Supported |
| Data Location | Primarily RAM; optional persistent store to disk |
| Typical Deployment | Between application tier and backend database (cache layer) |
| Integration | MariaDB, PostgreSQL, Oracle, NoSQL stores as backing store |
| Use Cases | Distributed caching, in-memory compute, hot data acceleration |
7. Layered Database Architecture for Linux Test Platforms
8. Linux Test Platform — Database Selection by Use Case
For a large-scale Linux Test Infrastructure platform storing skill files, Avocado test metadata, op-test metadata, Bugzilla defects, Jira tasks, CI/CD execution history, and AI-generated testcase mappings — the following mapping applies:
| Data Type | Recommended DB | Category | Reason |
|---|---|---|---|
| Skill Files | PostgreSQL / EDB | SQL | Structured metadata, versioning, relational queries |
| Avocado Test Metadata | PostgreSQL / MariaDB | SQL | Schema is known; relational joins across testcases |
| Op-test Metadata | MariaDB / PostgreSQL | SQL | Small/medium scale; traditional RDBMS sufficient |
| Bugzilla Defects | EDB / PostgreSQL | SQL | Relational data with complex queries and audit trail |
| Jira Tasks | EDB / PostgreSQL | SQL | Structured project data; reporting and analytics |
| CI/CD Execution History | CockroachDB / YugabyteDB | NewSQL | High volume writes; distributed across regions/sites |
| AI-Generated Testcase Mappings | YugabyteDB / CockroachDB | NewSQL | PostgreSQL-compatible + scales as AI data grows |
| Execution State / Real-time Cache | Aerospike / Redis | NoSQL | Sub-millisecond access; cache running test states |
| Hot Data Acceleration | GridGain | IMDG | In-memory compute layer for frequently-queried data |
9. Architect Decision Guide
10. Full Feature Comparison Matrix
| Feature | MariaDB | PostgreSQL | EDB | CockroachDB | YugabyteDB | MongoDB | Aerospike | Cassandra | Redis | GridGain |
|---|---|---|---|---|---|---|---|---|---|---|
| SQL Support | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ~ | ~ | ✗ | ✓ |
| ACID Transactions | ✓ | ✓ | ✓ | ✓ | ✓ | ~ | ~ | ✗ | ✗ | ✓ |
| Horizontal Scaling | ✗ | ✗ | ~ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Multi-Region | ✗ | ✗ | ~ | ✓ | ✓ | ✓ | ✓ | ✓ | ~ | ✓ |
| Sub-ms Latency | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✓ |
| Schema Flexibility | ✗ | ~ | ~ | ✗ | ~ | ✓ | ✓ | ✓ | ✓ | ~ |
| In-Memory Primary | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ~ | ✗ | ✓ | ✓ |
| Oracle Compat. | ✗ | ~ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Open Source | ✓ | ✓ | Core OSS | BSL/CCL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
✓ = Full support | ~ = Partial / via extension | ✗ = Not supported natively
11. Quick Mental Model — Memory Trick for Teams
| Database | One-Line Mental Model |
|---|---|
| MariaDB | Traditional SQL — reliable workhorse, simple operations |
| PostgreSQL | Advanced SQL — the most capable open-source RDBMS |
| EDB Postgres | PostgreSQL + Oracle migration + enterprise support contract |
| CockroachDB | Distributed SQL — cockroach survives anything; so does your database |
| YugabyteDB | PostgreSQL that scales horizontally across data centres |
| MongoDB | Document store — flexible JSON, no rigid schema needed |
| Aerospike | Speed demon — sub-millisecond, for real-time decisions |
| Cassandra | Write firehose — absorbs millions of events per second |
| Redis | RAM-first — fastest possible cache and state store |
| GridGain | Turbocharger — in-memory layer sitting in front of your real DB |
12. Architect Selection Checklist
- Define whether your workload is OLTP (transactional), OLAP (analytical), or mixed — this determines SQL vs NoSQL vs IMDG
- Assess current scale (rows/records count, QPS) and projected growth over 3 years
- Determine if multi-region or geo-distributed deployment is required (forces NewSQL or NoSQL choice)
- Identify if sub-millisecond latency is required (forces Aerospike or Redis into the stack)
- Evaluate whether Oracle compatibility or Oracle migration path is needed (forces EDB Postgres)
- Check if PostgreSQL wire-protocol compatibility is required for existing tooling (YugabyteDB > CockroachDB)
- Assess team SQL expertise — NoSQL requires different operational skills than RDBMS
- Review licensing: CockroachDB uses BSL/CCL; confirm compliance with your legal team
- Plan for dual-layer architecture: persistent DB (MariaDB/CockroachDB/EDB) + cache layer (Redis/Aerospike)
- Evaluate GridGain only if in-memory distributed compute acceleration is required — not as a primary store
- For a small/medium test platform: start with PostgreSQL/MariaDB for metadata, add Aerospike/Redis for execution state
- For large-scale distributed test platforms: upgrade persistent tier to CockroachDB or YugabyteDB as volume grows
13. Database Architecture Block Diagrams
Each diagram below illustrates the internal architecture and data flow of a specific database — from the client application down to persistent storage. Use these as a quick visual reference when evaluating options with your team.
13.1 MariaDB — Traditional Relational Architecture
13.2 PostgreSQL — Advanced RDBMS Architecture
13.3 EDB Postgres — Enterprise PostgreSQL Architecture
13.4 CockroachDB — Distributed SQL Architecture
13.5 YugabyteDB — Distributed PostgreSQL Architecture
13.6 MongoDB — Document Database Architecture
13.7 Aerospike — High-Performance Key-Value Architecture
13.8 Apache Cassandra — Wide-Column Store Architecture
13.9 Redis — In-Memory Key-Value Store Architecture
13.10 GridGain / Apache Ignite — In-Memory Data Grid Architecture
14. References & Further Reading
| Resource | URL |
|---|---|
| MariaDB Documentation | mariadb.com/kb/en/ |
| PostgreSQL Documentation | postgresql.org/docs/ |
| EDB Postgres Documentation | enterprisedb.com/docs/ |
| CockroachDB Documentation | cockroachlabs.com/docs/ |
| YugabyteDB Documentation | docs.yugabyte.com/ |
| MongoDB Documentation | mongodb.com/docs/ |
| Aerospike Documentation | docs.aerospike.com/ |
| Apache Cassandra Documentation | cassandra.apache.org/doc/ |
| Redis Documentation | redis.io/docs/ |
| GridGain / Apache Ignite Documentation | ignite.apache.org/docs/ |
