πŸ’Ύ Unit 11 β€’ Data Engineering

Database Systems: Relational SQL & Document NoSQL

The definitive engineering guide to data persistence architectures: ACID relational database design (PostgreSQL), DDL/DML grammar, multi-table JOIN mechanics, primary/foreign key cascades, B-Tree index optimization, connection pooling in Node.js, distributed NoSQL document models (MongoDB BSON), Mongoose ODM schema validation, and embedding vs. referencing design trade-offs.

Track Full-Stack Web Dev
Code & Level WD-111 β€’ Data Architecture
Core Topics 10 Modules + Capstone
Pedagogy Formula 50% Theory β€’ 30% Code β€’ 20% Flow
Curriculum Structure: 50% In-Depth Explanatory Theory 30% Verified SQL & Mongoose Code 20% Visual Architecture Flowcharts
01

Data Persistence Architectures: Relational (SQL) vs. Document (NoSQL)

At the core of software engineering lies data persistence: preserving volatile in-memory state across hardware crashes, server restarts, and operating system reboots. Modern engineering relies on two distinct database models: Relational (SQL) and Non-Relational (NoSQL).

πŸ›οΈ Relational Databases & The ACID Transaction Guarantee

Pioneered by Edgar F. Codd in 1970, Relational Database Management Systems (RDBMS) organize data into rigid tabular structures (rows and columns) governed by mathematical set theory. RDBMS systems (PostgreSQL, MySQL, SQLite) guarantee ACID Transactions:

  • Atomicity: All statements in a transaction succeed completely, or the entire operation rolls back to the beginning ("All or Nothing").
  • Consistency: Transactions can only transition the database from one valid state to another, strictly enforcing constraints (types, foreign keys, uniqueness).
  • Isolation: Concurrent transactions execute independently without dirty reads or phantom writes, simulating sequential execution.
  • Durability: Once a transaction commits, its changes are written to permanent non-volatile disk logs (Write-Ahead Logging / WAL), surviving power outages.
πŸƒ The NoSQL Paradigm & The CAP Theorem

As internet applications exploded in scale during the 2000s, rigid schemas and vertical server scaling hit financial and physical ceilings. NoSQL (Not Only SQL) enginesβ€”led by document stores like MongoDBβ€”prioritized horizontal cluster sharding, developer velocity, and dynamic JSON-native schemas.

According to Eric Brewer's CAP Theorem, a distributed data store can guarantee at most two out of three properties simultaneously:

  • Consistency (C): Every read receives the most recent write or an error.
  • Availability (A): Every non-failing node returns a response (without guarantee that it contains the latest write).
  • Partition Tolerance (P): The system continues operating despite arbitrary network dropped packets between server nodes. Distributed databases must choose either CP (e.g., MongoDB) or AP (e.g., Cassandra).
Architectural Dimension Relational SQL (PostgreSQL, MySQL) Document NoSQL (MongoDB)
Data Structure Strict tables, fixed columns, typed rows (tuples). Flexible hierarchical documents stored in BSON (Binary JSON).
Schema Rigor Strict Schema-on-Write (DDL migrations mandatory). Dynamic Schema-on-Read (polymorphic document structures).
Entity Relationships Normalized foreign keys reassembled via relational JOINs. Denormalized nested sub-documents (embedding) or ObjectId references.
Scaling Strategy Primarily Vertical Scaling (more CPU/RAM) + Read Replicas. Native Horizontal Scaling (automatic distributed sharding across nodes).
Best Production Fit Financial ledgers, e-commerce transactions, enterprise relational ERPs. High-velocity event logging, IoT telemetry, content management, polymorphic catalogs.
02

Relational Schema Definition & Table Architecture (DDL)

In SQL databases, defining the structure of data is accomplished via Data Definition Language (DDL). Tables define the contract: column names, static data types, and declarative constraints.

πŸ›‘οΈ Declarative Table Constraints
  • PRIMARY KEY: Uniquely identifies each record. Enforces non-nullness and automatically creates an underlying B-Tree index.
  • NOT NULL: Forbids the insertion of missing or unassigned values.
  • UNIQUE: Guarantees that no two rows possess identical values for the specified column (e.g., email addresses).
  • DEFAULT: Supplies an automated fallback value (e.g., DEFAULT NOW() or DEFAULT false).
  • CHECK: Evaluates a boolean expression before permitting row insertion (e.g., CHECK (salary > 0)).
πŸ’» Production PostgreSQL DDL Migration Script
-- Enable native cryptographic UUID generation extension
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

-- 1. Create Organizations Table (Parent)
CREATE TABLE organizations (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  name VARCHAR(100) NOT NULL,
  tier VARCHAR(20) DEFAULT 'starter' CHECK (tier IN ('starter', 'pro', 'enterprise')),
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- 2. Create Users Table with Referential Integrity & Constraints
CREATE TABLE users (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  org_id UUID NOT NULL REFERENCES organizations(id) ON DELETE CASCADE,
  email VARCHAR(255) NOT NULL UNIQUE,
  password_hash VARCHAR(60) NOT NULL,
  role VARCHAR(30) DEFAULT 'engineer',
  is_active BOOLEAN DEFAULT TRUE,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
PostgreSQL DDL Diagnostics
CREATE EXTENSION CREATE TABLE (organizations) CREATE TABLE (users) Integrity check: users.org_id strictly references organizations.id with ON DELETE CASCADE.
03

Data Manipulation (DML): INSERT, SELECT, UPDATE, DELETE & Filtering

Data Manipulation Language (DML) encompasses the four operational verbs used to query, mutate, and manage rows inside existing tables: INSERT, SELECT, UPDATE, and DELETE.

⚠️ The Trap of NULL and Three-Valued Logic in the WHERE Clause

In relational databases, NULL does not represent zero, an empty string, or false. NULL represents the complete absence of a value (Unknown).

Consequently, standard comparison operators fail against NULL:

-- FATAL BUG: Always returns 0 rows, even if status is NULL!
SELECT * FROM users WHERE status = NULL;

In SQL's Three-Valued Logic (True, False, Unknown), comparing anything to NULL yields Unknown. To filter NULLs safely, developers must always use the explicit predicates IS NULL or IS NOT NULL.

πŸ’» Verified SQL CRUD Operations & Pattern Matching
-- 1. INSERT: Create rows with RETURNING clause (avoids secondary roundtrip query!)
INSERT INTO users (org_id, email, password_hash, role)
VALUES 
  ('a1b2c3d4-0000-0000-0000-000000000001', 'shivan@devascent.me', '$2b$12$...', 'admin'),
  ('a1b2c3d4-0000-0000-0000-000000000001', 'alex@devascent.me', '$2b$12$...', 'engineer')
RETURNING id, email, created_at;

-- 2. SELECT: Filtering with WHERE, range matching, and sorting
SELECT id, email, role, created_at
FROM users
WHERE is_active = TRUE
  AND role IN ('admin', 'lead')
  AND email ILIKE '%@devascent.me' -- Case-insensitive pattern match
ORDER BY created_at DESC
LIMIT 20 OFFSET 0;

-- 3. UPDATE: Safe atomic modification (MANDATORY: Always supply WHERE!)
UPDATE users
SET role = 'principal', updated_at = NOW()
WHERE email = 'shivan@devascent.me';

-- 4. DELETE: Removing records
DELETE FROM users
WHERE is_active = FALSE AND created_at < NOW() - INTERVAL '90 days';
Query Execution Plan
INSERT 0 2 (returned 2 generated UUIDs) SELECT 1 rows (0.84ms) UPDATE 1 (updated_at synchronized to current timestamp) DELETE 0 (no records older than 90 days)
04

Primary Keys, Foreign Keys & Referential Integrity Cascades

The defining characteristic of relational systems is the ability to enforce Referential Integrity: ensuring that relationships between records in disparate tables remain valid, consistent, and immune to orphaned records.

πŸ”‘ Primary Keys (PK) vs. Foreign Keys (FK)
  • Primary Key (PK): A column (or combination of columns) whose values uniquely identify every row in a table. It cannot be NULL.
  • Foreign Key (FK): A column in a child table whose values point directly to a Primary Key in a parent table. The database engine physically rejects any attempt to insert a child row with a non-existent parent foreign key!
  • Referential Actions on Deletion (ON DELETE):
    • ON DELETE RESTRICT / NO ACTION (Default): Rejects deletion of the parent row if any child rows still reference it.
    • ON DELETE CASCADE: Deleting the parent automatically deletes all referencing child rows (e.g., deleting a User deletes all their Sessions).
    • ON DELETE SET NULL: Deleting the parent sets the child's foreign key column to NULL (e.g., deleting an Employee sets their assigned ticket's assignee_id to NULL).
REFERENTIAL INTEGRITY: 1:N CASCADE RELATIONSHIP TOPOLOGY FLOW-01
1. Parent Table: organizations
PK: id (UUID). Contains core company metadata.
βž” 1-to-Many βž”
2. Child Table: users
FK: org_id REFERENCES organizations(id) ON DELETE CASCADE.
βž” 1-to-Many βž”
3. Leaf Table: api_keys
FK: user_id REFERENCES users(id) ON DELETE CASCADE.
05

SQL JOINs: Recombining Normalized Data

In normalized relational databases, data is split across distinct tables to eliminate duplication (Third Normal Form / 3NF). When applications require composite data, SQL JOIN clauses reassemble these records on-the-fly based on related columns.

JOIN Type Venn Diagram Logic Left Table Behavior Right Table Behavior Standard Production Use Case
INNER JOIN Intersection (A ∩ B) Only rows with matching keys in Right table. Only rows with matching keys in Left table. Fetching users who strictly belong to an active organization.
LEFT JOIN All of A + (A ∩ B) 100% of rows preserved. Matching rows included; non-matches filled with NULL. Listing all users, including those who have never placed an order.
RIGHT JOIN All of B + (A ∩ B) Matching rows included; non-matches filled with NULL. 100% of rows preserved. Rarely used (identical to reversing tables in a LEFT JOIN).
FULL OUTER JOIN Union (A βˆͺ B) All rows preserved (non-matches filled with NULL). All rows preserved (non-matches filled with NULL). Reconciliation audits between billing ledgers and user records.
JOIN EXECUTION MATRIX: INNER JOIN VS LEFT OUTER JOIN FLOW-02
Table A: users
User 1 (Org 10), User 2 (Org 20), User 3 (Org NULL).
βž” INNER JOIN βž”
Result: 2 Rows
User 1 + Org 10, User 2 + Org 20. User 3 is excluded!
Table A: users
User 1 (Org 10), User 2 (Org 20), User 3 (Org NULL).
βž” LEFT JOIN βž”
Result: 3 Rows
User 1 + Org 10, User 2 + Org 20, User 3 + NULL Org name!
πŸ’» Multi-Table Aggregation Query with LEFT JOIN & GROUP BY
-- Aggregate query: List organizations with total member count and active status
SELECT 
  o.id AS org_id,
  o.name AS organization_name,
  o.tier,
  COUNT(u.id) AS total_members,
  COALESCE(MAX(u.created_at), o.created_at) AS latest_activity
FROM organizations o
-- Use LEFT JOIN so organizations with 0 users are still included in results!
LEFT JOIN users u ON o.id = u.org_id AND u.is_active = TRUE
GROUP BY o.id, o.name, o.tier, o.created_at
HAVING COUNT(u.id) >= 0
ORDER BY total_members DESC;
Query Results
org_id | organization_name | tier | total_members | latest_activity -------------------------------------+-------------------+------------+---------------+------------------------ a1b2c3d4-0000-0000-0000-000000000001 | Acme Enterprise | enterprise | 48 | 2026-10-15 16:45:00+00 b2c3d4e5-0000-0000-0000-000000000002 | Startup Lab | starter | 0 | 2026-10-12 09:12:00+00
06

Database Indexing Mechanics & The B-Tree Engine

Without an index, finding a single row in a table of 10 million rows forces the database engine to perform a Sequential Scan (Full Table Scan): reading every single data page from physical disk into memory, resulting in terrible O(n) algorithmic latency.

🌲 The B-Tree (Balanced Tree) Index Architecture

An Index is an auxiliary, ordered data structure that maintains sorted keys pointing to the physical disk location (Tuple ID / TID) of table rows.

By default, PostgreSQL and MySQL utilize B-Tree Indexes. B-Trees maintain self-balancing multi-way search trees:

  • Algorithmic Efficiency: Searching, inserting, and deleting executes in O(log n) time. In a table of 10,000,000 rows, a B-Tree locates any record within 3 to 4 disk page reads!
  • The Indexing Trade-Off (The Write Tax): Indexes are not free. While they accelerate SELECT queries, every index slows down INSERT, UPDATE, and DELETE operations. When a row is inserted, the engine must write the row to the table and traverse and rebalance every single B-Tree index on that table!
B-TREE INDEX LOOKUP: ROOT NODE βž” INTERNAL BRANCH βž” LEAF βž” DISK TUPLE FLOW-03
1. Root Node
Evaluates target key (e.g., id: 7420). Directs search to middle branch.
βž” pointer βž”
2. Branch Page
Narrows range (5000 - 8000). Directs pointer to specific leaf node.
βž” pointer βž”
3. Leaf Node (TID)
Points directly to disk page block #412, offset 6. Instant O(1) row fetch!
πŸ’» Creating Composite B-Tree Indexes & EXPLAIN ANALYZE Inspection
-- 1. Create a Composite Index (matches queries filtering on org_id AND is_active)
CREATE INDEX idx_users_org_active ON users (org_id, is_active);

-- 2. Performance Diagnostic: Inspect execution plan using EXPLAIN ANALYZE
EXPLAIN ANALYZE
SELECT id, email 
FROM users 
WHERE org_id = 'a1b2c3d4-0000-0000-0000-000000000001' 
  AND is_active = TRUE;
EXPLAIN ANALYZE Execution Plan Output
Bitmap Heap Scan on users (cost=4.30..12.50 rows=10 width=45) (actual time=0.038..0.042 ms) Recheck Cond: ((org_id = 'a1b2c3d4-...'::uuid) AND (is_active = true)) -> Bitmap Index Scan on idx_users_org_active (cost=0.00..4.30 rows=10 width=0) (actual time=0.021 ms) Execution Time: 0.072 ms (Index Scan selected; Sequential scan avoided!)
07

Using PostgreSQL in Node.js: Connection Pooling & Parameterized Safety

Connecting Node.js to PostgreSQL is managed via the official pg (node-postgres) client library. In high-concurrency production environments, initializing an individual database client connection for every incoming HTTP request causes severe connection exhaustion and crashes the database.

🏊 Connection Pooling Architecture

A Connection Pool maintains a warm, persistent cache of open TCP connections to PostgreSQL (e.g., 10 to 20 connections). When a request enters, it leases a pre-authenticated connection from the pool, runs the query, and releases the connection back to the pool instantly for reuse.

πŸ›‘ Eliminating SQL Injection via Parameterized Queries ($1, $2)

Never concatenate user input directly into SQL strings:

// VULNERABLE TO FATAL SQL INJECTION:
client.query(`SELECT * FROM users WHERE email = '${req.body.email}'`);

If an attacker supplies admin' OR '1'='1, the query dumps the entire database! Always use Parameterized Queries. The database treats parameterized arguments ($1, $2) strictly as literal values, never as executable SQL grammar.

πŸ’» Production PostgreSQL Connection Pool (db/postgres.js)
import pkg from 'pg';
const { Pool } = pkg;

// Configure warm connection pool
export const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 20, // Maximum 20 simultaneous connections
  idleTimeoutMillis: 30000, // Close idle clients after 30 seconds
  connectionTimeoutMillis: 2000, // Fail-fast if connection takes > 2 seconds
});

// Exported safe query runner with automatic client release
export async function query(text, params) {
  const start = Date.now();
  // pool.query borrows a client from pool, runs query, and releases client automatically
  const res = await pool.query(text, params);
  const duration = Date.now() - start;
  
  if (process.env.NODE_ENV === 'development') {
    console.log(`πŸ“Š [SQL] executed in ${duration}ms | rows: ${res.rowCount}`);
  }
  return res;
}
Pool Management Highlights
pool.query automatically catches dropped sockets, recycles dead connections, and guarantees zero connection leaks under high traffic load.
08

Introduction to MongoDB: Documents, Collections & BSON

MongoDB is the industry's leading open-source Document-Oriented NoSQL Database. Instead of storing data in tabular rows and columns, MongoDB stores data in self-describing, flexible hierarchical documents.

πŸ“„ The BSON Format & The 12-Byte ObjectId

While developers view and write data as standard JSON, MongoDB stores records internally in BSON (Binary JSON). BSON extends JSON with additional data types, including Date, 64-bit integers, binary buffers, and the universal primary key: ObjectId.

Every MongoDB document contains a unique _id field populated with a 12-byte hexadecimal ObjectId:

  • 4-Byte Timestamp: Unix epoch second of document creation (enables zero-cost temporal sorting without an extra created_at index!).
  • 5-Byte Random Value: Unique to the generating server host and process ID.
  • 3-Byte Incrementing Counter: Guarantees uniqueness for documents generated within the exact same second on the same machine.
MONGODB OBJECT HIERARCHY: DATABASE βž” COLLECTIONS βž” BSON DOCUMENTS FLOW-04
1. Database (cluster_prod)
Logical container holding sets of collections on disk.
βž” contains βž”
2. Collection (sensor_events)
Equivalent to an SQL table, but without schema enforcement.
βž” contains βž”
3. BSON Documents
Individual JSON records. Can have polymorphic keys and nested arrays.
πŸ’» Native MongoDB Shell (mongosh) CRUD Operations
// 1. INSERT: Insert document with nested objects and arrays
db.telemetry_logs.insertOne({
  nodeId: "node-east-41",
  metrics: { cpu: 42.8, ramUsage: 78.4 },
  tags: ["edge", "us-east", "v2"],
  timestamp: new Date()
});

// 2. QUERY: Deep nested property matching & array operators ($in, $gt)
db.telemetry_logs.find({
  "metrics.cpu": { $gt: 40 },
  tags: { $in: ["edge"] }
}).sort({ timestamp: -1 }).limit(5);

// 3. UPDATE: Atomic operators ($set, $inc, $push)
db.telemetry_logs.updateOne(
  { nodeId: "node-east-41" },
  { 
    $set: { "metrics.status": "healthy" },
    $push: { tags: "audited" } // Appends to array in-place
  }
);
MongoDB Native Output
{ acknowledged: true, insertedId: ObjectId("670e4e20b3f81e5a8d9a1001") } { matchedCount: 1, modifiedCount: 1 }
09

Mongoose ODM: Schemas, Models & Validation Lifecycle

While MongoDB's flexibility is powerful, unrestrained dynamic schemas in enterprise applications lead to corrupt, missing, or malformed fields. Mongoose is an Object Data Modeling (ODM) library for Node.js that brings structure, typecasting, validation, and middleware hooks to MongoDB.

πŸ—οΈ The Mongoose Architecture: Schema βž” Model βž” Document
  • Schema: Defines the blueprint: field types, default values, required flags, regex patterns, and custom validators.
  • Model: A compiled constructor wrapper created from a Schema. Provides the static interface to interact with the database (User.find(), User.create()).
  • Document: An individual instantiated instance of a Model representing a single MongoDB document, equipped with built-in helper methods (doc.save()).
  • Middleware Hooks (Pre/Post): Intercepts document lifecycles (e.g., executing code pre('save') to hash passwords or format fields before disk serialization).
πŸ’» Enterprise Mongoose Schema with Validation & Lifecycle Hooks
import mongoose from 'mongoose';

// 1. Define Schema Blueprint
const sensorLogSchema = new mongoose.Schema({
  nodeId: {
    type: String,
    required: [true, 'nodeId is mandatory'],
    trim: true,
    index: true // Automatically generates MongoDB B-Tree index!
  },
  temperature: {
    type: Number,
    required: true,
    min: [-50, 'Temperature cannot be below -50C'],
    max: [150, 'Temperature cannot exceed 150C']
  },
  status: {
    type: String,
    enum: ['nominal', 'warning', 'critical'],
    default: 'nominal'
  },
  tags: [String]
}, { 
  timestamps: true // Automatically creates createdAt and updatedAt fields
});

// 2. Lifecycle Pre-Save Hook (Middleware)
sensorLogSchema.pre('save', function(next) {
  // Business logic: automatically elevate status if temperature spikes
  if (this.temperature > 85) {
    this.status = 'critical';
  }
  next();
});

// 3. Compile Model Constructor
export const SensorLog = mongoose.model('SensorLog', sensorLogSchema);
Architecture Note
Mongoose automatically pluralizes the model name 'SensorLog' to target the 'sensorlogs' collection in the underlying MongoDB database.
10

NoSQL Schema Design: Embedding vs. Referencing

In MongoDB, the paramount schema design decision is: Should related data be embedded inside the parent document, or stored in a separate collection and referenced?

Pattern Data Representation Advantages Disadvantages Canonical Production Use Case
Embedding (Denormalization) Sub-documents or arrays directly inside parent document. Atomic single-document updates, zero joins, single high-speed disk seek. 16MB Document Limit; data duplication if sub-document is shared across entities. User addresses, e-commerce order line items, embedded blog comments (< 100).
Referencing (Normalization) Storing ObjectId pointing to another collection. Small parent documents, zero data duplication, unbounded scaling. Requires application-level joins (.populate()) or expensive $lookup aggregations. Author of an article, organization hierarchy, multi-million telemetry logs.
πŸ“ The 3 Cardinal Rules of MongoDB Schema Design
  1. One-to-Few (N < 100): Embed! (e.g., A user with 2 shipping addresses).
  2. One-to-Many (N < 5,000): Reference or embed with pagination! (e.g., A product with 50 customer reviews).
  3. One-to-Squillions (N > 10,000 to Millions): Parent-Reference! Do not put an array of 1,000,000 log IDs inside a Server document. Instead, store the serverId foreign reference inside each individual Log document!
11

Unit 11 Practical Capstone: Polyglot Database Architecture

Enterprise architectures frequently implement Polyglot Persistence: using PostgreSQL for strict transactional financial/user accounts and MongoDB for high-volume polymorphic sensor telemetry. Below is the complete verified integration:

πŸš€ Verified Polyglot Database Integration (polyglotService.js)
import pkg from 'pg';
import mongoose from 'mongoose';

// --- 1. PostgreSQL Connection Pool (Relational Core) ---
const { Pool } = pkg;
const pgPool = new Pool({ connectionString: process.env.POSTGRES_URL });

// --- 2. MongoDB Mongoose Connection (Telemetry Core) ---
await mongoose.connect(process.env.MONGO_URI);

// Mongoose Telemetry Model
const NodeTelemetry = mongoose.model('NodeTelemetry', new mongoose.Schema({
  clusterId: { type: String, required: true, index: true },
  metrics: Object,
  timestamp: { type: Date, default: Date.now }
}));

// --- 3. Unified Ingestion Controller ---
export async function recordTelemetryEvent({ organizationId, clusterId, metricsPayload }) {
  // Step A: Verify Organization existence & billing status in PostgreSQL (ACID check)
  const pgCheckQuery = `
    SELECT id, name, tier 
    FROM organizations 
    WHERE id = $1 AND tier IN ('pro', 'enterprise');
  `;
  const pgRes = await pgPool.query(pgCheckQuery, [organizationId]);

  if (pgRes.rowCount === 0) {
    throw new Error('Unauthorized: Organization not eligible for telemetry streams.');
  }

  // Step B: High-throughput ingestion of polymorphic sensor metrics into MongoDB
  const log = await NodeTelemetry.create({
    clusterId,
    metrics: metricsPayload
  });

  return {
    organization: pgRes.rows[0].name,
    telemetryId: log._id,
    ingestedAt: log.timestamp
  };
}
Polyglot Architecture Advantages
1. PostgreSQL handles strict relational constraints, billing tiers, and foreign key relations. 2. MongoDB ingests arbitrary JSON telemetry metrics without requiring expensive SQL table schema migrations on every sensor schema change.
12

University Exam & Technical Interview Viva Preparation Guide

Q1. Detail the four ACID properties of relational database transactions and provide a concrete banking example.
ACID stands for: (1) Atomicity: If transferring $500 from Account A to Account B, deducting from A and crediting to B must both succeed; if the server crashes after deducting from A, the entire transaction rolls back; (2) Consistency: Account balances cannot violate constraints (e.g., balance cannot drop below -$1000 overdraft limit); (3) Isolation: Concurrent transactions cannot view partial intermediate states of Account A; (4) Durability: Once the transaction commits, the updated balance survives a sudden hardware power crash because it is logged to disk (WAL).
Q2. What is the fundamental difference between an INNER JOIN and a LEFT OUTER JOIN?
An INNER JOIN returns strictly the intersection of both tables: rows are only included if the join condition matches on both sides. A LEFT OUTER JOIN preserves 100% of rows from the left table regardless of whether a match exists on the right table; if no matching row exists on the right side, all right-table columns are populated with NULL values.
Q3. How does a B-Tree index accelerate SQL queries, and what is the trade-off of over-indexing?
A B-Tree (Balanced Tree) maintains a sorted, multi-level hierarchy that permits binary search in O(log n) time, reducing millions of row scans to 3-4 disk page lookups. The trade-off is the Write Penalty: every index consumes physical disk space and forces the database engine to traverse, update, and rebalance the B-Tree whenever an INSERT, UPDATE, or DELETE executes. Tables with excessive indexes suffer severely degraded write throughput.
Q4. What is SQL Injection, and why do Parameterized Queries completely neutralize it?
SQL Injection occurs when untrusted user input is directly concatenated into a raw SQL query string, allowing an attacker to inject arbitrary commands (e.g., ' OR 1=1; DROP TABLE users; --). Parameterized Queries neutralize this by separating the query compilation phase from data transmission. The SQL statement template with placeholders ($1, $2) is compiled and plan-optimized first; the database treats the subsequent parameter payload strictly as literal strings/data, rendering executable code injection impossible.
Q5. Why is using a Connection Pool mandatory in production Node.js applications?
Establishing a new TCP/IP database connection requires a 3-way TCP handshake, a TLS cryptographic exchange, and an authentication credential validation roundtrip, consuming 50ms-100ms and substantial database CPU. If 1,000 concurrent HTTP requests attempt to open 1,000 separate database sockets, the database will exhaust available connections and crash. A connection pool maintains a warm, limited pool of reusable connections (e.g., 20), leasing them out for queries and recycling them immediately.
Q6. Compare Embedding vs. Referencing in MongoDB schema design. When should each be chosen?
Embedding stores related child data inside the parent document as nested objects or arrays. It enables single-disk-read retrieval and atomic updates, but is constrained by MongoDB's 16MB document size limit. Referencing stores child data in a separate collection and links them via ObjectId. Embedding is chosen for "One-to-Few" relationships (e.g., user addresses) where child data is always read alongside the parent. Referencing is chosen for "One-to-Squillions" (e.g., server event logs) or frequently updated independent entities to avoid huge document bloat.
Q7. What is the role of a Foreign Key constraint, and what does ON DELETE CASCADE accomplish?
A Foreign Key constraint guarantees referential integrity by ensuring that a value in a child table column must match an existing Primary Key in a parent table. ON DELETE CASCADE specifies an automated referential action: if a row in the parent table is deleted, the database engine automatically deletes all corresponding child rows referencing that parent, preventing dangling orphaned records.
Q8. What is the difference between JSON and BSON in MongoDB?
JSON is a human-readable, text-based data serialization format that supports only basic data types (strings, numbers, booleans, arrays, objects). BSON (Binary JSON) is a binary-encoded serialization format used internally by MongoDB. BSON is optimized for fast parsing and traversing, and introduces rich data types missing from JSON, such as ObjectId, Date (stored as 64-bit integer milliseconds), 64-bit integers (Long), and raw binary byte buffers.