<!--
Cluster DB Safety
Part of the Bles Software agent skills repository:
https://www.bles-software.com/claude-skills

Install for Claude Code:
  mkdir -p ~/.claude/skills/cluster-db-safety
  mv cluster-db-safety.md ~/.claude/skills/cluster-db-safety/SKILL.md

Install for Codex:
  mkdir -p ~/.codex/skills/cluster-db-safety
  mv cluster-db-safety.md ~/.codex/skills/cluster-db-safety/SKILL.md

For Gemini CLI and other agents, drop the body of this file into the
project instruction file the agent already reads.
-->

---
name: Cluster DB Safety
description: Atomic read-modify-write pattern for PM2 cluster mode database safety
---

# Atomic Read-Modify-Write Pattern for PM2 Cluster Database Safety

## Problem

When running Node.js applications in PM2 cluster mode (multiple workers), file-based JSON databases are vulnerable to race conditions:

```
Worker 1: READ settings (version A)
Worker 2: READ settings (version A)
Worker 1: MODIFY locally, WRITE (version B)
Worker 2: MODIFY locally, WRITE (version C) ← Overwrites Worker 1's changes!
```

This causes data loss - settings, creatives, gallery items disappear randomly.

## Solution: Atomic Mutator Pattern

Never read data outside the atomic lock. Use mutator functions that:
1. Acquire lock
2. Read FRESH data from disk (inside lock)
3. Pass data to callback for modification
4. Write atomically
5. Release lock

## Implementation

### Base `atomicWrite()` in db.js

```javascript
async function atomicWrite(mutator, operationName = 'unknown') {
  let lockId = null;
  try {
    if (pendingWrite) {
      clearTimeout(pendingWrite);
      pendingWrite = null;
    }
    lockId = await acquireLock(5000);

    // Read FRESH from disk inside lock
    const data = await fs.readFile(dbPath, 'utf-8');
    const s = JSON.parse(data);

    // Execute mutator with fresh data
    const result = await mutator(s);

    // Validate and write
    const jsonString = JSON.stringify(s, null, 2);
    JSON.parse(jsonString); // Validate
    await fs.writeFile(dbPath, jsonString);

    memoryCache = s;
    cacheTimestamp = Date.now();
    return result;
  } finally {
    if (lockId) await releaseLock(lockId);
  }
}
```

### Domain-Specific Mutators

#### User Settings
```javascript
async function mutateEffectiveSettings(userId, mutator, opName) {
  return atomicWrite((s) => {
    if (!s.userSettings) s.userSettings = {};
    if (!s.userSettings[userId]) s.userSettings[userId] = {};
    const settings = s.userSettings[userId];
    return mutator(settings);
  }, opName);
}
```

#### Discount Gallery
```javascript
async function mutateDiscountGallery(userId, mutator, opName) {
  return atomicWrite((s) => {
    if (!s.discountGalleries) s.discountGalleries = {};
    if (!s.discountGalleries[userId]) s.discountGalleries[userId] = [];
    const gallery = s.discountGalleries[userId].slice();
    const result = mutator(gallery);
    if (result && result.__skipSave) return result;
    s.discountGalleries[userId] = gallery;
    return result;
  }, opName);
}
```

#### Multiple Collections Atomically
```javascript
async function mutateBothGalleries(userId, mutator, opName) {
  return atomicWrite((s) => {
    // Initialize both
    if (!s.discountGalleries) s.discountGalleries = {};
    if (!s.discountGalleries[userId]) s.discountGalleries[userId] = [];
    if (!s.editedVideosGalleries) s.editedVideosGalleries = {};
    if (!s.editedVideosGalleries[userId]) s.editedVideosGalleries[userId] = [];

    const mainGallery = s.discountGalleries[userId].slice();
    const editedGallery = s.editedVideosGalleries[userId].slice();

    const result = mutator(mainGallery, editedGallery);
    if (result && result.__skipSave) return result;

    s.discountGalleries[userId] = mainGallery;
    s.editedVideosGalleries[userId] = editedGallery;
    return result;
  }, opName);
}
```

## Usage Pattern

### WRONG (Stale Read)
```javascript
// Race condition! Data read outside lock
const settings = await getEffectiveSettings(userId);
settings.creatives.push(newCreative);
await updateEffectiveSettings(userId, settings);
```

### CORRECT (Atomic Mutator)
```javascript
await mutateEffectiveSettings(userId, (settings) => {
  // settings is FRESH, read inside lock
  if (!settings.creatives) settings.creatives = [];
  settings.creatives.push(newCreative);
}, 'addCreative');
```

## Key Principles

1. **Never read outside lock** - Any `getSettings()` followed by `saveSettings()` is a race condition
2. **File I/O inside callback** - Read fresh data after acquiring lock
3. **Modify in-place** - The callback modifies the passed object directly
4. **Return values** - Callback can return data for the caller
5. **Skip save pattern** - Return `{ __skipSave: true, ...data }` to abort write but return data

## Testing

```bash
# Run Playwright tests for concurrent safety
npx playwright test

# Validate JSON integrity
node -e "JSON.parse(require('fs').readFileSync('autoposter.json'))"

# Check logs for race conditions
pm2 logs autoposter --lines 50 | grep -E "atomicWrite|LOAD ERROR"
```

## When to Use

Use atomic mutators for ANY operation that:
- Reads data, modifies it, then writes it back
- Could be called by multiple cluster workers simultaneously
- Operates on shared collections (settings, galleries, jobs, etc.)

Common patterns that need this:
- Adding/removing items from arrays
- Updating counters
- Toggling flags
- Batch operations