<!--
Multilingual Punctuation Debugging
Part of the Bles Software agent skills repository:
https://www.bles-software.com/claude-skills

Install for Claude Code:
  mkdir -p ~/.claude/skills/multilingual-punctuation-debugging
  mv multilingual-punctuation-debugging.md ~/.claude/skills/multilingual-punctuation-debugging/SKILL.md

Install for Codex:
  mkdir -p ~/.codex/skills/multilingual-punctuation-debugging
  mv multilingual-punctuation-debugging.md ~/.codex/skills/multilingual-punctuation-debugging/SKILL.md

For Gemini CLI and other agents, drop the body of this file into the
project instruction file the agent already reads.
-->

---
name: Multilingual Punctuation Debugging
description: Debug and fix unwanted punctuation in non-Latin scripts across multiple code layers
---

# Multilingual Punctuation Debugging Skill

Debug and fix unwanted punctuation in non-Latin scripts (Hebrew, Arabic, etc.) across multiple code layers.

## When to Use

Use this skill when:
- Hebrew/Arabic/RTL text appears with weird dots or periods between words
- Headlines or titles have unnatural punctuation: "פסטה. אסם" instead of "פסטה אסם"
- AI-generated images show broken text formatting
- Punctuation appears in the middle of product names or phrases

## The Pattern

**Root Cause:** Punctuation injection happens at MULTIPLE layers:

```
Layer 1: Data Formatting
  └─ parts.join('. ') → adds periods between components

Layer 2: AI Instructions
  └─ "replace hyphens with periods" → AI over-applies

Layer 3: Validation Rules
  └─ "use period/comma" → reinforces wrong pattern

Layer 4: Post-Processing
  └─ regex replaces dashes with periods → final corruption
```

## Diagnostic Steps

### Step 1: Find All Punctuation Injection Points

```bash
# Search for join with periods
grep -rn "join('\. ')\|join(\"\. \")" src/

# Search for period instructions in prompts
grep -rn "period\|comma\|punctuation" src/*.js | grep -v node_modules

# Search for dash-to-period replacement
grep -rn "→.*\." src/*.js
```

### Step 2: Trace the Text Flow

Map where text originates and how it transforms:
1. **Parser** - extracts components (usually correct)
2. **Formatter** - joins components (often adds periods)
3. **Prompt Builder** - instructs AI (may teach wrong patterns)
4. **Post-processor** - cleans output (may add more periods)

### Step 3: Check Language-Specific Rules

Search for Hebrew/Arabic specific instructions:
```bash
grep -rn "HEBREW\|Hebrew\|עברית\|\u0590-\u05FF" src/
grep -rn "ARABIC\|Arabic\|عربي\|\u0600-\u06FF" src/
```

## Common Fixes

### Fix 1: Data Formatting - Use Space Separator

```javascript
// WRONG
return parts.join('. ') || original;

// CORRECT
return parts.join(' ') || original;  // Space, not period
```

### Fix 2: AI Instructions - Fix Examples

```javascript
// WRONG - teaches AI to use periods
"לשבוע-אל" ❌ → "לשבוע. אל" ✅

// CORRECT - teaches AI to use spaces
"לשבוע-אל" ❌ → "לשבוע אל" ✅
```

### Fix 3: Validation Rules - Natural Flow

```javascript
// WRONG
"REMOVE dashes and use period/comma"

// CORRECT
"REMOVE dashes and use SPACE for natural flow"
```

### Fix 4: Post-Processing - Language-Aware Separators

```javascript
// WRONG - always uses period
return `${before}. ${after}`;

// CORRECT - Hebrew gets space, English gets period
const hasHebrew = /[\u0590-\u05FF]/.test(before) || /[\u0590-\u05FF]/.test(after);
return hasHebrew ? `${before} ${after}` : `${before}. ${after}`;
```

## Hebrew Character Detection

```javascript
// Hebrew Unicode range: \u0590-\u05FF
const hasHebrew = /[\u0590-\u05FF]/.test(text);

// Arabic Unicode range: \u0600-\u06FF
const hasArabic = /[\u0600-\u06FF]/.test(text);

// Check for RTL scripts
const hasRTL = /[\u0590-\u05FF\u0600-\u06FF]/.test(text);
```

## Key Principle

**RTL languages (Hebrew, Arabic) don't use periods as word separators.**

| Language | Word Separator | Sentence End |
|----------|---------------|--------------|
| English  | Space         | Period       |
| Hebrew   | Space         | Period (נקודה) |
| Arabic   | Space         | Period (.)   |

**Headlines and titles NEVER have periods between words in any language.**

## Verification

After fixing:
1. Restart service: `pm2 restart app`
2. Generate test content with Hebrew/Arabic text
3. Verify output shows: "פסטה אסם" NOT "פסטה. אסם"
4. Check all 4 layers are fixed (one layer fix won't be enough)

## Files Commonly Affected

| Layer | Common Files |
|-------|-------------|
| Data Formatting | `*Orchestrator.js`, `*Formatter.js` |
| AI Instructions | `promptBuilder.js`, `*Prompt*.js` |
| Validation | `promptBuilder.js`, `*Rules*.js` |
| Post-Processing | `reviewEngine.js`, `*Processor.js` |