vercel-labs/just-bash

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

just-bash is a TypeScript implementation of a bash interpreter with an in-memory virtual filesystem. Designed for AI agents needing a secure, sandboxed bash environment. No WASM dependencies allowed.

## Commands

```bash
# Build & Lint
pnpm build                 # Build TypeScript (required before using dist/)
pnpm typecheck             # Type check
pnpm lint:fix              # Fix lint errors (biome)
pnpm knip                  # Check for unused exports/dependencies

# Testing
pnpm test:run              # Run ALL tests (including spec tests)
pnpm test:unit             # Run unit tests only (fast, no comparison/spec)
pnpm test:comparison       # Run comparison tests only (uses fixtures)
pnpm test:comparison:record # Re-record comparison test fixtures
pnpm test:wasm             # Run WASM tests (python3, sqlite3, js-exec)

# Excluding spec tests (spec tests have known failures)
pnpm test:run --exclude src/spec-tests

# Run specific test file
pnpm test:run src/commands/grep/grep.basic.test.ts

# Run specific spec test file by name pattern
pnpm test:run src/spec-tests/spec.test.ts -t "arith.test.sh"
pnpm test:run src/spec-tests/spec.test.ts -t "array-basic.test.sh"

# Interactive shell
pnpm shell                 # Full network access
pnpm shell --no-network    # No network

# Sandboxed CLI (read-only by default)
node ./dist/cli/just-bash.js -c 'ls -la' --root .
node ./dist/cli/just-bash.js -c 'cat package.json' --root .
node ./dist/cli/just-bash.js -c 'grep -r "TODO" src/' --root .
```

### Sandboxed Shell Execution with `just-bash`

The `just-bash` CLI provides a secure, sandboxed bash environment using OverlayFS:

```bash
# Execute inline script (read-only by default)
node ./dist/cli/just-bash.js -c 'ls -la && cat README.md | head -5' --root .

# Execute with JSON output
node ./dist/cli/just-bash.js -c 'echo hello' --root . --json

# Allow writes (writes stay in memory, don't affect real filesystem)
node ./dist/cli/just-bash.js -c 'echo test > /tmp/file.txt && cat /tmp/file.txt' --root . --allow-write

# Execute script file
node ./dist/cli/just-bash.js script.sh --root .

# Exit on first error
node ./dist/cli/just-bash.js -e -c 'false; echo "not reached"' --root .
```

Options:
- `--root <path>` - Root directory (default: current directory)
- `--cwd <path>` - Working directory in sandbox (default: /home/user/project)
- `--allow-write` - Enable write operations (writes stay in memory)
- `--json` - Output as JSON (stdout, stderr, exitCode)
- `-e, --errexit` - Exit on first error

### Debug with `pnpm dev:exec`

Reads script from stdin, executes it, shows output. Prefer this over ad-hoc test files.

```bash
# Basic execution
echo 'echo hello' | pnpm dev:exec

# Compare with real bash
echo 'x=5; echo $((x + 3))' | pnpm dev:exec --real-bash

# Show parsed AST
echo 'for i in 1 2 3; do echo $i; done' | pnpm dev:exec --print-ast

# Multi-line script
echo 'arr=(a b c)
for x in "${arr[@]}"; do
  echo "item: $x"
done' | pnpm dev:exec --real-bash
```

## Architecture

### Core Pipeline

```
Input Script → Parser (src/parser/) → AST (src/ast/) → Interpreter (src/interpreter/) → ExecResult
```

### Key Modules

**Parser** (`src/parser/`): Recursive descent parser producing AST nodes

- `lexer.ts` - Tokenizer with bash-specific handling (heredocs, quotes, expansions)
- `parser.ts` - Main parser orchestrating specialized sub-parsers
- `expansion-parser.ts` - Parameter expansion, command substitution parsing
- `compound-parser.ts` - if/for/while/case/function parsing

**Interpreter** (`src/interpreter/`): AST execution engine

- `interpreter.ts` - Main execution loop, command dispatch
- `expansion.ts` - Word expansion (parameter, brace, glob, tilde, command substitution)
- `arithmetic.ts` - `$((...))` and `((...))` evaluation
- `conditionals.ts` - `[[ ]]` and `[ ]` test evaluation
- `control-flow.ts` - Loops and conditionals execution
- `builtins/` - Shell builtins (export, local, declare, read, etc.)

**Commands** (`src/commands/`): External command implementations

- Each command in its own directory with implementation + tests
- Registry pattern via `registry.ts`

**Filesystem** (`src/fs.ts`, `src/overlay-fs/`): In-memory VFS with optional overlay on real filesystem

- `real-fs-utils.ts` - Shared security helpers for real-FS-backed implementations
- `OverlayFs` / `ReadWriteFs` - Both default to `allowSymlinks: false` (symlinks blocked)
- Symlink policy is enforced at central gate functions (`resolveAndValidate`, `validateRealPath_`) so new methods get protection automatically
- Pass `allowSymlinks: true` only when symlink support is explicitly needed

**AWK** (`src/commands/awk/`): AWK text processing implementation

- `parser.ts` - Parses AWK programs (BEGIN/END blocks, rules, user-defined functions)
- `executor.ts` - Executes parsed AWK programs line by line
- `expressions.ts` - Expression evaluation (arithmetic, string functions, comparisons)
- Supports: field splitting, pattern matching, printf, gsub/sub/split, user-defined functions
- Limitations: User-defined functions support single return expressions only (no multi-statement bodies or if/else)

**SED** (`src/commands/sed/`): Stream editor implementation

- `parser.ts` - Parses sed commands and addresses
- `executor.ts` - Executes sed commands with pattern/hold space
- Supports: s, d, p, q, n, a, i, c, y, =, addresses, ranges, extended regex (-E/-r)
- Has execution limits to prevent runaway compute

**Python** (`src/commands/python3/`): CPython compiled to WebAssembly via Emscripten

- `python3.ts` - Command entry point, arg parsing, worker lifecycle, timeout with worker termination
- `worker.ts` - Worker thread: loads CPython WASM, HOSTFS/HTTPFS bridges, defense-in-depth
- `sync-fs-backend.ts` / `protocol.ts` - SharedArrayBuffer protocol for sync FS calls from WASM
- `fs-bridge-handler.ts` - Main thread: processes FS requests from worker
- Security: isolation by construction (no JS bridge, no ctypes, no dlopen, no NODEFS)
- Defense-in-depth: `Module._load` blocking at file scope (before WASM loads), `WorkerDefenseInDepth` after
- WASM binary at `vendor/cpython-emscripten/` — `python.cjs` has `__emscripten_system` patched to return -1
- `-m MODULE` names are validated with `/^[a-zA-Z_][a-zA-Z0-9_.]*$/` to prevent code injection
- Worker is terminated on timeout via `workerRef` pattern
- WASM memory capped at 512MB (`-sMAXIMUM_MEMORY=536870912`)
- Tests: `pnpm test:wasm` (excluded from `pnpm test:unit` by default due to WASM load time)

### Adding Commands

Commands go in `src/commands/<name>/` with:

1. Implementation file with usage statement
2. Unit tests (collocated `*.test.ts`)
3. Error on unknown options (unless real bash ignores them)
4. Comparison tests in `src/comparison-tests/` for behavior validation

### Testing Strategy

- **Unit tests**: Fast, isolated tests for specific functionality
- **Comparison tests**: Compare just-bash output against recorded bash fixtures (see `src/comparison-tests/README.md`)
- **Spec tests** (`src/spec-tests/`): Bash specification conformance (may have known failures)

Prefer comparison tests when uncertain about bash behavior. Keep test files under 300 lines.

### Comparison Tests (Fixture System)

Comparison tests use pre-recorded bash outputs stored in `src/comparison-tests/fixtures/`. This eliminates platform differences (macOS vs Linux). See `src/comparison-tests/README.md` for details.

```bash
# Run comparison tests (uses fixtures, no real bash needed)
pnpm test:comparison

# Re-record fixtures (skips locked fixtures)
RECORD_FIXTURES=1 pnpm test:run src/comparison-tests/mytest.comparison.test.ts

# Force re-record including locked fixtures
RECORD_FIXTURES=force pnpm test:comparison
```

When adding comparison tests:
1. Write the test using `setupFiles()` and `compareOutputs()`
2. Run with `RECORD_FIXTURES=1` to generate fixtures
3. Commit both the test file and the generated fixture JSON
4. If manually adjusting for Linux behavior, add `"locked": true` to the fixture

## Filesystem Security: Default-Deny Symlinks

`OverlayFs` and `ReadWriteFs` default to `allowSymlinks: false`. This means:

- `symlink()` throws EPERM
- Any path traversing a real-FS symlink is rejected (ENOENT/EACCES)
- `lstat()` and `readlink()` still work on symlinks (they inspect without following)
- `readdir()` lists symlink entries but operations through them fail

**How it works**: Central gate functions (`resolveAndValidate` in ReadWriteFs, `validateRealPath_` in OverlayFs) compare `realPath.slice(root.length)` vs `canonical.slice(canonicalRoot.length)`. A mismatch means a symlink was traversed — zero extra I/O cost.

**TOCTOU protection**: `readFile`, `writeFile`, and `appendFile` in ReadWriteFs use `O_NOFOLLOW` (when `allowSymlinks: false`) to prevent symlink-swap attacks between validation and I/O. `writeFile`/`appendFile` also re-validate paths after `mkdir()` to catch parent-directory-swap attacks.

**When adding new FS methods**: Route all real-FS access through the existing gates. Never call `fs.promises.stat()`, `fs.realpathSync()`, or similar directly on unvalidated paths. For data I/O (read/write), prefer `fs.promises.open()` with `O_NOFOLLOW` over `fs.promises.readFile()`/`writeFile()` to close TOCTOU gaps. The gate-based design means any method that goes through the gate is automatically protected.

**In tests**: Pass `allowSymlinks: true` to the constructor when testing symlink behavior. The `cross-fs-no-symlinks.test.ts` file tests the default-deny behavior and O_NOFOLLOW TOCTOU protection.

## Prototype Pollution Prevention

All `Record<string, T>` objects must use null prototypes to prevent `__proto__` lookups from traversing the prototype chain. This is enforced by the banned-patterns linter (`pnpm lint:banned`).

**For static lookup tables**, use `nullPrototype()` from `src/commands/query-engine/safe-object.ts`:

```typescript
import { nullPrototype } from "../query-engine/safe-object.js";
const COLORS = nullPrototype<Record<string, string>>({ red: "#f00", blue: "#00f" });
```

**For empty accumulators**, use `Object.create(null)`:

```typescript
const map: Record<string, string> = Object.create(null);
```

**For bundled workers** (can't import safe-object), use inline pattern:

```typescript
const TABLE: Record<string, string> = Object.assign(
  Object.create(null) as Record<string, string>,
  { key: "value" },
);
```

**For self-referential types** (where `Object.assign` breaks type inference), use `Object.setPrototypeOf` with a `@banned-pattern-ignore` comment:

```typescript
// @banned-pattern-ignore: prototype nulled below; self-referential type prevents Object.assign pattern
const MAP: Record<string, Fn> = { ... };
// @banned-pattern-ignore: defense-in-depth null-prototype for static lookup table
Object.setPrototypeOf(MAP, null);
```

**Always guard bracket access** with `Object.hasOwn()` or use `nullPrototype` objects — never do `obj[userInput]` on a plain `{}`.

## Development Guidelines

- Read AGENTS.md
- Use `pnpm dev:exec` instead of ad-hoc test scripts (avoids approval prompts)
- Always verify with `pnpm typecheck && pnpm lint:fix && pnpm knip && pnpm test:run` before finishing
- Assert full stdout/stderr in tests, not partial matches
- Implementation must match real bash behavior, not convenience
- Dependencies using WASM are not allowed (exception: sql.js for SQLite, approved for security sandboxing)
- We explicitly don't support 64-bit integers
- All parsing/execution must have reasonable limits to prevent runaway compute