File System (fs) Module

Q: How would you read a large file efficiently using a stream rather than loading it entirely into memory with fs.readFile()?

fs.createReadStream() reads a file incrementally in chunks, emitting 'data' events (or usable via .pipe() or async iteration) as each chunk becomes available, rather than buffering the entire file's contents in memory at once -- essential for processing files larger than available memory, or simply for reducing memory pressure and improving responsiveness when handling large files like log files or video uploads.

5 questions found

What is the difference between the synchronous, callback-based, and promise-based variants of Node's fs module methods?

Beginner

Every core fs operation is available in three forms: a synchronous version (like fs.readFileSync, which blocks the entire event loop until it completes -- appropriate only for startup scripts or CLI tools, never for a running server handling concurrent requests), a callback-based version (like fs.readFile, non-blocking, using the classic error-first callback), and a promise-based version (via require('node:fs/promises'), non-blocking, usable directly with async/await, generally the preferred style for modern application code).

const fsSync = require('node:fs');
const fs = require('node:fs/promises');

// Blocks the event loop -- avoid in request handlers
const data1 = fsSync.readFileSync('config.json', 'utf-8');

// Non-blocking, modern async/await style
const data2 = await fs.readFile('config.json', 'utf-8');

Real-world example A web server accidentally used fs.readFileSync() inside a request handler to serve an uploaded file, causing every other concurrent request to stall while any single file read was in progress; switching to fs.readFile() from fs/promises resolved the bottleneck by letting the event loop continue serving other requests during the I/O wait.

Common follow-ups: In what specific scenario is fs.readFileSync() actually the appropriate, correct choice rather than a mistake?;How does the promise-based fs/promises API compare in performance to the callback-based version, given both are non-blocking?

Async Patterns;Event Loop & Non-blocking IO

How would you read a large file efficiently using a stream rather than loading it entirely into memory with fs.readFile()?

Intermediate

fs.createReadStream() reads a file incrementally in chunks, emitting 'data' events (or usable via .pipe() or async iteration) as each chunk becomes available, rather than buffering the entire file's contents in memory at once -- essential for processing files larger than available memory, or simply for reducing memory pressure and improving responsiveness when handling large files like log files or video uploads.

const fs = require('node:fs');

const readStream = fs.createReadStream('huge-log-file.txt', { encoding: 'utf-8', highWaterMark: 64 * 1024 });

for await (const chunk of readStream) {
  processChunk(chunk); // processes incrementally, without loading the whole file into memory
}

Real-world example A log-analysis tool that previously used fs.readFile() and crashed with an out-of-memory error on multi-gigabyte log files is rewritten to use fs.createReadStream(), processing the file incrementally in manageable chunks and successfully handling files of any size regardless of available memory.

Common follow-ups: What does the highWaterMark option control, and how does it affect the tradeoff between memory usage and the number of 'data' events emitted?;How would you combine a read stream with a Transform stream to process and write output incrementally in a single pipeline?

Streams & Buffers;Performance Optimization & Profiling

How do you safely watch a directory or file for changes using fs.watch(), and what are its known cross-platform limitations?

Intermediate

fs.watch() emits events ('change', 'rename') when the watched file or directory is modified, but its behavior is notoriously inconsistent across operating systems -- the specific event types fired, whether filenames are always provided, and whether changes are reported reliably or occasionally missed/duplicated all vary between Linux (inotify), macOS (FSEvents), and Windows (ReadDirectoryChangesW), which is why many production tools (like nodemon) instead use a polling-based or hybrid approach, or a dedicated cross-platform library like chokidar, for more predictable behavior.

const fs = require('node:fs');

const watcher = fs.watch('./config', { recursive: true }, (eventType, filename) => {
  console.log(`${eventType} detected on ${filename}`);
  reloadConfig();
});

// For more reliable cross-platform behavior, many teams use chokidar instead:
// const chokidar = require('chokidar');
// chokidar.watch('./config').on('change', reloadConfig);

Real-world example A configuration hot-reload feature that worked reliably in development on macOS started missing some file-change events once deployed to a Linux-based Docker container using a mounted volume; the team switched to chokidar, which handles these platform and virtual-filesystem inconsistencies more robustly than the raw fs.watch() API.

Common follow-ups: Why do file changes inside a Docker volume sometimes behave differently with fs.watch() than changes on a native filesystem?;What's the difference between a polling-based and an event-based file watching strategy in terms of reliability versus resource usage?

CLI Tools & Scripting with Node.js;Path & OS Modules

What are Node.js file descriptor limits, and how might a Node.js application unintentionally exhaust them?

Advanced

Every open file, network socket, or pipe consumes a file descriptor, and the operating system imposes a limit on how many a single process can have open simultaneously (often 1024 by default on many systems) -- an application that opens files or streams without properly closing them (a common bug when error paths forget to close a stream, or when a loop opens many file handles concurrently without bounding concurrency) can exhaust this limit, causing subsequent file or network operations to fail with an EMFILE error.

// Leak: file handle never closed if an error occurs mid-processing
const fd = await fs.open('data.txt', 'r');
await processFile(fd); // if this throws, fd.close() below is never reached
await fd.close();

// Fixed: guarantees the descriptor is always released
const fd = await fs.open('data.txt', 'r');
try {
  await processFile(fd);
} finally {
  await fd.close();
}

Real-world example A batch job processing thousands of files started failing partway through with 'EMFILE: too many open files' errors, traced to a bug where an error thrown during processing skipped the file-close call for that iteration; wrapping the close call in a finally block ensured file descriptors were always released regardless of whether processing succeeded or failed.

Common follow-ups: How would you check and adjust the OS-level file descriptor limit (ulimit) for a Node.js process that legitimately needs to handle many concurrent connections?;How does concurrency-limiting a batch operation (as discussed for async task queues) also help prevent this kind of file descriptor exhaustion?

Error Handling;Performance Optimization & Profiling

What is the difference between fs.rename() and manually copying then deleting a file, particularly across different filesystems or drives?

Intermediate

fs.rename() performs an atomic move operation when the source and destination are on the same filesystem -- the file appears to move instantly with no window where it exists in neither location, which is important for operations that must never leave a file in a half-moved, inconsistent state. However, rename() fails with an EXDEV error when moving across different filesystems or drives (common in some cloud/container storage setups), in which case a manual copy-then-delete sequence is required instead, which is not atomic and can leave the operation partially complete if it's interrupted midway.

const fs = require('node:fs/promises');

try {
  await fs.rename('/tmp/upload.tmp', '/data/final.txt'); // atomic, if same filesystem
} catch (err) {
  if (err.code === 'EXDEV') {
    await fs.copyFile('/tmp/upload.tmp', '/data/final.txt'); // fallback: not atomic
    await fs.unlink('/tmp/upload.tmp');
  } else {
    throw err;
  }
}

Real-world example A file-upload service writes incoming uploads to a temporary file first, then uses fs.rename() to atomically move it into its final location only once the upload is fully complete and verified, ensuring no partially-written file ever appears in the location where other parts of the system expect to find fully-uploaded files.

Common follow-ups: Why does an atomic rename specifically prevent other processes from ever observing a partially written file?;What common deployment scenario (like a Docker volume spanning different underlying storage) triggers the EXDEV cross-filesystem error in practice?

File Uploads & Media Processing;Error Handling