中文

JavaScript Practice: a Node.js CLI Tool

AI-assistedDrafted with AI assistance and reviewed before publishing; examples are version-pinned. Verify against official docs when in doubt.

Node.js 22 LTS

This chapter assembles the series into one program: a command-line tool that scans Markdown and reports broken external links. The bar is what separates a script from a tool — clear arguments, a documented exit code, an installable command name, and output a CI job can parse.

What it must do

linkscan <dir> behaves like this:

  1. Recursively find every .md file under the directory.
  2. Extract http:// and https:// links, deduplicated across files.
  3. Probe them concurrently (5 at a time by default), recording status and duration.
  4. Print only failures by default; print the full result with --json.
  5. Exit 1 when links are broken and 2 when the arguments are wrong.

Project skeleton

linkscan/
  package.json
  bin/linkscan.js
  src/scan.js
  src/links.js
{
  "name": "linkscan",
  "version": "0.1.0",
  "private": true,
  "type": "module",
  "bin": { "linkscan": "./bin/linkscan.js" },
  "scripts": {
    "start": "node bin/linkscan.js"
  },
  "engines": { "node": ">=22" }
}

The bin field is what makes linkscan exist after npm link or npm i -g. The entry file starts with a shebang:

#!/usr/bin/env node

It must be the very first line — no BOM, no blank line above it — and the file needs to be readable and executable.

Parsing arguments

parseArgs is built into node:util, so argument handling needs no dependency:

// bin/linkscan.js
#!/usr/bin/env node
import { parseArgs } from "node:util";
import { scan } from "../src/scan.js";

const USAGE = `Usage: linkscan <dir> [options]

Options:
  -c, --concurrency <n>  parallel probes, default 5
      --timeout <ms>     per-link timeout, default 8000
      --json             print the full result as JSON
  -h, --help             show this help`;

class UsageError extends Error {}

function parseCli(argv) {
  const { values, positionals } = parseArgs({
    args: argv,
    allowPositionals: true,
    options: {
      concurrency: { type: "string", short: "c", default: "5" },
      timeout: { type: "string", default: "8000" },
      json: { type: "boolean", default: false },
      help: { type: "boolean", short: "h", default: false },
    },
  });

  const concurrency = Number(values.concurrency);
  const timeout = Number(values.timeout);

  if (!Number.isInteger(concurrency) || concurrency < 1) {
    throw new UsageError("--concurrency must be a positive integer");
  }
  if (!Number.isFinite(timeout) || timeout < 1) {
    throw new UsageError("--timeout must be a positive number of milliseconds");
  }
  if (positionals.length !== 1) {
    throw new UsageError("exactly one directory is required");
  }

  return { dir: positionals[0], concurrency, timeout, json: values.json, help: values.help };
}

Two conventions worth keeping: option values stay strings unless you convert them deliberately, and a dedicated UsageError lets the top level tell “you called it wrong” apart from “the tool failed”.

parseArgs stops at the first positional argument by default, so linkscan docs --json works but linkscan --json docs/extra is a usage error rather than a silent misread.

// src/links.js
const URL_PATTERN = /https?:\/\/[^\s<>"'`)\]]+/g;

export function extractLinks(markdown) {
  const withoutCode = markdown.replace(/```[\s\S]*?```/g, "");
  const matches = withoutCode.match(URL_PATTERN) ?? [];
  return matches.map((url) => url.replace(/[.,;:!?]+$/, ""));
}

Stripping fenced code first avoids probing the example URLs that documentation is full of. Trimming trailing punctuation turns see https://example.com/x. into a valid URL.

This is not a parser and should not become one. If you need reference-style links, front matter or HTML blocks handled correctly, use remark and walk the AST — the regex stays as the fast path.

Walking the tree

// src/scan.js
import { readdir, readFile } from "node:fs/promises";
import path from "node:path";
import { extractLinks } from "./links.js";

async function* walk(dir) {
  const entries = await readdir(dir, { withFileTypes: true });

  for (const entry of entries) {
    if (entry.name.startsWith(".") || entry.name === "node_modules") continue;

    const full = path.join(dir, entry.name);
    if (entry.isDirectory()) yield* walk(full);
    else if (entry.isFile() && entry.name.endsWith(".md")) yield full;
  }
}

withFileTypes avoids one stat call per entry. Skipping dot-directories and node_modules is not optional: scanning a repository root otherwise crawls tens of thousands of irrelevant files.

Symlinks are deliberately ignored here — entry.isSymbolicLink() is neither a file nor a directory in this check. Following them risks infinite loops on cyclic links; if you need to support them, track visited real paths.

async function checkLink(url, timeout) {
  const started = performance.now();
  try {
    const res = await fetch(url, {
      method: "GET",
      redirect: "follow",
      signal: AbortSignal.timeout(timeout),
      headers: { "user-agent": "linkscan/0.1 (+https://example.com/linkscan)" },
    });
    return { url, status: res.status, ok: res.ok, ms: Math.round(performance.now() - started) };
  } catch (err) {
    const status = err.name === "TimeoutError" ? "timeout" : "error";
    return {
      url,
      status,
      ok: false,
      ms: Math.round(performance.now() - started),
      message: err.message,
    };
  }
}

async function mapLimit(items, limit, worker) {
  const results = new Array(items.length);
  let cursor = 0;

  async function run() {
    while (cursor < items.length) {
      const index = cursor++;
      results[index] = await worker(items[index]);
    }
  }

  await Promise.all(Array.from({ length: Math.min(limit, items.length) }, run));
  return results;
}

Why GET and not HEAD: enough servers reject or misroute HEAD that you would report false positives. GET costs more bandwidth but answers the actual question. A --head flag is a reasonable option for users who care about traffic.

Why a custom user-agent: some sites block default clients, and an honest identifier with a contact URL is the polite version of automation.

The failure classification matters too. timeout, a network error and HTTP 404 are three different problems, and collapsing them into “broken” makes the output useless for whoever has to fix the links.

Assembling the scan

export async function scan({ dir, concurrency, timeout }) {
  const files = [];
  for await (const file of walk(dir)) files.push(file);

  const byUrl = new Map();

  for (const file of files) {
    const content = await readFile(file, "utf8");
    for (const url of extractLinks(content)) {
      const seen = byUrl.get(url) ?? [];
      seen.push(path.relative(process.cwd(), file));
      byUrl.set(url, seen);
    }
  }

  const urls = [...byUrl.keys()];
  const checked = await mapLimit(urls, concurrency, (url) => checkLink(url, timeout));

  return {
    files: files.length,
    links: urls.length,
    results: checked.map((item) => ({ ...item, files: byUrl.get(item.url) })),
  };
}

Keeping the URL-to-files map is what makes the report actionable: knowing that https://old.example.com/guide is dead is only useful together with the four documents that still link to it.

Output and exit codes

function report(summary, asJson) {
  if (asJson) {
    process.stdout.write(JSON.stringify(summary, null, 2) + "\n");
    return;
  }

  const failed = summary.results.filter((item) => !item.ok);

  for (const item of failed) {
    const reason =
      item.status === "timeout" ? "timeout"
      : item.status === "error" ? item.message
      : `HTTP ${item.status}`;

    console.error(`x [${reason}] ${item.url}`);
    for (const file of item.files) console.error(`    ${file}`);
  }

  console.log(`\nscanned ${summary.files} files, ${summary.links} links, ${failed.length} failures`);
}

async function main() {
  let options;
  try {
    options = parseCli(process.argv.slice(2));
  } catch (err) {
    console.error(err.message);
    console.error(USAGE);
    process.exitCode = 2;
    return;
  }

  if (options.help) {
    console.log(USAGE);
    return;
  }

  const summary = await scan(options);
  report(summary, options.json);
  process.exitCode = summary.results.some((item) => !item.ok) ? 1 : 0;
}

main().catch((err) => {
  console.error("linkscan failed:", err);
  process.exitCode = 1;
});

The three decisions that make it scriptable:

  • process.exitCode instead of process.exit(). The latter can truncate buffered stdout when output is piped.
  • Streams have jobs. Human summaries go to stdout, failure details to stderr, and with --json stdout contains JSON and nothing else, so linkscan --json | jq '.results[]' works.
  • Exit codes carry meaning. 0 clean, 1 broken links, 2 bad usage. A CI step can distinguish “the docs are stale” from “the tool was misconfigured”.

Installing and running it

npm link                              # registers the linkscan command
node bin/linkscan.js ./docs --json
linkscan ./src/content -c 8

Inside a repository you do not need a global install — a script entry is enough:

{
  "scripts": {
    "check:links": "linkscan ./src/content --json"
  }
}

In CI, treat a failure as information rather than a crash:

npm run check:links || echo "dead links found"

Common mistakes

1. Missing shebang or permissions

npm link succeeds and the command still fails with Permission denied. The shebang must be line one, and the file must not start with a BOM.

2. Path resolution drift

Build paths with path.join, never string concatenation. process.cwd() is where the command was run, which is not necessarily the project root; derive the root from import.meta.dirname when it must be fixed.

3. Concurrency that gets you rate limited

Five is deliberately conservative. When a host answers 429, respect retry-after and back off instead of pushing harder — otherwise the tool creates the problem it reports.

4. Counting a timeout as a dead link

TimeoutError and 404 are different diagnoses. Report them separately or the owner of the link cannot tell a slow server from a removed page.

5. Leaking file handles

readFile closes for you; createReadStream does not. If you switch to streaming large files, handle error and close explicitly.

6. Mixing logs into JSON output

One stray console.log and jq fails. Structured output gets one channel and nothing else.

7. Shipping without --help

The help text is the tool’s first documentation, and it is the only documentation users read.

Summary

  • parseArgs plus a UsageError blocks bad input at the entry point.
  • Walk with withFileTypes and skip dot-directories and node_modules.
  • Probe with a worker pool for concurrency and AbortSignal.timeout for deadlines.
  • Exit codes: 0 clean, 1 broken links, 2 usage error.
  • Human output on stdout, diagnostics on stderr, JSON alone on stdout.

Next: the same state discipline applied to a browser application with no framework at all.

Tags