From: Dmitry Volyntsev Date: Wed, 20 May 2026 23:27:46 +0000 (-0700) Subject: Added agent developer guide. X-Git-Tag: 1.0.0~55 X-Git-Url: http://git.kaiwu.me/postgresql/log/contrib/postgres_fdw/postgres_fdw.c?a=commitdiff_plain;h=621f7cf9918aaf532cad1bf9244b5d668225fa63;p=njs.git Added agent developer guide. The new top-level AGENTS.md (with CLAUDE.md symlinked to it) is the canonical index for agent and contributor instructions. Detailed guides live under docs/agent/: docs/agent/engine-dev.md building, testing, debugging the engine and the nginx modules. docs/agent/js-dev.md writing JavaScript for either engine, with the common nginx API surface and engine differences. docs/agent/js-dev-njs.md specifics of the deprecated njs engine and migration to QuickJS. --- diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..f1008db0 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,79 @@ +# njs — agent instructions + +njs is a JavaScript engine integrated with NGINX. It ships as: + +- a standalone CLI (`build/njs`) for testing and scripting, +- two NGINX modules: `ngx_http_js_module` and `ngx_stream_js_module`, +- two interchangeable JS engines selectable per location/server via the + `js_engine` directive: + - **njs** — the built-in engine, deprecated since 1.0.0. + - **QuickJS** — recommended. Set `js_engine qjs;` in nginx.conf. + +This file is the index. Detailed instructions live under [`docs/agent/`](docs/agent/). + +## Pick your task + +| If you are doing... | Read | +|---|---| +| Editing C in `src/`, `external/`, `nginx/` — engine, modules, build system | [docs/agent/engine-dev.md](docs/agent/engine-dev.md) | +| Writing JavaScript that runs in njs (CLI or NGINX), targeting either engine | [docs/agent/js-dev.md](docs/agent/js-dev.md) | +| Writing JavaScript that must run on the deprecated njs engine | [docs/agent/js-dev-njs.md](docs/agent/js-dev-njs.md) | + +## 1. Engine and module development (C) + +You are extending or fixing the engine, the QuickJS integration, or the +nginx modules. + +Quick facts: + +- **Build (CLI):** `./configure && make njs` → `build/njs`. Rebuild is fast. +- **Build (NGINX):** configure NGINX with `--add-module=/nginx` (static) + or `--add-dynamic-module=/nginx` (dynamic) in a separate NGINX tree. +- **Dual engine = dual code.** Most external modules ship both an `njs_*.c` + and a `qjs_*.c` implementation. If you change behavior on one side, change + it on the other. +- **Tests:** `make unit_test`, `make lib_test`, `make test262`. NGINX + integration tests under `nginx/t/` run with + `prove -I nginx/t/`. +- **Code style:** NGINX conventions — 4 spaces (no tabs), 80-column limit, + no trailing whitespace, newline after closing brace, `-Werror` build. +- **Commit subjects:** past tense, prefixed + (`HTTP:`, `Stream:`, `Core:`, `QuickJS:`, `Tests:`, …), ≤67 characters. + +Full details, sanitizer builds, VM architecture, and object model: +[docs/agent/engine-dev.md](docs/agent/engine-dev.md). + +## 2. Writing JavaScript for njs (CLI or NGINX) + +You are writing `.js` modules that run inside `js_content` / `js_filter` / +`js_set` / `js_access` / `js_preread` handlers, or under the standalone CLI. + +Orientation: + +- **Default to the QuickJS engine** (`js_engine qjs;`). The built-in njs + engine is deprecated since 1.0.0; write new code for QuickJS. +- **Language baseline.** QuickJS is ES2023; the njs engine is ES5.1 strict + with a curated ES6+ subset. See the + [compatibility page](https://nginx.org/en/docs/njs/compatibility.html). +- **Nginx drives the engine, not the JS.** Code only runs from + directive-bound entry points (HTTP: `js_content`, `js_access`, + `js_header_filter`, `js_body_filter`, `js_set`, `js_periodic`). +- **Quick test (CLI):** `./build/njs -c ''` or `./build/njs file.js`. +- **Test inside NGINX:** `prove -I nginx/t/.t` with + `TEST_NGINX_GLOBALS_HTTP='js_engine qjs;'` (and the same for `_STREAM`). + +Everything else — full integration-point semantics, `nginx.conf` wiring +(`js_shared_dict_zone`, `resolver` + `js_fetch_*`, `js_import` / +`js_path` / `js_engine`), bindings (`r`, `s`, `ngx.fetch`, `ngx.shared`, +`crypto`, …), engine-only features, do/don't recipes: +[docs/agent/js-dev.md](docs/agent/js-dev.md). For code that must run on +the deprecated njs engine, also see +[docs/agent/js-dev-njs.md](docs/agent/js-dev-njs.md). + +## Resources + +- [njs official documentation](https://nginx.org/en/docs/njs/) +- [Reference (API surface)](https://nginx.org/en/docs/njs/reference.html) +- [Compatibility](https://nginx.org/en/docs/njs/compatibility.html) +- [Engine selection](https://nginx.org/en/docs/njs/engine.html) +- [njs-examples repo](https://github.com/nginx/njs-examples/) diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/docs/agent/engine-dev.md b/docs/agent/engine-dev.md new file mode 100644 index 00000000..74227852 --- /dev/null +++ b/docs/agent/engine-dev.md @@ -0,0 +1,304 @@ +# Engine and module development (C) + +This document covers C development inside the njs repository: the engine +core (`src/`), the external module wrappers (`external/`), the NGINX +modules (`nginx/`), and the build system (`auto/`, `configure`). + +For per-task orientation see the top-level [AGENTS.md](../../AGENTS.md). + +## Building + +njs has no autotools/cmake. The shell-based `configure` script generates +`build/Makefile`. Always run `make clean` before reconfiguring with +different options — the Makefile is not regenerated in place. + +### Standalone CLI + +```bash +./configure +make -j$(nproc) njs # build/njs +``` + +### njs with QuickJS backend + +QuickJS is built separately and linked into njs. + +```bash +# Build libquickjs.a in the QuickJS source tree +( cd && CFLAGS=-fPIC make libquickjs.a ) + +# Configure njs to use it +make clean +./configure \ + --cc-opt='-I' \ + --ld-opt='-L' +make -j$(nproc) njs +``` + +### NGINX module (static / dynamic) + +njs builds as an NGINX module from a separate NGINX source tree. + +```bash +# Static +cd +./auto/configure --add-module=/nginx --with-stream --with-debug +make -j$(nproc) + +# Dynamic +./auto/configure --add-dynamic-module=/nginx --with-stream +make -j$(nproc) modules +``` + +Adding `--with-cc-opt='-I'` and +`--with-ld-opt='-L'` enables QuickJS in the NGINX module. + +### AddressSanitizer build + +```bash +./auto/configure --add-module=/nginx --with-stream --with-debug \ + --with-cc=clang \ + --with-cc-opt='-O0 -fsanitize=address' \ + --with-ld-opt='-fsanitize=address' +make -j$(nproc) +``` + +The njs `configure` exposes `--address-sanitizer=YES` directly when +building the CLI; prefer `clang` on arm64 (gcc ASan is slow there). + +### Configure options (njs) + +| Option | Purpose | +|---|---| +| `--cc=FILE` | C compiler (default: gcc) | +| `--cc-opt=OPTIONS` | Additional CFLAGS | +| `--ld-opt=OPTIONS` | Additional LDFLAGS | +| `--debug=YES` | Runtime checks | +| `--debug-memory=YES` | Memory allocation tracing | +| `--debug-opcode=YES` | Per-instruction execution trace | +| `--debug-generator=YES` | Bytecode generator trace | +| `--address-sanitizer=YES` | AddressSanitizer (use with `clang`) | +| `--with-quickjs` | Require QuickJS to be present | +| `--no-openssl` / `--no-libxml2` / `--no-zlib` | Drop optional deps | + +Run `./configure --help` for the complete list. + +## Testing + +```bash +make unit_test # 5800+ language and API tests +make lib_test # internal data structures (hash, rbtree, unicode) +make test262 # ECMAScript test262 compliance suite +make test # shell tests + unit_test + test262 +``` + +NGINX integration tests live under `nginx/t/` and use Perl's `prove` +harness against `Test::Nginx`. + +```bash +TMPDIR=$(mktemp -d) \ +TEST_NGINX_BINARY= \ + prove -I nginx/t/ +``` + +Useful environment variables: + +| Variable | Effect | +|---|---| +| `TEST_NGINX_BINARY` | Path to the nginx binary (required) | +| `TEST_NGINX_VERBOSE=1` | Verbose harness output | +| `TEST_NGINX_LEAVE=1` | Keep test artifacts in `$TMPDIR/nginx-test-*` | +| `TEST_NGINX_CATLOG=1` | Dump `error.log` after the run | +| `TEST_NGINX_GLOBALS=` | Inject global-scope config (e.g. `load_module ...`) | +| `TEST_NGINX_GLOBALS_HTTP='js_engine qjs;'` | Run http tests under QuickJS | +| `TEST_NGINX_GLOBALS_STREAM='js_engine qjs;'` | Same, for stream tests | + +Use a per-run `TMPDIR=$(mktemp -d)` to isolate artifacts across concurrent +runs and avoid destructive `rm -fr /tmp/nginx-test*`. + +For more on the harness see `/Test/Nginx.pm`. + +## Validation checklist + +Before submitting a change: + +1. `./configure && make -j$(nproc)` compiles without warnings (`-Werror`). +2. `make unit_test` and `make lib_test` pass. +3. If you touched `src/`, also run `make test262`. +4. If you touched `nginx/`, run `prove -I nginx/t/`, + once with the default engine and once with + `TEST_NGINX_GLOBALS_HTTP='js_engine qjs;'`. +5. New source files: update `auto/sources` (njs core), + `auto/modules` (njs external modules), or + `auto/qjs_modules` (QuickJS external modules). +6. Dual-engine: if you added/changed behavior in an `njs_*.c` module, + mirror it in the corresponding `qjs_*.c` (and vice versa). + +## Code style and commits + +NGINX coding style: + +- 4 spaces, no tabs. +- 80-column line limit. +- No trailing whitespace. +- Newline after closing brace. +- Comments explain *why*, not *what*; avoid em-dashes. +- `-Werror` is on by default — fix all warnings. + +Commit messages: + +- Past tense subject (`Added X`, `Fixed Y`). +- Subject ≤67 chars, body wrapped to ~80 chars. +- Subject prefix: `HTTP:`, `Stream:`, `Core:`, `QuickJS:`, `Tests:`, + `Modules:`, etc. +- One logical change per commit; rebase/squash before submitting. + +## Project layout + +``` +njs/ +├── configure # build entry point +├── auto/ # shell-based build system +│ ├── sources # njs core source list +│ ├── modules # njs external module list +│ ├── qjs_modules # QuickJS external module list +│ └── cc, options, ... # compiler/option detection +├── src/ # engine core (C) +│ ├── njs_vm.c / njs_vmcode.c # virtual machine +│ ├── njs_lexer.c # tokenizer +│ ├── njs_parser.c # parser +│ ├── njs_generator.c # bytecode generator +│ ├── njs_object.c / njs_array.c # built-in types +│ ├── njs_promise.c / njs_async.c +│ ├── njs_value.h # value representation +│ ├── njs.h # public C API +│ ├── qjs.c # QuickJS engine wrapper +│ └── test/ # C unit tests +├── external/ # extension modules +│ ├── njs_shell.c # CLI entry point (main()) +│ ├── njs_*_module.c # njs-engine modules (crypto, fs, ...) +│ └── qjs_*_module.c # QuickJS-engine counterparts +├── nginx/ # NGINX module integration +│ ├── ngx_http_js_module.c +│ ├── ngx_stream_js_module.c +│ ├── ngx_js.c # core nginx-JS bindings +│ ├── config # NGINX build glue +│ └── t/ # Perl integration tests +├── test/ # functional test suite +│ ├── js/ # JS language feature tests +│ ├── harness/ # test framework utilities +│ └── shell_test.exp # interactive shell tests (Expect) +└── ts/ # TypeScript type definitions +``` + +Public C API: `src/njs.h`. VM internals: `src/njs_vm.h`, +`src/njs_value.h`. CLI entry point: `external/njs_shell.c`. + +## VM architecture (njs engine) + +The QuickJS backend uses upstream QuickJS internals (see +[bellard.org/quickjs](https://bellard.org/quickjs/)). What follows is the +**njs engine** internals only. + +### Register-based VM + +Each instruction has operands that are immediate values or **indexes**. +An index is encoded as: + +``` +index | level_type (4 bits) | var_type (4 bits) +``` + +### Level types (storage location) + +``` +NJS_LEVEL_LOCAL = 0 // local variable in current frame +NJS_LEVEL_CLOSURE = 1 // closure variable from parent frame +NJS_LEVEL_GLOBAL = 2 // global variable +NJS_LEVEL_STATIC = 3 // static / absolute scope +``` + +Values are addressed as `vm->levels[NJS_LEVEL_*][index]`. + +### Variable types + +``` +NJS_VARIABLE_CONST = 0 +NJS_VARIABLE_LET = 1 +NJS_VARIABLE_CATCH = 2 +NJS_VARIABLE_VAR = 3 +NJS_VARIABLE_FUNCTION = 4 +``` + +### Bytecode example + +``` +$ ./build/njs -d +>> var a = 42; function f(v) { return v + 1 } + +shell:main + 1 | 00000 MOVE 0123 0133 + 1 | 00024 STOP 0033 + +shell:f + 1 | 00000 ADD 0203 0103 0233 + 1 | 00032 RETURN 0203 +``` + +`MOVE 0123 0133` copies the value at index `0x0133` to `0x0123`. +`ADD a b c` computes `a = b + c`. Indexes are printed in hex and encode +level and variable type. + +## Object model (njs engine) + +For performance and footprint, a JS object is split into a **local mutable +hash** for the current object and a **shared hash** holding inherited +properties. Built-ins are lazily materialized: the shared definitions stay +shared until first mutation. For functions, the first access copies the +function from the shared hash into the local mutable hash so the +per-object copy can be modified. + +Key entry points: + +- `njs_value_property()` — top-level property lookup. +- `njs_property_query()` — lookup with descriptor result. +- `njs_object_property_query()` — object-level walk including prototype. +- `njs_prop_private_copy()` — promotion from shared to local on write. + +## Debugging + +### CLI + +```bash +./build/njs -c '' # one-shot +./build/njs -d # interactive, with disassembly +./build/njs -d script.js # dump bytecode for a script +./build/njs -o script.js # opcode trace +./build/njs -h # full option list +``` + +Select the JavaScript engine with `-n ` (case-insensitive; default +is `njs`): + +```bash +./build/njs -n njs -c 'console.log(typeof Map)' # built-in engine +./build/njs -n QuickJS -c 'console.log(typeof Map)' # QuickJS backend +``` + +`-n QuickJS` requires the binary to be built with QuickJS linked in (see +[njs with QuickJS backend](#njs-with-quickjs-backend) above); otherwise +the CLI reports `unknown engine "QuickJS"`. + +### Opcode trace + +Built with `--debug-opcode=YES`, `./build/njs -o script.js` prints each +instruction as it executes — `ENTER`/`EXIT` for function boundaries, +opcode mnemonics for everything else. Useful for confirming control flow +through bytecode without a debugger. + +### Test failures (NGINX) + +With `TEST_NGINX_LEAVE=1`, each test leaves +`$TMPDIR/nginx-test-/` containing the generated `nginx.conf`, +`error.log`, and any artifacts. `TEST_NGINX_CATLOG=1` dumps the log to +stdout automatically. diff --git a/docs/agent/js-dev-njs.md b/docs/agent/js-dev-njs.md new file mode 100644 index 00000000..ac062dd2 --- /dev/null +++ b/docs/agent/js-dev-njs.md @@ -0,0 +1,78 @@ +# Writing JavaScript for the deprecated njs engine + +This document is for code that must run under the built-in **njs** +JavaScript engine. The njs engine is deprecated since 1.0.0; the +QuickJS engine is the recommended path for any new code. Read this only +if you maintain an existing njs-engine codebase that you cannot port +yet. For general JS development under njs (both engines) see +[docs/agent/js-dev.md](js-dev.md). + +## Language baseline + +The njs engine implements **ECMAScript 5.1 (strict mode)** plus a +curated set of ES6+ extensions. The authoritative list lives on the +[compatibility page](https://nginx.org/en/docs/njs/compatibility.html). + +What is shipped (highlights): + +- Arrow functions, `let`/`const`, template literals. +- `Promise`, full prototype methods. `async` / `await` inside `async` + functions (no top-level `await`). +- Rest parameters: `function f(...rest)`. +- Optional chaining `?.`, nullish coalescing `??`, logical assignments + `||=` / `&&=` / `??=` (since 0.9.6). +- ES2016 exponentiation operator `**`. +- `Symbol` subset (`for`, `keyFor`). +- ES modules: **default `import` / default `export` only**. Non-default + forms (`import { x } from "..."`, `import * as m from "..."`, + `import "..."`) are rejected with `Non-default import is not supported`. +- `require()` is still supported (deprecated; prefer `import`). + +## njs-engine-only features + +The notable ones are `js_preload_object`, `njs.dump()` / +`console.dump()`, and `require()`. None of them work on QuickJS. See +[Engine differences](js-dev.md#engine-differences-at-a-glance) and +[Engine-specific bindings](js-dev.md#engine-specific-bindings) in +`js-dev.md` for the full list and links; the section below covers how +to remove each one when porting. + +## Migration to QuickJS + +When porting an existing njs-engine module: + +1. Build NGINX with QuickJS linked in + (`--with-cc-opt=-I --with-ld-opt=-L`), and + set `js_engine qjs;` in the relevant `http { }` or `stream { }` + block. +2. Replace `require()` calls with `import` statements. +3. Remove calls to `njs.dump()` / `console.dump()`; switch to + `JSON.stringify` or a small helper. +4. If you used `js_preload_object`, fold the data into a regular module + that the code `import`s. The shared dictionary + (`ngx.shared` + `js_shared_dict_zone`) is the equivalent of preload + for cross-worker shared state. +5. Modernize syntax — destructuring, `class`, spread, `Map`/`Set` — + freely. They are all available under QuickJS. +6. Re-run the test suite under both engines until parity, then drop the + njs-engine variant: + +```bash +# njs (deprecated) +TEST_NGINX_BINARY= \ + prove -I nginx/t/.t + +# QuickJS +TEST_NGINX_GLOBALS_HTTP='js_engine qjs;' \ + TEST_NGINX_BINARY= \ + prove -I nginx/t/.t +``` + +## Resources + +- [Compatibility (full ECMAScript list)](https://nginx.org/en/docs/njs/compatibility.html) +- [Engine selection (deprecation note)](https://nginx.org/en/docs/njs/engine.html) +- [Reference (API surface — same on both engines)](https://nginx.org/en/docs/njs/reference.html) +- [TypeScript type definitions in `ts/`](../../ts/) — authoritative + per-symbol interface description, identical on both engines +- [General JS development guide (both engines)](js-dev.md) diff --git a/docs/agent/js-dev.md b/docs/agent/js-dev.md new file mode 100644 index 00000000..fdd54580 --- /dev/null +++ b/docs/agent/js-dev.md @@ -0,0 +1,367 @@ +# Writing JavaScript for njs (both engines) + +This document is for authors of `.js` modules that run inside `js_content` +/ `js_filter` / `js_set` / `js_access` / `js_preread` handlers, or under +the standalone `build/njs` CLI. It covers the common runtime, the nginx +API surface, and the points where the two engines differ. + +For per-task orientation see the top-level [AGENTS.md](../../AGENTS.md). +For code that must run on the deprecated **njs** engine, also read +[docs/agent/js-dev-njs.md](js-dev-njs.md). + +## Pick an engine + +njs ships two interchangeable JavaScript engines: + +| Engine | Language baseline | Status | When to pick | +|---|---|---|---| +| **QuickJS** | ES2023 | **Recommended** | All new code. Modern JS works as-is. | +| **njs** | ES5.1 strict + curated ES6+ | Deprecated since 1.0.0 | Only when maintaining existing njs-engine code that you cannot port yet. | + +Select per-context with `js_engine` in nginx.conf: + +```nginx +http { + js_engine qjs; # http-wide default + js_import http.js; + + server { + # js_engine inherits, can be overridden per server or location + } +} +``` + +The same `js_engine` directive exists for `stream { }`. From the CLI use +`-n njs` or `-n QuickJS` (the latter requires a build with QuickJS +linked in). + +## Engine differences at a glance + +| Feature | njs engine | QuickJS engine | +|---|---|---| +| `class` | ✗ | ✓ | +| Generators (`function*`, `yield`) | ✗ | ✓ | +| `async` / `await` | ✓ | ✓ | +| Spread in calls / array literals (`f(...a)`, `[...a]`) | ✗ | ✓ | +| Rest parameter (`function f(...rest)`) | ✓ (no destructuring) | ✓ | +| Destructuring (`{a,b} = x`, `[a,b] = x`) | ✗ | ✓ | +| Optional chaining `?.`, nullish `??`, `??=`/`&&=`/`\|\|=` | ✓ (since 0.9.6) | ✓ | +| `Map`, `Set`, `WeakMap`, `WeakSet` | ✗ | ✓ | +| `BigInt`, `Proxy`, `Reflect` | ✗ | ✓ | +| Template literals | ✓ | ✓ | +| `Promise`, full | ✓ | ✓ | +| `Symbol` subset (`for`, `keyFor`) | ✓ | ✓ | +| Module imports (`import`/`export`) | ✓ default only | ✓ | +| Non-default imports (`import {x}`, `import *`, `import "s"`) | ✗ | ✓ | +| `require()` | ✓ | ✗ (use `import`) | +| `njs.dump()`, `console.dump()` | ✓ | ✗ | +| `js_preload_object` | ✓ | ✗ | +| Native modules (`js_load_*_native_module`) | ✗ | ✓ | + +For the full ECMAScript compatibility list of the njs engine, see the +[compatibility page](https://nginx.org/en/docs/njs/compatibility.html). + +## Integration points (where JS runs) + +**JS code in njs does not run on its own.** There is no main loop, no +background thread, no startup script. Every JS function executes because +some `nginx.conf` directive bound it to a phase of request processing +and nginx invoked it. You cannot register an event listener from JS, +schedule work outside a directive-driven entry, or keep code running +after the handler returns. The single near-exception is `js_periodic`, +whose trigger is still nginx's timer — nothing self-starts from JS. + +Each directive below defines when the handler runs, what context object +is exposed, and how it terminates. + +**HTTP (`ngx_http_js_module`)** + +| Directive | When | Context | Termination | +|---|---|---|---| +| `js_content module.fn` | content phase, replaces upstream | `r` | `r.return()` or `r.send()` + `r.finish()` | +| `js_access module.fn` | access phase | `r` | `r.return(403\|...)` to deny; otherwise fall through | +| `js_header_filter module.fn` | response header filter | `r` (mutate `headersOut`) | synchronous return | +| `js_body_filter module.fn [buffer_type=string\|buffer]` | response body filter | `r`, plus `(data, flags)` | `r.sendBuffer(out, flags)` | +| `js_set $var module.fn [nocache]` | variable evaluation | `r` | return value (synchronous) | +| `js_periodic module.fn interval=...` | timer, no request | none | implicit return | + +**Stream (`ngx_stream_js_module`)** + +| Directive | When | Context | Termination | +|---|---|---|---| +| `js_preread module.fn` | before upstream connects | `s` | `s.allow()` / `s.deny()` / `s.done()` | +| `js_filter module.fn` | data filter, both directions | `s` (subscribe with `s.on()`) | `s.done()` | +| `js_access module.fn` | access | `s` | `s.allow()` / `s.deny()` | +| `js_set $var module.fn` | variable evaluation | `s` | return value (synchronous) | +| `js_periodic module.fn interval=...` | timer, no session | none | implicit return | + +Notes: + +- **Async support is not uniform.** `js_content` and `js_access` accept + fully async handlers (returning a `Promise` or using `await`). The + remaining HTTP handlers (`js_header_filter`, `js_body_filter`, + `js_set`) reject async work with `"async operation inside ... handler"` + if `await` hits the event loop. `js_set` may still return an + already-resolved `Promise`. +- **`js_body_filter`** is invoked once per response chunk; the last call + has `flags.last === true`. Pick the chunk shape with + `buffer_type=string|buffer`. +- **`js_header_filter` / `js_body_filter`** see the *response*; they + cannot read the request body via `r.requestText` / `r.readRequest*()`. +- **`js_periodic`** lives in a dedicated `location @name { }` block and + runs without a client request. Pin to specific workers with + `worker_affinity`. + +## Runtime model (common to both engines) + +- **Nginx drives the engine, not the JS.** Every execution begins at a + directive-bound entry point (see + [Integration points](#integration-points-where-js-runs)) and ends when + that handler resolves. There is no top-level long-lived script; work + that escapes the handler's lifetime is on its own. +- **Per-request isolation.** For each incoming HTTP request or stream + session, the JS module creates a fresh VM. State you put in module + scope is visible to subsequent requests on the same worker but cannot + be used to carry per-request data — it leaks across requests. +- **Worker isolation.** nginx is multi-process. Module scope is not + shared across workers. To share state across workers, use + [`ngx.shared`](https://nginx.org/en/docs/njs/reference.html#ngx_shared) + (shared dictionary). For request-scoped per-worker state, use a `Map` + / object keyed by some request identifier, but mind eviction. +- **Event loop.** Async work is driven by nginx's event loop; + `r.subrequest()`, `ngx.fetch()`, and `await` integrate with it. + `setTimeout` / `clearTimeout` are available in both the CLI and + inside nginx handlers (HTTP, stream, periodic). +- **Top-level `await`** — QuickJS engine only. The njs engine requires + `await` to appear inside an `async` function and reports + `await is only valid in async functions` otherwise. +- **Module imports load once per worker.** Don't perform expensive setup + in module scope unless it's truly initialization. + +## NGINX bindings (common API surface) + +The full surface is documented in the +[Reference](https://nginx.org/en/docs/njs/reference.html). The +TypeScript declaration files under [`ts/`](../../ts/) are the +authoritative per-symbol description and apply to both engines. +Highlights: + +- **`r` (HTTP request).** Inside `js_content` / `js_filter` / + `js_access` / `js_set` handlers. + - Body: + - `js_content` only: `r.requestText`, `r.requestBuffer` (synchronous + accessors; require the body to be in memory — set + `client_max_body_size` and `client_body_buffer_size` accordingly). + - `js_access` and `js_content`: + `await r.readRequestText()`, `await r.readRequestArrayBuffer()`, + `await r.readRequestJSON()`, + `await r.readRequestForm()` (parses form/multipart). The body + is read once and cached; subsequent reads resolve from cache. + - Reply: `r.return(status, [body])`, `r.send(chunk)`, `r.finish()`, + `r.error(msg)`, `r.warn(msg)`, `r.log(msg)`. + - Subrequest: `await r.subrequest(uri[, options])`. + - Headers/vars: `r.headersIn`, `r.headersOut`, `r.variables`, + `r.rawHeadersIn/Out`. + - Internal redirect: `r.internalRedirect(uri)`. +- **`s` (Stream session).** Inside `js_preread` / `js_filter` / + `js_access` for the stream module. + - I/O: `s.send(data[, options])`, `s.on(event, cb)`, + `s.allow()` / `s.deny()`, `s.done()`. +- **`ngx.fetch(url[, options])`** — async HTTP client (request body, + headers, timeouts, TLS). Always set explicit timeouts. +- **`ngx.shared`** — process-wide shared dictionary configured via + `js_shared_dict_zone`. +- **Built-in modules.** Import with `import`: + - `crypto`, `buffer`, `fs`, `querystring`, `xml`, `zlib`. + - `WebCrypto` is available at `crypto.subtle` (since 0.8.10). + - `TextEncoder` / `TextDecoder` globals (since 0.8.10). +- **`process`** — argv/env (since 0.8.8). + +## NGINX configuration (nginx.conf) + +What you need to wire up so the JS bindings work. Defaults below match +the current code; see the +[reference](https://nginx.org/en/docs/njs/reference.html) for full +grammar and scope. + +### Loading modules + +```nginx +http { + js_path "/etc/nginx/njs/"; # module search path + js_import utils.js; # imports default export as `utils` + js_import foo from helpers/foo.js; # explicit local name + js_engine qjs; # recommended (default: njs) +} +``` + +`js_import` is in `http` / `stream` scope. `js_engine` is in +`http` / `server` / `location` (HTTP) and `stream` / `server` (Stream). + +### Variables + +```nginx +js_var $cache_key ''; # writable variable, default empty +js_set $token auth.gen_token; # bind a $var to a JS function +``` + +`js_set` evaluates lazily on first reference and caches the result for +the lifetime of the request; append `nocache` to recompute on every +reference. + +### `ngx.shared` — cross-worker dictionary + +```nginx +http { + # zone=name:size [type=string|number] [timeout=t] [evict] + js_shared_dict_zone zone=cache:1m timeout=60s evict; + js_shared_dict_zone zone=counters:32k type=number; +} +``` + +```js +ngx.shared.cache.set('k', 'v'); // string zone +ngx.shared.counters.incr('hits', 1); // number zone +``` + +`type=string` is the default. `evict` lets the LRU drop entries when the +zone is full; without it, `set()` fails once the zone is exhausted. +`timeout=` sets the default TTL (per-key TTL can override it). + +### `ngx.fetch()` — outgoing HTTP client + +A `resolver` is **required** when fetching by hostname. The `js_fetch_*` +directives sit in `http` / `server` / `location` (and the matching +stream scopes). Defaults shown: + +```nginx +http { + resolver 127.0.0.1 ipv6=off; + resolver_timeout 5s; + + js_fetch_timeout 60s; # total request timeout + js_fetch_buffer_size 16k; # per-connection read buffer + js_fetch_max_response_buffer_size 1m; # response body cap + + # HTTPS + js_fetch_trusted_certificate /etc/ssl/ca.pem; + js_fetch_ciphers HIGH:!aNull:!MD5; + js_fetch_protocols TLSv1.2 TLSv1.3; + js_fetch_verify on; # default on + js_fetch_verify_depth 1; + + # Connection pool (default: disabled) + js_fetch_keepalive 32; # max idle connections / worker + js_fetch_keepalive_requests 1000; + js_fetch_keepalive_time 1h; + js_fetch_keepalive_timeout 60s; + + # Forward proxy for outgoing fetches + js_fetch_proxy http://user:pass@proxy:3128; +} +``` + +The same directives exist under `stream { }` for `ngx.fetch()` from +stream handlers. + +### `js_periodic` — timer jobs + +`js_periodic` lives in its own `location @name { }` block (no client +request reaches it): + +```nginx +location @cron { + js_periodic tasks.tick interval=10s; + js_periodic tasks.cleanup interval=1m jitter=5s worker_affinity=all; +} +``` + +`worker_affinity` accepts `all` (every worker) or a bitmask +(e.g. `0101` runs on workers 0 and 2). `jitter` randomizes start to +spread load across workers. + +Engine-specific directives (`js_preload_object`, `js_load_*_native_module`) +are covered in [Engine-specific bindings](#engine-specific-bindings) below. + +## Engine-specific bindings + +- **`js_preload_object`** — preload an immutable shared object at config + load. **njs engine only.** See + [Preloaded objects](https://nginx.org/en/docs/njs/preload_objects.html). +- **Native modules** (`js_load_http_native_module` / + `js_load_stream_native_module`) — load a shared library as a JS + module. **QuickJS only.** See + [Native modules](https://nginx.org/en/docs/njs/native_modules.html). +- **`njs.dump()`, `console.dump()`** — pretty-print with hidden + properties. njs-engine only. + +## How to test + +### Standalone + +```bash +./build/njs -c 'console.log(typeof Map)' # under default njs engine +./build/njs -n QuickJS script.js # under QuickJS (if linked in) +./build/njs -m module.mjs # load as ES module +``` + +### Inside NGINX + +```bash +TMPDIR=$(mktemp -d) \ +TEST_NGINX_BINARY= \ + prove -I nginx/t/.t +``` + +Run twice, once per engine: + +```bash +# njs engine (default) +prove -I nginx/t/.t + +# QuickJS engine +TEST_NGINX_GLOBALS_HTTP='js_engine qjs;' \ + prove -I nginx/t/.t +# (use TEST_NGINX_GLOBALS_STREAM for stream tests) +``` + +Examples of well-shaped test files: anything under `nginx/t/js_*.t`. + +## Do / Don't + +**Do** + +- Default to the QuickJS engine for new code. +- Use `import` / `export` (ES modules); never `require()`. +- Use `ngx.shared` for cross-worker state; document the zone's + `keys`/`value` size in nginx.conf. +- Use `await` in handlers — return a `Promise` (implicit via `async`) or + call `r.return()` / `r.finish()` to terminate. +- Keep module-scope work to true one-time initialization + (configuration, schema compilation, etc.). + +**Don't** + +- Don't keep per-request state in module scope — it leaks across + requests handled by the same worker. +- Don't assume workers share memory — they don't. Use `ngx.shared`. +- Don't try to outlive the handler. A `Promise` you don't `await`, a + `setTimeout` you queue after `r.finish()`, an `ngx.fetch()` you fire + and forget — none of that is guaranteed to complete. Once the + handler resolves, the request context goes away and pending JS work + is dropped. If you need recurring work, use `js_periodic`. +- Don't rely on engine-specific extensions in code that should run on + both engines: `njs.dump()` / `console.dump()`, `js_preload_object`, + native modules, top-level `await`, non-default imports. + +## Resources + +- [Reference (full API)](https://nginx.org/en/docs/njs/reference.html) +- [Compatibility (njs engine)](https://nginx.org/en/docs/njs/compatibility.html) +- [Engine selection](https://nginx.org/en/docs/njs/engine.html) +- [Preloaded objects (njs-only)](https://nginx.org/en/docs/njs/preload_objects.html) +- [Native modules (qjs-only)](https://nginx.org/en/docs/njs/native_modules.html) +- [TypeScript type definitions in `ts/`](../../ts/) — + `ngx_http_js_module.d.ts`, `ngx_stream_js_module.d.ts`, `ngx_core.d.ts`, + `njs_webapi.d.ts`, `njs_webcrypto.d.ts` (same surface on both engines) +- [njs-examples](https://github.com/nginx/njs-examples/)