Dmitry Volyntsev [Thu, 22 Oct 2026 04:49:20 +0000 (21:49 -0700)]
Parser: fix reference atom IDs on big-endian systems
Previously, references to local variables and function arguments
failed with ReferenceError on 64-bit big-endian systems. The parser
overwrote a uintptr_t reference atom ID through the overlapping
uint32_t value member, replacing the high half of the ID.
The issue was introduced in 9ff881e3 (1.0.2), which moved the
reference atom ID to the start of the parser node union.
Now, reference and value payloads share a uint32_t atom ID at the same
offset, initialized in the corresponding parser paths.
This removes the redundant value-member write and makes atom ID access
independent of byte order without increasing the parser node size.
Dmitry Volyntsev [Thu, 22 Oct 2026 23:27:06 +0000 (16:27 -0700)]
CI: split CLI and nginx checks into parallel jobs
Previously, check-pr runs all build configurations sequentially, so
an early failure prevents later configurations from being tested.
Split CLI checks into matrix configurations and nginx checks
into separate njs, QuickJS, and QuickJS-NG matrix jobs. Disable
fail-fast and use fresh build directories without make clean.
Preserve the nginx module and SSL configure flags, sanitizer and
debug-memory settings, QuickJS-NG version, checkout action revision,
test commands, and failed-test retries.
Make engine selection explicit: njs jobs no longer auto-detect
QuickJS-NG, and the QuickJS CLI build uses an explicit library.
Existing archives prevent the nested make from checking engine source
changes. QuickJS builds also omit libnjs.a and engine headers from
the module dependencies.
Changes to engine sources are now picked up by nginx make without
rerunning configure or removing the existing libraries.
Dmitry Volyntsev [Thu, 22 Oct 2026 00:37:10 +0000 (17:37 -0700)]
Modules: isolate engine builds from the standalone CLI
nginx and the CLI share build artifacts and configuration. Building
nginx replaces the CLI Makefile, while rebuilding the CLI can leave
nginx linking libraries compiled with incompatible flags.
Building nginx no longer changes the CLI configuration or libraries.
Rebuilding the CLI no longer breaks subsequent nginx builds.
Dmitry Volyntsev [Thu, 22 Oct 2026 14:14:14 +0000 (07:14 -0700)]
Configure: fix pkg-config paths for absolute build directories
The generated njs.pc prepends the source directory to an absolute
build directory, producing invalid include and library paths.
Preserve absolute build paths and resolve only relative ones against
the source directory. This also supports engine builds embedded in
nginx without changing the standalone CLI configuration.
Independent configure runs can select different QuickJS libraries.
Allow callers to pass the selected headers and linker options without
falling back to another library.
Dmitry Volyntsev [Thu, 22 Oct 2026 00:35:52 +0000 (17:35 -0700)]
Configure: add configure mode without a source-root Makefile
Configuring a separate build directory still replaces the source-root
Makefile and redirects subsequent CLI builds. Allow embedded builds
to leave the standalone build entry point unchanged.
Dmitry Volyntsev [Thu, 15 Oct 2026 06:06:08 +0000 (23:06 -0700)]
WebCrypto: return false for wrong-length ECDSA signatures
ECDSA signatures of an invalid length cause verify() to throw instead
of returning false. njs passes an empty DER signature to
EVP_PKEY_verify(), while QuickJS rejects the length during conversion.
The njs behavior dates back to 86c202c7 (0.7.7); QuickJS rejects these
signatures since 75ca26f8 (0.8.10).
Treat a length mismatch as an invalid signature, as required by
WebCrypto ECDSA Verify, while preserving diagnostics for conversion
failures. Test all three supported curves with invalid lengths and
a valid signature.
Dmitry Volyntsev [Tue, 13 Oct 2026 05:37:30 +0000 (22:37 -0700)]
WebCrypto: fix algorithm availability in nginx modules
Previously, configure probes disabled Ed25519, X25519, and AES-KW in
nginx modules because the engine is configured with --no-openssl,
although WebCrypto uses nginx's OpenSSL and these algorithms were
available.
The fix is to detect 25519 APIs from OpenSSL headers and look up AES-KW
ciphers at runtime, removing the dependency on engine configuration
and direct references to wrap functions absent in some OpenSSL flavors.
Dmitry Volyntsev [Sat, 10 Oct 2026 01:54:41 +0000 (18:54 -0700)]
Modules: treat expired shared dict entries as absent
Expired entries are hidden by read APIs but can still be found by
write operations. replace() can revive an expired entry, delete()
can report that it existed, and incr() can use its old value instead
of init. Without an explicit timeout, incr() can return a value that
a subsequent get() cannot retrieve.
Use a common write-side lookup to remove expired entries and handle
them as missing in both engines. incr() then creates a new entry
with init plus delta and the directive or explicit timeout.
Also return false instead of undefined when QuickJS delete() is
called for a missing key.
Dmitry Volyntsev [Sat, 10 Oct 2026 01:40:28 +0000 (18:40 -0700)]
Modules: fix shared dict state JSON separators
When saving a shared dict zone with timeout=, expired entries are
skipped. A comma is appended whenever another tree node follows a live
entry, even if all remaining nodes are expired. This can leave a
trailing comma in the state file, making it invalid JSON.
Insert commas before live entries only when a previous entry has been
rendered, so skipped entries do not affect the separators.
Dmitry Volyntsev [Sat, 10 Oct 2026 01:36:20 +0000 (18:36 -0700)]
Modules: initialize shared dict state entry expiration
When a shared dict state entry omits the optional expire field, its
expiration remains uninitialized. In a zone with timeout=, the entry
can be discarded as expired or loaded with an arbitrary expiration
instead of receiving the zone timeout.
Initialize expiration to zero before parsing the entry so an omitted
expire field is treated like an explicit zero. The entry then receives
the zone timeout and the state file is scheduled for saving.
Dmitry Volyntsev [Wed, 14 Oct 2026 23:39:07 +0000 (16:39 -0700)]
Modules: limit waiting for shared dict state save on exit
If a worker dies while saving a shared dict zone state, the writing
flag remains set in shared memory and survives reloads. Workers
with unsaved changes then receive NGX_AGAIN on every attempt and
keep rearming the save timer, so they never finish graceful shutdown.
Limit how long an exiting worker waits for an outstanding save to
60 seconds since the save started. The shared writing flag is not
cleared and the writer is not interrupted, so a slow save is not
disturbed. Running workers keep waiting as before.
Dmitry Volyntsev [Sat, 10 Oct 2026 01:32:25 +0000 (18:32 -0700)]
Modules: track shared dict state changes per worker
With frequent changes to a shared dict zone using state=, old workers
can remain in shutting down after a configuration reload and continue
consuming resources even though their changes have already been included
in another worker's snapshot. The shared dirty flag cannot distinguish
those changes from subsequent changes made by new workers, so old
workers can keep rearming their save timers while competing for writes.
Replace the shared dirty flag with a write_id counter and record its
value locally in each worker on mutation. Advance the shared counter
after preparing a snapshot so workers whose changes are covered can
stop retrying regardless of subsequent mutations. On I/O failure, the
writer records the advanced counter locally to retain responsibility
for retrying the save.
Dmitry Volyntsev [Sat, 10 Oct 2026 01:16:25 +0000 (18:16 -0700)]
Modules: limit shared dict state save frequency
Saving shared dict zone content to file is scheduled with a maximum
rate of one write per second in each worker. The total rate can be
larger by a factor of the number of running workers.
Postpone a zone save if the previous save was less than one second
ago, using a shared timestamp to limit the rate across all workers.
Dmitry Volyntsev [Sat, 10 Oct 2026 01:03:47 +0000 (18:03 -0700)]
Modules: fix shared dict state save error ordering
On save failure, writing is cleared before dirty is restored. Another
worker can observe a clean, idle zone and stop retrying. If the failed
writer is exiting, it also stops retrying and the changes remain unsaved.
Set dirty first and add a memory barrier before clearing writing to
publish the need for another save before releasing the writer.
vitalyn [Fri, 2 Oct 2026 21:50:43 +0000 (23:50 +0200)]
Modules: fixed shared dict state save stuck forever
The "dirty" and "writing" flags of a shared dict were bit fields of
the same word. ngx_js_dict_save() clears "writing" without holding
the lock, while set(), incr(), delete() and clear() set "dirty" under
the write lock in other workers. The two read-modify-write operations
could interleave and leave "writing" set for good.
After that, every save attempt returned NGX_AGAIN: the state file was
never written again, and the save timer, re-armed every second and not
cancelable, prevented any worker of the zone from finishing a graceful
shutdown, regardless of worker_shutdown_timeout. The flag lives in
shared memory, so it survived reloads.
The fix is to store the flags in separate words.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This fixes #1136.
Modules: abort shared dict state save on close failure
A close failure may report a delayed write error. Do not rename the
temporary file over the previous state file in this case. Invalidate
the descriptor before cleanup to avoid closing it twice.
Modules: initialize shared dict state file before allocation
Initialize the state file descriptor before allocating chain links and
buffers. Allocation failures otherwise reach cleanup with an
uninitialized descriptor and may close an unrelated file.
Dmitry Volyntsev [Tue, 11 Aug 2026 21:29:33 +0000 (14:29 -0700)]
Modules: preserve shared dict entries on failed updates
Shared dictionary updates temporarily remove timeout entries from the
expiration rbtree to prevent eviction while allocating a replacement.
ngx_rbtree_delete() clears the removed node key, so the failure path
reinserted the entry with a zero expiration and made its old value appear
expired.
Factor the shared-memory update into an engine-independent helper and
restore the previous expiration when allocation fails. Keep value
conversion in the njs and QuickJS wrappers.
Previously, $attr$ returned undefined for attributes containing entity
references, while $attrs returned only the first text fragment. libxml2
canonicalization expands the complete attribute value, creating an
interpretation difference for signed XML consumers.
Use xmlNodeGetContent() for both getters in the njs and QuickJS backends.
This preserves the existing explicit-attribute lookup and makes the
JavaScript value agree with canonicalization.
Previously, XML parser contexts were retained with their parsed documents.
When parsing failed in the njs engine, no cleanup was registered, leaking
the context and allowing repeated malformed XML input to grow memory usage
in long-lived NGINX worker processes.
The fix is to limit parser contexts to the parse operation and release
them immediately after parsing or error reporting. The VM cleanup now owns
only the parsed document.
VM: report import of a module removed after failed compilation
A module whose compilation fails is removed from the cache, but another
module compiled earlier in the same cycle may still import it. Executing
such an import returned an error without setting an exception, so the
shell reported an unrelated TypeError.
Throw "Cannot load module" instead, and add a shell test for a cyclic
dependency of a module that fails to compile.
VM: preserve dependency code after failed module compilation
Commit fd54d52d freed all bytecode added after the parent module's
compilation checkpoint, including successfully compiled dependencies.
Those dependencies remain cached, so a later import in the same VM
executes freed bytecode. This is reachable in the interactive shell
and in embedders that continue after a compilation error.
The fix is to remove eager bytecode cleanup while retaining removal of
the failed module from the cache so subsequent loads can retry. Keep
the removed module and its name in the VM pool because cyclic
dependencies and code metadata can still reference them. These
allocations are released when the VM is destroyed.
Add a shell regression test that retries a failed parent import and then
executes its cached dependency.
QuickJS exposes child elements as own properties, so a child named
like an XMLNode method hides the method on its prototype. Let methods
take precedence over bare child names, as on the njs engine. Keep the
child accessible through $tag$name and avoid deleting it when deleting
the method name.
Cover collisions for all XMLNode methods and prototype changes.
The existing $attr$name and $tag$name accessors compare local names
only, so x:ID and ID, or x:Issuer and Issuer, are indistinguishable.
The new XMLNode methods getAttributeNS(namespaceURI, localName) and
getChildNS(namespaceURI, localName) match both the namespace URI and
the local name.
Previously, property existence checks such as "'$attr$a' in node",
"'to' in node", or "'$tags$to' in node" created the property value and
never released it when no property descriptor was requested. A leaked
XMLNode also held a reference to its document, so the whole document
was never freed.
Also, qjs_xml_node_make() and qjs_xml_attr_make() incremented the
document reference count before creating the object, so a failed
JS_NewObjectClass() left the document permanently referenced.
The fix is to free the value when no descriptor is requested and to
take the document reference only after the object is created.
Previously, Expect treated timeouts as successful matches, and EOF
passed even if the REPL exited abnormally. Some checks could match
the input echo alone, while completion checks required a bell that
varies between libedit versions.
The fix is to fail on timeouts, premature EOF, and abnormal exit;
check the result and prompt for Unicode input and guard against a
repeated promise rejection. Send Unicode as ASCII escapes to work
around libedit 3.1.snap20150325-2.12 on SLES 15 SP7, which reports
a character count where the readline callback expects a byte count.
QuickJS: stop tracking rejections during final report
Converting the rejection reason to a log string may reject another
promise. Disable the tracker before reporting so new rejections
cannot leave references in the request context after its list has
been drained.
Previously, njs_engine_qjs_unhandled_rejection() kept a pointer to
the rejected promise list while converting the first rejection reason
to a string. The conversion may call user JavaScript code, which can
reject more promises and cause the list to be reallocated. After that,
the old pointer was used together with the new item count to free the
promise and reason values, which caused out-of-bounds reads and invalid
frees.
The fix is to take a reference to the first reason and free the list
before conversion, so that rejections created during conversion are
tracked in a new list.
The test/js/promise_rejection_tracker_recursive.t.js test is no longer
skipped for the QuickJS engine.
Promise: do not track handled rejections with no list
Previously, when a handled promise was reported and the rejected promise
list was NULL, the trackers fell through and added it as unhandled. In
the interactive shell, handling a promise after its rejection had been
reported caused another, false unhandled rejection. The QuickJS nginx
module could do the same while formatting a rejection reason.
The fix is to return for every handled notification, removing the promise
from the list only when the list exists. Apply this to both engines in the
shell and nginx module.
CHB: avoid overflow when enforcing the chain limit
Check the requested size against the remaining chain capacity instead of
adding it to the current total. This prevents the size check from wrapping
and bypassing the configured chain limit.
Buffer: reject oversized QJS alloc() and concat() sizes
Buffer.alloc() and Buffer.concat() converted sizes with JS_ToUint32(),
which wraps values modulo 2^32: Buffer.alloc(2 ** 32 + 16) returned a
16-byte buffer. Use JS_ToIndex() instead, so these sizes throw a
RangeError.
In Buffer.concat(), the total length of the list could also overflow the
32-bit accumulator. Keep it in 64 bits and check it against
QJS_BUFFER_MAX_LENGTH. The overflow now throws a RangeError instead of a
TypeError, as in Node.js.
Convert array-like lengths through checked floating-point conversion before
applying ToLength semantics. This avoids reading float-tagged QuickJS
numbers through the integer accessor.
The length is kept as a 64-bit value and passed to qjs_buffer_alloc()
unchanged, so an oversized length is rejected before any allocation
instead of being truncated.
Buffer: check QJS allocation sizes before narrowing
Take a 64-bit size in qjs_buffer_alloc() and reject sizes above
QJS_BUFFER_MAX_LENGTH before converting them. Previously, a caller
holding a wider size had to narrow it to size_t first, which truncated
it silently on 32-bit platforms.
Accept RangeError from Buffer.concat() revalidation.
Remove oversized Buffer.from() cases that make Node.js allocate 4 GiB.
Expect empty Buffer.fill() values to fail.
Use fs.rm() for recursive removal under Node.js.
Check non-enumerable fs.Stats date properties directly.
Use distinct readlink fixtures for concurrently run test cases.
Known failures for js_test --binary=node are limited to module
resolution in import and XML module tests for v25.2.1 on linux.
Previously, the local JavaScript tests, including WebCrypto tests, were
run by the test262 target, while the official TC39 Test262 suite was
run by njs_test262_test.
The fix is to rename the local test target to js_test and make test262
run the official suite. The engine-specific local targets are renamed
to js_test_njs and js_test_quickjs.
Memory pool: compare block addresses without overflow
Previously, the rbtree comparator converted the difference between two
addresses to intptr_t. On 32-bit systems, addresses that differ by at
least 2 GiB could produce an incorrect ordering and prevent a later block
lookup from finding its cluster.
The fix is to compare the addresses directly and return -1, 0, or 1.
This fixes a pool that remained non-empty after all allocations were freed
in the x86 memory pool test.
Previously, the memory pool was covered only indirectly by a single
flathsh configuration.
The fix is to add direct coverage for allocation, reuse, alignment, and
mixed allocation sizes across valid pool configurations. The test uses a
smaller workload in the regular matrix and a full stress workload in the
ASan job.
Library tests are included in the standard test target. The added ASan
configuration omits debug memory so that the complete test target and the
pool page and cluster paths are exercised. The library-test dependency
rule is also corrected for the new test executable.
Previously, input bytes were promoted to signed int before being shifted
into a hash word. A byte with its high bit set could make the 24-bit
shift undefined.
The fix is to cast input bytes to uint32_t before shifting. This
preserves the hash result and avoids undefined behavior under UBSan.
Previously, fetch cached fixed-length responses with bytes after their
declared body. Bodyless and chunked responses with such bytes did not
complete correctly.
The fix is to close a framed response connection with unconsumed bytes
instead of adding it to the keepalive cache.
TypedArray: coerce length argument once on buffer construction
Previously, njs_typed_array_alloc() passed the length argument through
njs_value_to_index() twice, before and after the detached-buffer check,
and kept only the second result. The argument's valueOf() was invoked
twice, which is observable.
The Date setters always converted their result as local time. Use the
setter's UTC flag instead.
Remove the intermediate range check in njs_make_date(). TimeClip already
checks the result after local-to-UTC conversion, which can move a value
across the valid time range boundary.
Fix the conversion of a property key to an array index
Previously, a double was converted to uint32_t before its range was tested,
so a property key that is NaN, negative or larger than 2^31 made the
conversion undefined. UndefinedBehaviorSanitizer reported it for every VM
creation, where an environment variable name is atomized.
The fix is to test the range first and to share one njs_number_is_atom_index()
helper between the four places that repeated the pattern.
The lambda structure is allocated at the end of module compilation,
and if the allocation fails, lambda is NULL. Since nothing fails after
a successful allocation, lambda is always NULL on the failed path.
Previously, assigning null or undefined to node.$tag$name removed
matching children in QuickJS, while the njs engine rejected every
assignment.
The fix is to distinguish property assignment from deletion in the
QuickJS exotic property handler. Assignments now throw TypeError in both
engines, while delete node.$tag$name remains supported.
Previously, text and structural mutators copied and replaced the receiver's
libxml2 node. Existing XMLNode wrappers continued to point at the detached
original, while subsequent lookups returned the replacement. Wrappers for
children preserved by the operation likewise pointed into the detached old
subtree. In QuickJS, mutating the detached receiver again could dereference
the NULL result of xmlReplaceNode().
The fix is to modify the existing receiver's child list directly. Existing
wrappers and subsequent lookups therefore continue to refer to the same
libxml2 node. Removed child lists remain readable until the QuickJS
document or njs VM is destroyed. A node is considered live only when
following its parent links reaches its xmlDoc. Replacement children and
cleanup records are prepared before the live tree is modified.
Changed behaviour:
- A removed child has an undefined $parent, while its descendants retain
their internal parent links.
- Mutating or canonicalizing a removed XMLNode throws TypeError.
- Text and attribute values must be primitive strings; null and undefined
retain their deletion semantics.
- C14N rejects an excluding node from another document or a detached list.
- addChild() and $tags reject namespace-bearing or unsupported source trees
before mutation.
- Named $tags assignments are consistently rejected by both engines.
While here, use an unsigned QuickJS $tags array length, support assigning
$tags on an empty njs XMLNode, and make deleting $tags from an empty node
a no-op.
Previously, text updates encoded content manually and then passed it to
libxml2. This duplicated libxml2 escaping rules and allocated the
encoded buffer before creating the text node.
The fix is to create a libxml2 text node from the unencoded value before
replacing the copied node's children. libxml2 now owns escaping, and a
failure leaves the original XML tree unchanged.
Value-preservation analysis recursively examines the upcoming expression before
the generator decides whether mutable-slot operands need temporary copies.
Deeply nested binary expressions and arrays could therefore exhaust the native
stack.
Move the generator-specific predicate out of the parser API and limit its
recursive walk to 128 levels. At the limit, conservatively require
preservation: an extra move is safe, whereas missing a deeper side effect can
change evaluation semantics.
Module compilation publishes a provisional module before parsing to support
recursive loading. Remove that module on failure and release code records,
line maps, and code buffers produced after the compilation checkpoint so a
failed load does not poison subsequent lookups.
Allocate non-global scopes, local variables, variable tree nodes, references,
and declaration arrays from the compilation pool. The global scope and its
symbol table remain persistent for runtime lookup and later compilation.
Build closure-index arrays temporarily, then copy the finalized indexes into
VM-owned storage referenced by each lambda. Preserve declaration-origin type
without retaining pointers to released block scopes.
Use the parser invocation pool for generator stack entries, copied contexts,
blocks, jump patches, index caches, and traversal scratch. Nested function
generators share the same temporary pool, while generated code and runtime
metadata remain in the VM pool.
Create a pool for each parser invocation and allocate parser nodes, array-item
storage, continuations, preread tokens, and active labels from it. Destroy the
pool after generation and on parser or generator failures.
Keep atoms, lambdas, regexp patterns, scopes, variables, bytecode, and runtime
constants in the VM pool. Clear persistent scope AST roots before releasing
the temporary pool.
Store the atom identifier and resolved variable directly in parser nodes and
move the undefined-reference flag into the common node flags. This removes the
separately allocated reference object and helps reduce parser nodes to the
64-byte memory-pool class.
The generic object-property lowering used six parser nodes and several
generator continuations for each array element. Store array values and indexes
in compact records and emit a dedicated immediate-index initializer instead.
This also avoids interning numeric element indexes and generates elements in
source order with constant continuation depth. Keep template backing arrays
flat because template concatenation requires contiguous storage.
Large literals switch to slow arrays above the fast-array capacity limit, but
the previous initialization path left their indexed values unset. Initialize
slow arrays directly so literals above 32760 elements preserve their contents.
Add generated workloads for large array literals, declarations, references,
closures, and labels. These tests are intended to expose catastrophic parser
scaling and compare compilation changes; source construction and Function()
compilation are both included in the measured interval.
Fix Array.prototype.sort() with huge sparse arrays
Previously, sort deleted every index from the compacted output position to
the original length. A sparse array with length near 2^32 therefore caused
billions of property-delete operations and unbounded memory growth.
The fix is to retain the direct cleanup loop for fast arrays and, for slow
arrays, enumerate and delete only existing indexed properties in the range.
Previously, njs_value_to_primitive() retained a pointer to the caller's value
slot while valueOf(), toString(), or property getters executed user code.
Reentrant code could overwrite that slot, after which the conversion treated
the replacement primitive as the original object.
The fix is to snapshot the input value before the first observable property
lookup and use the stable copy throughout conversion.
Previously, integer-indexed exotic validation was restricted to keys marked
as strings. Numeric atoms and the canonical string "NaN" could bypass the
check and be installed as ordinary typed-array properties.
The fix is to validate numeric atoms and canonical numeric strings after key
conversion, and to remove the obsolete source-type flag.
The numeric-key bypass was introduced in b28e50b1 (0.9.0); the incomplete
canonical-index handling dates to bf208048 (0.3.8).
Previously, PCRE name-table capture indexes were decoded through plain char.
On signed-char targets, bytes with the high bit set were sign-extended and
produced a wild or incorrect capture index used to read a match array.
The fix is to decode unsigned bytes through a checked output parameter and to
validate every named capture index against the compiled capture count.
Fix Object.defineProperty() with reentrant descriptors
Previously, njs_object_prop_define() retained a pointer to a target hash
property while descriptor getters executed user code. Reentrant mutation of
the target could resize the flat hash and leave the retained pointer dangling.
Descriptor getters were also evaluated again after fast-array conversion.
The fix is to convert the descriptor before querying the target and to place
the retry point after descriptor conversion.