=encoding utf-8 =head1 Name ngx_stream_lua_module - Embed the power of Lua into Nginx stream/TCP Servers. This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty. I See [the installation instructions](#installation). =head1 Status Production ready. =head1 Version This document describes ngx_stream_lua L, which was released on 21 May, 2023. =head1 Synopsis events { worker_connections 1024; } stream { # define a TCP server listening on the port 1234: server { listen 1234; content_by_lua_block { ngx.say("Hello, Lua!") } } } Set up as an SSL TCP server: stream { server { listen 4343 ssl; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers AES128-SHA:AES256-SHA:RC4-SHA:DES-CBC3-SHA:RC4-MD5; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/cert.key; ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; content_by_lua_block { local sock = assert(ngx.req.socket(true)) local data = sock:receive() -- read a line from downstream if data == "thunder!" then ngx.say("flash!") -- output data else ngx.say("boom!") end ngx.say("the end...") } } } Listening on a UNIX domain socket is also supported: stream { server { listen unix:/tmp/nginx.sock; content_by_lua_block { ngx.say("What's up?") ngx.flush(true) -- flush any pending output and wait ngx.sleep(3) -- sleeping for 3 sec ngx.say("Bye bye...") } } } =head1 Description This is a port of the L to the Nginx "stream" subsystem so as to support generic stream/TCP clients. The available Lua APIs and Nginx directives remain the same as those of the ngx_http_lua module. =head2 Directives The following directives are ported directly from ngx_http_lua. Please check the documentation of ngx_http_lua for more details about their usage and behavior. =over =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =back The L directive in the Nginx "http" subsystem is missing in the "stream" subsystem. As such, ngx_stream_lua_module uses the C directive for this purpose instead. B the lingering close directive that used to exist in older version of C has been removed and can now be simulated with the newly added L API if necessary. =head2 preread_by_lua_block B I B I B I Acts as a C phase handler and executes Lua code string specified in C for every connection (or packet in datagram mode). The Lua code may make L and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). It is possible to acquire the raw request socket using L and receive data from or send data to the client. However, keep in mind that calling the C method of the request socket will consume the data from the buffer and such consumed data will not be seen by handlers further down the chain. The C code will always run at the end of the C processing phase unless L is turned on. This directive was first introduced in the C release. =head2 preread_by_lua_file B Ipath-to-lua-script-fileE> B I B I Equivalent to L, except that the file specified by C<< >> contains the Lua code or LuaJIT bytecode to be executed. Nginx variables can be used in the C<< >> string to provide flexibility. This however carries some risks and is not ordinarily recommended. When a relative path like C is given, it will be turned into the absolute path relative to the C path determined by the C<-p PATH> command-line option given when starting the Nginx server. When the Lua code cache is turned on (by default), the user code is loaded once at the first connection and cached. The Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching L C in C to avoid having to reload Nginx. This directive was first introduced in the C release. =head2 log_by_lua_block B I B I B I Runs the Lua source code specified as C<< >> during the C request processing phase. This does not replace the current access logs, but runs before. Yielding APIs such as C, C, C, or C are B available in this phase. This directive was first introduced in the C release. =head2 log_by_lua_file B Ipath-to-lua-script-fileE> B I B I Equivalent to L, except that the file specified by C<< >> contains the Lua code or LuaJIT bytecode to be executed. Nginx variables can be used in the C<< >> string to provide flexibility. This however carries some risks and is not ordinarily recommended. When a relative path like C is given, it will be turned into the absolute path relative to the C path determined by the C<-p PATH> command-line option given when starting the Nginx server. When the Lua code cache is turned on (by default), the user code is loaded once at the first connection and cached. The Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching L C in C to avoid having to reload Nginx. This directive was first introduced in the C release. =head2 lua_add_variable B I B I Add the variable C<$var> to the "stream" subsystem and makes it changeable. If C<$var> already exists, this directive will do nothing. By default, variables added using this directive are considered "not found" and reading them using C will return C. However, they could be re-assigned via the C API at any time. This directive was first introduced in the C release. =head2 preread_by_lua_no_postpone B I B I Controls whether or not to disable postponing L directives to run at the end of the C processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the C phase. This directive was first introduced in the C release. =head2 Nginx API for Lua Many Lua API functions are ported from ngx_http_lua. Check out the official manual of ngx_http_lua for more details on these Lua API functions. =over =item * L =back This module fully supports the new variable subsystem inside the Nginx stream core. You may access any L provided by the stream core or other stream modules. =over =item * L C, C, and etc. =item * L C, C, and etc. =item * L =item * L =item * L =back =over =item * L =back Only raw request sockets are supported, for obvious reasons. The C argument value is ignored and the raw request socket is always returned. Unlike ngx_http_lua, you can still call output API functions like C, C, and C after acquiring the raw request socket via this function. When the stream server is in UDP mode, reading from the downstream socket returned by the C call will only return the content of a single packet. Therefore the reading call will never block and will return C when all the data from the datagram has been consumed. However, you may choose to send multiple UDP packets back to the client using the downstream socket. The raw TCP sockets returned by this module will contain the following extra method: =head2 reqsock:receiveany B I B I, ngx.timer.E<42>, ssl_certificate_by_luaE<42>> This method is similar to L method This method was introduced into C since C. =head2 tcpsock:shutdown B I B I> Shuts down the write part of the request socket, prevents all further writing to the client and sends TCP FIN, while keeping the reading half open. Currently only the C<"send"> direction is supported. Using any parameters other than "send" will return an error. If you called any output functions (like L) before calling this method, consider use C to make sure all busy buffers are complely flushed before shutting down the socket. If any busy buffers were detected, this method will return C will error message C<"socket busy writing">. This feature is particularly useful for protocols that generate a response before actually finishing consuming all incoming data. Normally, the kernel will send RST to the client when L is called without emptying the receiving buffer first. Calling this method will allow you to keep reading from the receiving buffer and prevents RST from being sent. You can also use this method to simulate lingering close similar to that L for protocols in need of such behavior. Here is an example: local LINGERING_TIME = 30 -- 30 seconds local LINGERING_TIMEOUT = 5000 -- 5 seconds local ok, err = sock:shutdown("send") if not ok then ngx.log(ngx.ERR, "failed to shutdown: ", err) return end local deadline = ngx.time() + LINGERING_TIME sock:settimeouts(nil, nil, LINGERING_TIMEOUT) repeat local data, _, partial = sock:receive(1024) until (not data and not partial) or ngx.time() >= deadline =head2 reqsock:peek B I B I> Peeks into the L buffer that contains downstream data sent by the client without consuming them. That is, data returned by this API will still be forwarded upstream in later phases. This function takes a single required argument, C, which is the number of bytes to be peeked. Repeated calls to this function always returns data from the beginning of the preread buffer. Note that preread phase happens after the TLS handshake. If the stream server was configured with TLS enabled, the returned data will be in clear text. If preread buffer does not have the requested amount of data, then the current Lua thread will be yielded until more data is available, L<`preread_buffer_size`|https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_buffer_size> has been exceeded, or L<`preread_timeout`|https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_timeout> has elapsed. Successful calls always return the requested amounts of data, that is, no partial data will be returned. When L<`preread_buffer_size`|https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_buffer_size> has been exceeded, the current stream session will be terminated with the L C<400> immediately by the stream core module, with error message C<"preread buffer full"> that will be printed to the error log. When L<`preread_timeout`|https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_timeout> has been exceeded, the current stream session will be terminated with the L C<200> immediately by the stream core module. In both cases, no further processing on the session is possible (except C). The connection will be closed by the stream core module automatically. Note that this API cannot be used if consumption of client data has occurred. For example, after calling C. If such an attempt was made, the Lua error C<"attempt to peek on a consumed socket"> will be thrown. Consuming client data after calling this API is allowed and safe. Here is an example of using this API: local sock = assert(ngx.req.socket()) local data = assert(sock:peek(1)) -- peek the first 1 byte that contains the length data = string.byte(data) data = assert(sock:peek(data + 1)) -- peek the length + the size byte local payload = data:sub(2) -- trim the length byte to get actual payload ngx.log(ngx.INFO, "payload is: ", payload) This API was first introduced in the C release. =over =item * L =item * L =item * L =item * L This call currently ignores the C argument and always wait for all the pending output to be completely flushed out (to the system socket send buffers). =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L Always takes the Lua string value C<"stream"> in this module. =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =item * L =back =head1 TODO =over =item * Add new directives C and C. =item * Add C to emulate the L directive. =back =head1 Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: =over =item * 1.25.x (last tested: 1.25.1) =item * 1.21.x (last tested: 1.21.4) =item * 1.19.x (last tested: 1.19.3) =item * 1.17.x (last tested: 1.17.8) =item * 1.15.x (last tested: 1.15.8) =item * 1.13.x (last tested: 1.13.6) =back Nginx cores older than 1.13.6 (exclusive) are I tested and may or may not work. Use at your own risk! =head1 Installation It is I recommended to use L which bundle Nginx, ngx_http_lua, ngx_stream_lua, (this module), LuaJIT, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with Nginx yourself since it is tricky to set up exactly right. Note that Nginx, LuaJIT, and OpenSSL official releases have various limitations and long standing bugs that can cause some of this module's features to be disabled, not work properly, or run slower. Official OpenResty releases are recommended because they bundle L and [Nginx/OpenSSL patches](https://github.com/openresty/openresty/tree/master/patches). Alternatively, ngx_stream_lua can be manually compiled into Nginx: =over =item 1. LuaJIT can be downloaded from the L. The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above =item 2. Download the latest version of ngx_stream_lua L =item 3. Download the latest supported version of Nginx L (See L) =back Build the source with this module: wget 'https://nginx.org/download/nginx-1.13.6.tar.gz' tar -xzvf nginx-1.13.6.tar.gz cd nginx-1.13.6/ # tell nginx's build system where to find LuaJIT 2.1: export LUAJIT_LIB=/path/to/luajit/lib export LUAJIT_INC=/path/to/luajit/include/luajit-2.1 # Here we assume Nginx is to be installed under /opt/nginx/. ./configure --prefix=/opt/nginx \ --with-ld-opt="-Wl,-rpath,/path/to/luajit-or-lua/lib" \ --with-stream \ --with-stream_ssl_module \ --add-module=/path/to/stream-lua-nginx-module # Build and install make -j4 make install You may use C<--without-http> if you do not wish to use this module with the "http" subsystem. ngx_stream_lua will work perfectly fine without the "http" subsystem. =head1 Community =head2 English Mailing List The L mailing list is for English speakers. =head2 Chinese Mailing List The L mailing list is for Chinese speakers. =head1 Code Repository The code repository of this project is hosted on GitHub at Lstream-lua-nginx-module|https://github.com/openresty/stream-lua-nginx-module>. =head1 Bugs and Patches Please submit bug reports, wishlists, or patches by =over =item 1. creating a ticket on the L, =item 2. or posting to the L. =back =head1 Acknowledgments We appreciate L for kindly sponsoring L on the following work: =over =item * Compatibility with Nginx core 1.13.3. =item * Development of L to make code sharing between this module and L possible. =item * C, C, C and C phases support. =item * L<`reqsock:peek`> API support. =back =head1 Copyright and License This module is licensed under the BSD license. Copyright (C) 2009-2019, by Yichun "agentzh" Zhang (章亦春) Eagentzh@gmail.comE, OpenResty Inc. Copyright (C) 2009-2016, by Xiaozhe Wang (chaoslawful) Echaoslawful@gmail.comE. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: =over =item * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. =back =over =item * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. =back THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. =head1 See Also =over =item * L =item * L =item * L =back