• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

openresty/lua-nginx-module: Embed the Power of Lua into NGINX HTTP servers

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称(OpenSource Name):

openresty/lua-nginx-module

开源软件地址(OpenSource Url):

https://github.com/openresty/lua-nginx-module

开源编程语言(OpenSource Language):

C 94.9%

开源软件介绍(OpenSource Introduction):

Name

ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers.

This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty.

This module is not distributed with the Nginx source. See the installation instructions.

This is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :)

Table of Contents

Status

Production ready.

Version

This document describes ngx_lua v0.10.19, which was released on 3 Nov, 2020.

Videos

You are welcome to subscribe to our official YouTube channel, OpenResty.

Back to TOC

Synopsis

 # set search paths for pure Lua external libraries (';;' is the default path):
 lua_package_path '/foo/bar/?.lua;/blah/?.lua;;';

 # set search paths for Lua external libraries written in C (can also use ';;'):
 lua_package_cpath '/bar/baz/?.so;/blah/blah/?.so;;';

 server {
     location /lua_content {
         # MIME type determined by default_type:
         default_type 'text/plain';

         content_by_lua_block {
             ngx.say('Hello,world!')
         }
     }

     location /nginx_var {
         # MIME type determined by default_type:
         default_type 'text/plain';

         # try access /nginx_var?a=hello,world
         content_by_lua_block {
             ngx.say(ngx.var.arg_a)
         }
     }

     location = /request_body {
         client_max_body_size 50k;
         client_body_buffer_size 50k;

         content_by_lua_block {
             ngx.req.read_body()  -- explicitly read the req body
             local data = ngx.req.get_body_data()
             if data then
                 ngx.say("body data:")
                 ngx.print(data)
                 return
             end

             -- body may get buffered in a temp file:
             local file = ngx.req.get_body_file()
             if file then
                 ngx.say("body is in file ", file)
             else
                 ngx.say("no body found")
             end
         }
     }

     # transparent non-blocking I/O in Lua via subrequests
     # (well, a better way is to use cosockets)
     location = /lua {
         # MIME type determined by default_type:
         default_type 'text/plain';

         content_by_lua_block {
             local res = ngx.location.capture("/some_other_location")
             if res then
                 ngx.say("status: ", res.status)
                 ngx.say("body:")
                 ngx.print(res.body)
             end
         }
     }

     location = /foo {
         rewrite_by_lua_block {
             res = ngx.location.capture("/memc",
                 { args = { cmd = "incr", key = ngx.var.uri } }
             )
         }

         proxy_pass http://blah.blah.com;
     }

     location = /mixed {
         rewrite_by_lua_file /path/to/rewrite.lua;
         access_by_lua_file /path/to/access.lua;
         content_by_lua_file /path/to/content.lua;
     }

     # use nginx var in code path
     # CAUTION: contents in nginx var must be carefully filtered,
     # otherwise there'll be great security risk!
     location ~ ^/app/([-_a-zA-Z0-9/]+) {
         set $path $1;
         content_by_lua_file /path/to/lua/app/root/$path.lua;
     }

     location / {
        client_max_body_size 100k;
        client_body_buffer_size 100k;

        access_by_lua_block {
            -- check the client IP address is in our black list
            if ngx.var.remote_addr == "132.5.72.3" then
                ngx.exit(ngx.HTTP_FORBIDDEN)
            end

            -- check if the URI contains bad words
            if ngx.var.uri and
                   string.match(ngx.var.request_body, "evil")
            then
                return ngx.redirect("/terms_of_use.html")
            end

            -- tests passed
        }

        # proxy_pass/fastcgi_pass/etc settings
     }
 }

Back to TOC

Description

This module embeds LuaJIT 2.0/2.1 into Nginx. It is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty.

Since version v0.10.16 of this module, the standard Lua interpreter (also known as "PUC-Rio Lua") is not supported anymore. This document interchangeably uses the terms "Lua" and "LuaJIT" to refer to the LuaJIT interpreter.

By leveraging Nginx's subrequests, this module allows the integration of the powerful Lua threads (known as Lua "coroutines") into the Nginx event model.

Unlike Apache's mod_lua and Lighttpd's mod_magnet, Lua code executed using this module can be 100% non-blocking on network traffic as long as the Nginx API for Lua provided by this module is used to handle requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services.

At least the following Lua libraries and Nginx modules can be used with this module:

Almost any Nginx modules can be used with this ngx_lua module by means of ngx.location.capture or ngx.location.capture_multi but it is recommended to use those lua-resty-* libraries instead of creating subrequests to access the Nginx upstream modules because the former is usually much more flexible and memory-efficient.

The Lua interpreter (also known as "Lua State" or "LuaJIT VM instance") is shared across all the requests in a single Nginx worker process to minimize memory use. Request contexts are segregated using lightweight Lua coroutines.

Loaded Lua modules persist in the Nginx worker process level resulting in a small memory footprint in Lua even when under heavy loads.

This module is plugged into Nginx's "http" subsystem so it can only speaks downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, WebSockets, etc...). If you want to do generic TCP communications with the downstream clients, then you should use the ngx_stream_lua module instead, which offers a compatible Lua API.

Back to TOC

Typical Uses

Just to name a few:

  • Mashup'ing and processing outputs of various Nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua,
  • doing arbitrarily complex access control and security checks in Lua before requests actually reach the upstream backends,
  • manipulating response headers in an arbitrary way (by Lua)
  • fetching backend information from external storage backends (like redis, memcached, mysql, postgresql) and use that information to choose which upstream backend to access on-the-fly,
  • coding up arbitrarily complex web applications in a content handler using synchronous but still non-blocking access to the database backends and other storage,
  • doing very complex URL dispatch in Lua at rewrite phase,
  • using Lua to implement advanced caching mechanism for Nginx's subrequests and arbitrary locations.

The possibilities are unlimited as the module allows bringing together various elements within Nginx as well as exposing the power of the Lua language to the user. The module provides the full flexibility of scripting while offering performance levels comparable with native C language programs both in terms of CPU time as well as memory footprint thanks to LuaJIT 2.x.

Other scripting language implementations typically struggle to match this performance level.

Back to TOC

Nginx Compatibility

The latest version of this module is compatible with the following versions of Nginx:

  • 1.19.x (last tested: 1.19.3)
  • 1.17.x (last tested: 1.17.8)
  • 1.15.x (last tested: 1.15.8)
  • 1.14.x
  • 1.13.x (last tested: 1.13.6)
  • 1.12.x
  • 1.11.x (last tested: 1.11.2)
  • 1.10.x
  • 1.9.x (last tested: 1.9.15)
  • 1.8.x
  • 1.7.x (last tested: 1.7.10)
  • 1.6.x

Nginx cores older than 1.6.0 (exclusive) are not supported.

Back to TOC

Installation

It is highly recommended to use OpenResty releases which bundle Nginx, ngx_lua (this module), LuaJIT, as well as other powerful companion Nginx modules and Lua libraries.

It is discouraged to build this module with Nginx yourself since it is tricky to set up exactly right.

Note that Nginx, LuaJIT, and OpenSSL official releases have various limitations and long standing bugs that can cause some of this module's features to be disabled, not work properly, or run slower. Official OpenResty releases are recommended because they bundle OpenResty's optimized LuaJIT 2.1 fork and Nginx/OpenSSL patches.

Alternatively, ngx_lua can be manually compiled into Nginx:

  1. LuaJIT can be downloaded from the latest release of OpenResty's LuaJIT fork. The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above
  2. Download the latest version of the ngx_devel_kit (NDK) module HERE
  3. Download the latest version of ngx_lua HERE
  4. Download the latest supported version of Nginx HERE (See Nginx Compatibility)
  5. Download the latest version of the lua-resty-core HERE
  6. Download the latest version of the lua-resty-lrucache HERE

Build the source with this module:

 wget 'https://openresty.org/download/nginx-1.19.3.tar.gz'
 tar -xzvf nginx-1.19.3.tar.gz
 cd nginx-1.19.3/

 # tell nginx's build system where to find LuaJIT 2.0:
 export LUAJIT_LIB=/path/to/luajit/lib
 export LUAJIT_INC=/path/to/luajit/include/luajit-2.0

 # tell nginx's build system where to find LuaJIT 2.1:
 export LUAJIT_LIB=/path/to/luajit/lib
 export LUAJIT_INC=/path/to/luajit/include/luajit-2.1

 # Here we assume Nginx is to be installed under /opt/nginx/.
 ./configure --prefix=/opt/nginx \
         --with-ld-opt="-Wl,-rpath,/path/to/luajit/lib" \
         --add-module=/path/to/ngx_devel_kit \
         --add-module=/path/to/lua-nginx-module

 # Note that you may also want to add `./configure` options which are used in your
 # current nginx build.
 # You can get usually those options using command nginx -V

 # you can change the parallelism number 2 below to fit the number of spare CPU cores in your
 # machine.
 make -j2
 make install

 # Note that this version of lug-nginx-module not allow to set `lua_load_resty_core off;` any more.
 # So, you have to install `lua-resty-core` and `lua-resty-lrucache` manually as below.

 cd lua-resty-core
 make install PREFIX=/opt/nginx
 cd lua-resty-lrucache
 make install PREFIX=/opt/nginx

 # add necessary `lua_package_path` directive to `nginx.conf`, in the http context

 lua_package_path "/opt/nginx/lib/lua/?.lua;;";

Back to TOC

Building as a dynamic module

Starting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the --add-dynamic-module=PATH option instead of --add-module=PATH on the ./configure command line above. And then you can explicitly load the module in your nginx.conf via the load_module directive, for example,

 load_module /path/to/modules/ndk_http_module.so;  # assuming NDK is built as a dynamic module too
 load_module /path/to/modules/ngx_http_lua_module.so;

Back to TOC

C Macro Configurations

While building this module either via OpenResty or with the Nginx core, you can define the following C macros via the C compiler options:

  • NGX_LUA_USE_ASSERT When defined, will enable assertions in the ngx_lua C code base. Recommended for debugging or testing builds. It can introduce some (small) runtime overhead when enabled. This macro was first introduced in the v0.9.10 release.
  • NGX_LUA_ABORT_AT_PANIC When the LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually result in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the v0.9.8 release.

To enable one or more of these macros, just pass extra C compiler options to the ./configure script of either Nginx or OpenResty. For instance,

./configure --with-cc-opt="-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC"

Back to TOC

Community

Back to TOC

English Mailing List

The openresty-en mailing list is for English speakers.

Back to TOC

Chinese Mailing List

The openresty mailing list is for Chinese speakers.

Back to TOC

Code Repository

The code repository of this project is hosted on GitHub at openresty/lua-nginx-module.

Back to TOC

Bugs and Patches

Please submit bug reports, wishlists, or patches by

  1. creating a ticket on the GitHub Issue Tracker,
  2. or posting to the OpenResty community.

Back to TOC

LuaJIT bytecode support

Watch YouTube video "Measure Execution Time of Lua Code Correctly in OpenResty"

Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup

As from the v0.5.0rc32 release, all *_by_lua_file configure directives (such as content_by_lua_file) support loading LuaJIT 2.0/2.1 raw bytecode files directly:

 /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.ljbc

The -bg option can be used to include debug information in the LuaJIT bytecode file:

 /path/to/luajit/bin/luajit -bg /path/to/input_file.lua /path/to/output_file.ljbc

Please refer to the official LuaJIT documentation on the -b option for more details:

https://luajit.org/running.html#opt_b

Note that the bytecode files generated by LuaJIT 2.1 is not compatible with LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added in ngx_lua v0.9.3.

Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked to LuaJIT 2.0/2.1 (or vice versa) will result in an Nginx error message such as the one below:

[error] 13909#0: *1 failed to load Lua inlined code: bad byte-code header in /path/to/test_file.luac

Loading bytecode files via the Lua primitives like require and dofile should always work as expected.

Back to TOC

System Environment Variable Support

If you want to access the system environment variable, say, foo, in Lua via the standard Lua API os.getenv, then you should also list this environment variable name in your nginx.conf file via the env directive. For example,

 env foo;

Back to TOC

HTTP 1.0 support

The HTTP 1.0 protocol does not support chunked output and requires an explicit Content-Length header when the response body is not empty in order to support the HTTP 1.0 keep-alive. So when a HTTP 1.0 request is made and the lua_http10_buffering directive is turned on, ngx_lua will buffer the output of ngx.say and ngx.print calls and also postpone sending response headers until all the response body output is received. At that time ngx_lua can calculate the total length of the body and construct a proper Content-Length header to return to the HTTP 1.0 client. If the Content-Length response header is set in the running Lua code, however, this buffering will be disabled even if the lua_http10_buffering directive is turned on.

For large streaming output responses, it is important to disable the lua_http10_buffering directive to minimise memory usage.

Note that common HTTP benchmark tools such as ab and http_load issue HTTP 1.0 requests by default. To force curl to send HTTP 1.0 requests, use the -0 option.

Back to TOC

Statically Linking Pure Lua Modules

With LuaJIT 2.x, it is possible to statically link the bytecode of pure Lua modules into the Nginx executable.

You can use the luajit executable to compile .lua Lua module files to .o object files containing the exported bytecode data, and then link the .o files directly in your Nginx build.

Below is a trivial example to demonstrate this. Consider that we have the following .lua file named foo.lua:

 -- foo.lua
 local _M = {}

 function _M.go()
     print("Hello from foo")
 end

 return _M

And then we compile this .lua file to foo.o file:

 /path/to/luajit/bin/luajit -bg foo.lua foo.o

What matters here is the name of the .lua file, which determines how you use this module later on the Lua land. The file name foo.o does not matter at all except the .o file extension (which tells luajit what output format is used). If you want to strip the Lua debug information from the resulting bytecode, you can just specify the -b option above instead of -bg.

Then when building Nginx or OpenResty, pass the --with-ld-opt="foo.o" option to the ./configure script:

 ./configure --with-ld-opt="/path/to/foo.o" ...

Finally, you can just do the following in any Lua code run by ngx_lua:

 local foo = require "foo"
 foo.go()

And this piece of code no longer depends on the external foo.lua file any more because it has already been compiled into the nginx executable.

If you want to use dot in the Lua module name when calling require, as in

 local foo = require "resty.foo"

then you need to rename the foo.lua file to resty_foo.lua before compiling it down to a .o file with the luajit command-line utility.

It is important to use exactly the same version of LuaJIT when compiling .lua files to .o files as building nginx + ngx_lua. This is because the LuaJIT bytecode format may be incompatible between different LuaJIT versions. When the bytecode format is incompatible, you will see a Lua runtime error saying that the Lua module is not found.

When you have multiple .lua files to compile and link, then just specify their .o files at the same time in the value of the --with-ld-opt option. For instance,

 ./configure --with-ld-opt="/path/to/foo.o /path/to/bar.o" ...

If you have too many .o files, then it might not be feasible to name them all in a single command. In this case, you can build a static library (or archive) for your .o files, as in

 ar rcus libmyluafiles.a *.o

then you can link the myluafiles archive as a whole to your nginx executable:

 ./configure \
     --with-ld-opt="-L/path/to/lib -Wl,--whole-archive -lmyluafiles -Wl,--no-whole-archive"

where /path/to/lib is the path of the directory containing the libmyluafiles.a file. It should be noted that the linker option --whole-archive is required here because otherwise our archive will be skipped because no symbols in our archive are mentioned in the main parts of the nginx executable.

Back to TOC

Data Sharing within an Nginx Worker

To globally share data among all the requests handled by the same Nginx worker process, encapsulate the shared data into a Lua module, use the Lua require builtin to import the module, and then manipulate the shared data in Lua. This works because required Lua modules are loaded only once and all coroutines will share the same copy of the module (both its code and data).

Note that the use of global Lua variables is strongly discouraged, as it may lead to unexpected race conditions between concurrent requests.

Here is a small example on sharing data within an Nginx worker via a Lua module:

 -- mydata.lua
 local _M = {}

 local data = {
     dog = 3,
     cat = 4,
     pig = 5,
 }

 function _M.get_age(name)
     return data[name]
 end

 return _M

and then accessing it from nginx.conf:

 location /lua {
     content_by_lua_block {
         local mydata = require "mydata"
         ngx.say(mydata.get_age("dog"))
     }
 }

The mydata module in this example will only be loaded and run on the first request to the location /lua, and all subsequent requests to the same Nginx worker process will use the reloaded instance of the module as well as the same copy of the data in it, until a HUP signal is sent to the Nginx master process to force a reload. This data sharing technique is essential for high performance Lua applications based on this module.

Note that this data sharing is on a per-worker basis and not on a per-server basis. That is, when there are multiple Nginx worker processes under an Nginx master, data sharing cannot cross the process boundary between these workers.

It is usually recommended to share read-only data this way. You can also share changeable data among all the concurrent requests of each Nginx worker process as long as there is no nonblocking I/O operations (including ngx.sleep) in the middle of your calculations. As long as you do not give the control back to the Nginx event loop and ngx_lua's light thread scheduler (even implicitly), there can never be any race conditions in between. For this reason, always be very careful when you want to share changeable data on the worker level. Buggy optimizations can easily lead to hard-to-debug race conditions under load.

If server-wide data sharing is required, then use one or more of the following approaches:

  1. Use the ngx.shared.DICT API provided by this module.
  2. Use only a single Nginx worker and a single server (this is however not recommended when there is a multi core CPU or multiple CPUs in a single machine).
  3. Use data storage mechanisms such as memcached, redis, MySQL or PostgreSQL. The OpenResty official releases come with a set of companion Nginx modules and Lua libraries that provide interfaces with these data storage mechanisms.

Back to TOC

Known Issues

Back to TOC

TCP socket connect operation issues

The tcpsock:connect method may indicate success despite connection failures such as with Connection Refused errors.

However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation.

This issue is due to limitations in the Nginx event model and only appears to affect Mac OS X.

Back to TOC

热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap