Introduction

Openresty is a high-performance web platform based on Nginx and Lua. Its main components are:

1. Nginx¹
2. Lua virtual machine
3. lua-nginx-module: A project that embeds the Lua virtual machine into Nginx and provides Nginx APIs for Lua to call. At the Lua level, you can achieve non-blocking effects by using these APIs, mainly thanks to cosocket and the nginx event model
4. stream-lua-nginx-module: Similar functionality to lua-nginx-module, the difference is that lua-nginx-module provides APIs for nginx’s http module, while stream-lua-nginx-module provides APIs for nginx’s stream module.
5. lua-resty-core: Uses FFI to provide a series of common APIs at the Lua level
6. lua-resty-*: Based on the above modules, a series of commonly used service modules are encapsulated. Such as: lua-resty-redis/lua-resty-mysql/lua-resty-http

Development

In Openresty-based development, you need to understand the working principles of both Nginx and Lua, then further understand the principles of lua-nginx-module. This way you can have a project with a good balance between performance and maintainability.

1. Nginx

In Openresty-based development, you first need to recognize Nginx’s role in the entire project. Typical roles include:

1. Reverse proxy server
2. Web Server itself
3. Forward proxy server

Considering that projects developed with Openresty generally lean toward the server side, we’ll only discuss the first two cases here

1.1 Reverse Proxy

This is Nginx’s typical usage, i.e., acting as a gateway or load balancer frontend, directly receiving external requests, doing simple processing, then using the upstream feature to proxy traffic to the backend.

+---------+---------+    +------------------------+
|         |         |    |                        |
|         |         |    |    Web server          |
|         |         |    |                        |
|         |         |    +------------------------+
|         |         |
|         |         |    +------------------------+
|         |         |    |                        |
|   Lua   |Upstream |    |    Web server          |
|         |         |    |                        |
|         |         |    +------------------------+
|         |         |
|         |         |    +------------------------+
|         |         |    |                        |
|         |         |    |    Web server          |
|         |         |    |                        |
+---------+---------+    +------------------------+

Here’s a typical architectural diagram. The Openresty part is abstracted into the gateway logic layer handled by Lua, and the Upstream functionality layer handled by Nginx.

Here we’ll focus on Nginx’s role in this architecture:

1. Receiving requests
   1. If nginx works in master-worker multi-process mode, multiple worker processes listening on the same port will face a “thundering herd” problem: multiple processes are awakened simultaneously by a connection event, but only one process actually handles the connection. Nginx uses the ngx_accept_mutex synchronization lock mechanism to solve this problem.²
   2. Load balancing between multiple processes: Uses load thresholds to represent process load conditions, thus dynamically balancing load between multiple processes
2. Simple routing functionality: This mainly refers to multiple levels of keywords set in nginx configuration files, such as http/server/location, providing simple routing functionality (of course complex routing can be set up, but I tend to hand off complex routing to Lua code)
3. Providing hook points for lua-nginx-module to load lua virtual machine and code: These are Nginx’s module mount points. These mount points subdivide the request lifecycle into multiple phases, each phase with clear purposes, facilitating modular program management.³ The above phase subdivision is provided to nginx module developers. Before developing specific functionality for a nginx module, there are roughly several things to do: initialize configuration directive data structures, module context, handle configuration directive conflicts, register the module, and only then can you start developing the module’s specific functionality. After openresty provides multiple hook points and exposes appropriate APIs at corresponding hook points, development costs are significantly reduced
4. Upstream functionality

1.2 Web Server

In this architecture, my general approach is:

Hand off the simple routing part to Nginx configuration files, then run Lua programs that respond to user requests through mount points provided by lua-nginx-module

server {
    listen 80;
    server_name example.com;

    location /backend/ {
        content_by_lua_block {
            local handler = require "handler"
            handler.go()
        }
    }
}

If you need to use an existing web framework here, you might see frameworks like Vanilla.

Personally, I prefer combining multiple lightweight modules:

1. Use simple lua tables for routing
2. Use openresty/lemplate for rendering pages or results
3. Use lua-resty-* series modules for accessing databases or making http requests

The advantages of web servers developed this way:

1. Easy packaging, no need for luarocks tool (actually openresty officially provides the opm tool to solve lua-resty-* dependencies)
2. Easy deployment
3. Simple modularization

2. Lua

There are simple analyses of Lua source code here, so I won’t elaborate further.

Lua source code series articles:

• Lua Source Code Reading Plan
• Lua Source Code Reading (Part 1)
• Lua Source Code Reading (Part 2)
• Lua Source Code Reading (Part 3)
• Lua Source Code Reading (Part 4)
• Lua Source Code Reading (Part 5)
• Lua Source Code Reading (Part 6)

1. Use table.new to allocate lua tables of known size, because table.new calls the lua_createtable() function which can be optimized in LuaJIT. Another advantage is pre-allocating table size, preventing resource consumption during table growth.
2. Use local variables to cache results returned by APIs provided by lua-nginx-module or lua-resty-core. This reduces consumption by reducing unnecessary stack operations.
3. Note the application of lua’s __gc metamethod in some modules to prevent strange bugs caused by triggering __gc⁴
4. Note that in openresty, lua level code should avoid blocking IO: such as using lua’s native os library to read/write local files or system calls, which will affect the entire nginx worker’s operation

3. lua-nginx-module

This section mainly discusses some important configurations in lua-nginx-module.

• lua_code_cache: on|off: Turning off code caching means each request runs a separate Lua VM instance, i.e., changes to lua code take effect immediately. This feature is recommended only during debugging. Some module functionality may depend on code caching
• lua_package_path / lua_package_cpath These two directives directly determine whether nginx can find the lua modules you want to reference, so they are very important

Other directive documentation can also be clearly found in the official openresty/lua-nginx-module documentation.

4. Tools

1. openresty/openresty-devel-utils: This repository has many convenient small tools to use during openresty-related development, such as:
   1. lua-releng: A wrapper around the luac command line, bringing multiple global variables provided by openresty into the correct scope
   2. reindex: Mainly a syntax format check for test files based on the Test::Nginx module
2. spacewander/luacov-console: Combined with the luacov tool, generates colored code coverage in the terminal, and with slight processing can be used as a code coverage data source in the CI tool chain
3. Test::Nginx: The data-driven testing framework officially used by openresty

See Also

1. OpenResty Best Practices
2. openresty/lua-nginx-module / openresty/stream-lua-nginx-module / openresty/lua-resty-core official documentation