Being a cross-platform and embeddable scripting language, Lua is well-known for its simplicity and ease of use. To facilitate contract development on CKB platform, we have recently integrated Lua 5.4 to CKB Virtual Machine (CKB-VM) and created CKB-Lua. This allows CKB developers to either run the standalone Lua interpreter on CKB or embed Lua code into the main CKB contracts, thus greatly lowering the barrier of contract development on CKB.

Introduction

CKB-Lua can be used as a shared library (embedded CKB-Lua) or a standalone interpreter. In the embedded mode, it is integrated into a hosting program and called when needed; in the standalone mode, CKB-Lua itself is the main program and loads Lua scripts from the provided arguments. Standalone CKB-Lua is highly ergonomic thanks to the simplicity and ease of use of the Lua programming language. Even programmers without prior experience in CKB contract development can write their first CKB-Lua contract in just a few hours. With the powerful dynamic loading mechanism of embedded CKB-Lua, developers can extend their main contracts in various ways, be it customizing the rules to issue tokens or writing an all-new NFT trading engine in Lua. To further facilitate the development of CKB contracts with Lua, we created a file system called Simple Lua File System that helps with code reuse. Files within this file system can be made available for Lua scripts to read and execute, e.g., running require('mymodule') and io.open('myfile'). This way, we can share common modules between different contracts.

The easiest way to try out CKB-Lua is by building a Lua project with the Capsule template that supports CKB-Lua. You may create an embedded CKB-Lua project with capsule new --template lua-embedded lua-demo or a standalone project with capsule new --template lua lua-demo.

You can obtain the source code of CKB-Lua from the GitHub repository, and build the binaries for both modes by running make all-via-docker. The shared library for the embedded CKB-Lua is ./build/libckblua.so, while the standalone CKB-Lua interpreter is./build/lua-loader.

Choosing between the embedded and standalone modes requires weighing the pros and cons, which are outlined below. It's necessary to consider the trade-offs when making a decision.

Embedded

👍 Time-sensitive operations can be completed in the hosting program

👎 Slightly steeper learning curve

Standalone

👍 Easy to learn

👎 May be not efficient enough for some computation-intensive use cases, such as signature verification

Using CKB-Lua As a Shared Library

In this section, we provide a simple example to show how to use CKB-Lua as a shared library. Since the usage of embedded CKB-Lua is almost a superset of the usage of standalone CKB-Lua, I will primarily focus on illustrating the embedded mode.

A sample program that evaluates Lua code with CKB-Lua shared library is dylib.c. Before we dive into the details, let me showcase how to load and unpack script args by a few lines in Lua below:

local _code_hash, _hash_type, args, err = ckb.load_and_unpack_script()
print(err)
if err == nil then
        ckb.dump(args)
end

Once familiar with this mode, you can unleash the full potential of Lua with your creativity. For instance, a more advanced and realistic example of implementing Simple UDT (Simple User Defined Tokens, or sUDT) in Lua can be found here.

Also notice that dylib.c is written in C, but both the main program and the test code can be alternatively written in Rust. Loading a shared library and performing unit tests by mocking transactions can be easily achieved in Rust (Here is an example).

I will now explain what the above sample code does and how to run it in three steps.

Step 1 Building CKB-Lua

In the given sample code, the execution of Lua relies on the libckblua.so shared library. The hosting program, ./build/dylibexample can be built from dylib.c with make all-via-docker.

Step 2 Preparing Shared Library

Packing the Shared Library Into the Dependent Cell Data

To use the CKB-Lua shared library, we need to store it as cell data and specify the cell with shared library data as a dependent cell. The simplest way to do that is to generate a Lua project with Capsule. You can also mock this with the CKB Rust SDK (like this).

Passing the Dependent Cell Information With Script Args

Then we pass the cell information to the args of the main C script by performing some standard operations to obtain the args of the current script, i.e., loading the script with ckb_load_script and then de-serializing the args with Molecule.

After obtaining the args, we can unpack the cell information contained within the args.

In our example, the args has the following format:

// <reserved args, 2 bytes> <code hash of the shared library, 32 bytes>
// <hash type of shared library, 1 byte>

With the size and offset of the fields, we can easily obtain the code hash and hash type of the dependent cell, and open the shared library by ckb_dlopen2.

Opening the Shared Library and Calling the Functions

    err = ckb_dlopen2(code_hash, hash_type, code_buff, code_buff_size, handle, &consumed_size);

[ckb_dlopen2] is a ckb-c-stdlib function to load the shared library from dependent cell(s) with specific code_hash and hash_type. Once the shared library is found, ckb_dlopen2 will write the shared library handle to the handle parameter. The handle parameter can then be used to obtain pointers to our Lua functions.

Step 3 Calling Lua Script

We use the following exported functions to call Lua scripts.

Exposed C Functions in the Shared Library

The following four functions are exposed in the shared library:

1. void *lua_create_instance(uintptr_t min, uintptr_t max)

2. int lua_run_code(void *l, const char *code, size_t code_size, char *name)

3. void close_lua_instance(void *L)

4. void lua_toggle_exit(void *L, int enabled)

lua_create_instance

With lua_create_instance, a new Lua instance can be created. In order to avoid the collision between Lua’s malloc and the host program’s malloc function, we need to specify the upper and lower bound of the memory used exclusively by Lua (the host program must not use this memory space when the Lua instance is alive). The host program can allocate this space in the stack or heap. The return value of lua_create_instance is an opaque pointer that can be used to evaluate Lua code with lua_run_code. We can reclaim the resources by calling lua_close_instance.

lua_run_code

The evaluation of Lua code can be initiated by calling lua_run_code. Both Lua source code and Lua bytecode are acceptable to lua_run_code. Evaluating binary code can be less resource-hungry in terms of both the cycles needed to run the code and the storage space required to store it. This function takes four arguments:

• l: the opaque pointer returned from lua_create_instance

• code: a pointer that points to the code buffer, which can be either Lua source code or Lua bytecode.

• code_size: the size of the code buffer

• name: the name of the Lua program. This may be helpful in debugging.

Lua script may early return control to the host program by calling ckb.exit_script(code), where the return code of this function is the parameter code. In the normal execution flow (i.e., no ckb.exit_script is called), the return value of lua_run_code will be 0 if the Lua script is executed successfully, or a negative number if the Lua script exited abnormally. Lua script should pass a non-negative number to ckb.exit_script in order to distinguish the Lua interpreter error code and deliberately set Lua script return code.

lua_close_instance

Calling lua_close_instance will release all the resources used by Lua, allowing the memory given to Lua reusable. It takes the opaque pointer returned from lua_create_instance as a parameter.

lua_toggle_exit

This can be used to toggle to enable/disable the ckb.exit feature. Since ckb.exit(code) can stop the execution of the whole VM and return code to CKB-VM, it may be undesirable in some situations, and we disabled it by default. Users may call lua_toggle_exit(l, 1) to enable it (the parameter l is the Lua instance).

Lua Functions

Functions in Lua Standard Library

Most of the functions in Lua standard library have been ported to CKB-Lua. All the platform-independent functions are supported, while some platform-dependent functions (e.g., most functions in the module os) are not.

CKB Specific Functions

Also, we have added a few ckb-specific helper functions to interact with CKB-VM, including those for issuing syscalls, debugging, and unpacking data structures. All are located in the ckb Lua module.

See the Appendix at the end of this post for a list of exported constants and functions.

Stitching Things Together

Running Lua code is quite easy. First, we load the functions from the previous returned shared library handle, then create a stack space for Lua to use. And finally, we can call the exported functions:

        CreateLuaInstanceFuncType create_func =
        must_load_function(handle, "lua_create_instance");
    EvaluateLuaCodeFuncType evaluate_func =
        must_load_function(handle, "lua_run_code");
    CloseLuaInstanceFuncType close_func =
        must_load_function(handle, "lua_close_instance");

    const size_t mem_size = 1024 * 512;
    uint8_t mem[mem_size];

    void* l = create_func((uintptr_t)mem, (uintptr_t)(mem + mem_size));
    if (l == NULL) {
        printf("creating lua instance failed\n");
        return;
    }

    int ret = evaluate_func(l, code, code_size, "test");

    if (ret != 0) {
        printf("evaluating lua code failed: %d\n", ret);
        return;
    }
    close_func(l);

The Lua code below demonstrates how to load CKB script and unpack it into code_hash, hash_type and args. In case of anything goes wrong during the process, an error would be returned as err. Since the script args can be arbitrary binary data, it may be hard to inspect. We have also added a dump function to dump the hex and ASCII representations of the data.

local _code_hash, _hash_type, args, err = ckb.load_and_unpack_script()
print(err)
if err == nil then
        ckb.dump(args)
end

Using Standalone CKB-Lua

Now that we’ve seen how to use CKB-Lua as a shared library, let’s move forward to the standalone method. For this, we need to specify the Lua code first, which is the main difference between the two modes of using CKB-Lua. The code can be either Lua source code or byte code.

Standalone CKB-Lua Args Format

The args passed to standalone CKB-Lua should be in the following format:

<lua loader args, 2 bytes> <data, variable length>

where the first two bytes are used by CKB-Lua interpreter internally and the format and meaning of data depend on lua loader args in the first two bytes.

Currently, the only accepted value for lua loader args is 0, and the remaining data should be of the following format:

<code hash of lua code, 32 bytes> <hash type of lua code, 1 byte>.

In this case, we load Lua code from the data of the cell with the given code hash and hash type.

Simple Lua File System

We can mount the Simple Lua File System so that CKB-Lua programs can reuse the files within the file system. All the files in the File System are readable, executable, and immutable. Moreover, the normal file system attributes (e.g., owner, readability) are irrelevant in CKB-Lua. We can thus represent the Simple Lua File System as a mapping from the file path to the file content. The file system can be stored in any cell whose data are accessible from Lua scripts, and Lua scripts may mount multiple file systems from multiple cells. We may use the script fs.lua to create a file system from given files, or vice versa, unpacking the file system into files. A file system is represented as a binary file which will be displayed soon below.

Mounting File System Within Lua Scripts

Calling ckb.mount(source, index) will mount the file system from the data of the cell with the specified source and index. For example, if you want to mount the file system within the first output cell, run ckb.mount(ckb.SOURCE_OUTPUT, 0). The returned value of ckb.mount will be nil unless an error occurred. In the latter case, an integer to represent the error will be returned. Afterward, you can read and execute files contained in this file system. By calling ckb.mount multiple times, you can mount several file systems.

Note that files from later mounts may override those from earlier mounts. For instance, if a file named a.txt is contained in two file systems, the file a.txt from a later mount is preferred over that of the earlier mount when reading.

Creating a File System

The binary representation of Simple Lua File System is simple. You may think of it as the C struct SimpleFileSystem as below:

struct Blob {
    uint32_t offset;
    uint32_t length;
}

struct Metadata {
    struct Blob file_name;
    struct Blob file_content;
}

struct SimpleFileSystem {
    uint32_t file_count;
    struct Metadata metadata[..];
    uint8_t blobs[..];
}

It consists of three parts:

• file_count: a number to represent the number of files contained in this file system

• metadata: an array of metadata used to obtain the filenames and file contents

• blobs (also blob): an array of binary objects to store the actual filenames and file contents.

To pack all Lua files within the current directory into $packed_file, run find . -name '*lua' -type f | lua "./utils/fs.lua" pack "$packed_file". Note that all file paths piped into the Lua script must be in the relative path format. The absolute path of a file in the current system is usually meaningless in the Simple Lua File System. An alternative way to pack files is running lua "./utils/fs.lua" pack "$packed_file" *.lua. However, the utility of the latter command is limited due to the maximum number of command line arguments in your OS.

Unpacking File System to Files

To unpack the files contained within a file system, you may run lua "./utils/fs.lua" unpack "$packed_file" "$directory", where $packed_file is the file that contains the file system, $directory is the directory to hold all the unpacked files.

Request for Comments

The development of the CKB-Lua prototype is now in the last stage and we welcome your feedback. If you have any use cases that are not currently addressed by the existing implementation, feel free to share your thoughts by leaving comments on the GitHub repo.

Appendix: Exposed Lua Functions and Constants in the CKB Module

Exposed Functions

Partial Loading

Partial loading refers to the process of loading a portion of a large resource into memory, in order to save memory and improve performance. CKB-Lua provides a partial-loading mechanism where Lua functions can obtain a part of the whole data by passing argument(s) length and/or offset.

Error Handling

Most args may return an error as the last argument. The error is a non-zero integer that represents the kind of error that happened during the execution of the function. Callers should check errors to be nil before using other arguments.

More Examples

See CKB syscall test cases.

Table of Functions

Note when partial loading support is enabled, the description for arguments length and offset is omitted. You may optionally pass length and offset in that case.

Exported Constants

While most constants here are directly taken from ckb_consts.h, some specific constants, like ckb.LUA_ERROR_INTERNAL, still have to be defined. These CKB-Lua constants are presented below.

ckb.SUCCESS
ckb.INDEX_OUT_OF_BOUND
ckb.ITEM_MISSING
ckb.LENGTH_NOT_ENOUGH
ckb.INVALID_DATA
ckb.LUA_ERROR_INTERNAL
ckb.LUA_ERROR_OUT_OF_MEMORY
ckb.LUA_ERROR_ENCODING
ckb.LUA_ERROR_SCRIPT_TOO_LONG
ckb.LUA_ERROR_INVALID_ARGUMENT

ckb.SOURCE_INPUT
ckb.SOURCE_OUTPUT
ckb.SOURCE_CELL_DEP
ckb.SOURCE_HEADER_DEP
ckb.SOURCE_GROUP_INPUT
ckb.SOURCE_GROUP_OUTPUT

ckb.CELL_FIELD_CAPACITY
ckb.CELL_FIELD_DATA_HASH
ckb.CELL_FIELD_LOCK
ckb.CELL_FIELD_LOCK_HASH
ckb.CELL_FIELD_TYPE
ckb.CELL_FIELD_TYPE_HASH
ckb.CELL_FIELD_OCCUPIED_CAPACITY

ckb.INPUT_FIELD_OUT_POINT
ckb.INPUT_FIELD_SINCE

ckb.HEADER_FIELD_EPOCH_NUMBER
ckb.HEADER_FIELD_EPOCH_START_BLOCK_NUMBER
ckb.HEADER_FIELD_EPOCH_LENGTH

✍🏻 Witten by Biao YI

---

You may also be interested in:

• Implementing EIP-712 on CKB-VM

• Enable Bitcoin Taproot on CKB (Part I)

• Enable Bitcoin Taproot on CKB (Part II)

• RVV (RISC-V V Extension): Key to the Efficient On-Chain Cryptography

By the same author:

• Enhance sUDT’s Programmability with xUDT