This post is a summary of my presentation on 18th Monthly Technical Session.

As I worked more on AWS Lambda for some feature of our service, I looked into developing AWS Lambda functions with languages not supported by Amazon. Recently, I'm working on some private project in Rust so it is worth a try to see whether it is possible to run Rust code on AWS Lambda.

How to run Rust code on AWS Lambda

According to the document of AWS Lambda, the function is run on Amazon Linux with some language runtime and library. It is also possible to spawn new process to run some executable. So there is really little limitation on how the function executes aside from the limitation on CPU, memory and network usage.

There are 2 possible ways to run code in languages not supported:

Spawn a new process to run some executable.

With this way, some standalone executable that is runnable on Amazon Linux is built. The AWS Lambda function creates a new process to run it.
Build the code into dynamic load library and load the library with language supported and call the function. (Python or Javascript)

Code written in Rust can be easily compiled into dynamically linked (shared object) library which only depends on GNU libc on a Linux distribution. Also, Python supports loading this kind of library naturally. So the second option is quite adoptable.

Sample project

Here is the code of a sample project to build an AWS Lambda function in Rust with Python code as a loader.

https://github.com/yxd-hde/lambda-rust-demo

The Python loader

To implement handler function in Rust, some code in Python is required to load the library built from Rust and call the function in the library.

Python's cdll package is used to load the library built from Rust (which is named with libhandler.so and uses standard C calling convention) and then the code call the handler function with event and context serialized into json strings.

The code is quite simple and uses simplejson library to do json serialization.

from ctypes import cdll

import simplejson as json

lib = cdll.LoadLibrary('./libhandler.so')


def handler(event, context):
    ret = lib.handle(json.dumps(event),
                     json.dumps(context, default=encode_context))
    return ret


def encode_context(context):
    return {
        "function_name": context.function_name,
        "function_version": context.function_version,
        "invoked_function_arn": context.invoked_function_arn,
        "memory_limit_in_mb": context.memory_limit_in_mb,
        "aws_request_id": context.aws_request_id,
        "log_group_name": context.log_group_name,
        "log_stream_name": context.log_stream_name

        # add others if it is required.
    }

The handler in Rust

In Rust code, the event and context parameters are in C string and need to be converted into Rust strings for further processing. Some unsafe conversion is required to be done before handling them. After that, all things can be done in Rust safely.

Here, Rust's libc library is used for the type of C char pointer, which represents a string in C.

extern crate libc;

use std::ffi::CStr;
use libc::c_char;

#[no_mangle]
pub extern "C" fn handle(c_event: *const c_char,
                         c_context: *const c_char) -> i32 {
    let event = unsafe { CStr::from_ptr(c_event).to_string_lossy()
                         .into_owned() };
    let context = unsafe { CStr::from_ptr(c_context).to_string_lossy()
                           .into_owned() };

    real_handle(event, context)
}

fn real_handle(event: String, context: String) -> i32 {
    println!("Event: {}", event);
    println!("Context: {}", context);

    // Do the real handling

    return 0;
}

Wind them together

To wind them together, there are some extra work to do.

Build the Rust code into an .so file

In Cargo.toml, it is required to tell Rust's build tool cargo that the build target is a dynamic library.

...

[lib]
name = "handler"
crate-type = ["dylib"]

...

Add all the dependencies

The dependencies in Rust are managed by cargo, while requirements.txt is used for Python though there is only simplejson required.

Make the zip package to deploy

Python code, simplejson library and libhandler.so file built from Rust needs to be packaged together in one zip file to deploy.

The libhandler.so file should be binary compatible with Amazon Linux. It is recommended to build it on a distribution that is binary compatible with Amazon Linux.

AWS Lambda settings

An AWS Lambda function with Python environment is required and the handler needs to be set to Python's handler function.

Others

logs

Rust function can just output to stdout for logging as this is supported by AWS Lambda.

error handling

For the code above, the return value of Rust's handler can be used to decide whether the execution succeeded.

Finally

Rust code is safe and supposed to be as performant as C and C++. With this way of implementation, it is possible to use Rust as the language to develop AWS Lambda function easily. It is worth a try if there are requirements to develop performant AWS Lambda function, especially when the function is computation focused.

HDE BLOG

コードもサーバも、雲の上

Building AWS Lambda function in Rust