This post is a summary of my presentation on 18th Monthly Technical Session.
As I worked more on AWS Lambda for some feature of our service,
I looked into developing AWS Lambda functions with languages not supported
by Amazon.
Recently, I'm working on some private project in Rust so it is worth a try
to see whether it is possible to run Rust code on AWS Lambda.
How to run Rust code on AWS Lambda
According to the document of AWS Lambda,
the function is run on Amazon Linux with some language runtime and library.
It is also possible to spawn new process to run some executable.
So there is really little limitation on how the function executes
aside from the limitation on CPU, memory and network usage.
There are 2 possible ways to run code in languages not supported:
Spawn a new process to run some executable.
With this way, some standalone executable that is runnable on Amazon Linux is
built. The AWS Lambda function creates a new process to run it.
Build the code into dynamic load library and load the library with
language supported and call the function. (Python or Javascript)
Code written in Rust can be easily compiled into dynamically linked
(shared object) library which only depends on GNU libc on a Linux distribution.
Also, Python supports loading this kind of library naturally.
So the second option is quite adoptable.
Sample project
Here is the code of a sample project to build an AWS Lambda function in Rust
with Python code as a loader.
https://github.com/yxd-hde/lambda-rust-demo
The Python loader
To implement handler function in Rust, some code in Python is required
to load the library built from Rust and call the function in the library.
Python's cdll package is used to load the library built from Rust
(which is named with libhandler.so and uses standard C calling convention)
and then the code call the handler function with event and context
serialized into json strings.
The code is quite simple and uses simplejson library to do json serialization.
from ctypes import cdll
import simplejson as json
lib = cdll.LoadLibrary('./libhandler.so')
def handler(event, context):
ret = lib.handle(json.dumps(event),
json.dumps(context, default=encode_context))
return ret
def encode_context(context):
return {
"function_name": context.function_name,
"function_version": context.function_version,
"invoked_function_arn": context.invoked_function_arn,
"memory_limit_in_mb": context.memory_limit_in_mb,
"aws_request_id": context.aws_request_id,
"log_group_name": context.log_group_name,
"log_stream_name": context.log_stream_name
}
The handler in Rust
In Rust code, the event and context parameters are in C string and
need to be converted into Rust strings for further processing.
Some unsafe conversion is required to be done before handling them.
After that, all things can be done in Rust safely.
Here, Rust's libc library is used for the type of C char pointer, which
represents a string in C.
extern crate libc;
use std::ffi::CStr;
use libc::c_char;
#[no_mangle]
pub extern "C" fn handle(c_event: *const c_char,
c_context: *const c_char) -> i32 {
let event = unsafe { CStr::from_ptr(c_event).to_string_lossy()
.into_owned() };
let context = unsafe { CStr::from_ptr(c_context).to_string_lossy()
.into_owned() };
real_handle(event, context)
}
fn real_handle(event: String, context: String) -> i32 {
println!("Event: {}", event);
println!("Context: {}", context);
return 0;
}
Wind them together
To wind them together, there are some extra work to do.
Build the Rust code into an .so file
In Cargo.toml, it is required to tell Rust's build tool cargo that the build
target is a dynamic library.
...
[lib]
name = "handler"
crate-type = ["dylib"]
...
Add all the dependencies
The dependencies in Rust are managed by cargo, while requirements.txt is used for Python
though there is only simplejson required.
Make the zip package to deploy
Python code, simplejson library and libhandler.so file built
from Rust needs to be packaged together in one zip file to deploy.
The libhandler.so file should be binary compatible with Amazon Linux.
It is recommended to build it on a distribution that is binary compatible with
Amazon Linux.
AWS Lambda settings
An AWS Lambda function with Python environment is required and
the handler needs to be set to Python's handler function.
Others
logs
Rust function can just output to stdout for logging as this is supported by AWS Lambda.
error handling
For the code above, the return value of Rust's handler can be used to decide
whether the execution succeeded.
Finally
Rust code is safe and supposed to be as performant as C and C++.
With this way of implementation, it is possible to use Rust as the language
to develop AWS Lambda function easily.
It is worth a try if there are requirements to develop performant AWS Lambda function,
especially when the function is computation focused.
