Developing AWS Lambda Functions Locally
How I develop AWS Lambda functions locally using a test-first approach
In this article I share an approach I use regularly to develop AWS Lambda functions (and other serverless projects) locally. It assumes a basic familiarity with AWS Lambda or a similar "serverless" service. The article will first introduce the approach, and then walk through a short example making use of it.
I've frequently seen other developers use the "guess and check" approach to AWS Lambda function development: Write some code; deploy and wait a long time; invoke the function; check the result; repeat. This isn't that surprising given how many guides to developing AWS Lambda functions have you repeatedly deploying or running a command line to invoke it. It's hardly controversial to say that deploying to check whether code is working as expected sucks big time. It's also unnecessary once you start treating your Lambda functions as just that — functions, which take simple input and return an output.
AWS Lambda functions are loosely like doing functional programming, albeit rarely pure. I suppose this was the premise in AWS naming the service "Lambda". Though the functions are hardly anonymous. No matter. When combined with AWS API Gateway, Lambda functions do a very functional-esque thing: map an input to an output. In principle, given some specific input, a Lambda function will always return some specific output that is somehow correlated, or dependent on the input. This makes for a tester's wet dream. Input goes into a box and out comes result which is generally the same each time!
How does this help us develop AWS Lambda functions locally? Time to reintroduce you to a buzzword from a few years ago. Our good ol' buddy pal TDD (test driven development). But as I'm not a full-blown TDD evangelist, let's just call it something a bit more humble: a (typically) test-first approach to coding. What does that mean? In short, it means we think about what we want our AWS Lambda function to return, then write a test for that result, and "finally" write the code which gives us said result. Before I write anymore blah blah, let's just look at some code instead.
In the following example we'll combine AWS Lambda with the AWS API Gateway service—a common coupling. AWS API Gateway listens for client HTTP requests and passes them on to AWS Lambda to process them, and then returns the functions result back to the client. It's a sort of giant map-reduce cycle box. The key point is that, our AWS Lambda function is invoked by AWS API Gateway with a request event and it's our job to return a result.
To follow along with the example, here's some quick setup to run which will scaffold a bare node project with the Jest test framework. I've chosen Jest because it's popular.
mkdir lambda-tests && cd lambda-tests
npm init -y
npm install jest --save-devThat out of the way, let's start with something super simple. We want to create a cliché "Hello World" AWS Lambda function. We want the function to return the string "Hello world". 😯 OoOOoOoooOoooooo. Knowing that, we can write our first test.
handler.test.js:
const { handler } = require('./handler')
describe('Lambda function', () => {
it('should return with programming cliché', async () => {
const result = await handler()
expect(result).toBe('Hello world')
})
})The test runs our AWS Lambda function code (handler()) and checks that the output is "Hello world". Let me rephrase that in AWS-speak: Our test invokes our Lambda handler function and checks that the result is "Hello world". When AWS documentation says that they are invoking your Lambda function, it's just a fancy way of saying that they're running your function, usually passing it some input they call the "event" payload.
We can try running this test now with the npx jest command. Just kidding. The test doesn't pass. But our AWS Lambda function code is simple:
handler.js:
exports.handler = async () => {
return 'Hello world'
}Running the test again with npx jest our test passes.
If this suddenly seems like just a incredibly tedious tutorial on how to write basic software tests, well, that's because it kind of is. There's really nothing magical here. We're just writing tests for our functions—they just happen to be AWS Lambda functions!
Let's look at something a little bit more realistic.
As mentioned earlier, AWS Lambda functions are often coupled with API API Gateway to handle incoming requests. If you're familiar with Express.js, in principle, what's happening is this:
app.get('/', (request, response) => {
response.send(yourLambdaHandlerFunction(request))
})AWS API Gateway is handling the Express.js bit, and your AWS Lambda function contains the logic for one of more requested routes. The key take-away here: again, the AWS Lambda function is mapping an input to an output.
Each AWS service which can invoke (run) AWS Lambda functions will include an event payload—the input to your function. For the AWS API Gateway service, the invocation event input's shape is something like the following (see here for a full example):
{
"path": "/path/in/url/to/get",
"httpMethod": "GET",
"queryStringParameters": {
"foo": "bar"
},
"headers": {
"Host": "1234567890.execute-api.eu-west-1.amazonaws.com",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko",
...
},
...
}This input data contains some interesting things. For example, the resource path requested by the client, in this case /path/in/url/to/get is necessary if we want to respond to different paths with different data. Also useful for responding to a GET request are the parameters in queryStringParameters. So, let's create an AWS Lambda function which handles such a request from AWS API Gateway. Let's say we want to implement an endpoint which says "Hi" and appends whatever name is provided in the request's query string. We also want to return a 404 Not Found message for any other URL paths which we haven't implemented. In other words, our function will have two branches of execution: The "Say Hi" route, and a 404 catch-all.
With this knowledge, we can write our test. As there are two branches in our code, we'll write one test for each result.
handler.test.js:
const { handler } = require('./handler')
describe('Lambda function', () => {
it('should return Not Found for non-existent routes', async () => {
const event = {
path: '/does/not/exist',
httpMethod: 'GET',
}
const result = await handler(event)
expect(result).toMatchObject({
statusCode: 404,
})
})
it('should say hi', async () => {
const event = {
path: '/say/hi',
httpMethod: 'GET',
queryStringParameters: {
name: 'Marco',
},
}
const result = await handler(event)
expect(result).toMatchObject({
statusCode: 200,
body: JSON.stringify('Hi Marco'),
})
})
})The first test invokes our AWS Lambda handler code with a dummy event containing a path which we won't recognize, e.g. a 404 Not Found. The key here is to realize that we're doing exactly what AWS Lambda would have done had a real request come in through the AWS API Gateway service. We're executing our handler function, passing in some event data—the input to the function. The result from our function is then what would be returned to AWS API Gateway which would send it on as the response to the client. But of course, this isn't happening since we're running out handler code locally in our test. However, we can use the result in our test to check that the handler function returned what we expected it to.
Our second test is essentially the same as the first, just with a different input event, and a different expected output. Tests written, we're ready to implement the actual logic in our handler function. It looks something like this:
handler.js:
exports.handler = async (event) => {
if (event.path === '/say/hi') {
return {
statusCode: 200,
body: JSON.stringify(
'Hi ' + event.queryStringParameters.name,
),
}
}
return {
statusCode: 404,
body: JSON.stringify('Not found'),
}
}We can now run our tests with npx jest and confirm that our AWS Lambda function will work as we expect. All our tests pass which means we know that, assuming we got the input event data shape right, it'll work as expected once deployed to AWS.
Granted, these code examples are simple. But it works. I've been using this approach for nearly half a decade now and it's useful with both small one-off AWS Lambda functions as well as large, production-grade APIs. Of course, there are solutions like Localstack or the serverless-offline plugin for the Serverless-framework for developing locally that will try to simulate the AWS environment, and of course, I make use of these when locally developing a frontend app that uses an AWS Lambda-based backend. But, Localstack is huge and slow—it's like having a full local AWS on your computer. As well, Serverless-offline tends to encourage the guess-and-check style of software development leaving you still needing to write tests at the end. The test-first approach I've described means developing locally can be quite lightweight, and it leaves you more confident in your code as at the end you get a full test suite almost as a bonus.
What if your AWS Lambda function is invoked on a schedule, or by some other asynchronous event and doesn't have any output? The thing to realize is that, your function does have an output: it's just undefined. There's no reason you can't return a useful result against which you can write tests, even though once deployed, the result is discarded by AWS.
You might ask, are these unit tests? Functional tests? Integration tests? My answer to that is: Yep; they sure are tests. In other words, it doesn't matter. They're useful, so let's move on.
What about more complex AWS Lambda functions that interact with databases or other AWS services? Do you write mocks or interact with those services directly? Mocking means your tests will probably run faster, especially as you write more tests. But, they start to stray from reality. Moreover, good mocking quickly becomes a pain in the ass. In this context, since we're writing tests to develop AWS Lambda functions, I prefer to stay closer to reality — and thus not using mocks unless it's just simpler to do so (e.g. mocking email sending is simpler). I know that interacting with real services in tests can be controversial. But, in many cases, it's just... better. When you do want to mock some AWS service, there's a great library called aws-sdk-mock which wraps the AWS SDK which can be helpful (e.g. again, mocking an email inbox for AWS SES).
The purpose of this article was to show you how I develop AWS Lambda functions locally. The examples have been rather trivial, but the approach is useful whether the code is simple, or whether it's a full-scale production API. In the end, what all our AWS Lambda functions are doing is mapping input to output.
This article is part of my 30 days / 30 articles challenge where I've attempted to write thirty articles within thirty days.