**** Since I wrote this article, there is now another option, see here ***
In this article I will discuss the different configuration options for an environment that uses serverless architecture, and in particular AWS Lambda. I will first discuss the ways this was done with servers, back in the day, you know a few months ago… for me anyways. I will then list out a few different options of how this could be done using a stateless environment like AWS Lambda.
When building applications where one or more servers are involved, it was typical to store config data in a few different ways. One way was in a local config file, and having a duplicate copy of this for each server the app runs on. When doing this, several technology frameworks come with their own way of doing this. Using Microsoft .Net for example, one could easily configure the config file, encrypt it if necessary, and even set it to cache locally and reset any time the file changes. Another possible option would be to store the config data in a database, similar to how we will discuss below as an option for Serverless Architecture. With the latter approach, caching and encryption cold also be applied to this. In addition, another (not so widely used any more) option is to use something like the windows registry which is part of the core operating system, to store and secure config data. The registry option is obviously specific to Microsoft Windows, but you get the point I hope.
Now moving forward to new endeavors, let’s talk about how we would do this when using AWS Lambda. This approach uses what we call a stateless environment, which means there is now sharing of anything, between requests. Therefore trying to cache locally within Lambda itself would not provide any use, because the cache would be reloaded on each request.
So let’s review 3 ways to achieve this.
- Embed directly inside of your AWS Lambda Function
- Configuration data can be put inside directly inside of your code through the use of read only values or constants. The main benefit of this approach is that your config data would be versioned with that particular version of deployed code, assuming you have a through versioning process which I highly recommend. The concern with this approach, would be that in order to update any config values, you would have to deploy a new version of your lambda function. I would seriously consider the latter as a major concern and limitation. There will be times when you need to set a debug setting, or make another quick change and this method may take too long.
- Config file located in an S3 Bucket
- Configuration data can be put inside of a txt file and then stored inside of an s3 bucket. There are a few benefits to this such as being able to encrypt the file, using the built in file encryption feature that S3 already supports. Some other benefits are that it’s easy to access, and can also be locked down with an IAM role/policy. You also will not need to deploy another version of your Lambda Function when you need to make a simple update to the file. One main downside of this approach is that it may be easy to input malformed text, and essentially blow up your app and impact your customer experience. However with a proper CI/CD plan, unit and integration testing, “11th hour” mistakes can be avoided. I would also keep a backup of your prior file for your old friend “Justin Case”.
- Store in the database
- This is my preferred method, and what I would consider the easiest approach for storing config data for your AWS Lambda Functions. I would also recommend using DynamoDB, AWS’s NoSQL product. Config data can be stored in the format of a key/value pair or even as a (JSON) document. Doing this, you can the AWS Console web interface to manage the config data, which is already built into the product. The data can even be encrypted using your favorite encryption library. You can also track who is accessing and modifying this data through AWS CouldTrail. You will not need to redeploy your Lambda Function if you need to make any changes to the config data. The only down side to this I can think of would be usage of your RCU (Read Capacity Units), however with an added caching layer, and some real time cache invalidation logic, you can minimize this.
As with “skinning the cat”, there are probably more options for accomplishing this, however these are the top 3 that I can think of at this time.