Let’s have a closer look at a basic description of the AWS Lambda Service. It was created in 2014 and it brought a massive number of functionality, including running a code in response to events, and provided somewhat of a ‘bridge’ between services. Acquiring start impulse can invoke block of Python, C#, Node.js or Java code like a machine invoking internal devices after recording certain temperature. Of course, there are many more examples, but we won’t focus on pointing them out in this article. Finishing lambda task can result with some end reports but also can pass a new impulse to trigger a new service (or a new lambda). That’s why I used the word ‘bridge’ – we get trigger event from one side, passing it to another.
Advantages of Using Lambda
It’s worth looking at different ways of creating software. We can try to use a centralized system or to break it into smaller parts and follow a decentralized scheme. These two opposite approaches have their pros and cons but there is one major disadvantage of the first one and this needs to be mentioned. Perhaps, one big module is better if we look at it from a programmer’s side. It is easier to manage one service and finding bugs but unpredictable errors on single service will also stop other functionalities. This raises the question, why not cut project into slices, just like we cut bread in the morning? This is precisely why we’re using lambda… Providing functionality of reacting and passing events, we get a tool which help us divide modules into parts and establish good communication between them. But why AWS lambda? Why can’t I write my local decentralized system with all necessary functionalities? Of course, you can, but try to bind together database, mail service, and main part of service using bare Python code… I believe you’d consider using Lambda faster than you’d think, especially if two of these services are provided by AWS.
Common Examples of Lambda Use
Yes, yes, yes, we have lambda, we have a decentralized system so can we use it for something else? Of course, now, there are many situations where lambda rules the roost.
Operating Serverless Websites
We could start off by taking advantage of hosting the web frontend on S3. This frontend can send requests to Lambda functions via API Gateway. Lambda can handle application logic and channel the data to our database (one example would be DynamoDB for non-relational databases). So… we tie the frontend and backend using just one service. Great facilitation, don’t you think?
Log Analysis on the fly
What’s more you could easily create a Lambda function to check log files from Cloudtrail or Cloudwatch. Lambda can search through the logs for specific entries and send out notifications via SNS. Of course, SNS is not the only service we can handle with lambda – we can use it to send messages with any other service using their own API.
Automated Backups and Everyday Tasks
You could use boto3 Python libraries to quickly implement tasks that create backups, check for idle resources, generate reports and do any other fancy stuff. I’m aware that most people tend to forget about important things, like backups and work reports but we shouldn’t! They are the things protecting us from some long-term problems with applications.
Processing Uploaded S3 Objects
You can use S3 object event notifications to immediately start processing your files by Lambda once they land in S3 buckets. This is another type of invoking event, now connected with S3 buckets service.
Filtering and transforming Data on the fly
Because Lambda is highly scalable, it is great for transitioning data between S3, Redshift, Kinesis and database services, as well as filtering on the fly. So, Lambda is not just a ‘thing’ that is loaded between services to provide communication, but it can also work with data, transform it and pass it further if and when needed.
I’m sure we can agree that’s quite a lot, but the examples mentioned were textbox examples with theory. Now let’s take a look at some practical usage.
AWS Lambda Function Handler in Python
At the time you create a Lambda function, you specify a handler which is a function in your code, that AWS Lambda can invoke when the service executes your code. Use the following general syntax structure when creating a handler function in Python.
event – this parameter is used to pass event data to the handler. This is usually a ‘dict’ type but can be of other types as well (this includes the NoneType).
context – used to provide runtime information to your handler. This parameter is of the LambdaContext type.
Example of defining our handler function is shown below:
Code above depicts a single function called my_handler. The function returns a message containing data from the event it received as input.
AWS Lambda Layers
Layer is a completely new feature developed in 2018, it is essentially a zip archive which contains libraries and some dependencies. It helps you to keep your deployment package small and by doing that it makes the development stage easier. A typical way of lambda deployment is to pack the code into a zip file and upload it. It fulfills its role when we think about small pieces of code with little dependencies, otherwise we need to be prepared for uploading the entire zip file every time, even if our changes are not that significant.
The layer feature gives us the possibility to divide our code into pieces and upload only parts which were modified or simply upload newly created code using new layers. Another great feature that is worth mentioning is the possibility to define programing language in runtime parameter to find corresponding libraries. If we are talking about runtime, we can introduce last big advantage: defining default runtime. We can use it to prepare work environment for people e.g. without knowledge of dependencies in Python language.
Layers are extracted to the /opt directory execution environment in function. Each runtime looks for libraries in a different location under /opt, depending on the language.
Make sure your layers are structured properly, so that the libraries can be accessed without any need of additional configuration. /opt is added to the $PATH environment variable.
Up to 5 layers can be specified in your function’s configuration. You choose a specific version of a layer to use. To add layers to your function, use the update-function-configuration command.
Creating your own layers
It’s a straightforward process, consisting of following steps:
- Creating virtual environment and installing required dependencies
pipenv --python 3.6 pipenv shell pipenv install troposphere pipenv install awacs
- Building a deployment package
PY_DIR='build/python/lib/python3.6/site-packages' mkdir -p $PY_DIR pipenv lock -r > requirements.txt pip install -r requirements.txt --no-deps -t $PY_DIR #Install cd build zip -r ../tropo_layer.zip . #Zip cd .. rm -r build
- Creating a layer (and a test AWS function)
aws lambda update-function-configuration --function-name tropo_function_test --layers arn:aws:lambda:eu-central-1:xxxxxxxxxxxx:layer:tropo_test:1
- Adding a local library to your layer
cp -r local_module 'build/python/lib/python3.6/site-packages'
- Updating Lambda layer
aws lambda update-function-configuration --function-name tropo_function_test --layers arn:aws:lambda:eu-central-1:xxxxxxxxxxxx:layer:tropo_test:2
Congratulations to everyone who was able to get to the end of this article! I hope you found, at least, some of the information useful for your needs. AWS lambda is one of the fastest growing services and it’s started being used in many software programs. I hope the article encouraged you to use it on your own, or at very least, just to give lambda tool a chance.