Article 1:How to manage AWS Lambda Functions with Serverless Framework

AWS Lambda doesn’t require provisioning and managing of servers. You can also edit functions in the online editor inside the AWS Management Console, so it’s easy to get started. On the other hand, you may face difficulties if you don’t manage them carefully.

In this article, I will explain how to manage AWS Lamdba Functions with Serverless Framework.
AWS Lambda functions and its configuration can be complicated but Serverless Framework helps to manage them simply.

Other methods

（Skip here for the main contents.）

As Lambda is one of the AWS services, you can control it via the AWS CLI. Also, you can write shell scripts that use it.

If you want to go with the AWS Way, you will be using AWS CodePipeline, AWS CodeBuild, AWS CloudFormation, as this page explains.

Still, there wouldn’t be people saying “This seems easy. Let’s try.”

So I searched and found tools to make the process easier. They are called Lamvery and Apex(the maker of Lamvery rivaled Apex but gave up at the end).

I tried both. Still, those two had some problems such as less flexibility or lack of some features.

What is Serverless Framework

Serverless Framework (I will say “Serverless” after here) allows you to manage AWS Lambda, Azure and GCP easily via CLI.

I often use AWS, but I see many customers use Azure, so Serverless is helpful in that it allows me to manage various infrastructures in the same way.

Unless I don’t specify, the topics from here are limited about AWS.

What you can do

You can manage the settings below with the configuration in YAML.

Configurations of Lambda functions (you need to write code for function body).
Triggers of AWS Lambda Functions
Resources for AWS Lambda Functions
- S3
- DynamoDB
- SNS Topic
- Other
IAM role

Serverless uses CloudFormation under the hood.
Since creating and updating the Stack is transparent, you don’t have to be aware of them.

Install Serverless

Installation

You need npm installed. Installation guide of npm is below.

Use the packagef or Window
Use Homebrew for Mac

Hit npm command to install .

npm install -g serverless

Create IAM group

Permission is required to deploy AWS Lambda Functions or to create and manage resources with Serverless.

I created an exclusive IAM group and put the necessary permissions in it. Regarding what’s necessary, there are various discussions in this issue, so you can refer to this if you want to implement fine graind permissions.

Below is what I did for one of our projects.

I attached the following AWS managed policies to the group:

AWSLambdaFullAccess
AmazonS3FullAccess
AWSCloudFormationReadOnlyAccess

And created custom policies that allow the following actions, then attached them to the group:

cloudformation:ValidateTemplate
cloudformation:Describe*
cloudformation:List*
cloudformation:Get*
cloudformation:PreviewStackUpdate
cloudformation:CreateStack
cloudformation:DeleteStack
cloudformation:UpdateStack

After this steps, add IAM users in the IAM group.

Install and set up the AWS CLI

Other than this, you have to install and configure AWS CLI in your local developing machine. For the detail, refer this page.

Definition of Service

What is “Service”

The Serverless documentation says it is like a “Project”. At first, you can create one Service without thinking about the details. Then after you get used to it, you can divide it into different services as necessary.

Structure of a Service

The minimum settings are …

myService/ directory
serverless.yml
Lambda Functions program

You have to specify necessary settings on serverless.yml.

Create Service

Hit the command below to create Service.

sls create --template --path myService

This time, I used “aws-python3” as a template. For other templates, refer this page.

Various settings

Here, I will explain about various configurations in serverless.yml.

I wrote already what you can manage by Serverless(see below). I will explain about each one.

Configuration of Lambda Functions(Program as usual in the contents of Functions )
Triggers of AWS Lambda Functions
Every resource that Lambda Functions use
IAM Role

All available properties for serverless.yml are covered in [this page].(https://serverless.com/framework/docs/providers/aws/guide/serverless.yml/)
Please refer this page for better understanding.

General settings

Specify general information in the service and provider section.

service:
name: myService

provider:
name: aws
runtime: python3.6
region: ap-northeast-1
role: arn:aws:iam::nnnnnnnnnnnn:role/Foo_role
stage: ${opt:stage, 'dev'}

deploymentBucket:
# Packages created by Serverless will be placed in this bucket.
# If you don’t specify the name, the bucket can be automatically generated with a random name. (Not sure）
name: my-project-lambda-deployment
serverSideEncryption: AES256 # when using server-side encryption

“stage” is one of the categories such as development, staging, and production. I will describe it later.

Function: Settings of AWS Lambda Functions

Settings of Lambda Functions have to be written in function section. So here, I define only function name and other settings. I write actual program code in the other file.

functions:
my_awesome_function: # key
handler: handler.my_awesome_function
name: my_awesome_function # name
events:
- xxx

I will explain a few points here.

About handler section, I mentioned that “write actual program code in the other file”, but the function specified on hander section will be called.

For example, I specified “my_awesome_function” in the handler section. In this case, You need to create the handler.py file and define a function name my_awesome_function in the file. The filename doesn’t have to be “hander.extention”.

You don’t have to write anything in the name section. In such case, key value will be used. Still, I recommend you to specify the name.

On the reference page of serverless.yaml, as an example, “name” includes Stage name, 「${self:provider.stage}-lambdaName」. As we try in this way – Stage name include Function name, we faced problems. Instead, we decided to use the same name as a key name which I will explain later.

I will explain the events in the next chapter.

Event: Triggers of Lambda Functions

Triggers of Lambda functions must be defined in the events sub-section in the function section. Serverless allows you to configure Lambda Functions to be triggered by events from various AWS services.

For the details, refer to serverless.yml reference page.

In our project, uploading files to S3 is the trigger. Here is one thing to be considered. Because S3 bucket is under the management of Serverless, even if you specify “S3” in the event section, you cannot use existing S3 bucket.

This seems to be by design (for the detail, refer to this issue) but to avoid this inconvenience, there are plugins to enable existing S3 bucket. I will write about this later.

Resource: Resources for Lambda Functions

You have to specify resources for AWS Lambda Functions in resources section such as S3 bucket and Dynamo DB. Here, you may need to follow the syntax of CloudFormation Template. Please refer to AWS reference.

AWS Resource Types Reference – AWS CloudFormation

Edit Lambda Functions

Just edit program then mostly finished.

# handler.py

# Just execute “import “
import boto3
# pip install required
import requests

# Lambda Functions
def my_awesome_function(event, context):
greet("hello world!")

# You can also define the different Lambda Functions in the same file.
def my_great_function(event, context):
greet("hello world!!!")

# It is better to create Functions for common use.
def greet(str)
print(str)

You can use libraries with “import” statement. Some libraries may not installed. In such cases, you need to execute “pip install”.

Packaging and Deployment

Install the necessary python package

As I mentioned before, you need to install non-default Python packages by executing “pip install”. Note that they need to be located in the service directory or one of its subdirectories. The easiest way is to place it in the same directory as handler.py. Below is the command:

This is how to write.

cd /path/to/myService
pip install (package name) -t.

Postscript: I posted a better way as a separate article.

Serverless で Python のパッケージを使った Lambda 関数をデプロイ | もばらぶエンジニアブログ

Deploy

This is how to deploy.

sls deploy

Hit this command to create packages of Python program and configuration, and they will be uploaded to S3 bucket. Then it is deployed through CloudFormation.

Conclusion

Serverless Framework allows you to manage AWS Lambda Function and other resources with the simple configuration and program code. I recommend this way because it is so much easier compared to the official(?)AWS method.

About the image on the top

The image on the top of the page is licensed under the MIT license.

July 25, 2018 - posted by renn19

← 前の投稿

Rails 5.2のcredentialsを使うとDeviseからエラーが発生する問題と応急処置

次の投稿 →

Spark の DataFrame のテスト

Article 1:How to manage AWS Lambda Functions with Serverless Framework

Introduction