serverless

Drupal 8 as a Static Site: AWS API Gateway, Lambda and SES Form Processing

nigel — Wed, 02 Jan 2019 10:38:46 +0000

Drupal 8 as a Static Site: AWS API Gateway, Lambda and SES Form Processing nigel Wed, 02/01/2019 - 10:38

Serverless Installation

The intention is to build our AWS ecosystem using the serverless product. This will enable us to configure our AWS provisioning in a YML file which is then translated into AWS CloudFormation orchestration. In the same codebase our PHP Lambda function can be built out. It isn't my intention to provide heavy documentation here since the Contact Form architecture was defined in an earlier blog, and I have previously written many articles on the serverless / lambda / API Gateway combination of technologies to deliver HTTP content and handle form POSTs.

Notwithstanding this, I will of course document points of interest along the way so it all makes sense!

The first step is to install serverless on your system. A dependency is node, so if you don't have that then please follow tutorials elsewhere on the Internet before executing the command below to install globally serverless.

$ npm install serverless -g

We will be using Andy Raines' excellent PHP for AWS Lambda via Serverless Framework repo as our stating point. Issuing the following command will install our PHP serverless project into a directory called d8-contact-form.

$ serverless install --url https://github.com/araines/serverless-php -n d8-contact-form
Serverless: Downloading and installing "serverless-php"...
Serverless: Successfully installed "d8-contact-form"

We then cd into the directory and list.

$ cd d8-contact-form/
$ ls -las
total 48
0 drwxrwxr-x 16 501 dialout  512 Jan  2 11:14 .
0 drwxr-xr-x 14 501 dialout  448 Jan  2 11:14 ..
4 -rwxr-xr-x  1 501 dialout  660 Feb 14  2018 buildphp.sh
4 -rw-r--r--  1 501 dialout  569 Feb 14  2018 CHANGELOG.md
4 -rw-r--r--  1 501 dialout  430 Feb 14  2018 composer.json
0 drwxrwxr-x  3 501 dialout   96 Jan  2 11:14 config
4 -rw-r--r--  1 501 dialout 1011 Feb 14  2018 dockerfile.buildphp
4 -rw-r--r--  1 501 dialout   40 Feb 14  2018 .gitattributes
4 -rw-r--r--  1 501 dialout  120 Feb 14  2018 .gitignore
4 -rw-r--r--  1 501 dialout 1330 Feb 14  2018 handler.js
4 -rw-r--r--  1 501 dialout  829 Feb 14  2018 handler.php
4 -rw-r--r--  1 501 dialout 1103 Feb 14  2018 LICENSE
4 -rwxr-xr-x  1 501 dialout  133 Feb 14  2018 php
4 -rw-r--r--  1 501 dialout 3705 Feb 14  2018 README.md
4 -rw-r--r--  1 501 dialout 3008 Jan  2 11:14 serverless.yml
0 drwxrwxr-x  5 501 dialout  160 Jan  2 11:14 src

Since I don't have Git LFS installed, the PHP executable hasn't been downloaded from the repo correctly. I will download this manually, and add it to the directory. Once completed, we can see it there with:

$ ls -lash | grep " php"
 27M -rwxr-xr-x  1 501 dialout  27M Jan  2 11:22 php

Now we need to install the PHP dependencies using composer. This does have a dependency on PHP 7.1.

$ composer install -o --no-dev
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
Loading composer repositories with package information
Updating dependencies
Package operations: 9 installs, 0 updates, 0 removals
  - Installing psr/log (1.1.0): Downloading (100%)         
  - Installing monolog/monolog (1.24.0): Downloading (100%)         
  - Installing symfony/polyfill-ctype (v1.10.0): Loading from cache
  - Installing symfony/filesystem (v4.2.1): Downloading (100%)         
  - Installing symfony/config (v4.2.1): Downloading (100%)         
  - Installing symfony/contracts (v1.0.2): Downloading (100%)         
  - Installing psr/container (1.0.0): Loading from cache
  - Installing symfony/dependency-injection (v4.2.1): Downloading (100%)         
  - Installing symfony/yaml (v4.2.1): Downloading (100%)         
Writing lock file
Generating optimized autoload files

Any Raines' code comes with a very basic Lambda function called hello. To prove the environment is working correctly, we should invoke it locally (no need for deploying to AWS yet) and if everything is correct, it'll look like this:

$ serverless invoke local -f hello
Got event [] []
{
    "statusCode": 200,
    "body": "Go Serverless v1.0! Your function executed successfully!"
}

In an earlier blog I replaced the references of Hello with demo. I have changed it to D8ContactForm for the contact form. By deploying the code to AWS, and submitting the contact form, you will see the same screenshot as shown at the end of the previous blog.

SES Validation of email address

To use the AWS SES service, your email address will need to be verified. This is easy - navigate to Simple Email Service and click on email addresses, then "Verify a new email address". Add the email address and a verification email message will be sent for you to click back and confirm the verification process. Once verified, you'll see the second screenshot above.

Serverless configuration

The next step is to add the SES configuration to the serverless.yml file. Add a custom area for your own variables - in this case the email address.

custom:
  SENDER_EMAIL: xxxx@xxxxxxxx
  RECIPIENT_EMAIL: xxxx@xxxxxxxx

And add the SES config to the provider area so it looks like this in its entirety:

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  region: eu-west-1
  environment:
    SENDER: ${self:custom.SENDER_EMAIL}
    RECIPIENT: ${self:custom.RECIPIENT_EMAIL}
    DOMAIN: "*"
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "ses:SendEmail"
      Resource: "*"

Then deploy

$ sls deploy
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...

Add AWS SDK to the codebase using composer

The SDK will make it much easier to build out the SES interface code and can be installed to the codebase in the usual composer way

$ composer require aws/aws-sdk-php
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
Using version ^3.90 for aws/aws-sdk-php
./composer.json has been updated

Add SDK PHP required libraries

The AWS PHP SDK requires the filter and simplexml libraries that aren't in the PHP executable we deploy to AWS by default. Thankfully there is a way of recompiling the PHP executable and I covered this in an earlier blog. So the tl;dr is

Add the filter and simplexml libraries to dockerfile.buildphp
Make a note of the php creation date and filesize.
Make sure buildphp.sh is executable on the command line and run it.

Add the Serverless and SDK handler code

Now the AWS SDK is in the code base we can build out the functionality to integrate the SDK. Our first work is to ensure that the recipient and sender email addresses are available to the code. To achieve this, they were defined in the serverless.yml in the environment section which can be picked up in handler.php and assigned to the $event array which is made available to the handler class. Add the following lines before the call to handle

$event['sender_email'] = getenv('SENDER');
$event['recipient_email'] = getenv('RECIPIENT');
?>

Now the handler itself. A few things to note.

I have added the necessary use clauses to the AWS SDK client and the exception class
The protected $client_config holds the configuration we have - but note that further down I am getting the credentials from the CredentialProvider
The recipients are an array and so it is possible if required to add multiples here.

D8ContactFormHandler.php

namespace Raines\Serverless;

require 'vendor/autoload.php';

use Aws\Ses\SesClient;
use Aws\Exception\AwsException;

class D8ContactFormHandler implements Handler
{
    protected $client_config = [
        'region' => 'eu-west-1',
        'version' => '2010-12-01',
        'credentials.cache' => TRUE,
        'validation' => FALSE,
    ];

    /**
     * {@inheritdoc}
     */
    public function handle(array $event, Context $context)
    {
        $logger = $context->getLogger();
        // Comment out for debugging
        //$logger->notice('Got event', $event);

        // Set up AWS SDK
        $this->client_config['credentials'] = \Aws\Credentials\CredentialProvider::env();
        $SesClient = new SesClient($this->client_config);

        $sender_email = $event['sender_email'];
        $recipient_emails[] = $event['recipient_email'];

        return [
            'statusCode' => 200,
            'body' => 'Go Serverless v1.0! Your function executed successfully!',
        ];
    }
}

?>

Email generation

Now we can insert the code to actually generate the email. All of this is self evident and really doesn't need further commentary from me. It should be inserted just before the return statement.
D8ContactFormHandler.php

        // Process the submitted form.
        // *TODO* This could do with more validation.
        $fields = [];
        parse_str($event['body'], $fields);

        if (!isset($fields['name'])) $fields['name'] = '{unknown name}';
        if (!isset($fields['mail'])) $fields['mail'] = '{unknown email address}';
        if (!isset($fields['subject'][0]['value'])) $fields['subject'][0]['value'] = '{unknown subject}';
        if (!isset($fields['message'][0]['value'])) $fields['message'][0]['value'] = '{unknown message}';

        $subject = '[badzilla.co.uk website feedback] '.$fields['subject'][0]['value'];
        $plaintext_body = 'From: '.$fields['name'].'  '.$fields['mail'].
            'Subject: '.$fields['subject'][0]['value'].
            ' Message: '. $fields['message'][0]['value'];
        $html_body =  ''.$fields['subject'][0]['value'].'
'.
            ''.'From: '.$fields['name'].'  .$fields['mail'].'">'.$fields['mail'].''.'
'.
            ''.$fields['message'][0]['value'].'
';
        $char_set = 'UTF-8';

        try {
            $result = $SesClient->sendEmail([
                'Destination' => [
                    'ToAddresses' => $recipient_emails,
                ],
                'ReplyToAddresses' => [$sender_email],
                'Source' => $sender_email,
                'Message' => [
                    'Body' => [
                        'Html' => [
                            'Charset' => $char_set,
                            'Data' => $html_body,
                        ],
                        'Text' => [
                            'Charset' => $char_set,
                            'Data' => $plaintext_body,
                        ],
                    ],
                    'Subject' => [
                        'Charset' => $char_set,
                        'Data' => $subject,
                    ],
                ],
            ]);
            $messageId = $result['MessageId'];
        } catch (AwsException $e) {
            // output error message if fails
            $logger->notice('Message', $e->getMessage());
            $logger->notice('AWS Message', $e->getAwsErrorMessage());
        }
?>

Redirect to thank you page

Once we've sent the email we want to redirect back to our thank you page. This is done with a 307 code which preserves the original POSTed values so they can be used on the forwarded page. Our use will be to make the message personalised. I have decided that I would like my redirect back to my originating page, but of course you could customise it as you see fit. For my case, I use the Referer header. The code below replaces the existing return functionality.
D8 ContactFormHandler.php

        return [
            'headers' => ['Location' => $event['headers']['Referer']],
            'statusCode' => 307,
        ];
codE>?>

Debugging

During the development process you will have to do multiple deploys and doubtless you will get numerous error responses from AWS. The best way of working these issues is to use CloudWatch, and a sample screenshot is shown above. The codebase has an excellent logging facility to check variables and these are written out to CloudWatch. Have a look at:

$logger->notice('Got event', $event);
?>

This can be amended for whatever you need to interrogate, but remember the second parameter must be an array.

Git Repository

The entire codebase discussed here is at https://github.com/sanddevil/serverless-php-contact-form - if you wish to extend its functionality, please send me a pull request.

blog terms

Static Site Drupal Drupal 8 PHP Development serverless serverless-php lambda

A Real World PHP Lambda App Part 7: Load Testing with serverless-artillery

nigel — Thu, 12 Apr 2018 08:05:09 +0000

A Real World PHP Lambda App Part 7: Load Testing with serverless-artillery nigel Thu, 12/04/2018 - 09:05

Load testing is an important aspect of delivering an enterprise solution. Load testing can monitor the system's response times for each of the transactions during a set period of time. Load testing can also raise attention to any problems in the application software and fix these bottlenecks before they become more problematic.

So we've established that load testing is necessary. Tools for load testing Lambda functions are scarce, but there is serverless-artillery, which is of course based on the artillery nodejs application. So this is our choice for load testing our voting app.

Installing serverless-artillery

First step is to install globally the parent artillery app

$ npm install -g artillery

Now lets install serverless-artillery

$ npm install -g serverless-artillery

And let's see what we've got on our system after that

$ ls -als /usr/local/lib/node_modules/
total 48
4 drwxr-xr-x 12 nigel www-data 4096 Apr 12 11:11 .
4 drwxr-xr-x  6 root  root     4096 Apr 17  2017 ..
4 drwxrwxr-x  7 nigel www-data 4096 Apr 12 10:07 artillery
4 drwxrwxr-x  4 nigel www-data 4096 Apr 17  2017 casper-chai
4 drwxrwxr-x  9 nigel www-data 4096 Apr 17  2017 casperjs
4 drwxrwxr-x  6 nigel www-data 4096 Apr 30  2017 localtunnel
4 drwxrwxr-x  6 nigel www-data 4096 Apr 17  2017 mocha
4 drwxrwxr-x  4 nigel www-data 4096 Apr 17  2017 mocha-casperjs
4 drwxrwxr-x  4 nigel www-data 4096 Nov 26 19:30 n
4 drwxrwxr-x 11 nigel www-data 4096 Nov 26 19:26 npm
4 drwxrwxr-x  6 nigel www-data 4096 Dec 23 11:15 serverless
4 drwxrwxr-x  6 nigel www-data 4096 Apr 12 11:10 serverless-artillery

Your directory listing of the available packages will obviously be different - but it looks like we are good to go.

Configuring serverless-artillery

We now need a home for our serverless-artillery configuration. I opted (perhaps confusingly) to create a directory called serverless-artillery in the usual place for my projects - /var/www/html. Once that is created, we can run the configure command.

$ cd /var/www/html
$ mkdir serverless-artillery
$ cd serverless-artillery
$ slsart configure

Now if we look at the directory we can see the normal artefacts we would expect in a severless project.

$ ls -las
total 112
 4 drwxrwxr-x   3 nigel    www-data  4096 Apr 12 11:27 .
 4 drwxrwxr-x  17 www-data www-data  4096 Apr 12 11:16 ..
48 -rw-rw-r--   1 nigel    www-data 46843 Apr 12 11:27 handler.js
 4 drwxrwxr-x 152 nigel    www-data  4096 Apr 12 11:27 node_modules
 4 -rw-rw-r--   1 nigel    www-data   222 Apr 12 11:27 package.json
44 -rw-rw-r--   1 nigel    www-data 41915 Apr 12 11:27 package-lock.json
 4 -rw-rw-r--   1 nigel    www-data  1329 Apr 12 11:27 serverless.yml

The serverless.yml needs editing because the AWS region is completely missing so it will default to us-east-1 unless we change it.

$ head -20 serverless.yml | tail -6
provider: # Using Node JS v4.3 on AWS
  name: aws
  region: eu-west-1
  runtime: nodejs6.10
  iamRoleStatements:
    - Effect: "Allow"

Above you can see I have now added a region of eu-west-1 which reflects the region I am operating out of.

Deploying the serverless-artillery Lambda Function

Something I hadn't appreciated when I first installed and configured serverless-artillery is that it is a standalone Lambda function in its own right. I assumed incorrectly it somehow was attached to a pre-existing Lambda function as a plugin. The documentation for serverless-artillery didn't make this clear so I added a GitHub issue and the problem will be addressed by the project's maintainer. So now we know it's a standalone function, it needs deploying. The deployment is a little different - see below. The screenshot above shows the function listed on the AWS console.

$ slsart deploy
 
	Deploying Lambda to AWS...
 
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
.....
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (5.45 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...............
Serverless: Stack update finished...
Service Information
service: serverless-artillery-Bk1YNhhsM
stage: dev
region: eu-west-1
stack: serverless-artillery-Bk1YNhhsM-dev
api keys:
  None
endpoints:
  None
functions:
  loadGenerator: serverless-artillery-Bk1YNhhsM-dev-loadGenerator
 
	Deploy complete.

Create New 'test' Stage

Now would be a really good time to create a new stage called test to keep my testing environment separate from my development stage. To enable us to take advantage of the ability to flip between different stages on the command line, we have to make a minor change to the serverless.yml file. Change the stage to the output from the cat / grep combination below.

$ cd /var/www/html/serverless-vote
$ cat serverless.yml | grep stage | grep dev
  stage: ${opt:stage, 'dev'}

This effectively means take the stage from the command line option stage but if that doesn't exist, use dev as the default. Now we can do a regular deploy with a switch for the stage. Don't forget we will also need to sync assets for our new test stage.

$ sls deploy --stage test --no-color
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (12.53 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...........................................................................
Serverless: Stack update finished...
Service Information
service: vote
stage: test
region: eu-west-1
stack: vote-test
api keys:
  None
endpoints:
  GET - https://bnz9qsvtm1.execute-api.eu-west-1.amazonaws.com/test/
  POST - https://bnz9qsvtm1.execute-api.eu-west-1.amazonaws.com/test/
  POST - https://bnz9qsvtm1.execute-api.eu-west-1.amazonaws.com/test/thank_you
functions:
  vote_get: vote-test-vote_get
  vote_post: vote-test-vote_post
  thank_you: vote-test-thank_you
$ sls syncToS3 --stage test
Serverless: s3,sync,dist/,s3://vote-test-webapps3bucket-ncaevy1jcsry/
upload: dist/images/favicon.ico to s3://vote-test-webapps3bucket-ncaevy1jcsry/images/favicon.ico
upload: dist/css/default.css to s3://vote-test-webapps3bucket-ncaevy1jcsry/css/default.css
upload: dist/images/badzilla-logo32x32.png to s3://vote-test-webapps3bucket-ncaevy1jcsry/images/badzilla-logo32x32.png
 
Serverless: stderr undefined
Serverless: Successfully synced to the S3 bucket

It should now be possible to point a web browser at the url provided by AWS, and by navigating to the Lambda function list on the AWS console, the test stage functions can be seen (see screenshot above).

Auto Scale DynamoDB

We are going to load test the vote_post function, so we should ensure that the DynamoDB Auto Scaling feature is enabled, along with minimum and maximum read and write capacity parameters. Navigate to Services->DynamoDB->Tables->{vote_test}->Capacity and you can see in the screenshot above I've elected to put the minimum capacity for read and writes to by 10 per second. I am not intending to really stretch DynamoDB too far for my load testing since this is a hobby project and DynamoDB costs can escalate out of hand quickly without prudence.

Create the Artillery Script

serverless-artillery uses Artillery scripts to run load tests. They are easy to create and are constructed in yml format. Below is the script for our vote_post function load test. We are going to simulate the submission of our POSTed vote form multiple time. Note that despite the test will be run from our serverless-artillery directory, I have created the script in the serverless-vote directory so it is included in our vote project's git repo.
$ pwd /var/www/html/serverless-vote $ mkdir artillery $ cd artillery $ cat vote_post.yml config: # this hostname will be used as a prefix for each URI in the flow unless a complete URI is specified target: "https://bnz9qsvtm1.execute-api.eu-west-1.amazonaws.com" phases: - duration: 300 arrivalRate: 1 rampTo: 10 scenarios: - name: "Post form" flow: - post: url: "/test" form: first_name: "Fred" last_name: "Blogs" optradio: "phpstorm"

Running the Load Test

Now the time has come to execute our test. We navigate back to the serverless-artillery directory and invoke the Lambda function with a path back to our yml script.

$ cd ../serverless-artillery/
$ slsart invoke -p ../serverless-vote/artillery/vote_post.yml 
 
	Invoking test Lambda
 
 
	Your function has been invoked. The load is scheduled to be completed in 300 seconds.

Ok so now our test will run for five minutes.

Analysing the Load Test Results

Once the load test completes there are monitoring screens that can be examined to determine the results. By navigating to the Metrics tab on the AWS Console DynamoDB vote_test, we can see the topmost screenshot above. The red line on the graph indicates where I have set the minimum provisioning thresholds. If we look at the Write Capacity metric we can also see a small blue line of the number of writes per second during the test. We didn't quite achieve ten per second. It's also useful to look at the rightmost metric - Throttled Write Requests. Thankfully the count is zero and this is important. I have noticed empirically that with throttling, there comes a nasty side effect - the AWS SDK putItem can timeout thus causing an exception to be thrown in the model code.

The second screenshot shows a zoomed view of the Write Capacity with five minute intervals. The most writes per second reached 8 diring our test.

By navigating to the Items tab (screenshot 3) we can see that we have a whole swathe of records that have been created in the DynamoDB database - this is exactly what we would expect.

Next we navigate to Services->Lambda->Functions->{vote_test_vote_post}->Monitoring and we can see three metrics - all of which are important. We achieved over 1500 invocations of the function in the five minutes (leftmost graphic). The Invocation Duration is interesting - we averaged a completion time of 67 milliseconds which is very fast. There was a maximum of 343 milliseconds - still very acceptable, and clearly juding where the maximum peak is in comparison to the average, there can't have been too many functions that were near the top end of that maximum. The rightmost metric - Invocation Errors - is crucial. If we got time outs caused by DynamoDB write throttling, we would see some activity on this metric. Thankfully there is nothing so we know all is good.

We can also use the command line - for instance to get the number of records created during the run we can issue:

$ aws dynamodb scan --table-name=vote_test
{snipped output}
        {
            "optradio": {
                "S": "phpstorm"
            }, 
            "datetime": {
                "S": "2018-04-13T16:10:18"
            }, 
            "surname": {
                "S": "Blogs"
            }, 
            "id": {
                "S": "6949cf80-336f-4536-a86d-4be164861597"
            }, 
            "forename": {
                "S": "Fred"
            }
        }
    ], 
    "ScannedCount": 1596, 
    "ConsumedCapacity": null
}

Hopefully this has given you a good taster for using artillery and serverless-artillery. As you can see, it really is quite easy.

blog terms

serverless serverless-php devops lambda PHP

A Real World PHP Lambda App Part 6: Local Development

nigel — Tue, 10 Apr 2018 08:31:50 +0000

A Real World PHP Lambda App Part 6: Local Development nigel Tue, 10/04/2018 - 09:31

One of the drawbacks of developing AWS Lambda apps is the time it takes to deploy any changes to code. In addition there are financial costs to repeatedly uploading the lambda executable to an AWS S3 bucket, exacerbated by the need in our case to upload a PHP image. The upload triggers GET Lambda costs and storage costs so it is a double whammy.

Ideally a development regime should be implemented whereby most development is undertaken locally and only deployed to AWS when the developer has a high degree of confidence there are no outstanding bugs.

In an earlier blog we discussed how to set up a local copy of DynamoDB - so now we have the ammunition to perform our development locally. Up until now I have been spoon-feeding all my code - but in reality development is an iterative process of building out functionality and fixing bugs. Let's see how we can do this.

Persist Local DynamoDB Data

For us to do anything meaningful we are really going to have to persist our DynamoDB data. The earlier blog started the local instance of DynamoDB with the -inMemory flag. Let's check it isn't still running before we do anything:

$ ps -ef | grep java
nigel     1142     1  0 Apr09 ?        00:00:00 /usr/bin/daemon --name=jenkins --inherit --env=JENKINS_HOME=/var/lib/jenkins --output=/var/log/jenkins/jenkins.log --pidfile=/var/run/jenkins/jenkins.pid -- /usr/bin/java -Djava.awt.headless=true -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
nigel     1143  1142  0 Apr09 ?        00:07:12 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
nigel     8748  8694  0 10:13 pts/1    00:00:00 grep --color=auto java

Ok so whilst the Java runtime is there - it actually corresponds to my Jenkins server and not DynamoDB. So lets start DynamoDB with a path to a directory where we want the database file to be.

$ cd /usr/lib/dynamodb/
$ java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb -dbPath /home/nigel
Initializing DynamoDB Local with the following configuration:
Port:	8000
InMemory:	false
DbPath:	/home/nigel
SharedDb:	true
shouldDelayTransientStatuses:	false
CorsParams:	*

Now let's create our database.

$ aws dynamodb create-table --table-name=vote_dev --attribute-definitions AttributeName=id,AttributeType=S --key-schema AttributeName=id,KeyType=HASH --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5  --endpoint-url http://localhost:8000
{
    "TableDescription": {
        "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/vote_dev", 
        "AttributeDefinitions": [
            {
                "AttributeName": "id", 
                "AttributeType": "S"
            }
        ], 
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0, 
            "WriteCapacityUnits": 5, 
            "LastIncreaseDateTime": 0.0, 
            "ReadCapacityUnits": 5, 
            "LastDecreaseDateTime": 0.0
        }, 
        "TableSizeBytes": 0, 
        "TableName": "vote_dev", 
        "TableStatus": "ACTIVE", 
        "KeySchema": [
            {
                "KeyType": "HASH", 
                "AttributeName": "id"
            }
        ], 
        "ItemCount": 0, 
        "CreationDateTime": 1523353381.749
    }
}

Check the database is there ok

$ aws dynamodb list-tables --endpoint-url http://localhost:8000
{
    "TableNames": [
        "vote_dev"
    ]
}

Setting the DynamoDB Endpoint in AWS PHP SDK

Under normal circumstances (i.e. when doing regular deploys to AWS) the DynamoDB endpoint doesn't need to be specified in the AWS PHP SDK. However when we are developing locally, the SDK does need to know where we have our local instance running. To achieve that we need to set the endpoint in the client configuration. The line of code would look like:

'endpoint' => 'http://localhost:8000'
?>

This needs to be added to the existing client configuration but must be set conditionally, i.e. only when we are running locally. So that means we can't do
VoteModel.php **WRONG**

    protected $client_config = [
        'region' => 'eu-west-1',
        'version' => '2012-08-10',
        'credentials.cache' => TRUE,
        'validation' => FALSE,
        'endpoint' => 'http://localhost:8000',
        'scheme' => 'http'
    ];
?>

We could create a new custom environment variable and surface it at the DynamoDB client creation step. But we already have a candidate - the stage variable can be used - and we could create a fictitious new stage of local to be used in a conditional.
VoteModel.php **CORRECT**

    protected function loadClient()
    {
        $this->client_config['credentials'] = \Aws\Credentials\CredentialProvider::env();
        if ($this->data['requestContext']['stage'] == 'local') {
            $this->client_config['endpoint'] = 'http://localhost:8000';
        }
        return(new DynamoDbClient($this->client_config));
    }
?>

Creating the Lambda Data

When we invoke a Lambda function locally, we need to ensure the environment is set up correctly for it. This includes the routing information (since we don't have a local copy of API Gateway) and the run time data that the Lambda function will be asked to process. In our case that would be the submitted POSTed form data. So how do we build this? We can handcraft some JSON which includes all this information, but it's time consuming. A much easier approach is to copy the data directly from a previous AWS invocation in the CloudWatch logs - this is where our Got event debug trace pays dividends!

Navigate to the Cloudwatch logs and you'll see a list of all the Lambda functions (see first screenshot above). Select the form POST function which in my case is /aws/lambda/vote-dev-vote_post. This takes us to the list of previous invocations (second screenshot) - I've done quite a lot which is a side effect of blogging! Select the top one and open up a Got event log entry (third screenshot). Select the dumped JSON object and copy it to your clipboard.

This can then be saved in a file. I created a new subdirectory under the project's top directory called data and I named this file post.json

Set the Stage

Now edit the post.json file so that the stage is set to local - see below
data/post.json

"stage": "local",

Local Invocation

We are now ready to run our code locally. The syntax for this is below:

$ sls invoke local -f vote_post --no-color -p data/post.json

The -f flag value sets the Lambda function name which in our case is vote_post. We use the --no-color switch for readability, and the -p flag value is a path to our data file. Running it gives us the following output (for brevity some output removed)

Got event {"resource":"/","path":"/","httpMethod":"POST","headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br","Accept-Language":"en-GB,en-US;q=0.9,en;q=0.8","cache-control":"no-cache","CloudFront-Forwarded-Proto":"https","CloudFront-Is-Desktop-Viewer":"true","CloudFront-Is-Mobile-Viewer":"false","CloudFront-Is-SmartTV-Viewer":"false","CloudFront-Is-Tablet-Viewer":"false","CloudFront-Viewer-Country":"GB","content-type":"application/x-www-form-urlencoded","Cookie":"_ga=GA1.2.1152355093.1509819171; mousestats_vi=d4e8e34af07e2372c27c","Host":"n4kofna4l1.execute-api.eu-west-1.amazonaws.com","origin":"https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com","pragma":"no-cache","Referer":"https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/","upgrade-insecure-requests":"1","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36","Via":"2.0 bd161094d4b1a9657f26409a791c36ef.cloudfront.net (CloudFront)","X-Amz-Cf-Id":"qbxtM7siu0PhYiY58aPKYVXrhG5RO8S4r8XfXOUG-u-IwuTjL_2Vpw==","X-Amzn-Trace-Id":"Root=1-5acbbc81-1c0ce7281e876d1c307f0d98","X-Forwarded-For":"94.3.136.181, 216.137.62.46","X-Forwarded-Port":"443","X-Forwarded-Proto":"https"},"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"resourceId":"lp67djpm80","resourcePath":"/","httpMethod":"POST","extendedRequestId":"FFpkSGhmjoEFlug=","requestTime":"09/Apr/2018:19:18:25 +0000","path":"/dev/","protocol":"HTTP/1.1","stage":"local","requestTimeEpoch":1523301505841,"requestId":"c6eb6aa6-3c2a-11e8-8346-13d846b589a2","identity":{"cognitoIdentityPoolId":null,"accountId":null,"cognitoIdentityId":null,"caller":null,"accessKey":null,"cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":null,"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36","user":null},"apiId":"n4kofna4l1"},"body":"first_name=Joe&last_name=Soap&optradio=phpstorm","isBase64Encoded":false,"static_url":"https://s3-eu-west-1.amazonaws.com/vote-dev-webapps3bucket-1htbi30lx4nii","dynamodb_table":"vote_dev"} []
{
    "headers": {
        "Location": "https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/local/thank_you"
    },
    "statusCode": 307,
    "body": ""
}

Note that the body contains the submitted data and we should now have a record in our local DynamoDB containing that informations. Also note that we got a HTTP response of 307 - this is a redirect which would, with API Gateway being present, redirect us to the thank_you page. Obviously running locally terminates the Lambda function once its job has been completed. There are two methods for determining whether the write to DynamoDB worked - the cli and the local console shell.

DynamoDB Scan Results - CLI Method

The easiest method to ensure your records are being created correctly is to use the AWS CLI. The command below doesn't really require further explanation.

$ aws dynamodb scan --table-name=vote_dev --endpoint-url http://localhost:8000
{
    "Count": 1, 
    "Items": [
        {
            "optradio": {
                "S": "phpstorm"
            }, 
            "forename": {
                "S": "Joe"
            }, 
            "surname": {
                "S": "Soap"
            }, 
            "id": {
                "S": "4679de4d-1726-436e-b24b-755415b6298a"
            }, 
            "datetime": {
                "S": "2018-04-11T09:45:54"
            }
        }
    ], 
    "ScannedCount": 1, 
    "ConsumedCapacity": null
}

DynamoDB Scan Results - Console Shell Method

This is more complicated than the command line, but is more powerful and would be ideally suited to someone with a good understanding of JavaScript. Go to your console shell at http://{your_ip_address}:8000/shell/ and click the button as shown in the first screenshot. This will provide a list of JavaScript templates. Click the Scan template and it will expand to show the sample JavaScript as per the second screenshot. Click the thick left arrow and the console will appear for you to edit code and run it (the third screenshot).

Now here's the (slightly) fiddly bit. The code has a great many parameters which can be supplied, but we don't need that. We just want to dump all records in their entireity. So cut all the parameters we don't need. For convenience I have pasted below what you need to leave in. It's quite obvious, but copy this over the entire code in the console.

var params = {
    TableName: 'vote_dev',
    Limit: 10, // optional (limit the number of items to evaluate)
    Select: 'ALL_ATTRIBUTES', // optional (ALL_ATTRIBUTES | ALL_PROJECTED_ATTRIBUTES | 
                              //           SPECIFIC_ATTRIBUTES | COUNT)
    ConsistentRead: false, // optional (true | false)
    ReturnConsumedCapacity: 'NONE', // optional (NONE | TOTAL | INDEXES)
};
dynamodb.scan(params, function(err, data) {
    if (err) ppJson(err); // an error occurred
    else ppJson(data); // successful response
});

Now click on the play shaped button.

The first screenshot above shows the console after it has completed the playback of the JavaScript code. The second screenshot shows a zoomed view of the results. The results are exactly what we put into the local invocation so it is fine.

Further Functions

The example I have shown above relates to the form's POSTed submission. To develop the original form GET and the redirect to the thank_you page, simply change the function names in the local invocation. The POST is the most complex since it performs the database write. The others shouldn't be too onerous in comparison!

blog terms

serverless serverless-php lambda PHP devops

A Real World PHP Lambda App Part 5: Models

nigel — Mon, 09 Apr 2018 09:31:50 +0000

A Real World PHP Lambda App Part 5: Models nigel Mon, 09/04/2018 - 10:31

Now it's time to create our model. This will be easy at this stage since we are only going to be writing a record to our DynamoDB table. However since the scope of the project could increase over time, it would make sense to create an abstract class which will deal with the AWS SDK Client configuration and creation. So to start we should create a directory and two PHP files - one for the abstract class and one to write the record.

$ pwd
/var/www/html/serverless-vote
$ mkdir -p src/Models
$ touch src/Models/VoteModel.php
$ touch src/Models/PutVote.php

Installing AWS PHP SDK

We are going to use the AWS PHP SDK for writing our records out to DynamoDB since it contains a very convenient DynamoDB Client. To install it, use composer:

$ composer require aws/aws-sdk-php
Using version ^3.54 for aws/aws-sdk-php

The PHP Abstract Class

VoteModel.php

namespace Vote\Models;


use Aws\DynamoDb\DynamoDbClient;


abstract class VoteModel {


    protected $data;
    protected $logger;

    protected $client_config = [
        'region' => 'eu-west-1',
        'version' => '2012-08-10',
        'credentials.cache' => TRUE,
        'validation' => FALSE,
        'scheme' => 'http'
    ];


    public function __construct($logger, $data)
    {
        $this->logger = $logger;
        $this->data = $data;
    }


    public function __set($key, $value)
    {
        $this->$key = $value;
    }


    public function __get($key)
    {
        return $this->$key;
    }

    /**
     * Get an aws sdk client for extended classes
     *
     * @return \Aws\DynamoDb\DynamoDbClient
     */
    protected function loadClient()
    {
        $this->client_config['credentials'] = \Aws\Credentials\CredentialProvider::env();
        return(new DynamoDbClient($this->client_config));
    }

}
?>

This shouldn't hold too many surprises. The constructor requires the data and logger which is made available to any class that extends from it - we will need both since the logger object is used to notify our status through CloudWatch, and the data will contain the submitted form data. There are setters and getters as you would expect to provide a consistent way of dealing with the class's properties.

The loadClient method instantiates the DynamoDB client, and note that we are getting the necessary credentials from the environment which will be passed to the client.

PHP DynamoDB PutVote Class

PutModel.php

namespace Vote\Models;

use Aws\DynamoDb\Exception\DynamoDbException;



class PutVote extends VoteModel {


    private $id = '';


    /**
     * Put request to AWS DynamoDB API
     *
     *
     * @param array $payload
     */
    public function put($payload)
    {
        $client = $this->loadClient();

        $this->id = $this->generateUUID();
        $current_datetime = date('Y-m-d\TH:i:s');

        $item = [
            'id'               => ['S' => $this->id], // Primary Key
            'forename'        => ['S' => $payload['first_name']],
            'surname'        => ['S' => $payload['last_name']],
            'datetime'        => ['S' => $current_datetime],
            'optradio'      => ['S' => $payload['optradio']],
        ];

        try {
            $client->putItem(array(
                'TableName' => $this->data['dynamodb_table'],
                'Item' => $item
            ));
        } catch (DynamoDbException $e) {
            $this->logger->notice('DynamoDB Put Failed', array(
                'table'         => $this->data['dynamodb_table'],
                'id'             => $this->id,
                'RequestID'     => $e->getAwsRequestId(),
                'ErrorType'     => $e->getAwsErrorType(),
                'ErrorCode'     => $e->getAwsErrorCode(),
                'ErrorMessage'    => $e->getAwsErrorMessage(),
                'StatusCode'     => $e->getStatusCode()
            ));
            return TRUE;
        }
        return FALSE;
    }


    /**
     * Create a UUID - PHP doesn't have a good function for this, so hand-rolled
     *
     * @see https://rogerstringer.com/2013/11/14/generate-uuids-php/
     *
     * @return string generated UUID
     */
    protected function generateUUID()
    {
        return sprintf( '%04x%04x-%04x-%04x-%04x-%04x%04x%04x',
            mt_rand( 0, 0xffff ), mt_rand( 0, 0xffff ),
            mt_rand( 0, 0xffff ),
            mt_rand( 0, 0x0fff ) | 0x4000,
            mt_rand( 0, 0x3fff ) | 0x8000,
            mt_rand( 0, 0xffff ), mt_rand( 0, 0xffff ), mt_rand( 0, 0xffff )
        );
    }
}
?>

The PutVote class is also very simple since the legwork is undertaken by the AWS PHP SDK. The primary key can be auto-increment but I elected to use a UUID which is created in the generateUUID method. The payload passed to the put method is an array of the POSTed data of the form which is bundled into an record before being passed to the SDK putItem.

DynamoDB Configuration

There are a number of changes we need to inform to our codebase about DynamoDB. The first place is serverless.yml- we need to set an environment variable with the name of the DynamoDB table, so we will stick with the convention of appending the stage to the end of it. Add it to the end of the environmental section as below:
serverless.yml

  environment:
    STATIC_URL: https://s3-${self:provider.region}.amazonaws.com/${self:custom.s3StaticBucket}
    DYNAMODB_TABLE: vote_${self:provider.stage}

Further down in the Resources section we need to define our DynamoDB resource.

    ## Specify the DynamoDB Table
    DynamoDbTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: vote_${self:provider.stage}
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 5
          WriteCapacityUnits: 5
    DynamoDBIamPolicy:
      Type: AWS::IAM::Policy
      DependsOn: DynamoDbTable
      Properties:
        PolicyName: lambda-dynamodb
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - dynamodb:GetItem
                - dynamodb:PutItem
              Resource: arn:aws:dynamodb:*:*:table/vote_${self:provider.stage}
        Roles:
          - Ref: IamRoleLambdaExecution

Finally, in the handler.php file that interfaces with the JavaScript shim there is a section of the code that sets PHP variables from the environment - this will make sure our model has the table name surfaced to use when a record is written out to DynamoDB. Make sure this section looks like mine below:
handler.php

// Get the handler service and execute
$handler = $container->get(getenv('HANDLER'));
$event['static_url'] = getenv('STATIC_URL');
$event['dynamodb_table'] = getenv('DYNAMODB_TABLE');
?>

DynamoDB Deploy

Once the serverless.yml file has been updated it makes sense to do a deploy to make sure the table has been created correctly in AWS. The deply is below, and you should end up with the screenshot above. To get there, navigate to Services->DynamoDB->Tables and you should see vote_dev

$ sls deploy
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (12.53 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
................................
Serverless: Stack update finished...
Service Information
service: vote
stage: dev
region: eu-west-1
stack: vote-dev
api keys:
  None
endpoints:
  GET - https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/
  POST - https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/
  POST - https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/thank_you
functions:
  vote_get: vote-dev-vote_get
  vote_post: vote-dev-vote_post
  thank_you: vote-dev-thank_you
Serverless: Removing old service versions...

Controller Modifications

Now the model exists, we need to made changes to our controllers so the models are instantiated and invoked. Firstly the save method in the FormController needs to be rewritten - see below. Once we get the submitted POSTed data, it is parsed into an array ready for the model to use. The model is called, and if it returns TRUE then we reload the index page with an error message.
FormController.php

    public function save()
    {
        // Get the data ready for for the model and the redirect
        $data = $this->__get('data');
        parse_str($data['body'], $params);
        $model = $this->createModel('PutVote');

        // Bail on Error
        if ($model->put($params)) {
            return $this->_index(
                'vote.html',
                [
                'static_url' => $data['static_url'],
                'title' => 'Badzilla\'s Lambda Vote Form',
                'error' => 'Something has gone wrong with your submission. Please try later.'
            ]);
        }

        // Construct the url using the Lambda data we have
        $url = $data['headers']['X-Forwarded-Proto'].'://';
        $url .= $data['headers']['Host'];
        if (isset($data['requestContext']['stage']) && $data['requestContext']['stage'] != '') {
            $url .= '/'.$data['requestContext']['stage'];
        }
        $url .= '/thank_you';        

        // Set the required http settings for a redirect resubmit POST
        $this->__set('statusCode', 307);
        $this->__set('headers', array('Location' => $url));

        // No body text to return 
        return '';
    }
?>

The ThankYouController requires a small modification - we need to parse the submitted data from the form and then pass the first name and last name to the Twig template. This occurs in the index method.
ThankYouController.php

    public function index()
    {
        parse_str($this->data['body'], $params);

        return $this->_index(
            'thank-you.html',
            [
                'title' => 'Badzilla\'s Big Thank You',
                'static_url' => $this->data['static_url'],
                'first_name' => htmlspecialchars($params['first_name'], ENT_QUOTES, 'UTF-8'),
                'last_name' => htmlspecialchars($params['last_name'], ENT_QUOTES, 'UTF-8'),
            ]
        );
    }
?>

Composer Autoloader

We will need to inform composer of our new autoloader requirements for the model. So make sure your composer has the following entries under psr-4 in composer.json

  "autoload": {
    "psr-4": {
      "Raines\\Serverless\\": "src",
      "Vote\\Controllers\\": "src/Controllers/",
      "Vote\\Models\\": "src/Models/",
      "Twig\\": "vendor/twig/twig/lib"
    }
  },

Then update with:

$ composer dump-autoload
Generating autoload files

Deploy and Add Records

The code can now be deployed (as per the instructions earlier) and by pointing a browser to the dev instance of the site, records can be successfully added to the DynamoDB database. By navigating in the AWS Console to Services->DynamoDV->Tables->{select vote_dev}->Items we can see that the records are being created successfully.

The solution is now functionally ok but there is a long way to go before we are production ready. Also I have spoon fed the code, but in the real world there would have been plenty of trial and error, and problems to solve. To cut down the endless deploy to AWS/test cycle, it is better to develop and test as far as possible locally. We prepped for this in the earlier blog A Real World PHP Lambda App Part 4: Setting up DynamoDb Locally and in the next blog I'll show you how to develop locally.

blog terms

serverless serverless-php PHP lambda devops

A Real World PHP Lambda App Part 4: Setting up DynamoDb Locally

nigel — Sun, 04 Feb 2018 16:47:33 +0000

A Real World PHP Lambda App Part 4: Setting up DynamoDb Locally nigel Sun, 04/02/2018 - 16:47

We will be using DynamoDb to save our vote data, but since this adds additional complexity to the development process, it will also slow it down. There is a solution - DynamoDb can be installed locally so that there isn't the need for continuous deploys to AWS. In fact, the local version of DynamoDb is probably SQLite but it will do the job whilst we are testing our code. It also requires a working Java runtime so that needs to be installed first. This can be done on the command line. I am using Ubuntu 16.04 so for my system I do:

$ sudo apt-get update
$ sudo apt-get install default-jre

The JAVA_HOME environment variable will need to be set for the local DynamoDb to work correctly. Firstly we must determine where Java has been installed.

$ sudo update-alternatives --config java
There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
Nothing to configure.

That's easy - Java is available at /usr/bin/java. Now edit /etc/environment and add the following line at the bottom:

JAVA_HOME="/usr/bin/java"

Now we need to reload and check the variable is set

$ source /etc/environment 
$ echo $JAVA_HOME
/usr/bin/java

Install DynamoDb

I downloaded the DynamoDb tarball from the AWS site and moved it into my /tmp directory.

$ cd /usr/lib
$ sudo mkdir /dynamodb
$ cd dynamodb
$ tar zxvf /tmp/dynamodb_local_latest.tar.gz

Now list the directory and you'll see the jar file we will need to run.

$ ls -las
total 3660
   4 drwxr-xr-x  4 root root    4096 Feb  4 17:55 .
   4 drwxr-xr-x 69 root root    4096 Feb  4 17:54 ..
3628 -rw-r--r--  1 root root 3713946 Feb 16  2017 DynamoDBLocal.jar
   4 drwxr-xr-x  2 root root    4096 Feb  4 17:55 DynamoDBLocal_lib
  12 -rw-r--r--  1 root root    8644 Feb 16  2017 LICENSE.txt
   4 -rw-r--r--  1 root root     795 Feb 16  2017 README.txt
   4 drwxr-xr-x  2 root root    4096 Feb  4 17:55 third_party_licenses

Start DynamoDB

To start DynamoDb, you need to be sure you are in the directory listed above, and then issue the following command

$ java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb -inMemory
Initializing DynamoDB Local with the following configuration:
Port:   8000
InMemory:       true
DbPath: null
SharedDb:       true
shouldDelayTransientStatuses:   false
CorsParams:     *

Note I have set the flag inMemory so I am not writing to a database held on disk - all data is stored in volatile RAM. This is useful for quickly testing a solution, but obviously the data won't persist. The sharedDb flag tells DynamoDB to use one database across all regions and credentials - it makes it much easier to manage on a local test installation. To check it is running ok, point a browser at {your IP address}:8000/shell/ and you should see the shell as shown above.

Create a Local DynamoDb Database

Once DynamoDb is up and running, you could write some code to create a database. However it is much easier to use the aws command line. All the options I list below are mandatory. I decided to define an attribute named id which I will generate when I write records.

$ aws dynamodb create-table --table-name=vote_dev --attribute-definitions AttributeName=id,AttributeType=S --key-schema AttributeName=id,KeyType=HASH --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5  --endpoint-url http://localhost:8000
{
    "TableDescription": {
        "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/vote_dev", 
        "AttributeDefinitions": [
            {
                "AttributeName": "id", 
                "AttributeType": "S"
            }
        ], 
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0, 
            "WriteCapacityUnits": 5, 
            "LastIncreaseDateTime": 0.0, 
            "ReadCapacityUnits": 5, 
            "LastDecreaseDateTime": 0.0
        }, 
        "TableSizeBytes": 0, 
        "TableName": "vote_dev", 
        "TableStatus": "ACTIVE", 
        "KeySchema": [
            {
                "KeyType": "HASH", 
                "AttributeName": "id"
            }
        ], 
        "ItemCount": 0, 
        "CreationDateTime": 1517770607.687
    }
}

Let's double check everything is ok

$ aws dynamodb list-tables --endpoint-url http://localhost:8000
{
    "TableNames": [
        "vote_dev"
    ]
}

blog terms

serverless serverless-php PHP lambda devops

A Real World PHP Lambda App Part 3: Controllers

nigel — Sun, 28 Jan 2018 10:36:46 +0000

A Real World PHP Lambda App Part 3: Controllers nigel Sun, 28/01/2018 - 10:36

The next step in our odyssey to build a real world Lambda app is to create the controllers. We will need two - one for the form and one for the thankyou page. We should - and therefore will - create an interface for those two. In addition since we want to be able to test our controllers out, we will need to create some placeholder views. The views will be constructed using the Twig template engine so that will have to be added to our build too.

First lets create our controllers.

Navigate to the root directory of the project and issue:

$ pwd
/var/www/html/serverless-vote
$ mkdir src/Controllers
$ touch src/Controllers/FormController.php
$ touch src/Controllers/ThankyouController.php
$ touch src/Controllers/VoteInterfaceController.php

Twig Installation

Twig installation is by composer and thus simplicity itself.

$ composer require "twig/twig:^2.0"
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 2 installs, 0 updates, 0 removals
  - Installing symfony/polyfill-mbstring (v1.7.0) Downloading: 100%         
  - Installing twig/twig (v2.4.4) Downloading: 100%         
Writing lock file
Generating autoload files
$

Interface

The interface is listed below. Note that we construct with the two properties we are going to need to do our work - the data array structure, and the logger object we can use to report to stderr which will be picked up by CloudWatch. We are setting the default HTTP status code to 200 and the headers will always return HTML by default. We have two protected methods in there - the createModel uses the new operator which dooms unit testing so it has been placed outside of the main code, and the _index method which also performs instantiation thus making unit testing difficult.

namespace Vote\Controllers;



abstract class VoteInterfaceController
{

    // Lambda context
    public $data;
    public $logger;


    // http protocol stuff - set default values
    protected $statusCode = 200;
    protected $headers = [
        'Content-Type' => 'text/html'
    ];


    public function __construct($logger, $data)
    {
        $this->logger = $logger;
        $this->data = $data;
    }


    public function __set($key, $value)
    {
        $this->$key = $value;
    }


    public function __get($key)
    {
        return $this->$key;
    }


    /**
     * Instantiate a model.
     * Performing this in it's own function makes unit testing easier
     *
     * @link https://stackoverflow.com/questions/7760635/unit-test-for-mocking-a-method-called-by-new-class-object
     *
     * @param string model
     * @return object
     */
    protected function createModel($model)
    {
        $model = "\\Vote\\Models\\".$model;
        return new $model($this->logger, $this->data);
    }


    /**
     * Wrapper for index controller
     *
     * @param string filename of the template
     * @param array key/values to populate the template
     *
     * @return string rendered template
     */
    protected function _index($filename, $values)
    {
        $loader = new \Twig_Loader_Filesystem(array(dirname(__DIR__).'/Views'));
        $twig = new \Twig_Environment($loader, array('cache' => '/tmp/cache'));
        $content = $twig->load($filename);

        return $content->render($values);
    }

?>

Composer Autoloader

We will need to inform composer of our new autoloader requirements - both Twig and our controllers. So add the following entries under psr-4 in composer.json

  "autoload": {
    "psr-4": {
      "Raines\\Serverless\\": "src",
      "Vote\\Controllers\\": "src/Controllers/",
      "Twig\\": "vendor/twig/twig/lib"
    }
  },

Then update

$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 0 installs, 4 updates, 0 removals
  - Updating symfony/filesystem (v3.3.13 => v4.0.4) Downloading: 100%         
  - Updating symfony/config (v3.3.13 => v3.4.4) Downloading: 100%         
  - Updating symfony/dependency-injection (v3.3.13 => v3.4.4) Downloading: 100%         
  - Updating symfony/yaml (v3.3.13 => v3.4.4) Downloading: 100%         
Writing lock file
Generating autoload files

Controller Code

Here is the code for our controllers. The action index in the FormController merely presents the form to the end user. The save method doesn't actually save the POSTed date yet since we haven't created our models yet. That will be covered in a later tutorial. However it constructs the URL for the redirect to the thank you page using the environmental data which is provided by Lambda. It returns an HTTP status code of 307 - which is the code that says redirect and don't change the original HTTP method, which in our case was POST. Therefore the thank you page will have the POSTed data we want.

FormController.php

namespace Vote\Controllers;


class FormController extends VoteInterfaceController {

    /**
     * Load the form
     */
    public function index()
    {
        return $this->_index(
            'vote.html',
            [
                'title' => 'Badzilla\'s Lambda Vote Form',
                'static_url' => $this->data['static_url']
            ]
        );
    }


    /**
     * Save the captured form and send to DynamoDB then redirect to thank_you
     */
    public function save()
    {
        // Get the data ready for for the model and the redirect
        $data = $this->__get('data');

        // Construct the url using the Lambda data we have
        $url = $data['headers']['X-Forwarded-Proto'].'://';
        $url .= $data['headers']['Host'];
        if (isset($data['requestContext']['stage']) && $data['requestContext']['stage'] != '') {
            $url .= '/'.$data['requestContext']['stage'];
        }
        $url .= '/thank_you';        

        // Set the required http settings for a redirect resubmit POST
        $this->__set('statusCode', 307);
        $this->__set('headers', array('Location' => $url));

        // No body text to return 
        return '';
    }
}
?>

ThankyouController.php

namespace Vote\Controllers;




class ThankyouController extends VoteInterfaceController
{
    public function index()
    {
        return $this->_index(
            'thank-you.html',
            [
                'title' => 'Badzilla\'s Big Thank You',
                'static_url' => $this->data['static_url'],
            ]
        );
    }
}
?>

Custom Serverless Parameter static_url

You'll notice in the code above that I am passing the parameter $this->data['static_url'] to the Twig templates. The static URL contains the base path to the S3 bucket that uses CloudFront to serve our assets from the S3 bucket (via CloudFront) we set up in a previous tutorial. So we need to make sure the URL is set correctly from the configuration.

Firstly open up the file serverless.yml in your favourite editor. Under custom, add the key/value pair where the key is s3StaticBucket and the value is the identifier of your s3 asset bucket (and not the s3 bucket for your deploys. Then under environment, add the full path of that bucket using STATIC_URL as the key. It's best explained by referring to the first image above. This will populate the value of STATIC_URL in the environment which can be picked up in the PHP when it runs.

Now edit handler.php and set the array key $event['static_url'] as shown in the second image.

Views

The views need to be created. These will hold the raw HTML for sending back to the browser. There will be three in total - an overarching layout.html which holds the main HTML structure such as the html head and body tags, and then vote.html and thank-you.html which hold specific body content for the two pages in our app. The actual HTML is beyond the scope of these tutorials, but they are created with:

$ mkdir -p src/Views
$ touch src/Views/layout.html
$ touch src/Views/thank-you.html
$ touch src/Views/vote.html

You obviously now need to put the HTML code in those empty files, and if you are using Twig, then the placeholders for the variables you pass to the templates. You should also be adding your images and JavaScript and css to directories underneath /dist. The entire project should look like this right now.

$ tree -I vendor
.
├── buildphp.sh
├── CHANGELOG.md
├── composer.json
├── composer.lock
├── config
│   └── services.yml
├── dist
│   ├── css
│   │   └── default.css
│   └── images
│       ├── badzilla-logo32x32.png
│       └── favicon.ico
├── dockerfile.buildphp
├── docroot
│   └── layout.html
├── handler.js
├── handler.php
├── LICENSE
├── node_modules
│   └── serverless-single-page-app-plugin
│       ├── index.js
│       ├── package.json
│       └── README.md
├── package-lock.json
├── php
├── README.md
├── serverless.yml
└── src
    ├── Context.php
    ├── Controllers
    │   ├── FormController.php
    │   ├── ThankyouController.php
    │   └── VoteInterfaceController.php
    ├── Handler.php
    ├── Views
    │   ├── layout.html
    │   ├── thank-you.html
    │   └── vote.html
    └── VoteHandler.php
 
10 directories, 29 files

I've used the Bootstrap theme Cosmo which is hotlinked in the layout.html so I don't need too many assets. But what I do have still needs syncing up to the S3 bucket which, as per my earlier tutorial, is achieved with what is below, followed by my final deploy because we are now ready to test our app.

$ sls syncToS3 
$ sls deploy

Run

Ok so time to check out our app. Yes we still have a long way to go because we aren't saving the data as yet. But steady progress is being made.

blog terms

lambda serverless serverless-php PHP devops

A Real World PHP Lambda App Part 2: Routing and a Front Controller

nigel — Sat, 27 Jan 2018 14:06:02 +0000

A Real World PHP Lambda App Part 2: Routing and a Front Controller nigel Sat, 27/01/2018 - 14:06

Since we are making a PHP application we really should stick to the recognised right way of doing this; the MVC pattern is the established best practice. Sitting in front of this pattern is the Front Controller which will act as a controller to determine which C of the MVC will actually be called, and to do a bit of the routing job, and to act as the layer between our actual code and the hander,js / handler.php shim that comes with serverless-php.

In fact we get a starter Front Controller in the serverless-php repo already although it is very basic and will need a little extending to serve a purpose in a MVC web app.

The name of the game is to keep whatever we develop to be very lightweight. Whilst it would be good to get Symfony / Zend / Laravel running under serverless-php, that will come at a cost of execution time and development time.

So the 'Front Controller' is called HelloHandler.php in the src/ directory but you should've renamed it VoteHandler.php already and changed the name of the class it contains.

The second consideration is the AWS API Gateway - that too needs to know which routes we are defining. We can accomplish that by configuration in the serverless.yml file.

As previously mentioned our voting app is going to present a form (HTTP GET), then when submitted (HTTP POST) it will cache the name of the voter and that person's choice in a NoSQL data store, then redirect to a thank you page (HTTP POST). Note that the redirect must be a POST since Google don't take kindly to URL parameters containing personal data, and we want to display the voter's name on the thank you page.

serverless.yml

We need to define the functions - one for each of the transactions. Present the form; Submit the form; Redirect to thank_you. Note that since we are always going to channel the code through our Front Controller and therefore we are going to use the same service.

functions:
  vote_get:
    handler: handler.handle
    environment:
      HANDLER: handler.vote  # This is the service name which will be used (from services.yml)
    events:
      - http:
          path: /
          method: get
  vote_post:
    handler: handler.handle
    environment:
      HANDLER: handler.vote  # This is the service name which will be used (from services.yml)
    events:
      - http:
          path: /
          method: post
  thank_you:
    handler: handler.handle
    environment:
      HANDLER: handler.vote  # This is the service name which will be used (from services.yml)
    events:
      - http:
          path: /thank_you
          method: post

Now lets deploy to AWS

$ sls deploy
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (9.96 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
.....................................................................
Serverless: Stack update finished...
Service Information
service: vote
stage: dev
region: eu-west-1
stack: vote-dev
api keys:
  None
endpoints:
  GET - https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/
  POST - https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/
  POST - https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/thank_you
functions:
  vote_get: vote-dev-vote_get
  vote_post: vote-dev-vote_post
  thank_you: vote-dev-thank_you

If we log into the AWS Console and navigate to the API Gateway we can see our functions are declared correctly.

Front Controller

Below is the front controller src/VoteHandler.php. Most of it is quite self-explanatory but a few points are worthy of commentary. The $routing is hard-coded since we need to match a path / method against a controller. The $event is populated with the serverless runtime such as command line data, HTTP method and the route. In the main handle function we are returning whatever the controller passes back to us in terms of HTTP status code, headers and body. It is up to the controllers to determine this. Finally the function createController is used to avoid having a new operator in the handle function which would make it impossible to be unit tested,

namespace Raines\Serverless;

use Psr\Log\LoggerInterface;

class VoteHandler implements Handler
{
    /**
     * Routing table
     */
    protected $routing = [
        [
            'path' => '/',
            'method' => 'GET',
            'controller' => 'Form',
            'action' => 'index'
        ],
        [
            'path' => '/',
            'method' => 'POST',
            'controller' => 'Form',
            'action' => 'save'
        ],
        [
            'path' => '/thank_you',
            'method' => 'POST',
            'controller' => 'Thankyou',
            'action' => 'index'
        ],
    ];


    /**
     * {@inheritdoc}
     */
    public function handle(array $event, Context $context)
    {
        $logger = $context->getLogger();
        $logger->notice('Got event', $event);

        // get the controller and action based on the path and method
        $route = [];
        foreach($this->routing as $k => $r) {
            if ($r['path'] == $event['path'] && $r['method'] == $event['httpMethod']) {
                $route[] = $this->routing[$k];
            }
        }

        try {
            if (count($route) != 1) {
                throw new \Exception('Inconsistent routing provided');
            }

            // Construct the controller name
            $classname = "Vote\Controllers\\{$route[0]['controller']}Controller";

            // See if classname and action both exist
            if (!class_exists($classname) || !method_exists($classname, $route[0]['action'])) {
                throw new \Exception('Unknown classname or action provided');
            }

        } catch (\Exception $e) {
            return [
                'headers' => ['Content-Type' => 'application/json'],
                'statusCode' => 500,
                'body' => json_encode($e->getMessage(), JSON_HEX_QUOT),
            ];
        }

        // Create the controller
        $controller = $this->createController($classname, $logger, $event);

        // Execute the action
        $output = $controller->{$route[0]['action']}();

        return [
            'headers' => $controller->__get('headers'),
            'statusCode' => $controller->__get('statusCode'),
            'body' => $output,
        ];
    }


    /**
     * Instantiate the controller
     *
     * @param string $classname
     * @param LoggerInterface $logger
     * @param array $event
     *
     * @return object
     */
    private function createController(string $classname, LoggerInterface $logger, array $event)
    {
        return new $classname($logger, $event);
    }
}
?>

We also need to make a small change to handler.php, In the final line we need to make sure the json_encode() call is passed the JSON_HEX_QUOT parameter to ensure correct escaping.

$loader = require __DIR__ . '/vendor/autoload.php';

use Raines\Serverless\Context;
use Symfony\Component\Config\FileLocator;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Loader\YamlFileLoader;

// Set up service container
$container = new ContainerBuilder();
$loader = new YamlFileLoader($container, new FileLocator(__DIR__ . '/config'));
$loader->load('services.yml');

// Get event data and context object
$event = json_decode($argv[1], true) ?: [];
$logger = $container->get('logger');
$fd = fopen('php://fd/3', 'r+');
$context = new Context($logger, $argv[2], $fd);

// Get the handler service and execute
$handler = $container->get(getenv('HANDLER'));
$response = $handler->handle($event, $context);

// Send data back to shim
printf(json_encode($response, JSON_HEX_QUOT));
?>

Runtime Invocation

We are now ready to run our code from the command line locally. Of course it isn't going to do too much since we've not added the MVC part of the code yet! However I would expect it to get to the point it tries to instantiate a controller before throwing an exception.

The syntax for local invocation is below. Note the JSON structure passed using the flag --data - this will be interpreted by our Front Controller in the populated $event array. The -f flag denotes the Lambda function we are running.

$ sls invoke local -f vote_get --data '{"path":"/","httpMethod":"POST"}'
Got event {"path":"/","httpMethod":"POST"} []
{
    "headers": {
        "Content-Type": "application/json"
    },
    "statusCode": 500,
    "body": "Unknown classname or action provided"
}

Ok - an error but totally expected. Progress is being made!

blog terms

lambda serverless serverless-php PHP devops

A Real World PHP Lambda App Part 1: Extend to CloudFront + S3

nigel — Sat, 23 Dec 2017 17:02:11 +0000

A Real World PHP Lambda App Part 1: Extend to CloudFront + S3 nigel Sat, 23/12/2017 - 17:02

We are going to create a voting app to collect information from our users. This will require a HTML form to be initially rendered to the user using the HTTP GET method and then POSTed to the PHP backend for processing and the data saved into AWS DynamoDB. The page should then be redirected to a thank you page.

We shall call this app vote. So our first activity will be to refer back to my initial blog but instead of create a project demo, call it vote. Once you are ready check your function has been created correctly with the command below.

$ serverless invoke local -f vote
{
    "statusCode": 200,
    "body": "Go Serverless v1.0! Your function executed successfully!"
}
$

Note how I am changing my regime. I now want to do all my development locally and only rarely deploy to S3. This will cut down S3 costs (although they won't be exorbitant) and speed up development .As we go through the blogs I will point out techniques to aid the local development process.

Ok - so we've reached base camp and we will now build on our sample function.

Prerequisites

We will need the aws cli console to be able to upload our app's static assets (css, js, images) to S3 for them to be served by CloudFront. The console has a requirement for Python so that will need to be installed first if you don't already have it. If you already have Python installed, execute the command below.

# pip install awscli --ignore-installed six

Next we will need to install our first Serverless plugin - this will provide us with the facility to sync our local assets with the S3 bucket we are going to create for our app. The plugin we need to achieve this is serverless-single-page-app-plugin. Let's install it. Navigate to the root directory of our app and issue the following.

$ npm install --save serverless-single-page-app-plugin

This will place the serverless-single-page-app-plugin in the parent directory node_modules. We need to let serverless know about our plugin by adding the following to our serverless.yml file just after the service: section.

plugins:
  - serverless-single-page-app-plugin

We now need to include into the serverless.yml file the CloudFormation configuration for the S3 assets bucket and for it to serve pages through CloudFront. Thankfully this is provided in the serverless-single-page-app-plugin README.md file so simply copy and paste the following.

resources:
  Resources:
    ## Specifying the S3 Bucket
    WebAppS3Bucket:
      Type: AWS::S3::Bucket
      Properties:
        AccessControl: PublicRead
        WebsiteConfiguration:
          IndexDocument: index.html
          ErrorDocument: index.html
    ## Specifying the policies to make sure all files inside the Bucket are avaialble to CloudFront
    WebAppS3BucketPolicy:
      Type: AWS::S3::BucketPolicy
      Properties:
        Bucket:
          Ref: WebAppS3Bucket
        PolicyDocument:
          Statement:
            - Sid: PublicReadGetObject
              Effect: Allow
              Principal: "*"
              Action:
              - s3:GetObject
              Resource:
                Fn::Join: [
                  "", [
                    "arn:aws:s3:::",
                    { "Ref": "WebAppS3Bucket" },
                    "/*"
                  ]
                ]
    ## Specifying the CloudFront Distribution to server your Web Application
    WebAppCloudFrontDistribution:
      Type: AWS::CloudFront::Distribution
      Properties:
        DistributionConfig:
          Origins:
            - DomainName:
                Fn::Join: [
                  "", [
                    { "Ref": "WebAppS3Bucket" },
                    ".s3.amazonaws.com"
                  ]
                ]
              ## An identifier for the origin which must be unique within the distribution
              Id: WebApp
              CustomOriginConfig:
                HTTPPort: 80
                HTTPSPort: 443
                OriginProtocolPolicy: https-only
              ## In case you want to restrict the bucket access use S3OriginConfig and remove CustomOriginConfig
              # S3OriginConfig:
              #   OriginAccessIdentity: origin-access-identity/cloudfront/E127EXAMPLE51Z
          Enabled: 'true'
          ## Uncomment the following section in case you are using a custom domain
          # Aliases:
          # - mysite.example.com
          DefaultRootObject: index.html
          ## Since the Single Page App is taking care of the routing we need to make sure ever path is served with index.html
          ## The only exception are files that actually exist e.h. app.js, reset.css
          CustomErrorResponses:
            - ErrorCode: 404
              ResponseCode: 200
              ResponsePagePath: /index.html
          DefaultCacheBehavior:
            AllowedMethods:
              - DELETE
              - GET
              - HEAD
              - OPTIONS
              - PATCH
              - POST
              - PUT
            ## The origin id defined above
            TargetOriginId: WebApp
            ## Defining if and how the QueryString and Cookies are forwarded to the origin which in this case is S3
            ForwardedValues:
              QueryString: 'false'
              Cookies:
                Forward: none
            ## The protocol that users can use to access the files in the origin. To allow HTTP use `allow-all`
            ViewerProtocolPolicy: redirect-to-https
          ## The certificate to use when viewers use HTTPS to request objects.
          ViewerCertificate:
            CloudFrontDefaultCertificate: 'true'
          ## Uncomment the following section in case you want to enable logging for CloudFront requests
          # Logging:
          #   IncludeCookies: 'false'
          #   Bucket: mylogs.s3.amazonaws.com
          #   Prefix: myprefix
  ## In order to print out the hosted domain via `serverless info` we need to define the DomainName output for CloudFormation
  Outputs:
    WebAppS3BucketOutput:
      Value:
        'Ref': WebAppS3Bucket
    WebAppCloudFrontDistributionOutput:
      Value:
        'Fn::GetAtt': [ WebAppCloudFrontDistribution, DomainName ]

It's time to deploy to AWS but before you do, check your credentials are loaded, and if not then fix. In my case I added them to the bottom of my ~/.bashrc file then logged out and in for them to take effect.

$ env | grep AWS
AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXX
AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXX
$

Now deploy with

$ sls deploy
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (9.96 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...........................
Serverless: Stack update finished...
Service Information
service: vote
stage: dev
region: eu-west-1
stack: vote-dev
api keys:
  None
endpoints:
  GET - https://n4kofna4l1.execute-api.eu-west-1.amazonaws.com/dev/vote
functions:
  vote: vote-dev-vote
$

This took an eternity first time through! I went and made a cup of tea!

AWS Console - S3

If we now navigate to S3 in the AWS console we should be able to see our two buckets for this project; one for the assets and one for our deployed serverless code.

AWS Console - CloudFront

Now if we navigate to CloudFront we can see the service we have just created which will serve our S3 assets.

The plugin provides the serverless syncToS3 command which is run to perform the sync, but it requires configuration before we can use it. We need to set a custom variable s3LocalPath in the serverless.yml file.

$ cat serverless.yml | grep -A1 custom:
custom:
  s3LocalPath: dist/
$

I have decided to put my assets in a subdirectory under my root called dist/. I now need to create that local directory and whilst I am at it I'll create an image subdirectory with a sample Badzilla logo image I can test the sync with.

$ mkdir -p dist/images
$ cp ../meedjum/docroot/themes/custom/beezee8/badzilla-logo32x32.png dist/images/.

Now let's be adventurous and see if the sync works.

$ sls syncToS3
Serverless: s3,sync,dist/,s3://vote-dev-webapps3bucket-1htbi30lx4nii/
upload: dist/images/badzilla-logo32x32.png to s3://vote-dev-webapps3bucket-1htbi30lx4nii/images/badzilla-logo32x32.png
 
Serverless: stderr undefined
Serverless: Successfully synced to the S3 bucket
$

Yey! Looking good but let's check it properly!

S3 Sync

Navigate on the console to the S3 bucket, then through the filesystem and the images directory then click on badzilla-logo32x32.png. Looks Good.

CloudFront Browser Access

Now if we point a web browser at your Bazilla logo asset we see that it is rendered correctly (look closely!). Yippee! We now have a working S3 / CloudFront configuration for our industrial strength Lambda function.

blog terms

lambda serverless serverless-php PHP devops

A Real World PHP Lambda App

nigel — Sun, 26 Nov 2017 18:26:58 +0000

A Real World PHP Lambda App

My previous blogs on AWS Lambda, PHP, Serverless and Serverless-PHP should have given you enough of a basic understanding of the mix of technologies to attempt to build something much more useful - a fully fledged PHP app.

What would be the motivation for doing this? What type of sites lend themselves to be built with Serverless?

Well certainly nothing on a grand scale - it would be foolhardy to attempt to host the BBC website on Lambda!! However Lambda lends itself to websites that may be immensely popular over a relatively short period of time. These could include (but are not limited to)

News announcements for significant events (e.g. Oscar results for instance)
Promotional websites for new products
Registration / statement of interest sites
Advocacy / campaigns
Questionnaires / surveys / voting

Any small site that has the possibility of incurring scaling problems, significant orchestration and deployment costs is a candidate. Furthermore once the promotion has served its purpose, the site could be re-skinned for re-use at a later date.

Why PHP? If you already have PHP skills, or your work department does, then it frequently makes sense to standardise skills when building out products.

The tutorials will show how to create a PHP site using a basic MVC with Twig templates, saving and retrieving data in DynamoDB, use the POST and GET HTTP methods, the correct way to redirect using PHP on top of a JavaScript shim, store your assets on S3 and CloudFront amongst other neat tricks. Let the fun begin - pick your tutorial from the list below!

nigel Sun, 26/11/2017 - 18:26

blog terms

lambda serverless serverless-php PHP devops

Recompiling PHP for serverless-php and Lambda

nigel — Sun, 05 Nov 2017 15:54:48 +0000

Recompiling PHP for serverless-php and Lambda nigel Sun, 05/11/2017 - 15:54

The serverless-php git repo contains a pre-compiled PHP executable which is uploaded to AWS Lambda as part of the FaaS deploy. However, and deliberately to keep it lightweight, many PHP extensions are not included in this PHP image. I thought I may get through my development without having to rebuild this image, but alas not! I decided to use the Twig template engine and that has a dependency on the hash() function. I realised something was afoot when my execution failed.

Cloudwatch error diagnostic

The error was discovered by inspecting my Lambda function run log - you may be able to see able there is a uncaught exception leading to a fatal error.

Fatal error: Uncaught Error: Call to undefined function hash() in /var/task/vendor/twig/twig/lib/Twig/Environment.php:272"

I wasn't aware that hash() required a separate PHP extension, so I checked this on my local copy of PHP 7.1 on my Macbook with

$ php -m | grep hash
hash
$

Aha! So nothing else for it, we need to add this to the compiled PHP image.

Recompilation of PHP image

Thankfully the serverless-php repo contains two files to aid the recompilation of the PHP image - buildphp.sh which is a shell script to run the process, and dockerfile.buildphp which is the configuration for a docker container in which the actual recompilation will occur.

The dockerfile contains the list of PHP extensions to include - so I added hash to the list

RUN ./configure \
    --enable-static=yes \
    --enable-shared=no \
    --disable-all \
    --enable-json \
    --enable-libxml \
    --enable-mbstring \
    --enable-phar \
    --enable-soap \
    --enable-xml \
    --enable-hash \
    --with-curl \
    --with-gd \
    --with-zlib \
    --with-openssl \
    --without-pear \
    --enable-ctype

Now we need to check the PHP filesize and run the build script - most of the output of this has been cropped since it was copious.

$ ls -las php
49144 -rwxr-xr-x@  1 nigel  staff  25161497  2 Nov 14:03 php
$ chmod +x buildphp.sh 
$ ./buildphp.sh 
Build complete.
Don't forget to run 'make test'.
 
 ---> 7b602edcddb5
Removing intermediate container 6c3cd6936a6d
Successfully built 7b602edcddb5
Successfully tagged php-build:latest
2bdff18d52adad839b6cbdba9ee55e3263d5399326175827398f757cc9c74c44
$

Ok so let's check what we've got:

$ ls -las php
50736 -rwxr-xr-x  1 nigel  staff  25976432  4 Nov 16:34 php

A different size which is encouraging! Now let's deploy and run:

$ sls deploy

Results

If we now run our function from the API Gateway or point a web browser on our assigned URL we get a 200 success! Yey!

blog terms

PHP serverless-php serverless lambda Docker devops

serverless

Drupal 8 as a Static Site: AWS API Gateway, Lambda and SES Form Processing

'.$fields['subject'][0]['value'].'

'.'From: '.$fields['name'].' .$fields['mail'].'">'.$fields['mail'].''.'

A Real World PHP Lambda App Part 7: Load Testing with serverless-artillery

A Real World PHP Lambda App Part 6: Local Development

A Real World PHP Lambda App Part 5: Models

A Real World PHP Lambda App Part 4: Setting up DynamoDb Locally

A Real World PHP Lambda App Part 3: Controllers

A Real World PHP Lambda App Part 2: Routing and a Front Controller

A Real World PHP Lambda App Part 1: Extend to CloudFront + S3

A Real World PHP Lambda App

Recompiling PHP for serverless-php and Lambda