bash

AWS Fargate Static Analysis with PHP

nigel — Mon, 13 Jan 2020 14:28:49 +0000

AWS Fargate Static Analysis with PHP nigel Mon, 13/01/2020 - 14:28

In my earlier blog on AWS Fargate Automated Testing I spoke about creating an AWS Fargate task definition which would include a web app's codebase and the Cucumber / Gherkin / maven / Java runtime stack. This tutorial is an opportunity to look at the static analysis stage of the pipeline where code is inspected for vulnerabilities, coding standards, compilation errors, copy and paste detection amongst others. Since my background is in Drupal and PHP, my tests will be focused around the PHP language and will use the excellent Static Analysis Tools for PHP repo.

Looking at the jakzal/phpqa repo, it is clear there is a great deal of tooling available. The purpose of this tutorial is take a small sample of these tools and create a Fargate task to run them against my codebase. My somewhat arbitrary selection is:

phpcpd - Copy/paste detection
phpcs - Codesniffer that detects coding standards violations
phplint - PHP syntax error checker.

Ok - let's get started.

Create the Dockerfile and the Entrypoint

As I have previously mentioned, Fargate is a black box and it doesn't lend itself to command line invocation of the tooling. Typically locally on a laptop we would run the jakzal/phpqa Docker image like:

$  docker run -it --rm -v $(PWD):/project -w /project  jakzal/phpqa phpcpd web/core/modules/views/src
$  docker run -it --rm -v $(PWD):/project -w /project  jakzal/phpqa phpcs web/core/modules/views/src
$  docker run -it --rm -v $(PWD):/project -w /project  jakzal/phpqa phplint web/core/modules/views/src

This could obviously be placed in a shell script. Now in Fargate we don't have access to the command line and we have to create a Fargate definition up front before that can be provisioned and then run. So it's a little bit more involved.

The simplest approach is to create a Dockerfile based on the jakzal/phpqa repo, and an extrypoint script that will run each one of the tools I have identified.

The Dockerfile is defined below. Note that I have selected PHP version 7.2. This is the closest supported version of PHP to what I'm running on my production box. Think I need to upgrade that at some point..

FROM jakzal/phpqa:1.28.1-php7.2
ADD entry.sh /entry.sh
RUN /bin/bash -c 'chmod +x /entry.sh'
ENTRYPOINT ["/entry.sh"]

Next up is the entry.sh file. First thing to note is I have added the set +e flag, which is default anyway. This is included to ensure that even if I get some pernickety error messages the script won't terminate immediately, and it acts as an aide memoire that we don't want error termination so don't accidentally copy paste in a competing set -e flag.

There is then a check whether the shared ECS volume has been loaded - this is the responsibility of the codebase container, but because containers can start at varying times, I have to ensure I don't run the PHP analysis before the volume is ready.

The path to the source code to perform the analysis on is defined as /var/static_tests/web/core/modules/views/src. This has two components. The first part /var/static_tests/ is the mount point which will correspond to my codebase top level directory. The second component, web/core/modules/views/src points to the core contributed Drupal Views module. This is purely there as a reference to some arbitrary code. I am using the codebase to my own Badzilla which is a site build with no custom code and thus I have none of my own code to use in this sample.

#!/usr/bin/env sh
 
set +e
 
while [ ! -d /var/static_tests/web ]
do
  sleep 5
  echo "Waiting for volume to be mounted and files copied into position"
done
 
phpcpd /var/static_tests/web/core/modules/views/src
phpcs /var/static_tests/web/core/modules/views/src
phplint /var/static_tests/web/core/modules/views/src

Building the Docker image locally

It isn't mandatory to test everything works before constructing the Fargate task definition, but it makes sense to me to test my work locally first. If the Fargate task subsequently fails then I know the problem area will be localised to the AWS eco-system, and probably to Fargate. So I always build locally and run locally first. To build I cd into my Dockerfile / entry.sh entry and issue:

$ docker build -f Dockerfile -t static-fargate  .

To run it's

$ docker run -it --rm -v $(PWD):/var/static_tests  static-fargate

You can see above I am establishing the Docker volume which will be shared with my codebase.

The Docker Hub repo for this is at https://hub.docker.com/repository/docker/sanddevil/static-fargate and the GitHub repo for the code is at https://github.com/sanddevil/devops-static-php

Define the Fargate task

With everything working locally it's time to build the Fargate task. This is very similar to the script I created in my earlier automated testing blog. There are a few differences obviously. My container stack comprises of a combined PHP + Apache + my own codebase image, and the PHP static analysis jakzal/phpqa obviously. I use the shell script heredoc convention to define each container. You should note I have now expanded my codebase container to include two shared volumes - one for the automated tests (not discussed here but discussed previously) and the static tests.

#!/bin/bash
 
 
BRANCH="master"
REPO="php7.0-apache/${BRANCH}"
 
# Repo account and repo
ECR_ACCOUNT="xxxxxxxxxxx.dkr.ecr.eu-west-2.amazonaws.com"
ECR_REPO="${ECR_ACCOUNT}/${REPO,,}"
ECS_CLUSTER="FargatePrototype"
 
# Get the Task Role ARN for executing the task
TASK_ARN=`aws iam get-role --role-name ecsTaskExecutionRole | jq .Role.Arn -r`
 
 
# Execution ARN appears to be the same as the Task ARN
EXECUTION_ARN="${TASK_ARN}"
 
 
# Determine if the log group already exists: NOTE: "aws ecs register-task-definition" will not create the log group for you!
LOG_EXISTS=`aws logs describe-log-groups | jq ".logGroups[] | select(.logGroupName == \"/ecs/"${BRANCH}"\") | .logGroupName" -r | wc -l`
if [[ "${LOG_EXISTS}" -eq 0 ]]; then
	aws logs create-log-group --log-group-name "/ecs/${BRANCH}"
fi
 
CONTAINER_CODEBASE=$(cat <
)
 
CONTAINER_STATIC_PHP=$(cat <
)
 
 
CONTAINERS_STATIC=$(cat <
)
 
 
aws ecs register-task-definition \
  --family "STATIC-${BRANCH}" \
  --volumes "name=static_tests,host={}" "name=automated_tests,host={}" \
  --task-role-arn "$TASK_ARN" \
  --execution-role-arn "${EXECUTION_ARN}" \
  --network-mode "awsvpc" \
  --container-definitions "${CONTAINERS_STATIC}" \
  --cpu "256" \
  --memory "512" \
  --requires-compatibilities "FARGATE"

I'm not quite done yet though. Somehow I need to copy my codebase into the shared Docker volume so that the PHP static analysis container can see it. This is done in the entry.sh script of my PHP + Apache + Codebase image (not discussed here) but the entry.sh file is shown below for completeness.

#!/bin/bash
set -e
 
# Copy the automated tests into a shared area accessible to maven
cp -R /var/www/html/test  /var/automated_tests/.
 
# Copy the PHP codebase into a shared area accessible to the PHP static analysis container
cp -R /var/www/html/web  /var/static_tests/.
 
# Run Apache in the foreground to prevent the container from quitting
apache2-foreground

Here you can see the copying I do for both the automated tests and the static analysis.

Running the Fargate task

Normally I would invoke the Fargate task programmatically using AWS CLI but since that leads to a rather dry image-free blog, I will use the console in this instance. My previous blog shows how to navigate around the ECS dashboard to run the task, then you can see the task running in the first screenshot above. It will only run until the entrypoint script terminates as per the rules on how Docker works, so to capture that scrennshot I had to be quick. The second screenshot shows the CloudWatch logs for the task, and you can see there are a few copy/paste errors detected.

blog terms

devops AWS AWS CLI ECS Docker bash

AWS Fargate Automated Testing with Cucumber, Gherkin, Java, Maven, Selenium Grid Hub and Chrome and Firefox Nodes

nigel — Mon, 09 Dec 2019 08:37:50 +0000

AWS Fargate Automated Testing with Cucumber, Gherkin, Java, Maven, Selenium Grid Hub and Chrome and Firefox Nodes nigel Mon, 09/12/2019 - 08:37

Tutorial for creating an automated testing AWS Fargate task that will include Docker containers for a full testing framework:

Maven for build automation
Java Runtime to enable the execution of maven and the junit unit testing framework
Java Cucumber - a tool for BDD testing
Gherkin to define behaviours in the English language
Selenium Hub being the central loading point for the tests
Selenium Chrome - one of the nodes
Selenium Firefox - a second node
Apache / PHP - the web app and web server.

This is not a trivial exercise but by the end of the blog I will have a working automated testing environment which can be slotted into a DevOps pipeline. Please note this blog is not a tutorial on how to write Cucumber / Gherkin tests, however I will create a few sample Gherkin behaviours and their respective Java Cucumber stepdefs as a starting point. These can then be expanded out by your QA team.

Starting Point

The starting point will be the simple PHP/Apache web app I put together in my previous blog here - this is a single url website which merely shows the phpinfo() status page. But it's ideal for our automated testing because our testing framework is plenty complex already, and we can still run a few tests against this simple landing page such as whether the additional PHP extensions I included in the Dockerfile are loaded correctly.

Now we need to build out our testing tree structure.

At a sibling level to the docroot directory I created in the previous blog, create the following:

$ mkdir -p mytests/src/test/java/mytests
$ mkdir -p mytests/src/test/resources/mytests

Now it's time to create the files I need

$ cd mytests/
$ touch pom.xml
$ touch src/test/java/mytests/RunCucumberTest.java
$ touch src/test/java/mytests/Stepdefs.java
$ touch src/test/java/mytests/SetupTestDriver.java
$ touch src/test/resources/mytests/home_page.feature

This will create a tree structure like this:

$ tree
.
|-- pom.xml
`-- src
    `-- test
        |-- java
        |   `-- mytests
        |       |-- RunCucumberTest.java
        |       |-- SetupTestDriver.java
        |       `-- Stepdefs.java
        `-- resources
            `-- mytests
                `-- home_page.feature
 
6 directories, 5 files

Now we need to flesh out those files we created. Firstly the pom file which contains a list of the project's dependencies. Careful inspection will show you that I used the hellocucumber template as my starting point.
pom.xml

 version="1.0" encoding="UTF-8"?>
 xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    >4.0.0>
 
    >hellocucumber>
    >mytests>
    >1.0.0-SNAPSHOT>
    >jar>
 
    >
        >4.2.6>
        >UTF-8>
    >
 
    >
        >
            >io.cucumber>
            >cucumber-java>
            >${cucumber.version}>
            >test>
        >
 
        >
            >io.cucumber>
            >cucumber-junit>
            >${cucumber.version}>
            >test>
        >
 
        >
            >junit>
            >junit>
            >4.12>
            >test>
        >
 
    
    >
        >io.github.bonigarcia>
        >webdrivermanager>
        >3.7.1>
    >
 
	
	>
    		>org.seleniumhq.selenium>
    		>selenium-java>
    		>3.141.59>
	>
 
    
    >
        >org.seleniumhq.selenium>
        >selenium-chrome-driver>
        >3.141.59>
    >
 
    
    >
        >org.apache.maven.plugins>
        >maven-surefire-plugin>
        >2.22.2>
    > 
 
    >
        >org.slf4j>
        >slf4j-api>
        >1.5.6>
    >
 
    >
        >org.slf4j>
        >slf4j-log4j12>
        >1.5.6>
    > 
 
    >
 
    >
        >
            >
                >org.apache.maven.plugins>
                >maven-compiler-plugin>
                >3.7.0>
                >
                    >UTF-8>
                    >1.8>
                    >1.8>
                >
            >
        >
    >
>

Now my Gherkin tests themselves. These are rather simple - I am checking for whether my GD and Memecached PHP extensions have been loaded, i.e. they are listed on the front page which just shows the entire PHP configuration. Under normal circumstances Gherkin test tend to be GIVEN..WHEN..THEN constructs, but since I am not actually doing anything (e.g. clicking a button to go somewhere) I only need WHEN..THEN
home_page.feature

Feature: Are we on the Home Page?
 
 
  Scenario: On the home page
    When User is on the Home Page
    Then Message Displayed Memcached
 
    When User is on the Home Page
    Then Message Displayed GD

Next is the StepDef file where I convert the Gherkin into Cucumber Java containing the actual business logic behind the quasi-English Gherkin. I don't pretend to be an expert Java coder - I have the sum total of about 4 days experience. Therefore my code is probably sub-optimal, but it does the job and an in-house Java professional could whip it into shape. I have defined two browsers - Firefox and Chrome as per our Selenium configuration, although please note there is currently no semblance of parallel running of the tests in here. That again has been left to more talented Java practitioners than myself.
StepDefs.java

package mytests;
 
import base.SetupTestDriver;
 
import java.util.concurrent.TimeUnit;
 
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import io.github.bonigarcia.wdm.WebDriverManager;
 
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
 
import java.net.MalformedURLException;
 
import static org.junit.Assert.*;
 
 
public class Stepdefs {
 
	public static WebDriver[] drivers;
	public static String[][] browsers;
 
    @When("User is on the Home Page")
    public void user_is_on_Home_Page() throws Throwable {
       	browsers = new String[][] {
       		{"linux", "firefox", "http://127.0.0.1", "http://127.0.0.1:4444"},
            {"linux", "chrome", "http://127.0.0.1", "http://127.0.0.1:4444"}
        };
 
		drivers = new WebDriver[browsers.length];
        for (int i = 0; i < browsers.length; i++) {
        	drivers[i] = new SetupTestDriver(browsers[i][0], browsers[i][1], browsers[i][2], browsers[i][3]).getDriver();
        }
    }
 
    @Then("Message Displayed Memcached")
    public void message_displayed_memcached() throws Throwable {
        for (int i = 0; i < drivers.length; i++) {
        	Boolean exists = !drivers[i].findElements(By.name("module_memcached")).isEmpty();
        	drivers[i].quit();
        	assertTrue(exists);
        }
    }
 
	@Then("Message Displayed GD")
	public void gd() throws Throwable {
        for (int i = 0; i < drivers.length; i++) {
			Boolean exists = !drivers[i].findElements(By.name("module_gd")).isEmpty();
        	drivers[i].quit();
        	assertTrue(exists);
        }
	}
}

The StepDef file doesn't contain any Firefox or Chrome webdriver startup configuration. This has been placed in a separate Java class.
SetupTestDriver.java

// src/main/java/base/SetupTestDriver.java
package base;
 
import org.openqa.selenium.Platform;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.firefox.FirefoxDriverLogLevel;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.Dimension;
 
import java.net.MalformedURLException;
import java.net.URL;
import java.util.concurrent.TimeUnit;
 
public class SetupTestDriver {
    private WebDriver driver = null;
    private String browser = null;
    private String baseUrl = null;
    private String os = null;
    private String node = null;
 
    public SetupTestDriver(String os, String browser, String baseUrl, String node) throws MalformedURLException {
        this.browser = browser;
        this.os = os;
        this.baseUrl = baseUrl;
        this.node = node;
 
        Platform platform = Platform.fromString(os.toUpperCase());
        if(browser.equalsIgnoreCase("chrome")) {
            ChromeOptions chromeOptions = new ChromeOptions();
            chromeOptions.addArguments("--headless");
            chromeOptions.setCapability("platform", platform);
            this.driver = new RemoteWebDriver(new URL(node + "/wd/hub"), chromeOptions);
        } else if (browser.equalsIgnoreCase("firefox")) {
            FirefoxOptions firefoxOptions = new FirefoxOptions();
            firefoxOptions.setProfile(new FirefoxProfile());
            firefoxOptions.setCapability("platform", platform);
            firefoxOptions.setHeadless(true);
            this.driver = new RemoteWebDriver(new URL(node + "/wd/hub"), firefoxOptions);
        }
 
        this.driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
        // Maximizing the screen doesn't appear to work.
        this.driver.manage().window().setSize(new Dimension(1600,900));
        this.driver.get(baseUrl);
 
    }
 
    public String getOs() {
        return this.os;
    }
 
    public String getBrowser() {
        return this.browser;
    }
 
    public String getBaseUrl() {
        return this.baseUrl;
    }
 
    public String getNode() {
        return this.node;
    }
 
    public WebDriver getDriver() {
        return this.driver;
    }
}

Finally there's some code to perform the glue between Gherkin and Cucumber.
RunCucumberTest.java

package mytests;
 
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
 
@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}

Build the Docker Codebase + PHP/Apache

In my previous blog here I showed how to add the codebase to the php:7.2-apache image. The codebase has now changed, and I now have two top level sibling directories docroot and mytests - which contains the automated test code I've just added. There is a change we need to make to the Dockerfile in this revision, and the inclusion of an entrypoint script called entry.sh. The entry.sh file will copy the test code suite into a shared Fargate non-persistent drive so that the Maven container can access the codebase and run the tests against it. Once the copy of files has been completed, Apache must be run in foreground mode to prevent the container from quitting. Below we have the new version of the Dockerfile, and the entry.sh file. The image needs to be rebuilt, but since that was covered in the previous blog, it is omitted here.

Dockerfile

FROM php:7.2-apache
RUN apt-get update && apt-get install -y libmemcached-dev zlib1g-dev \
    libfreetype6-dev \
    libjpeg62-turbo-dev \
    libpng-dev \
    libwebp-dev \
    && pecl install memcached \
    && docker-php-ext-enable memcached \
    && docker-php-ext-configure gd --with-gd --with-webp-dir --with-jpeg-dir \
       --with-png-dir --with-zlib-dir --with-freetype-dir \
    && docker-php-ext-install gd
COPY --chown=www-data:www-data ./ /var/www/html/.
COPY entry.sh /usr/local/bin/.
ENV AH_SITE_ENVIRONMENT devops
ENV APACHE_DOCUMENT_ROOT /var/www/html/docroot
RUN sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf
RUN sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf
RUN /bin/bash -c 'chmod +x /usr/local/bin/entry.sh'

entry.sh

#!/bin/bash
set -e
 
# Copy the automated tests into a shared areaaccessible to maven
cp -R /var/www/html/mytests  /var/automated_tests/.
 
# Run Apache in the foreground to prevent the container from quitting
apache2-foreground

Maven:3.6.2-jdk-13

The Java runtime is bundled with the Maven image if the correct tag is selected. I have opted for maven:3.6.2-jdk-13 which is up to date and supported. This requires a small amount of customisation, so again a Dockerfile will be needed. In this instance I want to put a wait loop in the entrypoint script to ensure that there's no attempt to run the tests before the non-persistent shared drive containing the test suite has been mounted by the d8codebase container. At the end I exec the mvn clean install command to run the tests.

The easiest way of achieving this is to edit the mvn-entrypoint.sh script which is in /usrlocal/bin. I copied out of the image to edit and the Dockerfile copies it back into place.

Using my previous blog I built this, tagged and pushed to ECR using the image name d8maven. The instructions aren't copied here for brevity reasons - please refer to my previous blog if you need to.

Dockerfile

FROM maven:3.6.2-jdk-13
COPY mvn-entrypoint.sh /usr/local/bin/mvn-entrypoint.sh
RUN /bin/bash -c 'chmod +x /usr/local/bin/mvn-entrypoint.sh'

mvn-entrypoint.sh

#! /bin/sh -eu
 
# Copy files from /usr/share/maven/ref into ${MAVEN_CONFIG}
# So the initial ~/.m2 is set with expected content.
# Don't override, as this is just a reference setup
 
copy_reference_files() {
  local log="$MAVEN_CONFIG/copy_reference_file.log"
  local ref="/usr/share/maven/ref"
 
  if mkdir -p "${MAVEN_CONFIG}/repository" && touch "${log}" > /dev/null 2>&1 ; then
      cd "${ref}"
      local reflink=""
      if cp --help 2>&1 | grep -q reflink ; then
          reflink="--reflink=auto"
      fi
      if [ -n "$(find "${MAVEN_CONFIG}/repository" -maxdepth 0 -type d -empty 2>/dev/null)" ] ; then
          # destination is empty...
          echo "--- Copying all files to ${MAVEN_CONFIG} at $(date)" >> "${log}"
          cp -rv ${reflink} . "${MAVEN_CONFIG}" >> "${log}"
      else
          # destination is non-empty, copy file-by-file
          echo "--- Copying individual files to ${MAVEN_CONFIG} at $(date)" >> "${log}"
          find . -type f -exec sh -eu -c '
              log="${1}"
              shift
              reflink="${1}"
              shift
              for f in "$@" ; do
                  if [ ! -e "${MAVEN_CONFIG}/${f}" ] || [ -e "${f}.override" ] ; then
                      mkdir -p "${MAVEN_CONFIG}/$(dirname "${f}")"
                      cp -rv ${reflink} "${f}" "${MAVEN_CONFIG}/${f}" >> "${log}"
                  fi
              done
          ' _ "${log}" "${reflink}" {} +
      fi
      echo >> "${log}"
  else
    echo "Can not write to ${log}. Wrong volume permissions? Carrying on ..."
  fi
}
 
# Wait until we see the test files that are being copied from the PHP/Apache/Codebase container to a shared volume.
while [ ! -d /var/automated_tests/mytests ]
do
  sleep 2
done
 
owd="$(pwd)"
copy_reference_files
unset MAVEN_CONFIG
 
cd "${owd}"
unset owd
 
exec mvn -f /var/automated_tests/mytests -q clean install

Define the Fargate Task

I now have all parts I need apart from a shell script to actually define the Fargate task which will contain all the config for the containers. I have made extensive use of the heredoc convention in the shell script - one per container definition - which makes it easier to read, and also means when I come to make up other tasks (such as static analysis, and persistent playground) I can simple swap out those containers I don't need.

There are a few comments here to note.

You cannot create a Fargate Task without the IAM ecsTaskExecutionRole, but that will not be created for you. It is however created when you manually create your first Fargate task in the AWS console. Of course ecsTaskExecutionRole can be created programmatically too - and will be the subject of a block at another time.
The Fargate task definition will not create your CloudFront log group for you! Therefore I do a check in the script to see if it already exists, and if not then create.
I have defined a mount point to where I copy the test suite. This is because the test suite is in the code repository which is copied into the PHP/Apache container, but is needed by the Maven container also.
The Selenium Chrome browser is using a non-standard port 5556. This is because it would clash with Firefox otherwise which also defaults to port 5555. You must remember to use an environmental variable NODE_PORT to change the port as well as the hostPort and containerPort.
I have defaulted the task's compute to extremely high cpu and memory. They should be reduced using trial and error to avoid incurring high compute fees.

#!/bin/bash
 
# Rewrite branch name so no feature/ hotfix/ forward slash
BRANCH="selenium-testing"
REPO="d8codebase"
 
# Repo account and repo
ECR_ACCOUNT="XXXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com"
ECR_REPO="${ECR_ACCOUNT}/${REPO,,}"
ECS_CLUSTER="default"
 
# Get the Task Role ARN for executing the task
TASK_ARN=`aws iam get-role --role-name ecsTaskExecutionRole | jq .Role.Arn -r`
 
 
# Execution ARN appears to be the same as the Task ARN
EXECUTION_ARN="${TASK_ARN}"
 
 
# Determine if the log group already exists: NOTE: "aws ecs register-task-definition" will not create the log group for you!
LOG_EXISTS=`aws logs describe-log-groups | jq ".logGroups[] | select(.logGroupName == \"/ecs/"${BRANCH}"\") | .logGroupName" -r | wc -l`
if [[ "${LOG_EXISTS}" -eq 0 ]]; then
	aws logs create-log-group --log-group-name "/ecs/${BRANCH}"
fi
 
CONTAINER_CODEBASE=$(cat <
)
 
CONTAINER_SELENIUM_HUB=$(cat <
)
 
CONTAINER_SELENIUM_FIREFOX=$(cat <
)
 
CONTAINER_SELENIUM_CHROME=$(cat <
)
 
CONTAINER_MAVEN=$(cat <
)
 
 
CONTAINERS_AUTOMATIC=$(cat <
)
 
 
aws ecs register-task-definition \
  --family "AUTOMATION-${BRANCH}" \
  --volumes "name=automated_tests,host={}" \
  --task-role-arn "$TASK_ARN" \
  --execution-role-arn "${EXECUTION_ARN}" \
  --network-mode "awsvpc" \
  --container-definitions "${CONTAINERS_AUTOMATIC}" \
  --cpu "4096" \
  --memory "30720" \
  --requires-compatibilities "FARGATE"

Running the Fargate Task

Once the script is executed, the task will be defined and ready to run. This can be run by creating a new shell script (or extending the one above) or by using the console. Since this blog is becoming too code focused, I'll document the running of tasks programmatically in another blog later. For now you should be able to see your task in the console and run it. See screenshot 1 above.

Screenshot 2 show that the task has reached RUNNING state. If you are quick you can click on the task identifier and be taken to the more detailed screen shown in image 3 which shows all the containers that are running.

Once one of the Docker images completes its activity, a signal will be sent to all the other containers to stop. In the task described in this blog, the first task to complete is the running of the automated tests. Once the tests complete, then all other containers will stop. See image 4.

Did the Tests Actually Succeed?

A valid question! The output of the Maven container is written to a CloudWatch log stream which can be parsed and echoed back to the screen, or sent to a Slack channel (more on this in a later blog). For now I have created a sequence of commands using pipes which includes the open source awslogs utility.

$ aws logs describe-log-streams --log-group-name "/ecs/selenium-testing" --log-stream-name-prefix "ecs/Maven" | jq '.logStreams|=sort_by(-.creationTime)|.logStreams[0].logeamName' -r | xargs -I{} awslogs get "/ecs/selenium-testing" {} -s1d | cut -d ' ' -f 1,2 --complement
mvn
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running mytests.RunCucumberTest
Dec 10, 2019 3:43:46 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Dec 10, 2019 3:43:47 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Dec 10, 2019 3:43:49 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Dec 10, 2019 3:43:50 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Feature: Are we on the Home Page?
  Scenario: On the home page         # mytests/home_page.feature:4
    When User is on the Home Page    # Stepdefs.user_is_on_Home_Page()
    Then Message Displayed Memcached # Stepdefs.message_displayed_memcached()
    When User is on the Home Page    # Stepdefs.user_is_on_Home_Page()
    Then Message Displayed GD        # Stepdefs.gd()
1 Scenarios (1 passed)
4 Steps (4 passed)
0m6.835s
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.055 sec
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Troubleshooting Selenium Grid

Getting the Selenium stack working can be fiddly, although the CloudWatch logs are a very good way of diagnosing problems. If you want to satisy yourself that Selenium Hub and its two nodes (Firefox and Chrome) are actually there and available, there is a way but you'll have to work fast on the console since the Fargate tasks completes and shuts down once the tests are complete.

When the Fargate task transitions from PENDING to RUNNING, click on the task identifier. Copy the public IP address into your clipboard - you'll need it shortly. Now click on the ENI (Elastic Network Interface) link which will launch a new tab. Then click on the security group. Now you need to create a new rule. By default port 4444 is blocked on the public IP address - and to see Selenium working I need that to be open.

Edit the rules, create a new custom TCP/IP rule, add 4444 as the port and select anywhere for the CIDR, then save. See screenshot 1 above.

You now instantly have an open port 4444. Open a new browser tab, then paste the IP address you copied a few steps ago, then append /grid/console to it and enter. You should see your Grid + Node configuration as per the second screenshot above.

blog terms

devops ECS AWS AWS CLI Development Docker bash

Programmatically Create AWS ECR Repository and Commit Docker Image to ECR

nigel — Sat, 07 Dec 2019 07:56:15 +0000

Programmatically Create AWS ECR Repository and Commit Docker Image to ECR nigel Sat, 07/12/2019 - 07:56

This is a tutorial for using AWS CLI to programmatically create an AWS ECR repository and then commit a Docker image to that repository. This is the first step in a series of tutorials for creating an automated testing framework in AWS Fargate which can be one step of a DevOps pipeline.

As an example I am going to use the Docker php:7.2-apache image and to make it a little more complex, add a few more PHP extensions using a new Dockerfile so that my new image will be Drupal 8 compliant.

Let's get started.

Pull Base Image and Define Dockerfile

The base php:7.2-apache image needs to be pulled from Docker Hub. Issue the following command.

$ docker pull php:7.2-apache
7.2-apache: Pulling from library/php
000eee12ec04: Pull complete 
8ae4f9fcfeea: Pull complete 
60f22fbbd07a: Pull complete 
ccc7a63ad75f: Pull complete 
a2427b8dd6e7: Pull complete 
91cac3b30184: Pull complete 
d6e40015fc10: Pull complete 
240e21c03bb4: Pull complete 
504858e1e4aa: Pull complete 
9a0523b2d73f: Pull complete 
e3acb84829f4: Pull complete 
28a372733f87: Pull complete 
62ee66cdb80a: Pull complete 
c3f368ddc7aa: Pull complete 
Digest: sha256:602bef9acc8d54527ae5b7ee0f16e5cdb6ab81364310f5123b93b95e357f608d
Status: Downloaded newer image for php:7.2-apache
docker.io/library/php:7.2-apache
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
php                 7.2-apache          fec8a1aaffb8        31 hours ago        410MB
lambci/lambda       go1.x               8d962cde7e27        3 months ago        707MB
sls-docker          latest              8d962cde7e27        3 months ago        707MB

Yes the docker image is there. Now I need to create a directory for the Dockerfile and a sample index.php file which will prove our image is working.

$ mkdir automated && cd automated

By default the Docker php image uses /var/www/html as Apache's docroot. However, I'm going to alter this to something more standard for Drupal web apps - a directory called docroot will be created and will contain a simple index.php containing a call to phpinfo();

$ mkdir docroot && echo " > docroot/index.php

The Dockerfile should now be created and populated. There a number of activities that need to be performed on it.

Reference the base image with a FROM instruction
Install the libraries that support memcached and graphics manipulation
Enable memcached PHP extension
Configure the graphics PHP extension
Install the graphics PHP extension
Copy the files in the current directory and below into the Docker image at the /var/www/html ensuring the permissions are correct for the apache user and group
Set the environment variable AH_SITE_ENVIRONMENT to devops - this is an Acquia Cloud environment variable and only of any significance for those using Acquia as their host - or need a setting in my case to denote that devops is not running on Acquia
Change the default location for the docroot - that means setting an environment variable and using the sed editor to change the configuration in the sites files

Dockerfile

FROM php:7.2-apache
RUN apt-get update && apt-get install -y libmemcached-dev zlib1g-dev \
    libfreetype6-dev \
    libjpeg62-turbo-dev \
    libpng-dev \
    libwebp-dev \
    && pecl install memcached \
    && docker-php-ext-enable memcached \
    && docker-php-ext-configure gd --with-gd --with-webp-dir --with-jpeg-dir \
       --with-png-dir --with-zlib-dir --with-freetype-dir \
    && docker-php-ext-install gd
COPY --chown=www-data:www-data ./ /var/www/html/.
ENV AH_SITE_ENVIRONMENT devops
ENV APACHE_DOCUMENT_ROOT /var/www/html/docroot
RUN sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf
RUN sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf

Build the Image, Run the Container

Once the Dockerfile is complete, it is built with

$ docker build -t d8codebase .

and to confirm it's built correctly:

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
d8codebase          latest              c2193f9778a6        33 minutes ago      439MB
<none>              <none>              2a71964685e0        2 hours ago         432MB
php                 7.2-apache          fec8a1aaffb8        35 hours ago        410MB
lambci/lambda       go1.x               8d962cde7e27        3 months ago        707MB
sls-docker          latest              8d962cde7e27        3 months ago        707MB

Great it's there. Now to spin up the container.

$ docker run -d -p 8080:80 d8codebase

Note I am mapping port 8080 to the container's port 80 which probably won't be pertinent to you. Long story I will try to make short. I write my blogs on my Macbook on a local network in the 192.168 range assigned by my ISP's hub. But I develop and test all the practical work in my blogs in a VM in the IP 10. range. Routing between 192.168 and 10. is difficult without port forwarding - so I port forward 127.0.0.1 8080 to my VM's 10.0.2.15 8080, so when I run my Docker Apache container I must remember incoming is on port 8080 and map that against the container's port 80. Hope that makes sense!

Now check it's running:

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES
9a6dc2fb88ff        d8codebase          "docker-php-entrypoi…"   44 minutes ago      Up 44 minutes       0.0.0.0:8080->80/tcp   agitated_feistel

Yes, and we can see from the screenshot above that it's picking up the memcached and the gd graphics extensions. Success.

Programmatically Create the AWS ECR Repository

This is quite easy and I've put it into a shell script that requires one parameter - the repo name to be created. The work is done by the aws ecr create-repository call. The rest of the code is simply checking for error conditions. The screenshot above was taken on the AWS console after the shell script below was successfully run with a parameter of d8codebase.

#!/bin/bash
 
#Programmatically create an ECR repo
 
# Check whether we have the args
if [[ -z $@ ]]; then
	echo "usage: [repo_name]"
	exit 1
fi
 
# Create the repo
REPO_NAME=`aws ecr create-repository --repository-name "$1" | jq .repository.repositoryName -r`
 
# Check it was created ok. We already get a quality stderr msg from aws cli so just quit on error.
# Similarly, if already exists it will tell us
if [[ "${REPO_NAME}" != $1 ]]; then
	exit 2
fi
 
echo "Successfully created ${REPO_NAME}"

Now the script to automatically login to Docker and push the image to ECR. Note that AWS ECR does require you to use the docker login command, and therefore provides a AWS CLI ECR command get-login which returns a token than can be piped into the docker login command. The token retrieved from ECR has a large payload, and it is top and tailed with other metadata beyond just the password. Therefore that needs to be removed, and if you look closely at the script below, you can see that I pipe the entire returned payload through sed with a regex, and then the result is piped onwards to docker login. Hopefully the rest is quite self explanatory - and you can see the screenshot above that the push has worked and the image can now be included in ECS / Fargate task definitions.

#!/bin/bash
 
#Programmatically push to an ECR repo
 
# Check whether we have the args
if [[ -z $@ ]]; then
	echo "usage: [repo_name]"
	exit 1
fi
 
REPO_NAME=$1
ECR_ACCOUNT="XXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com"
ECR_REPO="${ECR_ACCOUNT}/$1"
 
# AWS Authentication - since we need to use docker push
# The sed regex will top and tail the additional meta data sent by the aws login request.
aws ecr get-login --region eu-west-2  | sed -e 's/^.*-p \(.*\)\s\-\e.*$/\1/' |  docker login --password-stdin -u AWS ${ECR_ACCOUNT}
 
# Get the image id of the Docker build. If there is a "latest" use that else get the first without latest
TAG=`docker images | grep -w "${REPO_NAME}" | grep latest | awk '{ print $3; }'`
if [[ -z ${TAG} ]]; then
        TAG=`docker images | grep -w "${REPO_NAME}" | awk '{ print $3; }'`
fi
 
docker tag "$TAG" "${ECR_REPO}"
docker push "${ECR_REPO}"

blog terms

AWS AWS CLI Docker devops Fargate ECS Development bash

Create Arbitrary Subdomains for AWS Fargate Tasks using AWS CLI

nigel — Fri, 01 Nov 2019 15:03:02 +0000

Create Arbitrary Subdomains for AWS Fargate Tasks using AWS CLI nigel Fri, 01/11/2019 - 15:03

AWS Fargate is a relatively new product offering the capability of running Docker bundles as an ECS compute service on EC2 without the necessity for the user to have to orchestrate the underlying EC2 infrastructure. It is perfect for DevOps, and was the technology I chose as DevOps Architect at my current client. The client wanted standard CI/CD capabilities such as spinning up containerised copies of their website environment per feature branch to perform static analysis, automated testing, and also significantly, a persisting playground that QA or development personnel could use for manual testing and troubleshooting a feature (or hotfix) branch they are currently working on.

This is similar to the offering of both Platform.sh and Amazee Labs hosting solutions - developers can spin up their feature branches. Alas my client is locked into another host yet needed on-the-fly feature branch Docker bundles.

AWS Fargate tasks can achieve this, but out of the box it isn't possible to create public domain registration via ECS Service Discovery. The Fargate task is issued a bare IPv4 address which will obviously work but our requirement is to have a subdomain created per branch that would be of the format {hotfix|feature}--{jira ticket}-{suitable branch name}.{top level domain}. So an example would be:

feature--doi-746-my-whizzy-new-feature.clientdomain.dev

So whilst we can't create a subdomain like that natively, we can do it by retrieving the IP address from the ECS Fargate task, then add that IP to a public Route 53 Hosted Zone. Furthermore, since the Fargate task is run via a Jenkins pipeline job that uses shell scripts populated with AWS CLI, the good news is we can indeed complete the registration of subdomain also using the AWS CLI *and* have the DNS propagate within 60 seconds! Amazing! Let's dive into how we achieve this.

Requirements + Assumptions

The solution will be built using the standard httpd Apache2 server sample Fargate task described on the AWS blog Tutorial: Creating a Cluster with a Fargate Task Using the AWS CLI. I added the task definition into the console, although it is also possible create a shell script and use AWS CLI. I have used the default cluster for brevity's sake, and the AWS account default VPC. I have also transferred a spare domain I had, saasidate.com, into Route 53 before I started the blog. This will be my tld and all the subdomains will be added off it. It's important that the ownership of the domain is transferred into Route 53 since otherwise subdomains will have authority issues if the parent domain is still registered to other domain providers such as GoDaddy or 123-Reg.

The solution I am putting together will use AWS CLI which by default returns JSON structured data. It is absolutely essential to use the Linux command line utility jq for parsing the JSON. Documentation on jq is readily available on the net.

Steps

The solution will need the following steps:

Get the user's default VPC (or create a custom VPC if preferred. We'll need this at a few points in the code.
Get a subnet from the VPC. Three subnets are created for use with the default VPC. I pick the first which is arbitrary.
Get a security group. A new security group is created per task when using the console but that is unnecessary for the shell scripts we will be building. So in our script, check whether our 'standard' security group has been created. If so use it, if not then create a security group with an ingress port of 80 and a CIDR of 0.0.0.0/0 which allows access worldwide on TCP port 80.
Run the Fargate task and loop / sleep until the task has a status of RUNNING.
Get the network interface id of the task and interrogate it for the IP address.
Get the hosted zone for the parent domain.
Add a record set containing the feature branch subdomain to the parent domain's hosted zone.
Create a new hosted zone using the feature branch.

Ok - let's crack on and put this into AWS CLI shell scripts.

Get the VPC

I'm using the default VPC so I can filter the list I get back from AWS using the IsDefault attribute.

VPC=`aws ec2 describe-vpcs | jq '.Vpcs[] | select(.IsDefault == true) | .VpcId' -r`

Note the use of the jq select statement. This is a familiar construct and will be used throughout this blog. Also note the -r flag - this removes the double quotes from the response.

Get the First Subnet in the Default VPC

Here I am using jq again but piping the results through head to get the first subnet. There wasn't an obvious jq query to do this natively within jq.

# Get the first subnet of the VPC
SUBNET=`aws ec2 describe-subnets | jq --arg VPC "$VPC" '.Subnets[] | select(.VpcId == $VPC) | .SubnetId' -r | head -n 1`

Get the Security Group

Here I check whether we already have a dedicated security group which gives us worldwide access to port 80. I search first for a unique security group description and if it doesn't exist, then create it.

# Check if our security group for port 80 webserver already exists. Search for our funky unique description
SECURITY=`aws ec2 describe-security-groups | jq ' .SecurityGroups[] | select (.Description == "Ingress Port 80 Anywhere") | .GroupId' -r`
 
# Let's inspect what we got back.
if [[ $? -ne 0 ]]; then
	echo "describe-security-groups failed with code $?"
	exit $?
fi
 
# If we didn't get a security group then create
if [[ -z "$SECURITY" ]]; then
	SECURITY=`aws ec2 create-security-group \
		--description "Ingress Port 80 Anywhere" \
		--group-name "Fargate Webserver Port 80" \
		--vpc-id "$VPC" | jq .GroupId -r `
 
 
	aws ec2 authorize-security-group-ingress \
		--group-id "$SECURITY" \
		--ip-permissions "FromPort=80,ToPort=80,IpProtocol=TCP,IpRanges=[{CidrIp=0.0.0.0/0}]"
fi

Run the Fargate Task

Next I run the Fargate task. This is the sample Fargate task covered in the AWS documentation and uses the httpd docker image. When I created it in the console I called it "first-run-task-definition" - for the life of me I can't remember why I chose such a crazy name! Note I am piping the result through jq as per normal and then onwards through sed to get the task id without all the arn information that precedes it. The sed command looks for the final forward slash in the task arn and crops anything before it.

# Run the task and parse the task ARN for polling later to determine its status
TASK=`aws ecs run-task \
	--cluster "default" \
	--task-definition "first-run-task-definition" \
	--network-configuration "awsvpcConfiguration={subnets=[${SUBNET}],securityGroups=[${SECURITY}],assignPublicIp=ENABLED}" \
	--launch-type FARGATE  | jq .tasks[0].taskArn -r | sed "s/.*\///" `
 
# Let's inspect what we got back. There should be a task identifier.
if [[ $? -ne 0 ]]; then
	echo "task-run failed with code $?"
	exit $?
fi
 
if [[ -z "$TASK" ]]; then
	echo "task-run could not create task"
	exit 1
fi

Loop and Wait for Fargate Task to get to RUNNING Status

Next I have to wait for the task to acquire RUNNING status, However I can't wait for ever, and thus I have wrapped a shell script rule into a timeout mechanism. I have allowed for 5 minutes for the task to get to RUNNING status and that should be more than enough.

# Loop around until we see a RUNNING status or timeout after 10 x 30 second sleeps
for i in 1 2 3 4 5 6 7 8 9 10 11
do
	if [[ $i -eq 11 ]]; then
		echo "Timed out before could establish task is running"
		exit 1
	fi
 
	STATUS=`aws ecs describe-tasks --cluster="default" --tasks $TASK  | jq .tasks[0].lastStatus -r`
 
	# Did it exit unexpectedly?
	if [[ $? -ne 0 ]]; then
		echo "describe-tasks failed with code $?"
		exit $?
	fi
 
	if [[ "$STATUS" = "RUNNING"  ]]; then
		break
	fi
 
	sleep 30
done

Obtain the Network Interface Id

The ENI is available in the Fargate task's description so I needed to parse this on my quest to get the public IP.

# Get the network Interface id
ENI=`aws ecs describe-tasks --cluster="default" --tasks $TASK  | jq '.tasks[0].attachments[0].details[] | select(.name == "networkInterfaceId") | .value' -r `
 
# Let's inspect what we got back. There should be a network interface id.
if [[ $? -ne 0 ]]; then
	echo "describe-tasks failed with code $?"
	exit $?
fi
 
if [[ -z "$ENI" ]]; then
	echo "describe-tasks could not establish eni"
	exit 1
fi

Get the Public IP Address

With the ENI it is possible to parse the the descriptions of all the network interfaces for the public IP address.

PUBLIC_IP=`aws ec2 describe-network-interfaces | jq --arg ENI "$ENI" '.NetworkInterfaces[] | select(.NetworkInterfaceId == $ENI) | .PrivateIpAddresses[0].Association.PublicIp' -r`
 
# Did we get an IP address?
if [[ $? -ne 0 ]]; then
	echo "describe-network-interfaces failed with code $?"
	exit $?
fi
 
if [[ -z "$PUBLIC_IP" ]]; then
	echo "describe-network-interfaces could not retrieve public IP address"
	exit 1
fi

Get the Parent Domain's Hosted Zone Id

The shell script needs to know about the parent domain and the subdomain name (which in my case is a feature branch name) - so those need to be retrieved from runtime arguments. This will go at the top of the shell script.

if [[ -z $1 ]] || [[ -z $2 ]]; then
	echo "usage: parent_domain_name feature_branch_name"
	exit 1
fi
 
BRANCH=$2.$1
PARENT=$1.

Then continuing at the end of the codebase, the following gets the parent domain

PARENT_ZONE=`aws route53 list-hosted-zones-by-name | jq --arg PARENT "$PARENT" '.HostedZones[] | select(.Name == $PARENT) | .Id' -r | sed "s/.*\///"`

Add the Feature Branch Name and Public IP Address to the Parent Hosted Zone

Now I created a new record set for the parent domain's hosted zone. This uses the AWS CLI command change-resource-record-sets which allows for insert, upsert and delete. I needed a DNS A record adding. The easiest way of providing the runtime parameters is via a JSON structure and that is created in shell script heredoc format to simplify shell script escaping.

BATCH=$(cat <
)
 
 
# Use the hosted zone of the tld
aws route53 change-resource-record-sets \
	--hosted-zone-id $PARENT_ZONE \
	--change-batch "$BATCH"

Create the Feature Branch Hosted Zone

The final AWS CLI call is to use create-hosted-zone, and here I needed to provide the feature branch name and an arbitrary unique caller reference for which I used a timestamp.

timestamp=$(date +%s)
 
# Now create a new hosted zone of the feature branch
aws route53 create-hosted-zone \
	--name "$BRANCH" \
	--caller-reference "$timestamp"

The Complete Script

The complete script is listed below. Note that for brevity and easy reading I have cut a few coding standards. There is repetition in the error condition checking. This should be rewritten with calls to shell script functions in a separate file.

#!/bin/bash
 
# Script to assign a public IP address issued by a Fargate task to a subdomain
 
if [[ -z $1 ]] || [[ -z $2 ]]; then
	echo "usage: parent_domain_name feature_branch_name"
	exit 1
fi
 
BRANCH=$2.$1
PARENT=$1.
 
 
# Get the default VPC.
VPC=`aws ec2 describe-vpcs | jq '.Vpcs[] | select(.IsDefault == true) | .VpcId' -r`
 
# Get the first subnet of the VPC
SUBNET=`aws ec2 describe-subnets | jq --arg VPC "$VPC" '.Subnets[] | select(.VpcId == $VPC) | .SubnetId' -r | head -n 1`
 
# Check if our security group for port 80 webserver already exists. Search for our funky unique description
SECURITY=`aws ec2 describe-security-groups | jq ' .SecurityGroups[] | select (.Description == "Ingress Port 80 Anywhere") | .GroupId' -r`
 
# Let's inspect what we got back.
if [[ $? -ne 0 ]]; then
	echo "describe-security-groups failed with code $?"
	exit $?
fi
 
# If we didn't get a security group then create
if [[ "$SECURITY" = "" ]]; then
	SECURITY=`aws ec2 create-security-group \
		--description "Ingress Port 80 Anywhere" \
		--group-name "Fargate Webserver Port 80" \
		--vpc-id "$VPC" | jq .GroupId -r `
 
 
	aws ec2 authorize-security-group-ingress \
		--group-id "$SECURITY" \
		--ip-permissions "FromPort=80,ToPort=80,IpProtocol=TCP,IpRanges=[{CidrIp=0.0.0.0/0}]"
fi
 
# Run the task and parse the task ARN for polling later to determine its status
TASK=`aws ecs run-task \
	--cluster "default" \
	--task-definition "first-run-task-definition" \
	--network-configuration "awsvpcConfiguration={subnets=[${SUBNET}],securityGroups=[${SECURITY}],assignPublicIp=ENABLED}" \
	--launch-type FARGATE  | jq .tasks[0].taskArn -r | sed "s/.*\///" `
 
# Let's inspect what we got back. There should be a task identifier.
if [[ $? -ne 0 ]]; then
	echo "task-run failed with code $?"
	exit $?
fi
 
if [[ -z "$TASK" ]]; then
	echo "task-run could not create task"
	exit 1
fi
 
# Loop around until we see a RUNNING status or timeout after 10 x 30 second sleeps
for i in 1 2 3 4 5 6 7 8 9 10 11
do
	if [[ $i -eq 11 ]]; then
		echo "Timed out before could establish task is running"
		exit 1
	fi
 
	STATUS=`aws ecs describe-tasks --cluster="default" --tasks $TASK  | jq .tasks[0].lastStatus -r`
 
	# Did it exit unexpectedly?
	if [[ $? -ne 0 ]]; then
		echo "describe-tasks failed with code $?"
		exit $?
	fi
 
	if [[ "$STATUS" = "RUNNING"  ]]; then
		break
	fi
 
	sleep 30
done
 
 
# Get the network Interface id
ENI=`aws ecs describe-tasks --cluster="default" --tasks $TASK  | jq '.tasks[0].attachments[0].details[] | select(.name == "networkInterfaceId") | .value' -r `
 
# Let's inspect what we got back. There should be a network interface id.
if [[ $? -ne 0 ]]; then
	echo "describe-tasks failed with code $?"
	exit $?
fi
 
if [[ -z "$ENI" ]]; then
	echo "describe-tasks could not establish eni"
	exit 1
fi
 
 
# Get the Public IP Address
PUBLIC_IP=`aws ec2 describe-network-interfaces | jq --arg ENI "$ENI" '.NetworkInterfaces[] | select(.NetworkInterfaceId == $ENI) | .PrivateIpAddresses[0].Association.PublicIp' -r`
 
# Did we get an IP address?
if [[ $? -ne 0 ]]; then
	echo "describe-network-interfaces failed with code $?"
	exit $?
fi
 
if [[ -z "$PUBLIC_IP" ]]; then
	echo "describe-network-interfaces could not retrieve public IP address"
	exit 1
fi
 
BATCH=$(cat <
)
 
PARENT_ZONE=`aws route53 list-hosted-zones-by-name | jq --arg PARENT "$PARENT" '.HostedZones[] | select(.Name == $PARENT) | .Id' -r | sed "s/.*\///"`
 
aws route53 change-resource-record-sets \
	--hosted-zone-id $PARENT_ZONE \
	--change-batch "$BATCH"
 
 
timestamp=$(date +%s)
 
# Now create a new hosted zone of the feature branch
aws route53 create-hosted-zone \
	--name "$BRANCH" \
	--caller-reference "$timestamp"

Invoking the Script

The script requires two parameters as previous discussed. Here's my example for the blog:

$ ./feature-branch.sh saasidate.com feature--doi-746-my-whizzy-new-feature

Currently the output of the script is the response from the AWS CLI route53 create-hosted-zone call, and here you can detect the full domain name.

Checking the Outcome

Obviously the first check is to copy and paste the url of the feature branch into a browser and see if it loads. An example of this is the blog heading image at the top of the page.

Also the AWS console is the place to check everything is as required. Navigate to ECS -> default -> Tasks and you should see a screenshot similar to the first one immediately above. This is the task we invoked from inside our shell script. Now navigate to Route 53 -> Hosted Zones -> {domain name} and you can see the feature branch and its IP address.

Steps left to do

New hosted zones will normally propagate in 60 seconds in Route 53 - but it would be good if the script above polls Route 53 every 10 seconds or so and reports back the subdomain url and the IP address once the propagation has completed. This should be surfaced on the command line as a minimum, but perhaps a notification on Slack would be even better. Both options are trivial.

Whilst the idea is that the Fargate tasks should persist, they shouldn't be around forever. Therefore there needs to be clear up activities to remove the feature subdomain from the parent domain's hosted zone, and the subdomain hosted zone. Also there obviously needs a step to stop the Fargate task.

Caveats

There are caveats to this solution - but for my use case, it's one of those rare occurrences in life that the caveats don't apply to me.

Firstly, this solution will only work with a Fargate task and not an ECS service running a Fargate task. Tasks should be run as services for production environments since services give you great things like replication, and health percentages against number of running tasks. There is therefore no scaling, no load balancing, no DDOS protection in what I'm offering here.

Fargate tasks run in isolation are perfect for spun up short living environments such as playgrounds for devs and QAs to run manual tests against or troubleshoot development issues - which is exactly my use case.

blog terms

AWS AWS CLI Docker devops Fargate ECS Development Route 53 bash