Home

Introduction

The Datashades ckan-aws-templates provide a set of templates that allow you to provision a scaleable HA CKAN stack in the AWS Cloud using the AWS CloudFormation and OpsWorks management tools. The README.md file within the repo provides a technical overview of the templates. This wiki will provide more of conceptual overview of the templates and what they produce.

Overview

The Documents directory contains an architecture diagram and draw.io xml source file of the diagram.

The OpsWorks stack created by these CloudFormation templates contains the following key components:

• A GFS Layer that provisions the Gluster File System for HA NFS file store.
• An Apache ZooKeeper layer to support SolrCloud
• An Apache Solr layer to provide SolrCloud
• A Web layer that provisions and runs the CKAN application.

The OpsWorks stack also contains the following Application definitions:

• Zookeeper
• Solr
• CKAN

The CKAN Extensions template creates further Application definitions that facilitate the deployment of CKAN extensions. Primarily these simply express what CKAN extensions should be deployed by the CKAN Extensions recipe, and the source URL of those extensions. To add a new extension, add a new Application, prefix the name with ckanext- and provide the source URL. A specialized section will cover extensions more thoroughly. For the time being be aware that the extensions recipe may need modification to cater for extensions that have "special needs". An example of this is where the extension name differs from the plugins line entry. Another is where extensions need extra OS packages to be installed etc.

To get a meaningful stack running you need the Datashades OpsWorks Cookbook. As of the public preview release the default Cookbook values point to an internal Github Enterprise repository for the cookbook. This will be changed to the Public repo.

A key concept to understand about this build is the use of Route53 to deal with the dynamic nature of hosts. Link Digital use an internal domain to create stack subdomains. IE: platform.links.com.au. During provisioning we provide platform.links.com.au as the TLD, and the stack will create hosted zone record sets like devzk1.platform.links.com.au. If a host goes offline or is stopped and started, these records are deleted and recreated.

Provisioning the templates, running the stack, and examining the cookbook recipes will provide great insights into what is done. The code has been kept as streamlined and simple as possible with reasonable commenting to assist making sense of it.

This wiki will evolve over time to provide more useful documentation and insight into the workings of the stack. Hopefully initial drafts are sufficient to get people started.