Using Apache Brooklyn on Stackato

 At ActiveState, we realize applications often require data services to enhance the user experience. That is why Stackato includes a set of system services like MySQL, PostgreSQL, RabbitMQ, and more. But there are times when services beyond those bundled are needed. This is where service brokers serve their purpose. This post demonstrates using the Brooklyn Service Broker to make one of Apache Brooklyn's broad range of blueprints available to Stackato as a service.

Also of note, a proposal has been made to bring the Apache Brooklyn service broker (and CLI-Plugin) into the Cloud Foundry Incubator. So be sure to keep a close watch as this project could soon play an impactful role in Cloud Foundry's services domain.

Apache Brooklyn

Apache Brooklyn is a framework for provisioning, managing, and monitoring applications. Applications are deployed using blueprints, which allow for one-click deployments through the GUI or REST API to both cloud and non-cloud solutions.

Brooklyn Blueprints

A Brooklyn blueprint is a YAML document written with declarative syntax and provides support for JVM plugins. A blueprint describes, among other things, where to deploy the application, and which services the application will interact with. Blueprints can be composed of either a single process or combination of processes. These processes can be anything from a simple web-application server running a WAR file to a cluster of J2EE application servers connected to a cluster of SQL database servers. For a reference of YAML blueprints refer to here.

Catalogs

Brooklyn offers a catalog, formed by a persistent set of versioned blueprints. Items from the catalog can be deployed directly or used in other blueprints. A catalog is defined using the same YAML structure as a blueprint with an additional brooklyn.catalog key. Blueprint descriptions for catalog related fields can be found here. Brooklyn's catalog starts with an extensive list of items and can be extended by adding your own custom blueprints to the catalog.

Service Brokers

A service broker is the bridge that connects an external service to a Cloud Foundry based environment, Stackato in this case. Once connected, the service becomes available for use just like a system service. Normally, a service that needs to be made available to a Cloud Foundry based ecosystem will require a custom service broker implementation of its own. With the combination of Brooklyn's design and the capabilities of the Brooklyn Service Broker, the full range of Brooklyn's blueprints can be made available without requiring a custom service broker for each service.

Stackato Integration

There are certain requirements that need to be downloaded and/or installed in order to have these Apache Brooklyn enabled services built and available to Stackato.

• Java Development Kit 8
• Gradle 2.2 or newer
• Maven
• Apache Brooklyn
• Brooklyn Service Broker
• An Amazon Web Services (AWS) account - this is optional, but this example will deploy a blueprint application to AWS

Installing JDK 8

To install JDK 8 follow these instructions. Installation using these instructions will not set or update the JAVA_HOME environment variable to reference the JDK 8 install, which will be required when building the service broker using Gradle. To set the JAVA_HOME environment variable append the following to ~/.bash_profile:

export JAVA_HOME=/path/to/jdk8

Alternatively, point to the JDK install directory directly without using the JAVA_HOME environment variable. This is done by creating a gradle.properties file in the service broker's root directory containing:

org.gradle.java.home=/path/to/jdk8

Installing Gradle

Follow these steps to install Gradle, which will be used to build the service broker project.

Installing and Running Apache Brooklyn

When downloading Apache Brooklyn, retrieve the files from Github instead of from the mirror sites as instructed here. As noted here, this is necessary because the Brooklyn Service Broker has a dependency on Brooklyn's client library, which is available only in the snapshot version from Github. During Maven's install phase the necessary components to run a Apache Brooklyn server will be built and the client library dependency needed to build the broker will be installed to the local Maven repository. After the project is built, the Brooklyn server can be started. In this example, the Brooklyn server will be running on localhost and the Brooklyn blueprint application (the service for Stackato) will be deployed to AWS.

To specify access credentials for AWS, create a brooklyn.properties file with the properties below as the content. This will create a named location, where the name of the location will also serve as the plan name of the service in Stackato. In Brooklyn, locations are used used to indicate where applications will be deployed and include choices of localhost, fixed IP address servers, and various clouds.

// Create a named location called aws-oregon
brooklyn.location.named.aws-oregon:us-west-2
brooklyn.location.named.aws-oregon.identity = AWS_ACCESS_KEY
brooklyn.location.named.aws-oregon.credential = AWS_SECRET_KEY

The project can now be built and the Brooklyn server started.

cd ~/example
git clone https://github.com/apache/incubator-brooklyn
cd incubator-brooklyn

// Build the project, this step will take a bit of time, even when test are skipped,
// so now would be a good time to grab a coffee, or tea if that's your fancy
mvn clean install -DskipTests

// Start the Brooklyn server
usage/dist/target/brooklyn-dist/bin/brooklyn launch --localBrooklynProperties path/to/brooklyn.properties

The Brooklyn server should now be accessible at localhost:8081. For the example, a Redis server will be added to the catalog to use with our application. To add a Redis server to the catalog, access the Brooklyn server in a web browser. A dialog box will appear, choose the "Catalog" tab , select the "Add to Catalog" button, pick "Entity" and then paste in the following YAML into the text area.

brooklyn.catalog:
  id: redis
  version: 1.0
  iconUrl: classpath://redis.png
  description: Redis is an open-source, networked, in-memory, key-value data store with optional durability
name: Redis

services:
- type: brooklyn.entity.basic.BasicApplication
  brooklyn.children:
  - type: brooklyn.entity.nosql.redis.RedisStore
        id: redis
        name: Redis Server

Installing and Running the Brooklyn Service Broker

Next, download, configure, and deploy the Brooklyn Service Broker. The service broker will also be deployed to the localhost.

cd ~/example
git clone https://github.com/cloudfoundry-community/brooklyn-service-broker
cd brooklyn-service-broker

There are some properties that must be defined within the application.properties file before we can run the broker.

brooklyn.uri=127.0.0.1:8081
brooklyn.username=brooklyn-username
security.user.name=simple-user
security.user.password=simple-password

The brooklyn.username property is required to start the broker, while security.user.name and security.user.password are used to authenticate to the broker.

// Start the broker, available at localhost:8080
gradle clean bootRun

// Verify the broker is working. A JSON response containing information about
// the Redis server added earlier will be returned
// Enter simple-password when prompted for a password
curl -u simple-user localhost:8080/v2/catalog

Add Apache Brooklyn Services to Stackato

Add the service broker to Stackato, which will expose the various catalog blueprints from Brooklyn. A service called "Redis" should now be visible when in the list of services available to Stackato. By default, all service plans are private so they must be set to public to allow organizations to use them.

// Add the broker to Stackato
stackato create-service-broker --username simple-user --password simple-password --url http://192.168.68.29:8080 brooklyn-service-broker

// Verify example catalog 'Redis' is available
stackato services

// Make service plan public, plan name = aws-oregon
stackato update-service-plan --public aws-oregon --vendor Redis

The service can now be bound to apps. In Stackato some services are assigned a *_URL environment variable containing the uri connection string to the service. For example, services such as a MySQL database or a RabbitMQ messaging queue will have a MYSQL_URL and RABBITMQ_URL environment variable respectively. The convenience of such an environment variable is not available to services exposed through a service broker. Instead, the VCAP_SERVICES environment variable will need to be parsed to retrieve the credentials to establish a connection to the service.

When stackato creates the service, Brooklyn will provision an AWS Elastic Computer Cloud instance with Redis in the the Oregon region (defined earlier using the brooklyn.properties file). Spinning up the instance will take a couple of minutes before it is ready.

In this example the service will be bound to a deployed instance of the Python Currency Converter app.

// Deploy the Currency Converter app
git clone https://github.com/Stackato-Apps/bottle-currency
cd ~/bottle-currency
git checkout brooklyn-example
stackato push -n --no-start

// Create and bind the Brooklyn-enabled Redis Server to the Currency Converter app
stackato create-service Redis --plan aws-oregon bottle-currency-db bottle-currency

We can also use redis-cli to connect directly to our Redis server to verify that things are working.

// Connect to the Redis server
redis-cli -h host.address -p redis.port

// Connection is established
> keys *
1) "CADUSD"
> get CADUSD
"0.79475461951123"

Below is a snippet of the wsgi.py file from our Currency Converter example app adapted for the Brooklyn enabled Redis server

vcap_services = json.loads(os.getenv('VCAP_SERVICES'))
...
credentials = vcap_services['Redis'][0]['credentials']['Redis Server']
host = credentials['host.address']
port = credentials['redis.port']
rdb = Redis(host=host, port=port)

Conclusion

Curious about the capabilities of Apache Brooklyn on Stackato? Download the Stackato Micro Cloud or sign up for a 20GB license and try for yourself.

Information in this blog was based on "Integrating Cloud Foundry with Apache Brooklyn Part 1: Service Broker" by Robert Moss.