At ActiveState, we realize applications often require data services to enhance the user experience. That is why Stackato includes a set of system services like MySQL, PostgreSQL, RabbitMQ, and more. But there are times when services beyond those bundled are needed. This is where service brokers serve their purpose. This post demonstrates using the Brooklyn Service Broker to make one of Apache Brooklyn's broad range of blueprints available to Stackato as a service.
Also of note, a proposal has been made to bring the Apache Brooklyn service broker (and CLI-Plugin) into the Cloud Foundry Incubator. So be sure to keep a close watch as this project could soon play an impactful role in Cloud Foundry's services domain.
Apache Brooklyn is a framework for provisioning, managing, and monitoring applications. Applications are deployed using blueprints, which allow for one-click deployments through the GUI or REST API to both cloud and non-cloud solutions.
A Brooklyn blueprint is a YAML document written with declarative syntax and provides support for JVM plugins. A blueprint describes, among other things, where to deploy the application, and which services the application will interact with. Blueprints can be composed of either a single process or combination of processes. These processes can be anything from a simple web-application server running a WAR file to a cluster of J2EE application servers connected to a cluster of SQL database servers. For a reference of YAML blueprints refer to here.
Brooklyn offers a catalog, formed by a persistent set of versioned blueprints. Items from the catalog can be deployed directly or used in other blueprints. A catalog is defined using the same YAML structure as a blueprint with an additional brooklyn.catalog key. Blueprint descriptions for catalog related fields can be found here. Brooklyn's catalog starts with an extensive list of items and can be extended by adding your own custom blueprints to the catalog.
A service broker is the bridge that connects an external service to a Cloud Foundry based environment, Stackato in this case. Once connected, the service becomes available for use just like a system service. Normally, a service that needs to be made available to a Cloud Foundry based ecosystem will require a custom service broker implementation of its own. With the combination of Brooklyn's design and the capabilities of the Brooklyn Service Broker, the full range of Brooklyn's blueprints can be made available without requiring a custom service broker for each service.
There are certain requirements that need to be downloaded and/or installed in order to have these Apache Brooklyn enabled services built and available to Stackato.
- Java Development Kit 8
- Gradle 2.2 or newer
- Apache Brooklyn
- Brooklyn Service Broker
- An Amazon Web Services (AWS) account - this is optional, but this example will deploy a blueprint application to AWS
Installing JDK 8
To install JDK 8 follow these instructions. Installation using these instructions will not set or update the JAVA_HOME environment variable to reference the JDK 8 install, which will be required when building the service broker using Gradle. To set the JAVA_HOME environment variable append the following to ~/.bash_profile:
Alternatively, point to the JDK install directory directly without using the JAVA_HOME environment variable. This is done by creating a gradle.properties file in the service broker's root directory containing:
Follow these steps to install Gradle, which will be used to build the service broker project.
Installing and Running Apache Brooklyn
When downloading Apache Brooklyn, retrieve the files from Github instead of from the mirror sites as instructed here. As noted here, this is necessary because the Brooklyn Service Broker has a dependency on Brooklyn's client library, which is available only in the snapshot version from Github. During Maven's install phase the necessary components to run a Apache Brooklyn server will be built and the client library dependency needed to build the broker will be installed to the local Maven repository. After the project is built, the Brooklyn server can be started. In this example, the Brooklyn server will be running on localhost and the Brooklyn blueprint application (the service for Stackato) will be deployed to AWS.
To specify access credentials for AWS, create a brooklyn.properties file with the properties below as the content. This will create a named location, where the name of the location will also serve as the plan name of the service in Stackato. In Brooklyn, locations are used used to indicate where applications will be deployed and include choices of localhost, fixed IP address servers, and various clouds.
// Create a named location called aws-oregon brooklyn.location.named.aws-oregon:us-west-2 brooklyn.location.named.aws-oregon.identity = AWS_ACCESS_KEY brooklyn.location.named.aws-oregon.credential = AWS_SECRET_KEY
The project can now be built and the Brooklyn server started.
cd ~/example git clone https://github.com/apache/incubator-brooklyn cd incubator-brooklyn // Build the project, this step will take a bit of time, even when test are skipped, // so now would be a good time to grab a coffee, or tea if that's your fancy mvn clean install -DskipTests // Start the Brooklyn server usage/dist/target/brooklyn-dist/bin/brooklyn launch --localBrooklynProperties path/to/brooklyn.properties
The Brooklyn server should now be accessible at localhost:8081. For the example, a Redis server will be added to the catalog to use with our application. To add a Redis server to the catalog, access the Brooklyn server in a web browser. A dialog box will appear, choose the "Catalog" tab , select the "Add to Catalog" button, pick "Entity" and then paste in the following YAML into the text area.
brooklyn.catalog: id: redis version: 1.0 iconUrl: classpath://redis.png description: Redis is an open-source, networked, in-memory, key-value data store with optional durability name: Redis services: - type: brooklyn.entity.basic.BasicApplication brooklyn.children: - type: brooklyn.entity.nosql.redis.RedisStore id: redis name: Redis Server
Installing and Running the Brooklyn Service Broker
Next, download, configure, and deploy the Brooklyn Service Broker. The service broker will also be deployed to the localhost.
cd ~/example git clone https://github.com/cloudfoundry-community/brooklyn-service-broker cd brooklyn-service-broker
There are some properties that must be defined within the application.properties file before we can run the broker.
brooklyn.uri=127.0.0.1:8081 brooklyn.username=brooklyn-username security.user.name=simple-user security.user.password=simple-password
The brooklyn.username property is required to start the broker, while security.user.name and security.user.password are used to authenticate to the broker.
// Start the broker, available at localhost:8080 gradle clean bootRun // Verify the broker is working. A JSON response containing information about // the Redis server added earlier will be returned // Enter simple-password when prompted for a password curl -u simple-user localhost:8080/v2/catalog
Add Apache Brooklyn Services to Stackato
Add the service broker to Stackato, which will expose the various catalog blueprints from Brooklyn. A service called "Redis" should now be visible when in the list of services available to Stackato. By default, all service plans are private so they must be set to public to allow organizations to use them.
// Add the broker to Stackato stackato create-service-broker --username simple-user --password simple-password --url http://192.168.68.29:8080 brooklyn-service-broker // Verify example catalog 'Redis' is available stackato services // Make service plan public, plan name = aws-oregon stackato update-service-plan --public aws-oregon --vendor Redis
The service can now be bound to apps. In Stackato some services are assigned a *_URL environment variable containing the uri connection string to the service. For example, services such as a MySQL database or a RabbitMQ messaging queue will have a MYSQL_URL and RABBITMQ_URL environment variable respectively. The convenience of such an environment variable is not available to services exposed through a service broker. Instead, the VCAP_SERVICES environment variable will need to be parsed to retrieve the credentials to establish a connection to the service.
When stackato creates the service, Brooklyn will provision an AWS Elastic Computer Cloud instance with Redis in the the Oregon region (defined earlier using the brooklyn.properties file). Spinning up the instance will take a couple of minutes before it is ready.
In this example the service will be bound to a deployed instance of the Python Currency Converter app.
// Deploy the Currency Converter app git clone https://github.com/Stackato-Apps/bottle-currency cd ~/bottle-currency git checkout brooklyn-example stackato push -n --no-start // Create and bind the Brooklyn-enabled Redis Server to the Currency Converter app stackato create-service Redis --plan aws-oregon bottle-currency-db bottle-currency
We can also use redis-cli to connect directly to our Redis server to verify that things are working.
// Connect to the Redis server redis-cli -h host.address -p redis.port // Connection is established > keys * 1) "CADUSD" > get CADUSD "0.79475461951123"
Below is a snippet of the wsgi.py file from our Currency Converter example app adapted for the Brooklyn enabled Redis server
vcap_services = json.loads(os.getenv('VCAP_SERVICES')) ... credentials = vcap_services['Redis']['credentials']['Redis Server'] host = credentials['host.address'] port = credentials['redis.port'] rdb = Redis(host=host, port=port)
Curious about the capabilities of Apache Brooklyn on Stackato? Download the Stackato Micro Cloud or sign up for a 20GB license and try for yourself.
Information in this blog was based on "Integrating Cloud Foundry with Apache Brooklyn Part 1: Service Broker" by Robert Moss.