DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. Deployment Challenges and Solutions For PCF Platform

Deployment Challenges and Solutions For PCF Platform

Want to learn more about the PCF platform and potential challenges? Check out this post where we explore its solutions and other microservices concepts.

Rajesh Bhojwani user avatar by
Rajesh Bhojwani
·
Nov. 07, 18 · Tutorial
Like (3)
Save
Tweet
Share
7.40K Views

Join the DZone community and get the full member experience.

Join For Free

Many users have started PCF as a platform to host their applications and microservices. The more you use the platform and uncover the issues and challenges. So today, I am going to talk about microservices deployment issues on the PCF platform.

I have been working with many teams who started breaking their monolith applications and converting them into microservices. I have a team that handles approximately 20+ microservices now. And, they need to deploy these microservices to multiple environments as part of the SDLC, e.g. Dev, System, Perf, QA, Prod, etc. Now, for deploying a microservice to PCF, you need to use the manifest.yml file. And, it is recommended to separate each manifest file for each environment. So, for these 20+ microservices, they need to maintain 20*5 =100+ manifest files. Can you imagine how hard it would be to handle this many files? One manual mistake can cause so much damage. Let's first talk about why you need a separate manifest.yml file for each environment.

  1. For each environment, your microservice would require different configurations of the resources. For example, in development, you would only need two instances of the app with 512 GB of memory. But in production, you might need four instances with 1 GB of memory.

  2. Each environment might be binding to a backing service with a different plan. For development, you might need only a basic/free plan of the DB service. But in production, you would need to advance the plan with more resources so your service might be different. You might want to ensure that you are putting the name of the service instance accordingly so that the team understands what is free and what is chargeable. 

  3. There might be few env variables that you would like to setup while deploying the application and that may vary based on environment, for example, spring and spring.profiles.active= dev

There might be many more reasons, as well. But let's go back to how to resolve the number of growing manifest files.

PCF used to have a feature called Inheritance for a manifest file where you can create a parent manifest YML file and inherit the common properties from there. But, this has been deprecated and, anyhow, that doesn't resolve the number of manifest files you create; however, it reduces the content of each file significantly.

Inheritance Is Replaced by Variable Substitution

This feature helps to create a template manifest file, put the placeholders for variables, and replace the values dynamically from an external file.

  1. Let's start with a simple example. Here is a sample manifest file for a GO app.

//manifest.yml
---
applications:
- name: sample-app
  instances: ((noofinstances))
  memory: ((memory))
  env:
    GOPACKAGENAME: go_calls_ruby
  command: go_calls_ruby

 

2. This template file has put a placeholder for the variables at lines 5 and 6. Let's create another file data.yml, which is going to have those variable values.

//data.yml
noofinstances: 2
memory: 1G


3.  Now, push the application to PCF using the cf push command and use the --vars-file argument to pass the data.yml file.

cf push -f ~/workspace/manifest.yml --vars-file ~/workspace/data.yml


Values of 'noofinstances' and 'memory' from the data.yml file will be replaced in a manifest file. This shows us how to use the variable substitution feature. Now, we can use this solution to replace multiple manifest files into one.

Step 1

First, we need to create a Template manifest file:

//manifest.yml
---
applications:
- name: sample-app
  instances: ((noofinstances))
  memory: ((memory))
  services:
    - mysql
    - newrelic
  env:
    spring.profile.active: ((env))


Step 2

Next, we will create a data.yml file, which will have data for each environment. I would recommend keeping only all non-prod environment data in one file. For production, you should always have a separate manifest.yml and data.yml file.

//dev env
dev_noofinstances: 2
dev_memory: 512M
dev_env: dev

//system env
system_noofinstances: 3
system_memory: 1G
system_env: system

//perf env
perf_noofinstances: 4
perf_memory: 2G
perf_env: perf

//qa env
qa_noofinstances: 3
qa_memory: 1G
qa_env: qa


Step 3

In a CI/CD Pipeline, like Bamboo and Jenkins, write a script. Most PCF deployments are configured through the pipeline only. This script will take the environment value as the input per pipeline standards, and based on the env value like 'dev,' 'system,' 'perf,' and 'qa,' it will read this data.yml file and retrieve all the values of the related environment. Please notice that the data.yml file variables have an env value as the prefix — we need to follow that standard. The script would create a temporary file <env>_data.yml file.

For example:

//dev_data.yml
noofinstances:2
memory: 512M
env: dev
//system_data.yml
noofinstances:3
memory: 1G
env: system


Step 4

Now, as per earlier steps, just push the application to PCF using the cf push command and pass the data file dynamically through the pipeline for each environment.

cf push -f manifest.yml --vars-file=<env>_data.yml


This solution has replaced four manifest files to 1 manifest file. You can even go one step further and club all related microservices' manifest data into one data.yml file, and it can reduce the count further. But please note, keep the production data.yml file separate to avoid any manual mistakes.

That's all for this blog! To understand more about PCF and microservices concepts, check out these additional blog posts!

Manifest file Data file microservice

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Mr. Over, the Engineer [Comic]
  • Real-Time Stream Processing With Hazelcast and StreamNative
  • Distributed Stateful Edge Platforms
  • 2023 Software Testing Trends: A Look Ahead at the Industry's Future

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: