Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

How Much Is the Cloud? In JSON, Please!

DZone's Guide to

How Much Is the Cloud? In JSON, Please!

Learn how you can pull JSON data to learn more about how much the cloud is costing you.

· Cloud Zone ·
Free Resource

Insight into the right steps to take for migrating workloads to public cloud and successfully reducing cost as a result. Read the Guide.

Pricing calculators for using Cloud services are great for roughly estimating monthly costs. But you might feel the need for something a little bit more tailored to your specific usage pattern. The great news is that cloud providers usually provide pricing data by means of an API. The bad news is that pricing APIs are far from being unified. This post summarizes the pricing APIs of Microsoft Azure, Amazon Web Services, and Google Cloud Platform.

While we're at it: for tinkering with APIs, before going into the actual coding, I am using Postman. It's a convenient tool that also allows you to fetch OAuth tokens, save requests and responses and all kinds of other neat things.

In case you're looking for an easy solution to visualize JSON data (other than syntax highlighted... well... JSON strings): I recommend http://json.bloople.net/.

Just copy/paste your JSON data there and you'll have it verified and displayed in a foldable, hierarchical view. If you're working with different tools you'd like to recommend, please add them to the comments.

Amazon Web Services

The AWS Price List API is quite simple to use. There's a dedicated index endpoint that serves as an entry page to the APIs specific to the offerings:

 https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/index.json 

  "formatVersion" : "v1.0",
  "disclaimer" : "This pricing list is for informational purposes only. All prices are subject to the additional terms included in the pricing pages on http://aws.amazon.com. All Free Tier prices are also subject to the terms included at https://aws.amazon.com/free/",
  "publicationDate" : "2016-11-05T00:27:24Z",
  "offers" : {
    "AmazonS3" : {
      "offerCode" : "AmazonS3",
      "versionIndexUrl" : "/offers/v1.0/aws/AmazonS3/index.json",
      "currentVersionUrl" : "/offers/v1.0/aws/AmazonS3/current/index.json"
    },
    "AmazonGlacier" : {
      "offerCode" : "AmazonGlacier",
      "versionIndexUrl" : "/offers/v1.0/aws/AmazonGlacier/index.json",
      "currentVersionUrl" : "/offers/v1.0/aws/AmazonGlacier/current/index.json"
    }
  }
}

Note that the Price List API is always hosted on https://pricing.us-east-1.amazonaws.com regardless of the region your infrastructure is hosted in.

The response hosts two URLs for each offering: one is linking to another list of URLs for querying outdated price lists and the other one for directly retrieving the current version.

{
  "formatVersion" : "v1.0",
  "disclaimer" : "This pricing list is for informational purposes only. All prices are subject to the additional terms included in the pricing pages on http://aws.amazon.com. All Free Tier prices are also subject to the terms included at https://aws.amazon.com/free/",
  "publicationDate" : "2016-10-15T00:01:53Z",
  "offerCode" : "AmazonS3",
  "currentVersion" : "20161015000153",
  "versions" : {
    "20151209181126" : {
      "versionEffectiveBeginDate" : "2015-10-01T00:00:00Z",
      "versionEffectiveEndDate" : "2016-01-01T00:00:00Z",
      "offerVersionUrl" : "/offers/v1.0/aws/AmazonS3/20151209181126/index.json"
    },
    "20161015000153" : {
      "versionEffectiveBeginDate" : "2016-09-01T00:00:00Z",
      "versionEffectiveEndDate" : "",
      "offerVersionUrl" : "/offers/v1.0/aws/AmazonS3/20161015000153/index.json"
    }
  }
}

The links in this response and the currentVersionUrl property from the previous code block - finally — shows the data you'd expect from an API like this.

{
  "formatVersion": "v1.0",
  "disclaimer": "This pricing list is for informational purposes only. All prices are subject to the additional terms included in the pricing pages on http://aws.amazon.com. All Free Tier prices are also subject to the terms included at https://aws.amazon.com/free/",
  "offerCode": "AmazonS3",
  "version": "20160107021538",
  "publicationDate": "2016-01-07T02:15:38Z",
  "products": {
    "FAJN8MSJS6UUHTEX": {
      "sku": "FAJN8MSJS6UUHTEX",
      "productFamily": "Data Transfer",
      "attributes": {
        "servicecode": "AWSDataTransfer",
        "transferType": "InterRegion Inbound",
        "fromLocation": "EU (Frankfurt)",
        "fromLocationType": "AWS Region",
        "toLocation": "Asia Pacific (Tokyo)",
        "toLocationType": "AWS Region",
        "usagetype": "EUC1-APN1-AWS-In-Bytes",
        "operation": ""
      }
    },
    "C3T7NJAY35EWRYKY": {
      "sku": "C3T7NJAY35EWRYKY",
      "productFamily": "Fee",
      "attributes": {
        "servicecode": "AmazonS3",
        "location": "Asia Pacific (Sydney)",
        "locationType": "AWS Region",
        "feeCode": "S3-Storage-SIA-SmObject",
        "feeDescription": "Fee for storing small objects (<128KB) in SIA Storage",
        "usagetype": "APS2-TimedStorage-SIA-SmObjects",
        "operation": ""
      }
    }
  },
    "terms": {
      "OnDemand": {
        "FAJN8MSJS6UUHTEX": {
          "FAJN8MSJS6UUHTEX.JRTCKXETXF": {
            "offerTermCode": "JRTCKXETXF",
            "sku": "FAJN8MSJS6UUHTEX",
            "effectiveDate": "2016-01-01T00:00:00Z",
            "priceDimensions": {
              "FAJN8MSJS6UUHTEX.JRTCKXETXF.6YS6EN2CT7": {
                "rateCode": "FAJN8MSJS6UUHTEX.JRTCKXETXF.6YS6EN2CT7",
                "description": "$0.00 per GB - EUC1 (FRA) data transfer from Asia Pacific (Tokyo)",
                "beginRange": "0",
                "endRange": "Inf",
                "unit": "GB",
                "pricePerUnit": {
                  "USD": "0.0000000000"
                },
                "appliesTo": [ ]
              }
            },
            "termAttributes": { }
          }
        },
        "C3T7NJAY35EWRYKY": {
          "C3T7NJAY35EWRYKY.JRTCKXETXF": {
            "offerTermCode": "JRTCKXETXF",
            "sku": "C3T7NJAY35EWRYKY",
            "effectiveDate": "2016-01-01T00:00:00Z",
            "priceDimensions": {
              "C3T7NJAY35EWRYKY.JRTCKXETXF.6YS6EN2CT7": {
                "rateCode": "C3T7NJAY35EWRYKY.JRTCKXETXF.6YS6EN2CT7",
                "description": "$0.019 per GB-Month of storage used for small objects",
                "beginRange": "0",
                "endRange": "Inf",
                "unit": "GB-Mo",
                "pricePerUnit": {
                  "USD": "0.0190000000"
                },
                "appliesTo": [ ]
              }
            },
            "termAttributes": { }
          }
        }
      }
    }
  }

The actual data consists of meta-information about the products (sub-nodes of "products") and the actual price per unit (sub-nodes of "terms"). In practical terms, this means that you identify the according SKU inside the "products" objects and then look for that inside the "terms" nodes.

One thing I didn't mention yet is that instead of .json, you usually can use .csv as an alternate extension, which provides the result in csv format. So, pick whichever flavor you prefer.

This data can grow huge (when speaking in API terms). One time I got a 69 MB JSON response, which crashed my Postman instance. Responses of that size not only trouble that poor, little app, they are also quite likely to cause lots of load on your side. So, think about caching them. The dates in the response of the index-call should make it easy for you to check for new versions without doing all the heavy-lifting of the full set of data.

For further information, please refer here for the original source.

Google Cloud Platform

Google's pricing API is as simple as it gets. It's a single, static JSON file that holds all the available data.

https://cloudpricingcalculator.appspot.com/static/data/pricelist.json 

It holds next to no metadata, which is why it is only 4kB in size.

{
  "comment": "If you've gotten here by mistake, this is the JSON data used by our pricing calculator. It is helpful for developers. Go to https://cloud.google.com/products/calculator/ to get back to our web calculator.",
  "version": "v1.10",
  "updated": "30-November-2016",
  "gcp_price_list": {
    "sustained_use_base": 0.25,
    "sustained_use_tiers": {
      "0.25": 1.0,
      "0.50": 0.8,
      "0.75": 0.6,
      "1.0": 0.4
    },
    "CP-COMPUTEENGINE-VMIMAGE-F1-MICRO": {
      "us": 0.008,
      "europe": 0.009,
      "asia": 0.009,
      "asia-east": 0.009,
      "asia-northeast": 0.010,
      "cores": "shared",
      "memory": "0.6",
      "gceu": "Shared CPU, not guaranteed",
      "maxNumberOfPd": 4,
      "maxPdSize": 3,
      "ssd": [0]
    },
    "CP-COMPUTEENGINE-VMIMAGE-G1-SMALL": {
      "us": 0.027,
      "europe": 0.030,
      "asia": 0.030,
      "asia-east": 0.030,
      "asia-northeast": 0.035,
      "cores": "shared",
      "memory": "1.7",
      "gceu": 1.38,
      "maxNumberOfPd": 4,
      "maxPdSize": 3,
      "ssd": [0]
    }
}

It's really a mixture of instance information, like the number of CPU cores or available memory, and the price per region. If you'd like to have some more information about how to use this data or whether there's some structure to it, check the link that's mentioned in the comment on top.

Turns out that this is actually the data that's used by Google's own Pricing Calculator, which comes with a nice UI and that adds all the information that's missing from the plain API data.

Microsoft Azure

Azure's Pricing API is different to the previous ones, in many ways. First: there's two of them.

  • Resource Usage: This API allows you to get consumption data for an Azure subscription

  • Resource RateCard:  Use this API to get meter (aka resource) metadata information along with prices.

Second: Azure requires you to do authenticated API calls instead of just plain GET requests. So, make sure you read through https://msdn.microsoft.com/en-us/library/azure/dn790557   before you jump in.

Why? According to the documentation, it is because this is "A REST API that customers and partners can use [...]". So, the information here is not intended for everyone, but just to customers and partners.

Third: the API is still listed as being in preview state. I'll show you what this means in a minute.

While there are two APIs available, we'll only concentrate on Resource RateCard API, as that's the one that resembles the previous two the most.

Ok, let's get going. I have taken the following API call from the documentation:

https://management.azure.com/subscriptions/

{subscription-Id}/providers/Microsoft.Commerce/

RateCard?api-version=2015-06-01-preview&$filter=OfferDurableId

eq ’MS-AZR-0003p’ and Currency eq ’USD’ and Locale eq ’en-US’

and RegionInfo eq ’US’

There are a couple of mandatory parameters:

  • subscription-Id

  • api-version

  • filter (including OfferDurableId, Currency, Locale and RegionInfo)

For one, this is the most complicated API to use among the three mentioned in this post. For another, it also seems to be the most sophisticated one, with extensive filtering possibilities available.

Subscription-Id and api-version are no-brainers. They are what they are and you can easily find information about them in the documentation.

OfferDurableId is different: you are required to look up the code at this human-readable page (ie non-json)  http://azure.microsoft.com/en-us/support/legal/offer-details .

And this is what I was talking about when I mentioned that the API is still in preview.

Back to the data, this is what you can expect as a response:

{
   "OfferTerms": [],
   "Meters": [
      {
         "MeterId": "1822fcc4-6059-4cbb-a132-54a187aaac46",
         "MeterName": "Compute Hours",
         "MeterCategory": "Virtual Machines",
         "MeterSubCategory": "Basic_D6 VM (Non-Windows)",
         "Unit": "Hours",
         "MeterTags": [],
         "MeterRates": {
            "0": 3.136
         },
         "EffectiveDate": "2015-02-01T00:00:00Z",
         "IncludedQuantity": 0.0
      },
      {
         "MeterId": "3c5324ad-eb8c-44c6-af9a-6741ae75fc90",
         "MeterName": "Data Transfer Out at 500 Mbps (GB)",
         "MeterCategory": "Networking",
         "MeterSubCategory": "ExpressRoute (IXP)",
         "Unit": "GB",
         "MeterTags": [],
         "MeterRates": {
            "0": 0.1
         },
         "EffectiveDate": "2014-08-01T00:00:00Z",
         "IncludedQuantity": 2048.0
      },

      {
         "MeterId": "9ee077eb-c902-46ef-b7f9-2caeade852e0",
         "MeterName": "Compute Hours",
         "MeterCategory": "Cloud Services",
         "MeterSubCategory": "A6 Cloud Services",
         "Unit": "Hours",
         "MeterTags": [],
         "MeterRates": {
           "0": 0.71
         },
         "EffectiveDate": "2013-12-01T00:00:00Z",
         "IncludedQuantity": 0.0
      },
   …
   ]
   "Currency": "USD",
   "Locale": "en-US",
   "IsTaxIncluded": false,
   "MeterRegion": "US West",
   "Tags": []
}

From a conceptual perspective, this shows more similarities with the AWS API than it does with the bare-bones approach of the Google API. As request parameters and response tend to be quite verbose, Microsoft thankfully provides quite extensive information in the documentation: https://msdn.microsoft.com/en-us/library/azure/mt219004

Summary

At first sight, the AWS API seems to be the easiest of the three to work with. On the other hand, it lacks the necessary filtering to work with it on-the-fly, as responses grow too large. Google pretty much provides what it says: the underlying data of their pricing calculator web app. No more, no less. According to what you're expecting to do, this might be good for you, or not. Microsoft's approach is definitely the most complex to get running. It took me a while to get through authentication and to get my first response.And while this was a real tedious thing to do, I think it's the approach that delivers the most useful data, at least once it's final. Manually looking up codes on an HTML page is definitely something that requires improvement.

Where's the Cloud Pricing Comparison Tool?

The folks over at Cloudorado do a pretty good job in comparing multiple cloud vendors. As they put it theirselves, it's hard to do decent comparison, due to the very different nature of what things are part of billing and how they are part of billing. For some, a year consists of 365 days, for others it's only 360, just to name an example. And I completely agree with that. While it is great that Cloud vendors provide an API for pricing estimation and calculation, the very different nature of the offerings themselves make it very hard to compare them.

How do you estimate cloud costs? Did you use information like that before moving your services to the Cloud? So, to sum up: it's great to have that data available. However, I'm still trying to find a convenient way of leveraging the power that's behind that for sure.

TrueSight Cloud Cost Control provides visibility and control over multi-cloud costs including AWS, Azure, Google Cloud, and others.

Topics:
api ,azure ,aws ,cloud ,google cloud ,cloud cost estimation

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}