DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. Using Stubs for the AWS SDK for Ruby

Using Stubs for the AWS SDK for Ruby

Code that makes use of external dependencies can be difficult to test. If you are using AWS, then you have no doubt run into issues in testing your code. This article takes you through how to get around this little hurdle by using AWS Stubs if you are using Ruby.

Jeff Dugas user avatar by
Jeff Dugas
·
Apr. 18, 16 · Tutorial
Like (2)
Save
Tweet
Share
5.52K Views

Join the DZone community and get the full member experience.

Join For Free

If you plan to write, or have already written, Ruby code that leverages Amazon Web Services, you should consider using AWS Stubs. The ability to stub service responses allows you to develop robust code more quickly and enables you to test your code without incurring any cost. Amazon’s SDK engineers knew that as well, and have provided the capability since the Ruby SDK’s v1 release.

This post dives into two ways of using the AWS SDK for Ruby to stub responses from AWS clients. The first approach will utilize the global Aws.config, providing a set of stub responses for the classes and methods you specify. The second approach will combine AWS stubs with classic mocking, giving your test code more fine-grained control over client stubs and stub responses.

Both approaches have this in common: stubbed AWS clients/methods do not require AWS keys or credentials, and in fact don’t access the internet at all. Their behavior is designed to allow you to stub as much or as little of the AWS ecosystem as you need to.

Choosing a Stubbing Mechanism

The simpler of these two stubbing mechanisms is to use the global Aws.config. This snippet shows how to stub the list_buckets method for all S3 client instances. This approach works great for picking and choosing which combination of classes/methods to stub, and it can be overridden for different cases. All non-stubbed class instances and methods will continue invoking HTTP requests to AWS (i.e., be live).

Aws.config[:s3] = {
  stub_responses: {
    list_buckets: {}
  }
}

The other approach supports stubbing all methods for a given client instance. This is done by constructing that client using the stub_responses argument.

s3_client = Aws::S3::Client.new(stub_responses: true)

Regardless of the mechanism, the real benefit comes from stubbing the responses in such a way that best tests your code’s logic.

Knowing What to Expect

Code that relies on API response objects need to know what data, if any, is returned from those API calls. While the Ruby SDK documentation is the most authoritative source, AWS client stubs provide a stub_data that returns the top level of the specified operation’s response topology.

stubbed_s3_client.stub_data(operation_name: ‘list_buckets’)
#<struct Aws::S3::Types::ListBucketsOutput buckets=[], owner=#<struct Aws::S3::Types::Owner display_name="DisplayName", id="ID">>

This highlights an important fact; the API responses are Ruby structs. This is significant because the stubbed response data you provide is mapped into response objects; unlike a Ruby hash, which can be extended by adding additional key/value pairs, structs are immutable with respect to their keys.

This snippet shows how to provide a stubbed response (and how not to):

# this hash will be mapped into the ListBucketsOutput struct
Aws.config[:s3] = {
  stub_responses: {
    list_buckets: { buckets: [ { name: ‘your_bucket’ } ] } 
}
Aws::S3::Client.new.list_buckets
#<struct Aws::S3::Types::ListBucketsOutput buckets=[#<struct Aws::S3::Types::Bucket name="your_bucket", creation_date=nil>], owner=nil>

# a similar hash, with a typo, will not be
Aws.config[:s3] = {
  stub_responses: {
    list_buckets: { bucketss: [ { name: ‘your_bucket’ } ] } 
}
Aws::S3::Client.new.list_buckets
ArgumentError: unexpected value at params[:bucketss]

# you can also use the native structs to stub responses
Aws.config[:s3] = {
  stub_responses: {
    list_buckets: { buckets: [ Aws::S3::Types::Bucket.new(name: ‘her_bucket’) ] } 
}
Aws::S3::Client.new.list_buckets
#<struct Aws::S3::Types::ListBucketsOutput buckets=[#<struct Aws::S3::Types::Bucket name="her_bucket", creation_date=nil>], owner=nil>

Putting Stubs to Use

Unit testing often involves substituting consumable APIs with mock APIs that your code can safely interact with. Using Aws.config, instead of creating your own mock classes that mimic the hundreds of AWS service methods available, you’re substituting the behavior of those clients without having to provide mocks.

This RSpec snippet assumes we have a class instance (my_instance) and are testing its get_all_buckets method. The get_all_buckets code internally creates an S3 client and invokes the client’s list_buckets method to return an array of buckets. Here we’re verifying that our method returns a bucket array as expected.

describe '#get_all_buckets' do
  when 'stubbing s3’s “list_buckets” method'
    it 'returns an array of bucket' do
      # create the stub response data:
      #   always respond with an object that contains the attribute 'buckets', where
      #   ‘buckets’ is an array of objects, each containing the attribute 'name'
      buckets_as_hashes = [ { name: 'your-bucket' }, { name: 'her-bucket' } ]
      Aws.config[:s3] = {
        stub_responses: {
          list_buckets: { buckets: buckets_as_hashes }
        }
      }

      # compare what we stub to what we expect
      expect(custom_instance.get_all_buckets.first.name)
        .to eql(buckets_as_hashes.first[:name])
    end
  end
end

A More Complex Case

Application code is seldom limited to being just a wrapper around existing API calls. This code snippet demonstrates a common API feature: paginated responses. Here the code must make multiple list_stack_resources API calls to assemble the return value when a CloudFormation stack has generated more than 100 resources.

Let’s start with the application code:

def retrieve_resources(stack_name:, region: ‘us-west-2’)
  cfn_client = Aws::CloudFormation::Client.new(region: region)
  resource_summaries = Array.new
  next_token = nil
  loop do
    if next_token.nil?
      options = { stack_name: stack_name }
    else
      options = { stack_name: stack_name, next_token: next_token }
    end
    response = cfn_client.list_stack_resources(options)
    resource_summaries += response.stack_resource_summaries
    break if response.next_token.nil?
    next_token = response.next_token
  end
  resource_summaries
end

In order to test the logic in our code, we can use a combination of mocking and stubs to exercise ‘retrieve_resources’. First, since we know that the CloudFormation client’s list_stack_resources method returns an array of StackResourceSummary objects, let’s provide a helper method to generate as many StackResourceSummary objects as we like.

def stubbed_stack_summaries(how_many: 1)
  stub_resource_summaries = []
  (1..how_many).each { |i|
    stub_resource_summaries +=  [
      Aws::CloudFormation::Types::StackResourceSummary.new(
        last_updated_timestamp: Time.new,
        logical_resource_id: "resourceid#{i}",
        physical_resource_id: "resourcename#{i}",
        resource_status: 'CREATE_COMPLETE',
        resource_status_reason: nil,
        resource_type: "Aws::Type::#{i}"
      )
    ]
  }
  stub_resource_summaries
end

Below, our RSpec code exercises the retrieve_resources method of our_class_instance that contains that method. Notice we’re stubbing multiple calls to the same method, and that those stubbed responses will be returned in the order listed, allowing us to simulate the looping condition.

The first call returns a struct with a non-null next_token value, triggering a second call that fetches the remaining 25 stack resources.

describe '#retrieve_resources and test paging logic' do
  # create a stubbed client instance (mock the one in our code), and stub data
  let(:stub_client) { Aws::CloudFormation::Client.new(stub_responses: true) }
  let(:stub_resource_summaries) { stubbed_stack_summaries(how_many: 125) }
  context 'when retrieving >100 resources for a large stack' do
    it 'makes multiple calls using next_token, and assembles the data internally' do
      # intercept the AWS client’s constructor and substitute our stub client
      expect(Aws::CloudFormation::Client).to receive(:new).and_return(stub_client)

      # mimic the condition that triggers > 1 call to the 'list_stack_resources' API:
      #   all ‘live’ calls will return a maximum of 100 StackResourceSummary objects
      stub_client.stub_responses(:list_stack_resources,
        Aws::CloudFormation::Types::ListStackResourcesOutput.new(
          stack_resource_summaries: stub_resource_summaries[0..99],
          next_token: 'there-are-more'
        ),
        Aws::CloudFormation::Types::ListStackResourcesOutput.new(
          stack_resource_summaries: stub_resource_summaries[100..124],
          next_token: nil
        )
      )

      # We expect retrieve_resources will call the API twice in order to return
      #   an array containing all the stubbed stack_resource_summaries
      expect(our_class_instance.retrieve_resources(stack_name: ‘anything’))
        .to eql(stub_resource_summaries)
    end
  end
end

Conclusion

Using AWS stubs is straightforward and allows your unit testing to cover more of your code in a more meaningful way. There are a few limitations to keep in mind, however:

  • Client stubs do not respond differently to different arguments. Generally you can get around this limitation by predicting the order in which any AWS methods will occur and write your stubs accordingly. Helper methods go a long way toward creating stub data and allow you to create a stubbed ‘AWS ecosystem’.
  • For deeply-nested response objects, I recommend stubbing using the native AWS structs instead of hashes. (In some cases, I encountered difficulties with equality matchers and resorted to a custom matcher that converted both the actual and expected values to JSON before comparing them.)
  • Stubbed clients don’t *do* anything – they don’t magically add stub objects to stub buckets just because you’ve stubbed the API that does it in the real world!

Check out the GitHub repository accompanying this post. If you have VirtualBox and Vagrant installed, it’s up and running with ‘vagrant up’.

AWS Stub (distributed computing) Software development kit

Published at DZone with permission of , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Playwright vs. Cypress: The King Is Dead, Long Live the King?
  • When AI Strengthens Good Old Chatbots: A Brief History of Conversational AI
  • How to Cut the Release Inspection Time From 4 Days to 4 Hours
  • Using QuestDB to Collect Infrastructure Metrics

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: