DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • Comparing Cloud Hosting vs. Self Hosting
  • Is Podman a Drop-in Replacement for Docker?
  • Competing Consumers With Spring Boot and Hazelcast
  • Microservices With Apache Camel and Quarkus

Trending

  • Comparing Cloud Hosting vs. Self Hosting
  • Is Podman a Drop-in Replacement for Docker?
  • Competing Consumers With Spring Boot and Hazelcast
  • Microservices With Apache Camel and Quarkus
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. 6 Tips for AWS Command Line Ninjas

6 Tips for AWS Command Line Ninjas

Level up on your operational efficiency with these simple tips.

Andreas Wittig user avatar by
Andreas Wittig
·
Jun. 13, 16 · Tutorial
Like (3)
Save
Tweet
Share
5.12K Views

Join the DZone community and get the full member experience.

Join For Free

The AWS Command Line Interface (CLI) allows you to manage AWS services. Using the CLI from your terminal interactively allows you to half-automate tasks and frees you from logging into the AWS Management Console. In addition, integrating the CLI into shell scripts allows you to automate your infrastructure and the configuration of EC2 instances during the boot process.

This article covers typical hurdles when using the AWS CLI.

Command Completion

When using the CLI interactively within your terminal, command completion is a killer feature you should not miss. When enabled, command completion allows you to use the TAB key to complete commands. This will speed up your CLI usage significant.

The following steps are needed to enable command completion for bash on OS X:

echo "complete -C aws_completer aws" > ~/.bash_profile  
source ~/.bash_profile  


The official documentation contains general instructions for other shells as well.

Filtering Results of Requests on Server-Side

By default, the CLI uses a page size of 1,000 and retrieves all available items. If you need to request items from a list of more than 1,000 items, or if you want to speed up your commands, it is a good idea to filter the results of your request on the server side.

Many describe-* and list-* commands support server-side filtering: --filter. For example, it is possible to filter EC2 instances by instance type:

$ aws ec2 describe-instances --filter Name=instance-type,Values=t2.nano


Filtering Output on Client-Side

Another useful feature of the CLI is filtering the output of any command on the client side: --query. The JMESPath query language is used for filtering.

The following example lists all VPCs within a region and filters the results by using a --query.

$ aws ec2 describe-vpcs --query "Vpcs[?VpcId == 'vpc-aaa22bbb'].CidrBlock"
[
    "94.194.0.0/16"
]


You might need the CIDR of a VPC as a variable in your shell script. The following example shows how to achieve that. Formatting the output as text by adding the parameter --output text removes " character from the JSON result.

#!/bin/bash
CIDR=$(aws ec2 describe-vpcs --query "Vpcs[?VpcId == 'vpc-aaa22bbb'].CidrBlock" --output text)  
echo $CIDR  


Wait for...

When writing shell scripts by using the CLI, there will be the need to wait for a specific condition from time to time. For example, after initiating an EBS snapshot your script might need to wait until the snapshot was completed. Waiting can be achieved with a polling loop and a describe-* command. But there is a simpler solution built into the CLI for this: aws <service> wait <condition>.

The following example contains a wait command that will block the script until the snapshot has been completed.

#!/bin/bash
echo "Waiting for EBS snapshot"  
aws ec2 wait snapshot-completed --snapshot-ids snap-aabbccdd  
echo "EBS snapshot completed"  


Assuming an IAM Role

The CLI supports assuming an IAM role. Very handy if you need to switch between multiple AWS accounts with the help of cross-account roles.

All you need to do is to configure two profiles in ~/.aws/config: an IAM user and an IAM role profile.

[profile iam-user]
output = json  
region = eu-west-1

[profile iam-role]
role_arn = arn:aws:iam::<ACCOUNT_ID>:role/<IAM_ROLE>  
source_profile = iam-user  
output = json  
region = eu-west-1  


Only the IAM user needs security credentials stored in ~/.aws/credentials.

[iam-user]
aws_access_key_id = ***  
aws_secret_access_key = ***  


Afterwards, you are able to assume the IAM role by adding --profile iam-role to your CLI commands.

Fine-Tuning S3 config

The AWS CLI includes transfer commands for S3: cp, sync, mv, and rm. You are able to fine-tune these commands with special configuration.

For example, if you need to sync a large number of small files to S3, increasing the following values added to your ~/.aws/config config file will speed up the sync process.

[profile default]
...
s3 =  
  max_concurrent_requests = 100
  max_queue_size = 10000
  use_accelerate_endpoint = true


The official documentation contains detailed information about additional S3 configuration values.

Feedback

Is anything missing? Looking forward to your feedback! @andreaswittig or andreas@widdix.de.

AWS Command (computing) Command-line interface

Published at DZone with permission of Andreas Wittig. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • Comparing Cloud Hosting vs. Self Hosting
  • Is Podman a Drop-in Replacement for Docker?
  • Competing Consumers With Spring Boot and Hazelcast
  • Microservices With Apache Camel and Quarkus

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: