Advanced Security Using AWS IAM on QDS

For Big Data analyses and processing, Qubole Data Service (QDS) orchestrates storage and compute resources owned in the customer’s account. To enable this, customers delegate the necessary permissions to QDS. With IAM Roles promoted as security best practice on AWS, customers no longer need to provide access and secret keys to QDS. Thereby, making access control more secure.

An IAM role is similar to a user, in that it is an AWS identity with permission policies that determine what the identity can and cannot access or do in AWS. A role is intended to be assumable by anyone who needs it and it does not have any credentials (password or access keys) associated with it.

IAM Role contains:

• Permission Policy: Permissions for a given user assuming the role
• Trust Policy: Who can assume the role

The mechanism for initiating AWS API calls: Obtain temporary credentials by assuming an IAM Role and use those credentials to initiate AWS API calls.

IAM Role vs. IAM Key

Generally speaking, sharing access keys with anyone or under any circumstances opens up doors for potential security hazard. Gaining unauthorized access to keys enables someone other than trusted and authorized entities to assume your identity. Instead, if a user is assigned to an IAM Role, access keys are created dynamically and provided to the user. So from security standpoint, IAM Role comes recommended as security best practice on AWS.

IAM Roles in QDS

With cross-account IAM Roles, you can delegate necessary access to QDS without providing it your access keys. Once the cross-account IAM Role is created, you share the associated Role ARN with QDS. As a result, QDS becomes an IAM user by assuming the given IAM Role and obtains temporary security credentials to initiate AWS API calls. This enables QDS to seamlessly manage clusters (bringing up and down nodes, Spot instance bidding, reading and writing data to S3, etc.) on your behalf without requiring your credentials.

To get started with creating cross-account IAM Role for QDS, click here. For setup and configuration details of IAM Roles and Role ARN in QDS, click here.

Advanced Security in QDS

In our continuous efforts to make QDS more secure for our customers, we’ve implemented additional layers of security with regards to IAM Roles.

Dual IAM Role

QDS allows for creating two IAM roles as part of IAM Role authentication for a single QDS user account.

• Cross-account IAM Role at account level as described in Authorizing AWS using IAM Roles and Creating a Cross-account IAM Role for QDS.
• IAM Role configured at cluster level specifically to interact with the data.

Note: QDS instances only assume cross-account IAM Role which limits QDS’ access just to the default S3 location. This model ensures that the data remains secure under the ownership of the customer.

To get started with creating dual IAM Roles, click here.

IAM Role Override (Per User IAM Role)

In QDS, multiple users may be given access to the same account. So, an account-wide IAM Role has the downside of being used by many users that end up sharing common access permissions. This may not be ideal or suitable for some organizations where individual users within a team require different access levels. To accommodate for this scenario, Qubole provides a way to override the cross-account’s IAM Role settings at user level in an account.

For details on IAM Role Override in QDS, click here.