Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Securely Uploading Files From an iOS/Android App to S3

DZone's Guide to

Securely Uploading Files From an iOS/Android App to S3

Mobile applications often need to securely upload files to your server Here we learn to use a lightweight API endpoint and let AWS S3 do the heavy lifting.

· Mobile Zone
Free Resource

Launching an app doesn’t need to be daunting. Whether you’re just getting started or need a refresher on mobile app testing best practices, this guide is your resource! Brought to you in partnership with Perfecto

Most of the mobile applications these days require some form of a backend. Usually, it is used for something trivial, like maintaining user profiles, settings, and scores. However, more often than not, the application needs to upload some files to your server. The following tutorial will show how to do it securely and efficiently using AWS S3. Why spawn powerful servers that can handle huge amounts of traffic when all they do is get the files and store them on S3 anyway? Files can be uploaded to S3 directly, and S3 infrastructure can scale with our application and adjust to the amounts of traffic required. All we need to do is implement a simple lightweight API endpoint that will instruct the client where to upload the heavy things and let S3 do the heavy lifting.

This solution has the following characteristics:

  • The uploading is a two-step process; first, the client requests a permission to upload, providing the server with the intent and the metadata of the file uploaded. This is especially helpful in applications where the server should know about the ongoing uploading while it may take the client some time and possibly few attempts to upload the data (due to slow or unreliable cellular networks).
  • The upload links are given upon request and only to authenticated users (outside the scope of this tutorial), they are single use and time limited.
  • Absolutely no AWS access/secret keys are embedded within the application. Even as hackers reverse engineer our application, they will not gain any knowledge or keys to try to poke around our servers/backend storage.
  • At the same time, the mobile application is left completely unaware of the complexities of uploading to S3. It doesn’t need to sign the requests or provide special headers. No client AWS libraries are required. All the application does is uploading raw data to a URL through either a POST or a PUT request. We can use any HTTP method or a library of our choice for that.
  • This mechanism gives us the flexibility to expand and change our backend architecture in the future without changing or deploying a new version of the mobile application. If we decide we do want the uploaded files to hit our servers first or route them to a different cloud provider, we simply change the URLs returned from the API in our backend and the files will flow through a different route.

This two-step approach obviously has some drawbacks, e.g double the API calls, however in some cases it may bring unexpected benefits. For example, at Bugsee, we upload and process thousands of crash reports generated on millions of devices. Being able to replay the last minute of everything that happened in the app, including video, network traffic, console logs, et cetera turns out to be really useful for our clients, and helps them resolve bugs and crashes they wouldn’t otherwise be able to root cause. However, for many cases, a single recording is enough, having thousands different sessions of the same particular crash doesn’t bring any additional value, no developer is going to watch them all. The two-step approach allows our client SDK first to register and intent that containing a metadata and some unique signature of the crash and Bugsee backend has the luxury to register the crash, increase the statistics counters but to instruct the client to drop and never upload the recording itself. This saves huge amounts of traffic both for the users and for us.

With that in mind, let’s get down to business:

Preparing AWS Resources

First, we will create a new S3 bucket for this and name it "bugsee-example." No need to make it public or change its settings, default is good enough for what we are about to do.

Create a new AWS user and give him the rights to only upload files. In order to achieve that, attach the following policy:

{     "Version": "2012-10-17",     "Statement": [         {             "Effect": "Allow",             "Action": [                 "s3:PutObject"             ],             "Resource": [                 "arn:aws:s3:::bugsee-example/upload/*"             ]         }     ] }


Building The Backend

The user can only upload files (S3: PutObject action) to the "bugsee-example," and only to the the /upload/ subfolder. He can’t get any other files, he can’t even list the contents of the bucket. That was exactly our intention.

For the sake of this example, we will build a new node.js backend server from scratch. The backend will only handle one API route (POST /users/:userId/object), which will be used by our mobile application to post the intent to upload a file to our backend and get back the pre-signed S3 URL for the actual upload.

If you already have a backend for your application, you might easily incorporate a similar approach into your existing framework. Implementing this code in any other language is pretty straight forward too; AWS SDK is available for all the common languages.

Since S3 will handle the actual uploads for us, we may decide to go completely server-less and use Amazon Lambda to handle the intent as well. This exercise, however, is outside the scope of this tutorial, if you are interested in building pure server-less backends using Amazon Lambda you might refer to this tutorial.

Create a new folder and initialize new node project:

mkdir server; cd server npm init

Install the required node modules:

npm install --save express npm install --save aws-sdk npm install --save uuid


Create the main body of the application:

var express = require('express') var AWS = require('aws-sdk') var uuid = require('uuid')   AWS.config.update({   region: 'us-west-2',   accessKeyId: 'AKIA************',   secretAccessKey: '******************' });   var S3 = new AWS.S3();   var bucket = 'bugsee-example';   var app = express() app.post('/users/:userId/objects', function (req, res) {   console.log('Generating presigned URL for ' + req.params.userId);     // Generate unique key for the new object   var key = uuid.v4();     // Record metadata in the DB, associate it with the user     // Construct a path where data will be stored in that bucket   var path = 'users/' + req.params.userId + '/objects/' + key;     // Construct a request   var request = {     Bucket: bucket,     Key: path,     Expires: 3600 // Valid for only 1 hour     };     // Ask   S3.getSignedUrl('putObject', request, function(err, result) {     res.send({         method: 'PUT',         url: result     });   }); })   app.listen(3000, function () {   console.log('Example app listening on port 3000!') })


Obviously, in a real production system, we would not hard code accessKeyId/secretAccessKey pair within the application itself, but rather pass it to the application through environment variables or have it distributed through a config file which is not stored in our source control.
The code above accepts requests from the client in the form of POST requests to /users/objects routes, creates a new unique ID for an object, and returns to the client a JSON with the URL to upload the actual file. The link is valid only for one hour.

You might be tempted to go a step further and not use AWS Users at all, relying instead on AWS Roles which auto-rotate credentials automatically. While in general, it is a very good practice, this might not work well in this particular case. The catch is you as a developer do not control when exactly role credential pairs are being rotated/expired, and as they do, so will the URLs you create.

Uploading The File (iOS)

Below is the Swift code that uses the API endpoint to get the URL and then upload the raw file to that URL:

    func uploadtoSignedUrl(_ data: Data, _ url: URL, _ method: String) {         var urlRequest = URLRequest.init(url: url, cachePolicy: .reloadIgnoringCacheData, timeoutInterval: 60);           urlRequest.httpMethod = method;         urlRequest.setValue("", forHTTPHeaderField: "Content-Type")         urlRequest.setValue("\(([UInt8](data)).count)", forHTTPHeaderField: "Content-Length")         urlRequest.setValue("iPhone-OS", forHTTPHeaderField: "User-Agent")         urlRequest.setValue("testImage", forHTTPHeaderField: "fileName")           URLSession.shared.uploadTask(with: urlRequest, from:data) { (data, response, error) in             if error != nil {                 print(error!)             }else{                 print(String.init(data: data!, encoding: .utf8)!);             }             }.resume();     }       func requestSignedUrl(completion: @escaping ((_ url: URL, _ method: String) -> Void)) {         var urlRequest = URLRequest.init(url: URL.init(string: "http://192.168.0.26:3000/users/testUser/objects")!, cachePolicy: .reloadIgnoringCacheData, timeoutInterval: 60);         urlRequest.httpMethod = "POST";           URLSession.shared.dataTask(with: urlRequest) { (data, response, error) in             if error != nil {                 print(error!)             } else {                   if let urlContent = data {                     do {                         let jsonResult = try JSONSerialization.jsonObject(with: urlContent, options:                             JSONSerialization.ReadingOptions.mutableContainers)                           if let jsonResult = jsonResult as? [String: Any] {                                 completion(URL.init(string: jsonResult["url"] as! String )!, jsonResult["method"] as! String)                         }                     } catch {                         print("JSON parsing faild")                     }                 }             }         }.resume()     }


And when we have the file to upload we would call this:

requestSignedUrl(completion: {(_ url: URL, _ method: String) -> Void in      uploadtoSignedUrl(data, url, method) })


Uploading The File (Android)

In our  Android example we will be using okhttp library to achieve the same thing, it makes it much easier to work with http requests:

    private class UploadTask extends AsyncTask<String, Void, AsyncTaskResult<Void>> {         private static final int NETWORK_TIMEOUT_SEC = 60;         private static final String OUR_SERVER_URL = "http://192.168.0.26:3000/users/testUser/objects";           @Override         protected AsyncTaskResult doInBackground(String... params) {             try {                 // Obtain the url                 OkHttpClient client = new OkHttpClient.Builder()                         .connectionSpecs(Arrays.asList(ConnectionSpec.MODERN_TLS, ConnectionSpec.CLEARTEXT))                         .connectTimeout(NETWORK_TIMEOUT_SEC, TimeUnit.SECONDS)                         .readTimeout(NETWORK_TIMEOUT_SEC, TimeUnit.SECONDS)                         .writeTimeout(NETWORK_TIMEOUT_SEC, TimeUnit.SECONDS)                         .build();                 Request getUrlRequest = new Request.Builder()                         .url(OUR_SERVER_URL)                         .post(RequestBody.create(MediaType.parse("text/plain"), ""))                         .build();                 Response getUrlResponse = client.newCall(getUrlRequest).execute();                 if (!getUrlResponse.isSuccessful())                     return new AsyncTaskResult(new Exception("Get url response code: " + getUrlResponse.code()));                   String responseJsonString = getUrlResponse.body().string();                 JSONObject getUrlResponseJson = new JSONObject(responseJsonString);                 String url = getUrlResponseJson.getString("url");                   // Upload the file                 String imagePath = params[0];                 Request uploadFileRequest = new Request.Builder()                         .url(url)                         .put(RequestBody.create(MediaType.parse(""), new File(imagePath)))                         .build();                 Response uploadResponse = client.newCall(uploadFileRequest).execute();                 if (!uploadResponse.isSuccessful())                     return new AsyncTaskResult(new Exception("Upload file response code: " + uploadResponse.code()));                   return new AsyncTaskResult(null);               } catch (Exception e) {                 return new AsyncTaskResult(e);             }         }           @Override         protected void onPostExecute(AsyncTaskResult result) {             super.onPostExecute(result);             if (result.hasError()) {                 Log.e(TAG, "UploadTask failed", result.getError());                 Toast.makeText(MainActivity.this, "UploadTask finished with error", Toast.LENGTH_LONG).show();             } else {                 Toast.makeText(MainActivity.this, "UploadTask finished successfully", Toast.LENGTH_LONG).show();             }         }     }


Handling The Completion

The only part that is left is notifying your backend once the file has been successfully uploaded. One option is to create another endpoint and to have client ping the server again to close the loop. That would mean another API call. Luckily, S3 can perform a custom action(event) for every object being created. The screen below show the available options. It can trigger a Lambda function, or it can send a message to SQS or SNS. Depending on our needs we can use one of these options to trigger further processing of the uploaded file, but that is outside the scope of this tutorial.

S3 event settings

We’ve built a complete solution to securely and efficiently upload files directly to AWS S3. The complete source code of this example is hosted on GitHub for your convenience. The code includes both the node.js server as well as clients for iOS and Android.

Keep up with the latest DevTest Jargon with the latest Mobile DevTest Dictionary. Brought to you in partnership with Perfecto.

Topics:
ios ,android ,aws s3 ,mobile app development ,mobile ,tutorial

Published at DZone with permission of Dmitry Fink. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}