Over a million developers have joined DZone.
Platinum Partner

rsyncing to an AWS Instance

· Integration Zone

The Integration Zone is brought to you in partnership with 3scale. Take control of your APIs and get a free t-shirt when you complete the 3step Challenge

I wanted to try running some of the machine learning algorithms that Jen and I have been playing around with on a beefier machine so I thought spinning up an AWS instance would be the best way to do that.

I built the JAR with the appropriate algorithms on my machine and then wanted to copy it up onto an AWS instance.

I could have used scp but I quite like the progress bar that you can get with rsync and since the JAR had somehow drifted to a size of 47MB the progress bar was useful.

When I provisioned the machine I created a public/private key pair and I was able to ssh into the machine like this:

ssh -l ubuntu -i ~/Downloads/machinenursery.pem ec2-54-242-108-142.compute-1.amazonaws.com

I needed to tell rsync to use the pen file which I initially tried to do with the following command:

rsync --progress 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machinenursery

It seemed to ignore the pem file and I got a permission denied error when I ran this:

Permission denied (publickey).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]

Eventually came across an article which explained a way around the problem using RSH instead of SSH:

rsync --progress --rsh 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machine nursery

As I understand it RSH isn’t secure but all I’m transferring is a JAR file so it didn’t seem like too much of an issue.

I’m sure there must be a way to transfer this file using SSH but I’ve tried all the different flags and I can’t figure it out so if you know how to please let me know!.

The Integration Zone is brought to you in partnership with 3scale. Learn how API providers have changed the way we think about integration in The Platform Vision of API Giants.

Topics:

Published at DZone with permission of Mark Needham , DZone MVB .

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}