Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

rsyncing to an AWS Instance

DZone's Guide to

rsyncing to an AWS Instance

· Integration Zone ·
Free Resource

SnapLogic is the leading self-service enterprise-grade integration platform. Download the 2018 GartnerMagic Quadrant for Enterprise iPaaS or play around on the platform, risk free, for 30 days.

I wanted to try running some of the machine learning algorithms that Jen and I have been playing around with on a beefier machine so I thought spinning up an AWS instance would be the best way to do that.

I built the JAR with the appropriate algorithms on my machine and then wanted to copy it up onto an AWS instance.

I could have used scp but I quite like the progress bar that you can get with rsync and since the JAR had somehow drifted to a size of 47MB the progress bar was useful.

When I provisioned the machine I created a public/private key pair and I was able to ssh into the machine like this:

ssh -l ubuntu -i ~/Downloads/machinenursery.pem ec2-54-242-108-142.compute-1.amazonaws.com

I needed to tell rsync to use the pen file which I initially tried to do with the following command:

rsync --progress 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machinenursery

It seemed to ignore the pem file and I got a permission denied error when I ran this:

Permission denied (publickey).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]

Eventually came across an article which explained a way around the problem using RSH instead of SSH:

rsync --progress --rsh 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machine nursery

As I understand it RSH isn’t secure but all I’m transferring is a JAR file so it didn’t seem like too much of an issue.

I’m sure there must be a way to transfer this file using SSH but I’ve tried all the different flags and I can’t figure it out so if you know how to please let me know!.

With SnapLogic’s integration platform you can save millions of dollars, increase integrator productivity by 5X, and reduce integration time to value by 90%. Sign up for our risk-free 30-day trial!

Topics:

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}