rsyncing to an AWS Instance
rsyncing to an AWS Instance
Join the DZone community and get the full member experience.
Join For FreeAre your API program basics covered? Read the 5 Pillars of Full Lifecycle API Management eBook
I wanted to try running some of the machine learning algorithms that Jen and I have been playing around with on a beefier machine so I thought spinning up an AWS instance would be the best way to do that.
I built the JAR with the appropriate algorithms on my machine and then wanted to copy it up onto an AWS instance.
I could have used scp but I quite like the progress bar that you can get with rsync and since the JAR had somehow drifted to a size of 47MB the progress bar was useful.
When I provisioned the machine I created a public/private key pair and I was able to ssh into the machine like this:
ssh -l ubuntu -i ~/Downloads/machinenursery.pem ec2-54-242-108-142.compute-1.amazonaws.com
I needed to tell rsync to use the pen file which I initially tried to do with the following command:
rsync --progress 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machinenursery
It seemed to ignore the pem file and I got a permission denied error when I ran this:
Permission denied (publickey). rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]
Eventually came across an article which explained a way around the problem using RSH instead of SSH:
rsync --progress --rsh 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machine nursery
As I understand it RSH isn’t secure but all I’m transferring is a JAR file so it didn’t seem like too much of an issue.
I’m sure there must be a way to transfer this file using SSH but I’ve tried all the different flags and I can’t figure it out so if you know how to please let me know!.
Establish API creation, publishing and discovery as a master practice with the API Management Playbook.
Published at DZone with permission of Mark Needham , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
{{ parent.title || parent.header.title}}
{{ parent.tldr }}
{{ parent.linkDescription }}
{{ parent.urlSource.name }}