This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new Linux.com!

Linux.com

Feature: Networking

ssh-xfer: Quickly grabbing files over an existing SSH connection

By Ben Martin on August 08, 2008 (9:00:00 AM)

Share    Print    Comments   

The Secure Shell (SSH) and Secure Copy (SCP) make remotely performing system administration and copying files across secure links a painless operation. SSH and SCP use the same SSH protocol to protect network communications, but they rely on users knowing if they want a shell or to copy a file beforehand. You cannot easily use an existing SSH shell connection to a remote machine and just grab one or two files; if you want the files, you'll have to make another SSH connection for the file copy using SCP -- unless you have ssh-xfer.

The ssh-xfer project uses the local SSH agent to allow you to easily grab files using an existing SSH shell connection. You do not have to modify either the SSH client or server programs to use ssh-xfer -- but you will need to patch your ssh-agent. Although having to patch the ssh-agent is not ideal, you do gain one major advantage by doing this: you can send a file through more than one SSH connection. So if you first connect to the firewall and then you connected to a remote server from there, and from that remote server to a remote desktop machine, and from the shell on the remote desktop machine you decide to grab /etc/foo.conf, you don't have to think about how you to got there from your desktop, or how to SCP the file back via all the intermediate hosts. Simply run ssh-xfer /etc/foo.conf from the shell on the remote machine and the file will appear on your local machine's ~/Desktop -- or you can change the XFER_DEST_DIR definition in the ssh-xfer patch to specify a different default directory for transfers. Of course you'll need the ssh-xfer program to be available on the remote machine, but you don't need to change the SSH installation on any of the servers at all.

ssh-xfer is not packaged for Fedora, Ubuntu, or openSUSE. In this article I'll use a 64-bit Fedora 9 machine and build from source using ssh-xfer version 0.15 built against OpenSSH version 5.0p1-3.fc9 from the Fedora 9 updates repository.

The rpmbuild --recompile command below prepares and builds OpenSSH, including all the patches that Fedora adds to OpenSSH. Because we intend to patch the SSH agent, it is a good idea to recreate its source as the distribution uses it before applying the ssh-xfer patch. I found that a few parts of the patch did not apply cleanly to the current source, but these mainly related to slightly different return values (returning zero instead of nothing) and other things that are a fairly easy fix for a human but which deter the patch from working cleanly. I have uploaded an updated patch generated using OpenSSH version 5.0p1-3.fc9 from Fedora 9 updates to save others the time of performing these fixes.

$ rpmbuild --recompile /FromWeb/openssh-5.0p1-3.fc9.src.rpm $ cd ~/rpmbuild/BUILD/openssh-5.0p1/ $ patch -p0 < /FromWeb/ssh-xfer-0.15.diff ... patching file ssh-agent.c ... Hunk #16 FAILED at 1417. 2 out of 16 hunks FAILED -- saving rejects to file ssh-agent.c.rej ... use my patch above or fix these issues by hand.

Once you have the sources patched, the below commands reconfigure the source tree and rebuild the ssh-agent and the new ssh-xfer program. Then you need to run the modified ssh-agent on your local desktop machine and have the ssh-xfer binary available on the remote machines that you might want to grab files from. If you are running the same distribution on remote machines as the one you built the ssh-agent and ssh-xfer, using SCP is probably the easiest way to get ssh-xfer onto the remote hosts.

$ ./config.status $ make all ssh-xfer $ su -l # chown root:root ssh-xfer # chmod 755 ssh-xfer # scp -pv ssh-xfer myserver:/usr/local/bin # install --mode 755 ssh-agent /usr/local/bin/ssh-agent-xfer

To use ssh-xfer you need to be running the modified SSH agent and have authentication agent connection forwarding enabled. There is a potential security issue with using agent forwarding, so it is not enabled by default. Basically, if there is a means to bypass file permissions on the machine you connect to with agent forwarding, then someone who can bypass the security on the remote machine can access your local SSH agent. This security issue makes sense if you consider that you are enabling a feature that forwards connections to the agent on the remote machine to the agent on your local machine.

With the commands shown below, I first create a new local bash shell using the patched ssh-agent that has ssh-xfer support, then I SSH into a remote machine, create a new file there, and then ssh-xfer it back to my desktop machine.

ben@desktop ~$ ssh-agent-xfer bash ben@desktop ~$ ssh -A ben@myserver ben@myserver ~$ date > myserver-testfile.txt ben@myserver ~$ ssh-xfer myserver-testfile.txt Sending file . done. ben@myserver ~$ exit ben@myserver ~$ cd ~/Desktop ben@desktop Desktop$ ls -lh -rw------- 1 ben ben 29 2008-07-21 21:11 myserver-testfile.txt ben@desktop Desktop$ cat myserver-testfile.txt Mon Jul 21 21:19:14 EST 2008

The use of SSH agent forwarding might make some avoid using ssh-xfer. The ssh-xfer homepage even describes it has hackish and states that the protocol used to transfer file bytes is not very sophisticated. Because ssh-xfer is agent-based, you can directly download a file through SSH connections that have been chained together (connect to host b, then from b to c to d). This keeps you from needing to remember exactly which connections you used to get to the host you are currently looking at in order to download a file.

In bygone days when one connected to a Unix machine using a dial-up connection and a terminal program, you could issue commands and initiate a download directly from the console. The idea of extending the SSH agent to allow for file transfers brings some of the power that was available in the old days to modern SSH connections by allowing file transfer to be initiated directly from a remote terminal.

Ben Martin has been working on filesystems for more than 10 years. He completed his Ph.D. and now offers consulting services focused on libferris, filesystems, and search solutions.

Share    Print    Comments   

Comments

on ssh-xfer: Quickly grabbing files over an existing SSH connection

Note: Comments are owned by the poster. We are not responsible for their content.

No real advantage over scp

Posted by: Anonymous [ip: 82.192.250.149] on August 08, 2008 11:14 AM
Going through the hassle of building a non-packaged app can be worthwhile, but in this case it doesn't seem worth the bother. From another terminal window on the machine inside the firewall, just
"scp user@remotemachine:filename ." copies the file. OK, it will prompt for your password, which the other method doesn't need. Is that really such a big deal?

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 24.14.51.60] on August 08, 2008 01:06 PM
You can also use openssh's controlmaster feature to do the same thing without the need to recompile ssh. It muxes multiple login sessions and scp traffic over a single socket authenticated once.

Oddly enough, it is the topic of a previous article here.
http://www.linux.com/articles/54498

#

Re: ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 67.169.94.222] on August 09, 2008 08:16 AM
control master is quite nifty, and works with scp and sftp as well as ordinary ssh sessions. Much nicer than this ugly hack.

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 135.222.110.184] on August 08, 2008 04:29 PM
can i really trust a consultant with a PhD who can't keep the clocks on two machines in sync using NTP?
myserver is 8 minutes ahead of mydesktop. :*)


ben@myserver ~$ date > myserver-testfile.txt
ben@myserver ~$ ssh-xfer myserver-testfile.txt
Sending file
. done.
ben@myserver ~$ exit
ben@myserver ~$ cd ~/Desktop
ben@desktop Desktop$ ls -lh
-rw------- 1 ben ben 29 2008-07-21 21:11 myserver-testfile.txt
ben@desktop Desktop$ cat myserver-testfile.txt
Mon Jul 21 21:19:14 EST 2008

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 76.111.32.2] on August 08, 2008 11:36 PM
too much of a pain. Just set up a key so you don't have login to your SSH remote machine with a password. Then, in Nautilus or in KDE's file manager, just type an address of:

sftp://root@www.myservername.com

(if you want to login as root)

save a shortcut to this address on your desktop. This will give you a nice gui to copy and paste from, or navigate around in.

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 213.112.87.76] on August 09, 2008 01:07 AM
Or you could just use zssh which require no patching of ssh.

http://zssh.sourceforge.net/

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 85.25.135.137] on August 09, 2008 02:40 AM
does it work with windows?

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 66.112.68.151] on August 09, 2008 04:11 AM
One feature where I believe this beats all the alternatives listed here (correct me if I'm wrong): the ability to work across multiple links. Sometimes, due to firewalls or network design, you cannot connect directly to another machine, but must go through some intermediate steps. This still gives a method to bring files from that machine back to you. Hopefully this article will bring this 3-year-old hack to the attention of OpenSSH developers, and they will find a way to integrate it properly.

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 75.75.8.40] on August 09, 2008 04:53 AM
wow, i had to do a double take at the complete uselessness of this entire article... um, "existing"? you mean like with my agent properly configured and my ssh tunnel? or perhaps my plink scripts from my windows clients? bah, feh... if you've that many files to "transfer", might I suggest a fresh new technology: nfs. I even hear tale of using automounters for just-in-time access... but, hey, IANAD...

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 86.150.23.108] on August 09, 2008 02:41 PM
If the file is reasonably small you can of course just use uuencode, like the old days. Works for binary files too:

cat myfile | compress | uuencode

... stuff appears in your terminal. copy/paste it to a file, then just uncompress the file and uuencode that. Your terminal buffer must be big enough to contain the file and fast enough scrolling so you aren't waiting forever. And you need some arguments on the uuencode command too I think, can't remember them now.

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 89.139.77.17] on August 09, 2008 06:11 PM
What really bothers me is that utilities like scp are horribly slow when transfering multiple small files over the network.

Yes I know, the accepted workaround for this is to tar the files before copying them or the more sophisticated way:
$ tar -czf - /some/file | ssh host.name tar -xzf - -C /destination

Now I wonder, is it *that* hard to add a flag to scp that does the above line automatically? I enjoy working with command lines, and I appreciate the flexibility demonstrated by this command, but that doesn't have to come at the cost of ease of use. a simple --use-tar flag in scp would kill no one.

#

ssh-xfer: Quickly grabbing files over an existing SSH connection

Posted by: Anonymous [ip: 199.18.139.129] on August 22, 2008 07:52 PM
boxB# ssh boxA
....
boxA#
~C
>-L5000:localhost:5000

boxA# uuencode -m some-file-or-files | nc -l -p 5000
~^Z
boxB# nc localhost 5000 | uuencode -m -c

I fabricated this log, but I do something similar to this from time to time.
It's easy to see how additional hops could be handled in this way, and
how furthermore a simple shell script could automate much of the session
set up and provide a front-end so that port numbers don't need to be
remembered. It'll do in a pinch anyway and it's as portable as you like.

Consider that something which does this more elegantly would be very
valuable in situations where login requires some token (OTP), and the
overhead of establishing a new session, keying and diffie is a little much
to just copy a few files..

tar is your friend too. The only difference between this and the previous
helpful comments is that it does not involve copy and paste...

Thanks

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya