Tuesday, September 29, 2009

SCP over SSH

I was asked by a special someone to convert a DV tape into digital format at the ETC. So I did.

The resulting file was a 9.35 GB Quicktime movie. Of course, I didn't at all think about how I was going to transfer such a large file off of the ETC computer.
  1. I captured it in Premiere Pro, which apparently cannot save to anything other than Quicktime format, nor does the ETC have any compression programs on their computers.
  2. I did not have my flash drive with me (which wouldn't have mattered since it's way too small.
  3. I did not have my laptop on me to transfer the file to.
  4. The only external hard drive the ETC has was checked out and overdue.
  5. I couldn't scp the file from the Mac to my computer, because for some unknown reason, I could not establish an ssh connection with it.

So what was I to do? I had exhausted all the file transfer possibilities I could think of. I decided to experiment a bit more and found out that, lo and behold, I could ssh into Matthew! Now, why I could ssh into my CSE account on Matthew but not into my personal computer is beyond me. I know I didn't leave my computer in Windows, and sshd should be running...so I checked. I ssh'ed from Matthew to my computer just fine! In fact, I even ended up SSHing to Matthew with X-forwarding and then setting up a VNC connection to my desktop. It worked.

This went to prove that I was in fact able to transfer information from the ETC computer to my desktop, via Matthew. But how to transfer a 9+ GB file? I did some testing and then tried this experiment:

First, I created the most simple Perl script oProxy-Connection: keep-alive
Cache-Control: max-age=0

20my desktop:
print while (<>);

It just spits out everything that is fed to it. Then I invoked this script by SSHing to Matthew, passing it a command to SSH to my desktop and execute the script, redirecting the standard output to a file on my desktop. I typed in a few lines of text and then sent ^D. I checked the file on my desktop, and it worked! This means I can utilize this system to transfer arbitrarily large files over SSH. I first had to enable passwordless logins from the ETC computer, and then I set it to work...

$ ls -lh Movie.mov
-rw-r--r--@ 1 etcmedia staff 9.4G Sep 29 16:04 MovieProxy-Connection: keep-alive
Cache-Control: max-age=0

$ cat Movie.mov | ssh -C jdenardo@matthew.cse.taylor.edu \
"ssh jdenardo@ \"/home/jdenardo/transfer.pl > \

A few minutes later, the command finished, and I checked the MD5 checksums of the input and output files:

MD5 (Movie.mov) = ed889ff1879832e46c9d6e68fd56657a
ed889ff1879832e46c9d6e68fd56657a Movie.mov

...they match! Despite not being able to directly SSH to my desktop, I was able to set up a route through another computer on the LAN and copy files! As a bonus, it was all done securely! This is certain

Wednesday, September 9, 2009

clips Gentoo ebuild

I needed the CLIPS platform for one of my AI classes, and of course it doesn't appear to exist in Portage. (Even trying to find the correct Linux source code is a nightmare with the way they have it organized on Sourceforge.)

So I found someone else's ebuild, made a small modification, and voila! It's now in my local repository:

> su
> mkdir -p /usr/local/portage/sci-misc/clips
> cd /usr/local/portage/sci-misc/clips
> wget http://cse.taylor.edu/~jdenardo/clips-2.64.ebuild
> ebuild clips-2.64.ebuild manifest
> emerge -a clips

This will build two executables, clips and xclips. And, it even builds on ~amd64 with no problem!

Tuesday, September 8, 2009


I don't feel inspired to write poetry anymore. In everything there is a silver lining...