Log in

No account? Create an account

a geeky post about external drives

« previous entry | next entry »
Jul. 21st, 2007 | 06:15 pm
music: Strangers on a Train - Lovage (Music to Make Love to Your Old Lady By)

Copying 4.1 Gibibytes1 (4,366,678,016 bytes) between different places.

Okay so I have two USB2.0/Firewire400 external enclosures. I really wanted to see how they performed because I've been moving some big files around lately and I've noticed some bizarre inconsistencies in speed. Both enclosures have 7200rpm Maxtor disks in them; Kinbote is 200 gigabytes and Lo is 300 gigabytes. The two Maxtor disks' speeds don't seem to differ at all but this is certainly not a Perfect Scientific Test, but more a Test Of My Particular Setup2. There are a number of things I could have done to make this more accurate, but it just wasn't worth the trouble.

For this test I used rsync3 to move several large avi files contained in a few directories. One test I would still like to do is see how Firewire compares to USB, and different copy utilities compare, when moving large numbers of small files. I tried one transfer in Finder, rsync, and cp, and found them to be almost the same when moving just a few large files, so for this test I just used rsync.

Test 1: Two Firewire Drives "daisy-chained" Together

Transfer Times To Different Places Using Firewire

Interestingly enough firewire's daisy chaining doesn't effect communication between disk and computer — read and write speeds were almost identical when using a drive directly plugged into my computer compared to a drive connected through another drive. It took roughly 2:20 to go from drive to laptop, and 2:40 to go from laptop to drive. Both my drives have two firewire plugs on them, and it appears it doesn't matter which plug I use for what. Drive to drive transfers were different, however. Going from Kinbote to Lo was the slowest at 4:25, I don't know why it was so much slower than the other direction, which took 3:49.

Test 2: One USB Drive, One Firewire Drive

Transfer Times To Different Places Using Mixed Firewire and USB 2.0

It appears what's happening here is I'm being limited by the writing speed of USB and Firewire. No matter where from it takes about 4:00 to write to Lo, 2:41 to copy from the Macbook to Kinbote (same as previous) and 2:50 to copy from Lo to Kinbote. The slowest direction was 3:56 to go from firewire Kinbote to USB Lo — still faster than going from firewire drive to firewire drive. It appears reading from USB and writing to Firewire is the fastest way to move files between the two external drives—this is what I decided to use to backup my massive music collection from one drive to the other.

Test 3: Two USB Drives

Transfer Times To Different Places Using Mixed Firewire and USB 2.0

Just for completeness I had to try it. I even tried it once with a USB hub. Yeah, it sucks. 3:25 to copy from a USB drive to my macbook. 4:31 to copy from Lo to Kinbote, 4:42 to go the other direction. From macbook to anything (USB hub or not) took around 3:55.

One interesting observation: it appears simply the presence of another USB device slows down read speed— With one USB drive attached I copied all the data in about 3:00, however even the presence of the second drive caused it to slow down to around 3:25. I'm pretty sure spotlight isn't to blame because I didn't notice any activity on the second disk. God I hate spotlight, though.

Next Up

The next time I need a new drive (and that looks to be pretty soon with my friggin' 135GB of music) I'm going to get an eSATA expresscard and a SATA disk, since my dad has an eSATA enclosure sitting around gathering dust. I'll have to do some more speed tests with that (hopefully much faster) setup. Surprisingly a 500GB SATA disk goes for around $120 and an eSATA expresscard is around $40 — still cheaper than the cheapest 500GB FW800 disk I could find. FW800 enclosures seem to go for $120 without a disk. I don't get it.

One final note:

The Prolific PL-3507 chipset sucks. No, really, even the latest firmware has a really massive bug where multiple drives give the same Firewire GUID (the firewire equivalent of a MAC address) which is never supposed to happen. On OS X when I plugged both my drives in via firewire kernel[0]: FireWire Error: Devices with identical unique ID: 0050770e 00071002 cannot be used. was printed in the console and both drives were immediately disconnected. I had to reboot my computer to get it to recognize any more firewire drives. I guess that crashed the kernel firewire driver. Googling around for that ID I found this blog entry which claimed it could be fixed by hex editing the drive firmware. So I downloaded the latest firmware and the romwrite utility from Prolific in Windows and plugged my drives in as USB. I flashed my drive with the latest firmware, then downloaded it to my disk and hex edited the only occurrence of 0x00071002 to 0x00071003 on one of my drives. After that they both worked and registered the new GUID in OS X's System Profiler. Note: I couldn't just edit the new firmware downloaded from Prolific because it had some sort of checksum, but if I flashed my drive and then downloaded it I could edit and re-flash the downloaded firmware. Jesus fucking headache.

1. Whenever you see the size of a folder on your disk, it's given in gibibytes, not gigabytes. Gigabytes is just a bullshit size used by hard disk vendors to sell you a 30GB iPod that actually only holds 28 Gibibytes.

2. Details: I have a 2.16Ghz 15" Macbook Pro Core 2 Duo — it was the first MPB generation to have core 2 duo. Both of my external enclosures are Kingwin enclosures which look a lot like this. They both have the Prolific PL-3507 chipset in them, flashed to the latest (hacked, see above) firmware — at this time the latest is dated 2006.04.20.149.

3. time rsync -aEP [SOURCE] [DESTINATION] to be exact. (the -E is an OSX-ism for copy extended attributes, i.e. resource forks)

| Leave a comment |

Comments {6}

Joshua Kwan

(no subject)

from: joshuak
date: Jul. 22nd, 2007 10:00 am (UTC)

it's unlikely but possible that OS X is keeping the buffers for the externals around for a while even after you unplug them. Are you sure this couldn't have potentially fucked with your results? (Specifically, the one where A->B was faster than B->A.)

Reply | Thread

Alex P.

(no subject)

from: wetzel
date: Jul. 22nd, 2007 06:43 pm (UTC)

I do not know enough about system programming to understand what you are saying. What do you mean by buffers, like the /dev/disk*s* nodes? Do you mean A->B faster than B->A when they're both firewire?

Reply | Parent | Thread


(no subject)

from: the_answer_is
date: Jul. 22nd, 2007 06:11 pm (UTC)

durr.r. pretty illustrations

Reply | Thread

Alex P.

(no subject)

from: wetzel
date: Jul. 22nd, 2007 06:44 pm (UTC)

Yeah I decided to go back and try and make graphs in illustrator like I once did ages ago and I discovered that illustrator can't handle time as a number to deal with -- I had to turn everything into seconds. wtf?

Reply | Parent | Thread

Botany Nerd

(no subject)

from: litespeed
date: Jul. 23rd, 2007 03:51 pm (UTC)


Reply | Thread

Very good info!

from: anonymous coward
date: Dec. 29th, 2007 05:55 am (UTC)

I was having the same issues as you. I needed to move 700G from one disk to another, and FW was giving me 9MB/s writes. While a 700MB file would move from FW to internal at ~25MB/s.

Anyway, With your hint I moved the reader to USB and the writing disk to FW. With this setup I get 20-23MB/s very reasonable!

Let us know if you get that SATA drive. I've been wondering if it would be worth the upgrade for the SATA drives that I have.


Reply | Thread