Advice on storage and back-up for a starter please!

Hi,

As Chris knows, I am a UK starter now happily ripping my CDs to a Macbook and playing back through ADM9s. I have already made one big mistake which meant I deleted my itunes library and had to import again. Now with around 100 Gb on the hard drive of my 250 Gb Macbook, I am starting to think about external storage and back-up!

I reckon I will need around 500 Gb or so for my whole CD collection. I also want to back up to minimise the risk of losing it all! There is so much info and so many products out there I am a bit confused and bewildered. I would like to stick with Apple products. Is it as simple as buying a 500 Gb external hard drive for my music - if so, does firewire offer any advantages over USB? Would Time Capsule be a good option for back up - I believe this is wireless - how exactly does it work? Any other obvious options, particularly to keep costs down?

By the way Chris, looking forward to your full review of ADM9.1s.

Phil

Hi Phil,

I think you and I are in about the same stage of this journey. I am likewise in the process of happily ripping all of my CDs into my system, using a Macbook. For your storage, I'm going to guess that Chris will suggest a NAS (networked attached storage) solution, which is sort of an all-in-one wireless or networked set of drives that automatically mirror each other and thereby provide a kind of instant backup and failure protection. (BTW, I think you will also see that the majority of folks around here like putting their external hard disks in another room, for noise reasons.) Brands I see mentioned a lot here are the Drobo and Thecus. Be aware that these are very slick solutions, but can get a little pricey.

I chose another route which I think is simpler and cheaper, although not as elegant. I bought an Apple Airport Extreme router into which I feed my internet connection, and to which I have directly attached via USB a 1TB hard drive. The brand of drive I went with is by Other World Computing (OWC) - not sure if they ship across the pond to you. It's a nice product and was Apple-ready right out of the box. As for backup of that drive, I plan to purchase a "generic" (Western Digital, etc) 1TB hard drive and periodically use it to backup the OWC drive, and store it either off-site or in my fireproof safe. There's a little work to do there as opposed to the NAS/automatic solution, but that's OK by me.

I have found the speed of the USB connection to the hard drive to be completely sufficient for music files. I did some of my initial ripping using the Macbook directly attached to the hard drive via firewire and it was fast - however, I have lately been ripping across wi-fi to the Airport Extreme into the hard drive and it's still pretty quick. I explored Time Capsule and decided against it. For one thing it sort of concerned me that if the hard drive failed, I'm not sure how easy it would be to fix, being an all-in-one solution. In other words, I like having the flexibility of my own hard drive and a separate router. Time Capsule is a neat idea though, and probably worth looking into.

TheOtherTim

The cheapest and easiest way to start is to get a USB drive from Maplin.

Hi Phil - If you want to stick with Apple products, the Time Capsule is super easy and you can store the disk outside your listening room. As long as you stay under the 1 TB limit you're totally fine.

If you want direct attached storage I suggest FireWire because it is faster than USB (in reality, not specs) and will not share any part of a USB bus that a USB DAC may reside on.

Of course you know I am a huge fan of NAS devices and can go down that path if you want to discuss it. But, it seems like you would be totally fine with a Time Capsule which is a NAS itself.

Chris et al.,

Think I may be on to an appropriate storage solution. Not wanting to break the bank, one can pick up a firewire 400/USB 1 terrabyte external HD made by Western Digital for around £100. I am thinking of getting two, one for my itunes library and one as a stand-alone back-up.

Here's the question. Is it pretty straightforward to have one's entire itunes library on an external HD? As long as the Mac is connected to the HD, does itunes automatically recognise where the library is? At the moment, I have 100 Gb in my itunes on my Macbook and I just turn the thing on and away I go - totally hassle free! Will the system work as smoothly with an external HD containing my itunes library? Any problems I should watch out for? Any advice folk think they should give me - it would be much appreciated!

Many thanks

Phil

Hello again Phil,

I'll take a stab at this and hope that Chris adds his opinions or corrections if he has a chance.

Sounds like you are taking a similar approach as I did, with an external drive and another one as backup. I'm happy with mine so far. Be forewarned that when I did my first backup of around 150GB it took a few hours. Future ones only adding what's new will hopefully take not as long.

As far as iTunes automatically recognizing where the music files are, I doubt if it will. When you get everything hooked up, one of the first things to do is go into iTunes>>Preferences>>Advanced and in the General tab see where it is looking for the music files. (The location of this screen may have a changed a bit in iTunes v.8.) You will probably need to browse and change it to the folder where your music is on your external drive.

The other thing that was different for me was that, on startup, your external drive may not "mount" or be recognized by the Macbook. When I start up my gear I just go to Finder and make sure it's connected, but Chris has provided some tools that will do this for you whenever the machine starts up. Here's a link to one such discussion.

http://www.computeraudiophile.com/automatically_mount_nas_drive_on_os_x

Now, this may happen mainly with wireless connections like mine, and I sense that you will be wired directly, so it may not be an issue for you.

Good luck and enjoy.

TheOtherTim

Thanks Tim, I will see if Chris adds any other comments. As far as itunes recognising where the files are, I assume once you have pointed it in the right direction under itunes>>preferences>>advanced it then remembers things, i.e., you don't have to do this every time you boot up?

Also, once I have my itunes library migrated to an external HD, I assume I can back up via the Mac to my back up HD? I realise my back up HD will have to be USB so I have enough ports to connect to (I only have one FW).

Phil

I just bought the WD Mybook Mirror and I love it. I got it from Newegg for 379 and it comes with two 1TB drives that are set up in RAID 1 so I have an automatic backup for everything I load on the drive. I've been dumping all my files on it today and it works like a champ. Highly recommended.

Phil, yes, once you point iTunes at your music files it will stay there until you tell it to go elsewhere. (I have seen a few posts here of some people who have been moving their files around or switching machines and thereby confusing iTunes and creating some problems where iTunes will try to revert back to a "default" folder, but I think if you take it one step at a time and point iTunes where you want it to go you should be fine.)

Yes, you should be able to back up as you mentioned. As far as software, you can try the Mac's own Time Machine software, although I've seen conflicting reports about its ease of use (unusual for an Apple product). Or there are lots of products out there (search your favorite computer store or search sites like cnet.com for shareware or freeware). Or there is this "automator" solution posted by Chris awhile back:

http://www.computeraudiophile.com/node/47?page=0%2C0

I also see that user darrenwm has a thread open right now called "Can Time Machine do this...". You might see what kind of responses he gets over time.

My solution is a bit different in that I will occasionally attach my backup drive to my Windows desktop and do my backup using a piece of Windows-based software across my network - no help to you. BTW I likewise use a USB connection for this, and it works fine.

The post before mine from bigjppop is a good one - there are some very slick mirroring products out there. This would be a no-brainer as far as the backup software tool issue. The tradeoff for me is that I like being able to keep the backup in a physically different place, like off-site or in a fire safe. Either solution is excellent and it's a matter of personal preference.

TheOtherTim

Thoughts that have been prompted by what happened to an unfortunate aquaintance of mine last week. His problem relates to his business - but it could equally apply to anyone with stored music files.

He works from a home office; has a very good NAS server in place; went on holiday; while he was away his house was broken into and ALL his personal and office computer equipment was stolen. Of course he is insured, but he had no offsite backup of his data.

This prompted a couple of thoughts, and then I found this thread - so I thought I should add my 10 cents worth! If you value your music files (and we obviously all do) don't just rely on a single storage and backup solution. Offsite storage is essential!

We read many comments here about the extremely low cost of hard drive storage in relation to using uncompressed or lossless compression files, but this comment applies equally to backing up your files.

After considering the various posts on this site and elsewhere, I'm now convinced that storage of music for actual playback should be done in an uncompressed format. However, there is no reason why an offsite backup shouldn't be done in lossless format to save disc space.

An external drive for this purpose would just need to be portable and have a USB connection. Speed wouldn't be a factor - but reliability would. As an example - a library of say 1000 cds at an average of 600meg = 600gig. Compressed in FLAC or ALAC, this would require around 250 gig of storage space. Cost

(in Aust) of a 500 gig external drive - around $120-130. Very cheap security!

Another less high tech alternative is to simply burn the compressed backup files to good-quality DVD-R discs as you download them - by far the cheapest alternative!! The above example = 60 discs. Cost - around $25!! PLus slimline cases if you want to be really exotic!! This would take up 30 cm of shelf space - I'm sure your children could spare that!!

There is another reason that backup of a main NAS server is essential. Everyone raves about RAID - but it is not 100% reliable! Sure - if you are using RAID 1 or 5, and a hard drive crashes, you can replace it, and your data will be re-created. However, if (for example) the RAID controller in your NAS dies all your data could be irretrievably lost. I was a computer consultant in a previous life, and in a business situation we would NEVER suggest that a client rely on a single NAS backup.

I am in the very early stages of building my first computer-based music system, and haven't decided on a final system yet. However - I have decided on my backup strategy! And yes - confession time - I once lost 15 months of business records due to a failure to have a reliable backup strategy in place. So now I am a little paranoid about it - but just because you're paranoid doesn't mean it won't happen!!

1. 4 gig NAS server - 4x1 terabyte drives in RAID 1 - music storage 2 terabytes. Files in uncompressed

format - WAV or AIFF.

This will be placed in my home office, well away from my listening room.

2. 1 terabyte internal hard drive on my business computer.

This computer will form part of my network, which will include my music computer, and will be used for downloading files, converting files, burning cds when necessary and general maintenance of the music database. My downloaded music files will be placed straight onto this drive before they are converted, and this drive will form the first line of backup.

3. 1 terabyte external USB drive.

This will form my offsite backup.

Overkill?? Maybe - but I don't think so! A cost of around $AU250 for the external drive, and $160 for the internal will give me complete peace of mind. Plus, of course, I will use 2 and 3 for backup for my business backups anyway!!

Edward CR

So Here I thought I was squared away with a RAID 1 array and now I have to buy something else! Boy can my hobbies get away from me.

I really like both of the above comments on offsite storage and I must admit, it was something that never crossed my mind. I think I'll find a 1 TB external and just dump everything on that and stick it in my safe at work. You can never be too careful!

Thanks for the suggestions!

Gosh chaps, some fantastic feedback and advice from you all here - thanks so much. A lot of food for thought. But at my stage of involvement, maybe I don't need to get too complicated. I am still erring towards the idea of two 1 Tb stand alone drives.

At the end of the day, the vast majority of my itunes library will be my own CD collection plus maybe the odd track bought off the itunes store (AAC, shock, horror, but more about that in another post I am about to make which will set the cat amongst the pigeons). This means I do still have the original CDs as a second back up, although obviously there is the time/effort in ripping them.

One final Q - is migrating the itunes library to an external drive straightforward and is all the album artwork carried over, etc?

Thanks

Phil

You're quite welcome Phil. I started as a complete nooby on my own server project awhile back, never having possessed an iPod much less a Mac, and the folks on this site were indispensible. I just hope to give back a little, and also hope I don't lead anyone down the wrong path! Not too long ago I finally got my basic setup hooked into my home A/V rig, and when I fired up everything with screen sharing/mirroring going between a couple of Macs and the big TV, using coverflow in iTunes, pulling uncompressed music from the remote hard drive, I got a definite "wow, that's very cool" from my wife who is the more sane and level headed one about these types of silly projects. Fed my fragile male ego quite nicely, thank you very much.

On moving your iTunes music, I have a couple of quick comments and then a cop-out. First, I might warn you on some terminology that I found to be very confusing. I even asked Chris about it in a post, since I know he's a big fan of "no such thing as a stupid question". The way I understand it, if you get really technical about it, there is a difference between your actual iTunes "library file" and your library of music. There is an actual library file (or couple of files) that resides on your computer that contains most of the informational data about your music - where they are, what they are, most of the tagging, etc.. Then, separately, there are your actual music files which are the big aiff files that are the songs themselves. So, I try to make a point in discussions about this topic to refer to either the iTunes library file or my music files. The thing being, people can discuss moving their iTunes library file from one computer to another, or, one can talk about moving their music files from one hard drive to another. Anyway, I don't mean to be talking down to you here, just thought you should know and I found it very confusing at first. And by the way, if anyone thinks I'm completely wrong here, please chime in.

So, to your question, if you are keeping iTunes itself (i.e. the library file) on the same computer, and if you just copy your music files to the new drive and point iTunes there, you should be fine. However, here is where I cop out. I would suggest searching this site and googling others for methodology to make sure your situation is taken care of. For example, there are ways to use the iTunes "Consolidate Library" function in the Advanced menu to copy the music files. And you'll probably stumble across some articles about where and whether all of the tagging/artwork/metadata are stored (I believe most is kept in the library file but some, like some artwork, can be attached to the music file itself. It can depend on where the original rip or download came from.) Don't mean to add any stress or frustration here, but thought a little digging could be helpful to you. I think the simple move you have planned should go smoothly.

TheOtherTim

Tim,

Thanks for all this - I need to go through it in detail - will get back to you. Your help is hugely appreciated!

Phil

A couple of questions for the group. I have a Mac Mini in my audio rack and I'm doing a wired connection to the audio system (Toslink now, maybe USB in the future). But the music library needs to reside on an external hard drive or drives. A couple of questions:

1. Can these external drives be on the wireless network and communicate wirelessly with the Mini, or is it better to get USB or FW drives and connect them directly to the Mini? (I know I can do it wirelessly, but might it compromise performance in any way?)

2. If I start with, say, a 1TB external drive (plus a backup), but find that I need more than 1TB a year from now, will I be able to add an additional drive and have the iTunes library distributed across two drives? I hate to think that I'll have to eat the current investment in storage if I outgrow it.

By the way, I appreciate the discussion above about offsite storage as it's something I've been thinking about. I hope there will be some elegant new solutions for this in the next couple of years. For now, it seems like a getting a USB drive that will fit nicely into a safe deposit box is a good strategy. But I'd rather back it all up online somewhere, as time-consuming and expensive as that's likely to be.

Dan

Dan,

You are wrestling with some of the same issues as me - with my itunes library growing, I am going to need extra storage and views on this differ. Some say compress music more, some say get a bigger HD in the Macbook, some say external storage. I think the latter is the way I will go and I believe (to answer your Q1) you can either hard wire it or go wireless - I don't think the wireless will impair sound quality. I will probably do it one step at a time - put my music on an external hard wired HD first and then maybe go wireless.

One thing to consider. I have just bought and set up an Airport Express which is connected to my sound system via toslink. This means Macbook can be mobile. You could do same and put your Mac somewhere else with wired external HD - this is what I plan to do. Airport Express gives as bit perfect sound as a directly wired Mac to my ears (and technical tests also show this).

As for hooking up more than one external HD, not sure on this one. Mind you, 1 TB is a lot of storage even with lossless!

Phil

I see two reasons for keeping the Mini hard wired to the audio system:

1. The Airport Express won't pass 24/96, but the Mini will. I don't have any of those files yet, but I expect I will.

2. USB out. Although I find USB inferior to Toslink with my current DAC, I know there are DACs that do USB asynchronously and the word is, they are great. Something to look forward to.

By the way, Toslink out sounds best to me when Audio Midi is set to 16/44. If I have 24/96 files, I'll have to change the Audio Midi setting just for those files. What a pain!

Dan

"If I have 24/96 files, I'll have to change the Audio Midi setting just for those files. What a pain!"

Right now yes. Soon ... :-)

That's good to know, Chris -- thanks. I was hoping you would also weigh in on my questions about storage strategy.

Hi Dan - I finally got around to reading this one again and answering your original storage questions :-)

1. Can these external drives be on the wireless network and communicate wirelessly with the Mini, or is it better to get USB or FW drives and connect them directly to the Mini? (I know I can do it wirelessly, but might it compromise performance in any way?)

There won't be a sonic performance compromise. For example there is no chance the bass will be muddy or anything like that. If your wireless network is not up to snuff then you could experience drop-outs. I use all 802.11n devices and my network hasn't had a drop-out since I switched to all 802.11n. Remember one device running 802.11 B/G/A will slow your whole network down to that speed.

2. If I start with, say, a 1TB external drive (plus a backup), but find that I need more than 1TB a year from now, will I be able to add an additional drive and have the iTunes library distributed across two drives? I hate to think that I'll have to eat the current investment in storage if I outgrow it.

This is a tough call. Yes, you can span multiple drives with an iTunes library. But, you lose some functionality like the Library Consolidation feature. A good option for your is this - Pick up the 1 TB drive now. If you out grow the drive pick up a multiple disk enclosure that you can copy your music to and reuse your existing drive inside the enclosure. If you're not into that type of this, you would have to remove the external drive etc..., then get a multiple drive enclosure now and only put a 1 TB hard drive in it now. When the time comes you can expand to two. Many options, no right answers. Let me know if you are leaning one way or the other and we can dig into the details.

By the way, I appreciate the discussion above about offsite storage as it's something I've been thinking about. I hope there will be some elegant new solutions for this in the next couple of years. For now, it seems like a getting a USB drive that will fit nicely into a safe deposit box is a good strategy. But I'd rather back it all up online somewhere, as time-consuming and expensive as that's likely to be.

The online options are tough right now. Amazon's S3 is OK, but spendy. The Thecus 5200B Pro can replicate to another Thecus unit over a network. I asked about replicating over the Internet, but they don't recommend it. My next storage move will be picking up an additional 5200B Pro to replicate all my music on a schedule.

Chris (or anyone),

Must be getting old because I can't remember if I ever specifically asked this question or if I got an answer!

With the storgae on my Macbook decreasing rapidly and a renewed interest in trying new music (yes Chris, just ordered a Pearl Jam album - the one that has a cover with what looks like a Llama nibbling a fence?) I need more storage and this means an external HD. Two key questions:

1. Is there a post on this site (or recommended post elsewhere on web) that gives clear step-by-step instructions on how to move a music library to external HD and how to then make sure itunes looks there and automatically stores further imported music there?

2. Will back-up via time machine on an existing external HD automatically back up the contents of another HD?

I noticed another post on quiet HDs. I have a 1 TB Western Digital HD for back up - seems reasonably quiet to me. Any better recommendations?

Thanks

Phil

Phil,

Here's the official apple version of how to do it:

http://support.apple.com/kb/HT1449

Basically, there's an iTunes preferences option where it allows the change to a new music file. You change your music file address to this new location. There are no files in it at this point. The process to copy the files from the original location to the new location is managed by iTunes. When this is complete, you consolidate the library, to get all the playcounts and W.H.Y. updated. Then you trash your old music file.

I'll be going through this procedure very shortly, since I'll be using my new drobo for storage soon. I chose this option, with 4 WD Green power 500GB drives, giving 1.4TB of usable space, because of the extra protection against disk failure. One disk that fails will not loose data, and a disk can be hot swapped to up the available storage. A drobo can adddress up to 16GB of storage, and is compatible with 1.5TB drives. Looks to me quite future proof.

I'll let you know how I get on with it, if you don't beat me to it.

cheers

Peter

Thanks Peter, I had also come across this article. I had also found some others which seemed to say basically the same thing, but in a much more complicated fashion! That is why I was asking advice - I was getting a little confused.

One question. This procedure moves the music files, but does it move the supporting library info such as track listings, artwork, etc? I have a feeling this still stays on the HD of the Mac. Assuming it does, I assume itunes has no problem correlating the two?

This must be an issue which most folk subscribing to this Forum have to address at some point. When I first started ripping CDs, there was a very useful note/guideline produced by Chris which covered the basics, which preferences to select, critical points like using error correction, etc. The issue of moving itunes to an external HD could do with a similar note if someone has the time to produce one?

Phil

Phil,

I've migrated once before, but that was PC -> Mac, I can't remember, now, how I did that. At that time I had no lossless files, I think I copied the iTunes music file to the windows compatible formatted hard drive, then copied to the internal hard drive of the macmini, then reformatted the drive then recopied. I won't need to do that this time.

The iTunes library file always stays in the same place, iirc. It's an .xml file with info about each song in your iTunes music file.

cheers

Peter

This is a tough call. Yes, you can span multiple drives with an iTunes library. But, you lose some functionality like the Library Consolidation feature. A good option for your is this - Pick up the 1 TB drive now. If you out grow the drive pick up a multiple disk enclosure that you can copy your music to and reuse your existing drive inside the enclosure. If you're not into that type of this, you would have to remove the external drive etc..., then get a multiple drive enclosure now and only put a 1 TB hard drive in it now. When the time comes you can expand to two. Many options, no right answers. Let me know if you are leaning one way or the other and we can dig into the details.

Drobo is appealing for just these reasons, but I do appreciate that I'd need to use it as DAS, which means it will sit in the room with the stereo, and that it's proprietary. DAS because, from what I have read, Drobo's NAS performance sucks. Getting a single 1TB drive to start makes sense, except for the lack of backup. Still undecided about whether I will go ALAC or AIFF (current small library is a mix), which will obviously affect my storage needs.