Amazon Cloud Drive vs. Amazon Glacier

[burnspbesq]

The unlimited tier of Amazon Cloud Drive is going away, as i learned to my chagrin when I logged on this morning to upload some recently acquired music. The new pricing is $59.99 per terabyte.

 

I did a quick price comparison vs. Amazon Glacier, and it looks like Cloud Drive is less expensive up to 3 TB, but once you cross that line you get hit for an entire additional terabyte. Glacier's pricing is by gigabyte ($0.004/GB), so it probably has a cost advantage between n.0 and n.5 TB.

 

Curious to hear users' impressions of the Glacier interface.

[plissken]

From the Wiki:

 

Glacier is part of the Amazon Web Services suite of cloud computing services, and is designed for long-term storage of data that is infrequently accessed and for which retrieval latency times of 3 to 5 hours are acceptable. Storage costs are a consistent $0.004 per gigabyte per month, which is substantially cheaper than Amazon's own Simple Storage Service (S3).^[3]

[rando]

The next step is starting a server farm of our very own?  I will begin compiling a list of suitably smirk inducing acronyms for this venture.  

 

All we need now is the right compression algorithm and "The Box"

 

[mjt5282]

rather than wait for amazon to change it's collective mind, I have cancelled my Amazon Cloud account and uploaded my data to Backblaze B2. It works perfectly with rclone (important for my pre-written scripts and workflow) and is affordable for offsite cloud cold storage.  It satisfies my need for off-site storage of music related files and even now is a repository for my Roon library backup offsite.  Crucially, it is focused on cloud backup and is unlikely to change pricing policies midstream ala ACD. It also has additional snapshotting functionality, which ACD lacked.

[ismewor]

Agree, B2 is the cheapest for now.

[Mike Pinkerton]

Thanks for the pointer, backblaze personal looks like a perfect replacement for Amazon Cloud Drive! And $60/year!

[astrotoy]

Thanks for the info on B2.  Just so I am getting the info computed correctly, the website says $.005 per Gigabyte per month for storage, and $.02 per Gigabyte for downloading.

 

Since I have 40TB of music files, I calculate it would cost

 

40000 GB x $.02/GB to download or = $800.

40000 GB x $.005/GB to store or = $200 per month or $2400/year. 

 

Am I off in my calculation?    Currently, 8TB external HD are about $200 each, or 40TB would be 5-6 drives or $1000 to $1200, or a little more than the cost to download or six months or less of the storage costs.  My bank gives me two large safe deposit boxes for free, but normally they are about $100/year.

 

Larry

[Dan Gravell]

Well it's 0.004/GB, so 20% less.

 

Glacier is not designed to be accessed often. You would only have to download normally under exceptional circumstances.

 

S3 is the service you should be comparing to, but then this isn't as "consumer friendly" either.

[mjt5282]

I have about 2.2TB stored at BackBlaze B2, for about $11/month.  Astroboy has at least 20X the amount of storage, so if you are looking for offsite cloud storage, it would be very expensive.  You are in the territory of "build your own backup server" because of the amount of files required for backup.  A ~90 TB home file server(s) is within the realm of possibility now, considering 10TB drives are shipping finally.  The disk and RAM requirements would be considerable - do you have a RoonServer pointed at the storage? 

[Mike Pinkerton]

Why consider B2 over Backblaze Personal? The Personal service sounds a lot less expensive, unlimited storage for a fixed price, and no up/down costs.

[mjt5282]

B2 is designed to backup servers over the internet / cloud , is compatible with rclone (which I already had written unix script wrappers), and has scalable, predictable pricing. B2 has a command line interface (which I don't use, because i use rclone's API and scripting).  I believe BB Personal is for Mac/PC backup.  I store all my audio files on Freebsd ZFS.

[joelha]

On 7/7/2017 at 0:17 AM, astrotoy said:

Thanks for the info on B2.  Just so I am getting the info computed correctly, the website says $.005 per Gigabyte per month for storage, and $.02 per Gigabyte for downloading.

 

Since I have 40TB of music files, I calculate it would cost

 

40000 GB x $.02/GB to download or = $800.

40000 GB x $.005/GB to store or = $200 per month or $2400/year. 

 

Am I off in my calculation?    Currently, 8TB external HD are about $200 each, or 40TB would be 5-6 drives or $1000 to $1200, or a little more than the cost to download or six months or less of the storage costs.  My bank gives me two large safe deposit boxes for free, but normally they are about $100/year.

 

Larry

astrotoy,

 

I'm trying to understand, why wouldn't the personal Backblaze option work in your situation?

 

It seems it would be dramatically less expensive.

 

I ask because I'm also considering that option myself.

 

Joel

[astrotoy]

Thanks, this is all new to me. I read about B2 on this thread.  Not aware of Back Blaze Personal. I'll check it out.  Never thought about my own server.  CUrrently I have a couple of NAS units.  One has 12 drives, with 10 of them being 8TB drives.  So it can store a lot.  However, I still am concerned about catastrophic failure and potentially a fire in my home.  I have several sets of external drives backing up everything, including a set at the bank.  However, they are not so convenient when I add or delete or modify files.  So something cloud based would be great.  Just need to make sure that the company running the backups is reliable and doesn't go bankrupt.   Larry

[foodfiend]

It all depends on where the stored files reside. Back Blaze Personal is meant for the back-up of one PC or Mac, and not a NAS. NAS Back-up is meant to be on B2, which is costlier. The data volume is the main reason for offering a "light" version for single PCs and Macs, which are expected to have from about 500 GB to 4 TB of data.

 

I am not sure if Back Blaze will scream if they discover that you have been using a Back Blaze Personal account to back-up 40 TB of data. Technically, you can, if it all belongs to one connected computer, rather than a NAS or server. Perhaps we should look into their T&Cs carefully...

[astropuppy]

So far, very cloud company has raised their prices substantially after offering cheap prices for couple years. Remember when google drive was cheap? Anyway, Amazon's announcement was the last straw for me. I back up to drives stored at our bank and swap them out a couple times a year. For my daily and weekly backup needs; I have replaced cloud storage with an (usb3) iosafe which is water and fire proof. It can also be permanently mounted as a theft deterrent.  The backup speeds are much faster then any cloud based service ever thought of  being. 6tb was expensive, but thats it, no continuing cost.

 

[foodfiend]

I am thinking of placing another NAS at my mum's house for the off-site back-up (the NAS at home being the on-site back-up). I agree that most cloud companies have raised their prices, but I think that has to do with the cost of running their data-centres. Also, it is far easier to lure users with a cheap price and raise it later. The effort of redoing the back-up is quite a deterrent for users of large back-ups to change providers.

[agladstone]

I'm currently seeking out solutions to this problem also! 

I currently have a growing number of 2TB, 3TB, and 4TB external drives and it's becoming a pain to manage. 

I've been storing everything manually in duplicates (1 copy to drive A and 1 to drive B as an example). 

I'm trying to find the best and easiest solution to having at least 2 if not 3 copies of my entire music collection, and I would like one of those copies to be off-site. 

I am trying to decide between:

1. several large external Hard drives (4 to 6 8TB external drives - I have about 16TB's of music so 2-3 copies of 2 8TB drives). 

2. A USB or thunderbolt external Raid Box ?

3. A Synology 916Play with 2-4 8TB drives 

4. Any of the above but copy number 3 being stored in the cloud (G-Drive, Amazon Cloud) - Note: I have a G-Drive now (only with about 1TB of files stored) and I find that the upload times are sooo slow, I could not imagine how many years it would take to upload 16TB's to it!

5. As another person posted here, sounds like a better alternative to cloud storage for backup may be to purchase 2 Synology 916 NAS, on on-site and a second as backup at my brothers or a friends house? 

What are the advantages to a NAS vs. an external RAID box? 

Is a raid Box or NAS superior to several external hard drives ? (8TB external HD's are the cheapest option for certain vs a NAS or RAID box). 

Thanks for any suggestions ! 

 

[burnspbesq]

On 7/14/2017 at 11:39 AM, astropuppy said:

So far, very cloud company has raised their prices substantially after offering cheap prices for couple years. Remember when google drive was cheap? Anyway, Amazon's announcement was the last straw for me. I back up to drives stored at our bank and swap them out a couple times a year. For my daily and weekly backup needs; I have replaced cloud storage with an (usb3) iosafe which is water and fire proof. It can also be permanently mounted as a theft deterrent.  The backup speeds are much faster then any cloud based service ever thought of  being. 6tb was expensive, but thats it, no continuing cost.

That’s all true, but when the Canyon 2 fire got within two miles of my house while I was 400 miles away, it validated the cloud decision AFAIC.

[DaQi]

How are you loading this amount of data to a cloud service? My internet provider limits me to about 300gb per month. Furthermore, moving that data would take forever initially. So, 2tb I would have to up my limit to do it and it would still probably take a while to load it all up to the cloud. I am thinking I like the idea of drives at the bank safety deposit box as the cheapest easiest method. 3 drives  (one in the box at the bank, one current and one back-up on-site and rotated) swapped out monthly would work. 

[foodfiend]

On 8/8/2017 at 11:30 AM, agladstone said:

I'm currently seeking out solutions to this problem also! 

I currently have a growing number of 2TB, 3TB, and 4TB external drives and it's becoming a pain to manage. 

I've been storing everything manually in duplicates (1 copy to drive A and 1 to drive B as an example). 

I'm trying to find the best and easiest solution to having at least 2 if not 3 copies of my entire music collection, and I would like one of those copies to be off-site. 

I am trying to decide between:

1. several large external Hard drives (4 to 6 8TB external drives - I have about 16TB's of music so 2-3 copies of 2 8TB drives). 

2. A USB or thunderbolt external Raid Box ?

3. A Synology 916Play with 2-4 8TB drives 

4. Any of the above but copy number 3 being stored in the cloud (G-Drive, Amazon Cloud) - Note: I have a G-Drive now (only with about 1TB of files stored) and I find that the upload times are sooo slow, I could not imagine how many years it would take to upload 16TB's to it!

5. As another person posted here, sounds like a better alternative to cloud storage for backup may be to purchase 2 Synology 916 NAS, on on-site and a second as backup at my brothers or a friends house? 

What are the advantages to a NAS vs. an external RAID box? 

Is a raid Box or NAS superior to several external hard drives ? (8TB external HD's are the cheapest option for certain vs a NAS or RAID box). 

Thanks for any suggestions !

@agladstone If you are only going to back-up your music (and nothing else), then the NAS and external RAID box are going to be similar in implementation. That said, I think that the NAS implementation might be cheaper, since it is quite a hotly contested market, compared with a DAS RAID box.

 

Of your first 3 options, the first of using multiple external drives is going to be the cheapest to implement, but probably the most messy. If you format the lot as JBOD, then you risk your entire data set going up in smoke if one HDD goes belly up. If you use them as separate independent drives, then you have the problem of keeping track where each file is stored (which drive). Not ideal for a big music collection.

 

Option 3, running a NAS, is probably next up in terms of cost. NAS boxes are like small computers running as servers, and can have many uses. If you are not going to use the other functions, then it might be a waste of the NAS' capabilities. Many can enable streaming of music over your network, for example. Also good about NASes is the in-built software suite (unless you are custom-building one from scratch), which may help in your back-up functions.

 

Option 2, is a DAS RAID box. There are some cheaper DAS RAID boxes, but they are limited by the variations of RAID that they run. Many of the more expensive versions have their own hardware RAID controllers. The advantage of using a DAS, is that you needn't load your home network with all this data.

 

Considering these 3 options, I think the NAS option is the most painless and cost competitive.

 

As for the off-site back-up, you could run a NAS at another location, but the back-up speed would depend on what your ISP offers. Of course, this is less of a pain, because you can preload the NAS with your first back-up, and all it needs to do is to keep up with the changes (new added files).

 

I have yet to find a cloud provider that is not going to charge an arm and a leg for backing up TBs of data. Some may offer cheap rates, but may throttle your speed, or do not give you immediate access to the files. Others offer cheap rates to attract you, and then increase the charges once you are "hooked". I like neither, and so I would stick to a NAS for now, until the Cloud back-up solutions become more settled.

 

For the off-site back-up, of course you could also use a set of HDDs to back-up, and then lock them away in a safety deposit box, for example. The hassle is that you would have to retrieve the HDDs on a periodic basis to add to the back-up. The longer the period between fetching your drives to back-up, the more vulnerable information sits at home.

 

Hope this helps...

[agladstone]

@foodfiend Thanks for your advice! 

As you may see, my post was 8/7/17. 

I actually purchased the Synology 916+ w/ 4 8TB WD Red drives running RAID 5 around 9/1/17. 

At the moment, I’m backing the entire NAS up to a Google G-Drive using the included CloudSync SW. 

I also have two external 8TB HD’s on-site that have a duplicate copy of all the music currently on the NAS. I also have three 4TB external drives with another 11TB’s of music on them that I have not yet copied over to the NAS (I’m slowly but surely working on trying to get it all organized and “cleaned up” before putting it in the NAS). I also have a total of about 40TB’s of music on the Google G-Drive (16TB’s are the NAS backup, the other 24TB’s are a mix of what is on the three 4TB’s not yet on my NAS and then about 15TB’s worth of live music (concerts), which I’m not interested in keeping on my on-site NAS or even having a back-up copy at home of. Since I have a Google G-Suite account, I have unlimited storage at the moment (I fear this will change in the future), so I just have been storing any live concerts or bootlegs directly to the Google Drive, but I don’t really listen to that stuff often and if I lost all of it for some reason, I would honestly care less. 

I’m now ready for stage 2! I need to get a more formal on-site (or remote as in bank or relatives home) backup of my NAS using the included backup SW and I need to hurry and get the remaining in-organized 11 TBs remaining on external drives onto my NAS. (It’s been slow because it’s really time consuming! I’m slowly working on getting all my music (27 TB’s!) into a organized database  such as:

file type- 

AIFF

FLAC

DSD

SACD-ISO 

and then by genre under each file type. 

This may end up being impossible to ever finish!  

I do have the 16TB’s on my NAS already sorted and organized into this (it makes managing a large music collection much easier!! )

thanks for all of your advice and help so far! 

 

 

[foodfiend]

@agladstone Is there a reason for you to maintain separate FLAC and AIFF libraries, and separate DSD and SACD-ISO ones too? I have converted all my PCM files to AIFF, and all my DSD to DSF. Not sure if I may live to regret it, but I would like to understand your rationale.

[agladstone]

On 10/31/2017 at 6:40 AM, foodfiend said:

@agladstone Is there a reason for you to maintain separate FLAC and AIFF libraries, and separate DSD and SACD-ISO ones too? I have converted all my PCM files to AIFF, and all my DSD to DSF. Not sure if I may live to regret it, but I would like to understand your rationale.

First: 

SACD-ISO library AND DSD library:

1. I too convert all SACD ISO to Stereo (2 channel) DSF. 

* I have an Aurender N100H with an internal HDD that I use for most playback:

2. On (internal HDD) My Aurender N100, I only store the 2 channel DSF, NO SACD ISO’s.

3. At least 50%-60% if my SACD-ISO’s have both 5.1 DSD and 2.0 DSD mixes. 

Thus, at some point in the future, I believe I will have and maybe most of “us” will have 5.1 DAC’s. So, in the future I want to be able to use the original SACD ISO to create a 5.1 DSF version. 

4. I also believe it’s very important to maintain an archive (the SACD ISO) of the DSF not only for a bit perfect backup, but also as an example, I encountered a DSF album I had on my Aurender HDD the other day with a track (song) that was corrupted. I was able to go back to the SACD ISO and Re-Convert the album to stereo DSF again, and it was fixed! If I did not keep backup copies of my SACD ISO’s, I would not have been able to do this! 

Last, I also have an OPPO BDP-95 CD/SACD/Blu-Ray Player. I am able to burn copies of the SACD-ISO’s I have to dual-layer DVD-R discs and play the 5.1 version using the OPPO as the player and my Integra 11.4 Pre-Pro (it actually has 5 channels of internal Sabre 9028 DACs and plays Native direct 5.1 DSD, so it’s not what I use for 2 channel listening (I have a Mytek DAC I use for stereo), but it’s not bad at all for multi-channel 5.1 DSD. 

AIFF and FLAC:

This is honestly less clear to me and I’m not 100% sure it’s necessary! 

My Aurender plays WAV, DSF, DFF, AIFF, ALAC, FLAC, mp3, MQA, etc. (So does my DAC). 

So, I only keep one copy on my Aurender internal HDD (same as I only keep DSF on the Aurender also). 

When I first started all this “computer audio”,  about 2 years ago, I first read Chris’s Computet Audiophile ripping guide. He suggests, when ripping a CD via dbPoweramp to automatically make a WAV, FLAC, and AIFF version of all CD’s ripped so you have a true WAV copy for archive and FLAC and AIFF copies for playback, etc. 

I decided from the start to skip the WAV since I thought AIFF is very similar to WAV, but it can hold Meta Data and be tagged. 

So, for all of my ripped CD’s, I have an AIFF copy and a FLAC copy. 

As of now, I keep 2 backup copies of every CD ripped in both AIFF and FLAC, however on my Aurender’s internal, I just keep the FLAC version (takes up less HDD space). 

I do also have a lot of HiRez digital downloads from HDTracks, Quboz, Pono, etc. and some of them are FLAC only, some are AIFF only, some are ALAC onky (I’ve experimented a bit trying to decide which version of HiRez download sounds best but it’s not easy to tell much of a difference), however this an additional reason why I have separate “AIFF” and “FLAC” folders for my digital music backup copies. 

So, I’m 100% convinced that it’s very important to have both a DSD (DSF) version and the original SACD ISO version for all of my SACD / DSD backups. 

However, I’m not really sure if having both AIFF and FLAC versions really is necessary? It could be I’m just wasting Hard Drive space and that all I need is AIFF only ? (I think if I had to choose between FLAC only or AIFF only, AIFF makes more sense since it is more “true lossless” vs. FLAC, it can store Meta and Tagging and Art, and it can be converted to FLAC at any time (although it’s definitely time consuming to convert AIFF to FLAC, so it’s convenient to have both versions stored, but at the cost of almost double the required storage space). 

Please let me know your thoughts (or anyone else) in regards to my backup and storage methodology. 

I’m definitely not convinced it’s the best, just what I’ve been doing. 

Also, the synology CloudSync to Google Drive is an absolute worst case scenario backup, because it takes soo much time to upload and download to and from Google Drive, it’s not actually practical for “real time” backup vs having a local backup copy on a NAS or external HDD’s. 

I plan to purchase a second NAS to backup my primary NAS some time in the future, but since I just recently purchased my first one, I need to wait a while to save up additional funds:) 

 

 

[Mike Pinkerton]

I do the exact same thing:

- convert ISOs to DSF and hold onto the original ISO for backup and eventual re-conversion to 5.1 DSF

- convert CDs to FLAC, skip having a WAV backup

 

All of these are kept on my NAS, which I used to back up manually to Amazon Cloud, but since their pricing went through the roof, I now only keep a single offsite backup of my NAS via a hard drive I keep in my desk at work. 

[foodfiend]

@agladstone I do understand the SACD-ISO and DSF file storage. Not so convinced about the AIFF and FLAC duplication, especially when you can actually create FLAC files with zero compression. WAV files can also store meta-data, by the way, although many players do not seem to retrieve this data properly.

 

As for Synology CloudSync back-up to Google Drive, are you problems related to the CloudSync app, or has it got to do with the Google Drive end? Have no experience with back-up to Google Drive, so I am interested to hear from you.