[Hidden-tech] Online backup for video production

[Tim Boudreau]

For what it's worth, if you're technically inclined:  There is a filesystem
called ZFS, originally developed for Sun Microsystems' Solaris (I used to
work for Sun), which is freely available for Linux (and Solaris, and maybe
Mac OS, but definitely not Windows), which was designed for
enterprise-class data-loss-free storage with redundancy and a bunch of
other nice features (the demo used to be a server streaming video to the
audience onstage while the guy doing the demo yanked hard drives out of the
machine and skidded them across the stage without the server missing a
beat) like being able to replace drives without a shutdown - i.e. drives
can fail-in-place with no loss of storage (assuming enough redundancy) and
you can replace them at your leisure, rather than it being a crisis.

It costs nothing but time and expertise to set it up, can handle larger
filesystems than anything else out there (it's a 128 bit filesystem).  You
don't need special RAID hardware (RAID firmware tends to hide what actually
went wrong when something does), and can use it on commodity hardware.

One of its features is "snapshots" - i.e. taking a picture of the state of
the filesystem at the current moment, which you can roll back to at any
point.  You can then take those snapshots and "zfs send" them to another
machine over the network/internet that also has ZFS - it uses ssh as a
transport, so it's secure.  The first snapshot is, of course sending
everything.  But *future snapshots* will only move the bits that have
changed.  You can set up a script to periodically do that, say, every day
at 3AM.  It's a bulletproof, enterprise-class backup strategy - you just
need two machines with equivalent amounts of storage, and a network
connection between them to do it.

This is all freely available stuff (I use this approach at home with an 8TB
Linux router/NAS machine I built, so I put my money where my mouth is).  It
does require a bit of technical expertise to set up, but I highly recommend
it - and unlike commercial solutions, you're not paying anyone to hold your
backups hostage.

-Tim


On Sun, Oct 12, 2014 at 4:12 PM, Al Williams <palwilliams at northamptontv.org>
wrote:

>
>
>
>
> Greetings,
>
> I would echo Dan. We have about 80TB of information on site at a time. I
> would say about 30TB of that is simply finished, exported digital video and
> and audio projects in a compressed state. So none of that includes any raw.
>
> There is no online solution of which I am aware  that can manage those
> kinds of needs at any reasonable price.. We use RAID1 (which is really a
> minimum security level) drives as well as a Drobo array which is RAID5. The
> RAID1 drives need to be spun up at least once a month to avoid freezing.
> And, of course, we need to regularly purchase and swap out drives as they
> fail.
>
> Best,
>
> Al
>
> On Sun, Oct 12, 2014 at 12:31 PM, Stefan Gonick <
> stefan at databasewebworks.com> wrote:
>
>>
>>
>>
>>
>>  Hey Everyone,
>>
>> I did some more research and found a service that looks good: *SOS
>> Online Backup*.
>> They have unlimited storage, fast upload, backs up open files, and they
>> got an
>> Editor's Choice award at PC Mag. In fact, PC Mag has a link in their
>> review that gets
>> you 25% off.
>>
>>  http://www.pcmag.com/article2/0,2817,2371307,00.asp
>>
>> It looks good!
>> Stefan
>>
>> At 08:28 AM 10/12/2014, you wrote:
>>
>>
>>
>>
>> The biggest issue that you're going to have is with having enough
>> upstream bandwidth in order to back up all that you need to overnight.
>> Because you're not going to want to have cloud backup running during the
>> day because it will hinder your productivity. So that means scheduling it
>> to start after business hours.  And if you have several workstations to
>> back up, or have the data from several users on one server, that's going to
>> take a lot of bandwidth. What's the upstream bandwidth of your Internet
>> connection currently?
>>
>> Backblaze is my favorite cloud backup service. $5/month for unlimited
>> data.
>>
>> Previously CrashPlan was my favorite. But I found that their Java-based
>> software is very slow and becomes downright  problematic when backing up a
>> large quantity of files. For example,when my clients would have 4 GB or
>> more to be packed up the software with frequently lose track of what it was
>> doing. And CrashPlan support couldn't always fix it
>>
>> That's when I switched to backblaze at my home office and at several
>> clients offices and have been very happy.
>>
>>
>> *Chris Hart *
>>     
>> * Computer Support & Technology Consulting *
>>
>> *        for Connecticut and Western Massachusetts *
>>
>> *            Tel: 860-291-9393 <860-291-9393> *
>>
>> *                 http://www.MyMacTech.com <http://www.mymactech.com/> *
>> On Oct 11, 2014, at 6:27 PM, David Korpiewski <davidk at cs.umass.edu>
>> wrote:
>>
>>
>>
>>
>> Crashplan.com   It has unlimited data storage for 5 bucks a month.
>> There is a business plan available too but I don't know how much that
>> costs.  I know a few businesses that signed up for the "personal" or
>> "family plan" and are backing up several hundreds of gigabytes.
>>
>>
>>
>> On 10/11/2014 8:41 AM, Sven Kielhorn wrote:
>>
>>
>>
>>
>>
>> Good morning all,
>>
>> As a video production company we have rather large file and project back
>> ups needs. A conversation about cloud backup quickly includes several TBs
>> per work station. Currently we use a system which includes external drives
>> with RAID arrays.
>>
>> Does anyone have any thoughts on affordable online backup up solutions?
>> ---
>> thank you
>>
>> Sven Kielhorn
>>
>> Creative & Marketing Director
>>
>> eTown Videos
>>
>> P 413-324-1130
>>
>> www.etownvideos.com
>>
>>
>>
>>
>> _______________________________________________
>> Hidden-discuss mailing list - home page:http://www.hidden-tech.net
>> Hidden-discuss at lists.hidden-tech.net
>>
>> You are receiving this because you are on the Hidden-Tech Discussion
>> list.
>> If you would like to change your list preferences, Go to the
>> Members
>> page on the Hidden Tech Web site.
>> http://www.hidden-tech.net/members
>>
>>
>> --
>>
>> *David Korpiewski*
>> Software Specialist I
>> CSCF - Computer Science Computing Facility
>> University of Massachusetts Amherst
>> 413-545-4319
>> _______________________________________________
>> Hidden-discuss mailing list - home page: http://www.hidden-tech.net
>>  Hidden-discuss at lists.hidden-tech.net
>>
>> You are receiving this because you are on the Hidden-Tech Discussion list.
>> If you would like to change your list preferences, Go to the Members
>> page on the Hidden Tech Web site.
>>  http://www.hidden-tech.net/members
>>
>> _______________________________________________
>> Hidden-discuss mailing list - home page: http://www.hidden-tech.net
>> Hidden-discuss at lists.hidden-tech.net
>>
>> You are receiving this because you are on the Hidden-Tech Discussion list.
>> If you would like to change your list preferences, Go to the Members
>> page on the Hidden Tech Web site.
>>  http://www.hidden-tech.net/members
>>
>>  =====================================================
>> Stefan Gonick, MS Computer Science
>> Database WebWorks: Dynamic web sites through database integration
>>  http://www.DatabaseWebWorks.com
>>  <http://www.databasewebworks.com/>
>>
>> _______________________________________________
>> Hidden-discuss mailing list - home page: http://www.hidden-tech.net
>> Hidden-discuss at lists.hidden-tech.net
>>
>> You are receiving this because you are on the Hidden-Tech Discussion list.
>> If you would like to change your list preferences, Go to the Members
>> page on the Hidden Tech Web site.
>> http://www.hidden-tech.net/members
>>
>
>
>
> --
> P. Al Williams
> Executive Director
> Northampton Community Television
> www.northamptontv.org
> www.paradisecitypress.org
> 413-587-3550
>
> _______________________________________________
> Hidden-discuss mailing list - home page: http://www.hidden-tech.net
> Hidden-discuss at lists.hidden-tech.net
>
> You are receiving this because you are on the Hidden-Tech Discussion list.
> If you would like to change your list preferences, Go to the Members
> page on the Hidden Tech Web site.
> http://www.hidden-tech.net/members
>



-- 
http://timboudreau.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.hidden-tech.net/pipermail/hidden-discuss/attachments/20141014/7ab0511d/attachment.html