We process more data now than ever before and as video resolutions increase,
the requirement for more space will continue to grow exponentially.
Technology is helping us to adjust the way we store, operate and manage our
information in order to keep up with the rapid growth of data. Some solutions
include clever software like CAT VD or Dalet to help us index, categorise and
search the content. Storage appliances are also becoming more advanced as
our requirements change.
Adopting a more holistic approach to data storage is also important to manage
sustainable (often collaborative) workflows and keep long-term storage costs to
a minimum.
When it comes to storage here are few things to consider:
1. More space isn’t always the answer, our data normally magically grows to fill
up any unallocated terabytes.
2. Professional storage is not the same as consumer storage. When it comes to
performance, reliability and data integrity, professional products are worth the
upfront investment. If you don’t believe that, just ask anyone that has had to
pay for data recovery.
3. Define the purpose of the storage before you buy – there are thousands of
alternatives out there. Choose what you need according to your priorities. Think
seriously about a tiered storage approach and archiving, it’s a very effective way
to “recycle’ your existing storage.
4. What after-sales support exists in South Africa for the product? Is it supplied
and installed by a reputable reseller and backed by an established distributor?
How long would it take to bring the system back online if there was a problem?
So let’s look at how one might introduce a tiered structure with different storage
systems, each with a unique purpose. The idea is that content is moved to less
expensive storage over time in order to free up space for current project
requirements.
Before we go any further it’s important to clarify that archiving is different to
backing up. Backing up entails copying information and protecting against data
loss. Archiving is simply moving data to a more suitable long-term storage
medium.
Online storage is typically a really fast RAID (Redundant Array of Independent
Discs) where content for current projects is stored, compiled and edited. Single
users will normally connect with Thunderbolt or SAS, whereas a team of users
needing to access the content concurrently will probably access the storage via
Fibre Channel or 10GBe.
TBOX is a great example of an online storage appliance that offers up to 16
users concurrent access to the storage, support for all major editing workflows
(including AVID) and enough performance to edit 2K footage. A single box ships
with up to 64TB of space and is scalable to 320TB by adding expansion units.
TBOX also includes a basic MAM called ProjectStore that allows users to index
and search the content, create low res proxy videos and archive projects to
alternative storage on the network.
This is where near line storage becomes useful – it’s essentially a slower type of
disc-based storage that is also accessible to users on the network and is
significantly more cost effective per gigabyte than online storage.
Rather than continually expanding the online storage volume whenever it runs
out of space, non-current content is moved over to the near line volume (where
it is still available to users) and the online storage can effectively be redeployed
for new projects. The easiest way for all users to have access to near line
storage is via a network, which is why many companies opt for Network
Attached Storage (NAS). Thecus manufacture an excellent range of NAS
products, some of which have 10GBe capabilities and can be deployed as very
basic online storage units.
When it comes to long-term storage of media, the best solution is still LTO tape.
Although it’s cost effective to store old projects on portable hard drives, there
are several down sides to this approach. Hard drives have a short shelf life and
many will not detect after standing on a shelf for five years, not to mention that
it’s also very difficult to keep track of what has been saved on each drive.
LTO tapes are a little different to hard drives because they require a tape drive
for reading and writing data and store content in .TAR or .LTFS file formats.
LTO6 tapes have a capacity of 3TB and have a shelf life of at least 25 years,
they are ideal for keeping data safe for long periods of time. This is ideal for raw
footage and completed projects that are not needed on a regular basis. Good
archiving policies will free up volumes of space in online and near line storage
arrays.
The process of archiving to tape is made easier with software like Bru PE.
Xendata is an advanced software solution that gives LTO archives the
functionality of disc based storage which means that all content can be
managed by MAM software, archived to tape, easily searched or retrieved
almost instantly.
