Storage

Caution !

It is common to obtain a statement of the storage duration for a digital video recorder. However, a statement of the storage duration is meaningless UNLESS the following parameters for the recorded images are also specified:

• Resolution in pixels of the recorded video images, such as 704x480 (4CIF), 640x480 (VGA), 352x240 (1CIF), 320x240 (QVGA), 172x120 (QCIF), or 160x120 (QQVGA)
• Clarity of the recorded video images, such as very high, high, medium, low or very low, but since these terms are poorly defined, it should include the data rate in bits per second for a video stream, or the number of bytes per frame, and the type of compression used, as wavelets, motion JPEG, MPEG-2 or MPEG-4
• Frame rate for the images recorded for each given camera with specified resolution and clarity
• Continuous or intermittant (event-based) recording

Introduction

Any system that records a natural (CCTV) video stream with high quality for a long period of time requires a large amount of storage. Any system that records a large number of natural video streams with high quality for a long period of time requires a vast amount of storage. Therefore, any system that handles a large number of cameras and/or records for a long period of time must have the ability to manage a huge amount of storage.

How much storage is that? That question is answered here.

The Boundless Security System™ with Boundless' Storage Operating System™ uniquely provides a reliable, fault-tolerant, redundant solution with vast storage capacity at low cost.

The Boundless Security System™ also uniquely provides the means to optimize the capture of each camera. It provides multiple digital video streams per camera simultaneously with different resolutions, frame rates, quanitizations and data rates to satisfy the competing needs for image quality, storage duration, communications bandwidth, and video display for archival searching, investigations, monitoring and first-responders.

The Rule of Ones™

We developed a Rule of Thumb, the Rule of Ones™ to help estimate how much storage is required for digital video recording, regardless of the technology used:

The new math for storage is: 1 x 1 x 1 = 1

1 camera recorded continuously x 1 Mbps x 1/4 of a year = 1 TB (terabyte, a trillion bytes, 1024 gigabytes) of storage

In other words:

1 camera x

1 million bits/second x

60 seconds/minute x 60 minutes/hour x 24 hours/day x 365/4 days divided by 8 bits/byte =

982,800 million bytes = 1 TB (approx)

We can easily count the number of cameras we're recording. We can easily choose the amount of time we'd like to keep the recordings. The hard item to estimate is the data rate, which is subject to many factors. But with this Rule of Thumb, we can quickly determine, for example, that a lot of video must be lost by a 16-channel DVR that has only 160 GB of storage but claims a recording duration of 30 days, for which we would need 16 x 1 x 1/3 x 1 TB = 5.3 TB to record all cameras continuously at full resolution, high clarity and medium frame rate.

Example Storage Numbers

Using the Rule of Ones™, one can easily compute the storage requirements for:

An office building with a 16-channel DVR with every camera recorded continuously at 1/4 Mbps (e.g., 1 fps at 640x480 with high sharpness) for 2 weeks = 16 x 1/4 x 2/13 x 1 TB = 0.6 TB = 600 GB

A government building with a 16-channel DVR with every camera recorded continuously at 1 Mbps for 1 month = 16 x 1 x 1/3 x 1 TB = 5.3 TB

A large airport with 1,000 cameras recorded continuously at 1 Mbps for 1 year = 1,000 x 1 x 4 x 1 TB = 4 PB

While most of these figures assume a constant 1 Mbps/camera, clearly, a lot of storage can be required in large systems.

Recording Movies

Before we estimate how much storage a digital video recorder needs, we need to determine what data rate to use with the Rule of Ones™.

A movie DVD, with 720 pixels/line x 480 lines x 30 frames/second x 24 bits/pixel needs about 6 Mbps of compressed video. The amount of raw video is 249 Mbps, so this requires compression of 249/6 = 41.5.

So, using the Rule of Ones™, if we record one movie at 6 Mbps (instead of 1 Mbps) for 1/4 of a year, we need 6 TB of storage.

A movie DVD has scenes that change rapidly, which requires a lot more data than a security camera that sees a relatively steady scene. However, the images in a movie are filtered and compressed with human help to reduce the data rate. We can't do any such filtering of a security camera because we lose sharpness. We must operate continuously in real time, so we can't have human help to compress the images.

As a result, the data rates are similar, 6 Mbps, for similar compression technology, image quality and amount of motion if a high-resolution CCTV camera is used.

Recording Surveillance Video

Using the Rule of Ones™, we can rapidly estimate the amount of storage required by a DVR to record 16 cameras continuously at full resolution, very high clarity and full frame rate. The amount of storage required for one month's storage is:

16 cameras (instead of 1) x 6 Mbps (instead of 1 Mbps) x (1/3 x 1/4) year = 64 TB

However, the maximum amount of internal storage that one can buy for a DVR is only 4 TB, and usually is much less. In other words, using these parameters, a DVR with 4 TB could only store video for 4/64 of a month = 2 days.

It may be possible to attach a Network Attached Storage (NAS) unit to a DVR to increase the amount of storage. However, the price for 64 TB of such disk storage is much higher than using internal storage.

Many systems simply discard information, such as by reducing resolution below the resolution needed to capture all the information from a camera, or by dropping frames. "Compression" factors of 1,000 are more are claimed for some systems, when in fact, they are simply throwing information away.

There's a close relationship between frame rate and data rate. If we reduce the frame rate from 30 frames/second to 5 frames/second, we reduce the amount of storage to about 1/6. We would still need 10 TB -- three times more than the 4 TB one can buy, but a lot less than 64 TB.

There's also a close relationship between resolution and data rate, although it's not linear because most of the edges are still present in the lower-resolution image. Sometimes we can reduce the resolution slightly, such as from 720x480 to 640x480, without losing information. Even though we're handling fewer pixels, no video information is lost if the TVL (TV lines) of resolution of the CCTV camera is moderate. This reduces the amount of storage slightly, by about 11%, so now we "only" need 9 TB.

The Storage Operating System™ in the Boundless Security System™ can handle up to 1 PB (1,000 TB) of disk storage per LAN. In addition, the system is fault-tolerant and redundant to compensate for transient and permanent network and server problems. Furthermore, its Multi-Servers, which record and forward video data under remote control of its CamServers, are optimized for storing video. As a result, its Multi-Servers use inexpensive, low speed, low power, high density disks. This provides low power dissipation and low cost, which can't be provided in general purpose, Network Attached Storage devices that are intended for transaction processing because they would be too slow.

Factors that Affect the Data Rate of Compressed Video

Other factors that affect the data rate are the type of compression, the amount of motion, and the frequency of key frames. Since the security video is recorded continuously, the compression must run in real time. In contrast, a movie can be processed repeatedly and have manual intervention to improve the compression and reduce the data rate, but we can't do this for security video.

We use MPEG-4 VBR/CQ, which is variable bit rate (also known as constant quality), for the highest resolution (4CIF or VGA) video streams, which are used for archival searches and investigations. The quality of the image is constant regardless of whether there's a little motion or a lot of motion. If a camera only sees motion for a part of a day, such as the 12 hours of daylight, then the amount of storage required may reduce by about a factor of 2 if the camera images at night are not noisy.

The frequency of key frames has a significant impact on the amount of data required when there's not much motion. Systems that lose packets, such as those that use multi-casting protocols, require redundancy within the video streams to compensate, and a higher frequency of key frames, increasing the data rate for a given scene. The Boundless Security System™ assures accurate recording, enabling relatively infrequent key frames to be used in the highest quality streams, reducing the data rate and storage requirements.

With the Boundless Security System™, one can also reduce the amount of storage required by reducing the number of pixels per line that is digitized, according to the number of lines of resolution of the camera used. There's no need to record 720 pixels per line, or even 480 lines, if the CCTV camera only has 330 TV lines (TVL) of resolution. There's no need to record 720 pixels per line even with the 450 TV lines of resolution for some of the newest, "high resolution" CCTV cameras.

Another way of reducing the data rate in the Boundless Security System™ is to increase the quantization, Q, of the compression for a given digital video stream for a given camera. However, the larger Q is, the more data and image quality are thrown away. Other systems use different terminology, such as low, medium, high, very high, and best quality, for a given resolution.

Some vendors claim a video compression factor that may be in the 100's or even 1,000. Usually, most of the so-called "compression" is simply due to the fact that they are throwing away data, not compressing it. The only way to know for sure how different systems perform is to capture a video test sequence onto a DVD with very high quality, and to run it through several systems. This ensures that the tests can be repeated and that all systems have the same video input.

Many consider MPEG-4 to be the best way to compress surveillance video. One of the reasons is that it gives a several-to-one improvement in data rate compared to compression techniques, such as wavelets and motion JPEG, that compress each frame individually, rather than taking advantage of similarities between frames to reduce the amount of data required. Thus for a given data rate, MPEG-4 produces sharper images than wavelets or motion JPEG.

Gating Recording According to Motion

Many DVR's gate recording based upon the presence of motion to reduce storage requirements. They generally store a brief period of video before and after the motion. The problem is that it is difficult to ensure that the motion detectors, or motion detection software, capture all events. Some courts of law may not allow video that has not been recorded continuously if one cannot guarantee that all events were recorded. If motion is continuous, then there's no advantage gained from this technique.

The Boundless Security System™ records continuously but achieves a similar reduction in storage requirements as if recording were gated according to motion. The Boundless Security System™ uses the variable bit rate / constant quality version of MPEG-4 for the archival searching and investigations streams. It captures changes from one frame to the next, automatically reducing the data rate as the amount of motion decreases. This ensures that no events are missed.