Data Storage, Impact by the Numbers, and What It Means for Application Performance Management (APM)

I love statistics—who doesn't? Stats are a great way to identify important trends, behaviors, and market challenges—which is precisely why a recent infographic published on ReadWriteWeb about data storage grabbed my attention.

The infographic incorporates some astonishing stats on the growth of digital data, noting that in 1986, only 1 percent of data was stored digitally, growing to 94 percent by 2007. As ReadWriteWeb journalist Joe Brockmeier accurately notes, this growth is no surprise since "instead of papers, punchcards, vinyl records, and Polaroids, we now cram most of our lives into digital storage of some kind." It's true—everything from family photos to company IP is now stored digitally.

While digital storage certainly helps people and enterprises streamline access and clean out file drawers, the overwhelming amount of data is actually creating some unforeseen issues, which are becoming more and more difficult to solve. In fact, the infographic shows that 47 percent of enterprise survey respondents ranked data growth in their top three challenges, according to leading analyst firm Gartner.

The problem with handling such volumes of data—frequently referred to as Big Data—is that IT staff have to deal with corresponding increases in traffic, transactions, and overall complexity. This, combined with new technologies like cloud computing and virtualization, directly impact enterprise applications, especially with regards to application performance management (APM). Simply put, all this adds up to applications that are prone to latency, task incompletion, data loss, and a whole host of other application-performance issues.

Luckily, there is a solution. Passive, network-based APM solutions are able to circumvent Big Data-induced issues through superior visibility and proactive early warning. With such a solution in place, companies can more easily handle the IT complexities that result from growing data volumes to prevent and address minor issues before they become major, business-crippling catastrophes.

Knowing there's a solution available may ease companies concerns about data growth, but looking further into the infographic, it's astonishing to see just how big Big Data really is. According to the chart, some interesting points jump out:

• Enterprises store approximately 6 exabytes of data while consumers store around 7 exabytes
• Individuals generate about 75 percent of all digital information, but enterprises are accountable for 80 percent of it
• The manufacturing industry stores the most amount of digital data, at 966 petabytes, while the education sector stores the least, at 269 petabytes
• The U.S. stores the most digital data of any geography at 3500 petabytes, with Europe coming in second at 2000 petabytes

What's interesting to note, as our CEO Jesse Rothstein mentioned recently in his Virtual Strategy Magazine article (Application Performance Management for Big Data), is some of the measurements used to relay these figures. The last few statistics mentioned above are in terms of petabytes, which have only recently become a reality. To put this in perspective, one petabyte is equal to approximately one million gigabytes. And, if you think that's big, a recent IDC report notes that data is expected to grow by as much as 44 times in the next 10 years, amounting to 35.2 zettabytes globally—and, one zettabyte is equal to one million petabytes!

With no hint of data growth slowing down, having a proper application performance management solution in place that can scale to match such large volumes becomes all the more crucial to ensure the seamless performance of applications and services. It'll certainly be interesting to see how Big Data impacts companies in years to come and what new stats will astound us next.