Introduction
Everywhere you turn in enterprise IT, you hear the buzz about big data: billboards, commercials and likely your own conference rooms and inboxes. The variety, volume and velocity of data streaming through enterprise systems is on the rise, and so is the amount of discussion about the best way to handle it.
But while the big data phenomenon is giving organisations a broad range of new options for data analysis, it also compounds the business challenges associated with collecting, managing, organising and protecting data. Successfully leveraging big data for analytics demands that companies develop strategies to reduce the cost of managing data and reduce the risk involved in organising and protecting that data.
In addition to these business challenges, enterprises face technical challenges in managing rapidly expanding volumes of all types of data. The sheer number of continually proliferating data sources introduces complexity into data lifecycle management processes. If IT departments cannot manage information appropriately – from the moment it is created to the point when it can be archived or defensibly disposed — they risk violating legal and regulatory compliance requirements.
Big Data for Development Projects
Innovation Life Cycle
It’s obvious but important: the better the data, the better the results. When the New Intelligent Enterprise-a joint partnership between the MIT Sloan Management Review and the IBM Institute of Business Value-conducted a survey on analytics, it found that organizations that used analytics for competitive advantage were 2.2 times more likely to substantially outperform their industry peers.1 Ensuring the accuracy of the data used for analytics is a key factor in enhancing overall organizational productivity.
But achieving that accuracy requires a strong commitment to data lifecycle management-the process of managing business information throughout its existence, regardless of where that information is stored. That means ensuring that large data volumes do not inhibit application performance in production environments, and acquiring the ability to both offload the data into archived systems and maintain accessibility to that archived data for retention and business continuity.
The emergence of big data only reinforces the need for effective data lifecycle management. Big data is more than just information stored in an Apache Hadoop based framework; it is also the structured data within data warehouses, databases and standalone applications. The size of the data source-whether a single database, an entire data warehouse or a Hadoop framework-doesn’t matter. In the world of big data, all data sources are crucial and must be managed throughout their lifecycles.