By John D'Angelo, Managing Director of RealFoundations John D‚ÄôAngelo is a frequent presence at NAREIM meetings and presented at this year‚Äôs 20/20 Investor Summit. D‚ÄôAngelo sheds light on the hidden opportunities in the data you are already keeping, and through systematic collection, it could start working for you.   Summary While the term ‚Äúbig data‚Äù captures the imagination, it‚Äôs poorly understood outside of the realm of data science and in the instances where it exists. Real Estate Investment Managers don‚Äôt have big data, but most certainly DO have a data opportunity or ‚Äúdark data.‚Äù There is value to be found in addressing data opportunities and putting the dark data to work.   What is ‚ÄúBig Data‚Äù? Gartner, a world leading information technology research and advisory company, defines big data as "high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision-making.‚Äù What does that mean? The term ‚ÄúBig Data‚Äù describes a data set that is so vast that it taxes the ability of anything but specifically designed database management tools to effectively store, manage, operate and support analytics. Examples of Big Data sets are the billions of hours of subscriber viewing activity that Netflix captures, grocery store item sales, and of course, Google‚Äôs unfathomably large collection of individual web search data. Each of these examples describes data sets with hundreds of millions, billions, even trillions of individual records. Big data initiatives or big data work is focused on mining and analyzing such gigantic data sets to discover insight, trends, opportunities and meaning from the data. It‚Äôs a captivating concept ‚Äì by probing a huge data set, what can be discovered about behavior, trends, patterns, opportunities, risks, and threats? By now it should be clear that most real estate investment managers do not have what has been historically defined as big data. Nor do they have a big data problem because they don‚Äôt have big data in the first place. But this isn‚Äôt to say that there isn‚Äôt a data opportunity! There most certainly is. And it isn‚Äôt to say that the term and concept of Big Data isn‚Äôt both important and useful to real estate investment managers - it is! The opportunities lie untapped in ‚Äúdark data.‚Äù   Dark Data vs. Big Data What most real estate investment managers do have currently is ‚Äúdark data‚Äù. This data may be hidden within the silo of a fund, outside the enterprise at various service providers or joint venture partners, on a disconnected spreadsheet on a desktop, in a document on a shared server, within functional silos, or simply spread across multiple applications. In order to be considered ‚Äúdark‚Äù, this data is unstructured, disconnected, or simply unavailable. Although fit for the specific purpose for which it was created and intended, this data CANNOT physically be combined for analytics, or SHOULD NOT because it‚Äôs not consistent or trusted for any purpose than for which it was created. Even worse, it is common for business critical data to be duplicated in several places, and although you‚Äôd think it would be identical in each place, it isn‚Äôt. Without well-defined data standards, the result is vast amounts of unusable data for larger purposes. Gartner treats ‚Äúdark data‚Äù thusly: ‚ÄúMost of this data already belongs to organizations, but it is sitting there unused‚Ķ. Similar to dark matter in physics, dark data cannot be seen directly, yet it is the bulk of the organizational universe.‚Äù   The Data Opportunity Here‚Äôs a good acid test question for those of you who have an office portfolio. Are you any better today at answering the question about exposure to a given single tenant or NAICS code than you were on Lehman weekend? If the (honest) answer is no, and you‚Äôre tired of looking at your shoes when you think about this scenario, you probably have a dark data opportunity. Converting this dark data opportunity isn‚Äôt complicated; it just requires work, discipline, and often a different mindset about how data can and should be used throughout the enterprise. Chances are VERY good that the expertise and experience required to recognize the opportunity is being exercised on a routine basis in your investor reporting, portfolio management and asset management functions. Out of necessity, highly compensated and very capable colleagues are compiling data from multiple sources in a spreadsheet in a labor-intensive way, then working to apply rules to the compiled data (e.g., are names misspelled, should a given tenant name include or exclude ‚Äú,llc‚Äù, is it ‚ÄúUSPS‚Äù or ‚ÄúUnited States Postal Service‚Äù, etc.). What is the end result? Is this data aggregated into a single data set? Do numbers of each individual asset manager add up to the numbers being used by portfolio managers? If an asset manager makes an adjustment in his or her spreadsheet, does it flow through to investor reporting? How often are reports ‚Äì every month or every quarter? You get the idea. To realize the data opportunity, you first need organized, reliable data. Sources of data should be made available when appropriate and governed so ownership is clear when a question or dispute arises. In addition, data should be reviewed by knowledgeable resources or tools so that it can be trusted. It is also important to structure data for various uses. But is all data equally valuable? Not so much.   Findings & Challenges There is value in looking at how big data efforts have succeeded or failed to capitalize on the dark data opportunities at a REIM. A key theme from whitepapers and case studies written about big data suggest that you start with understanding the value expected before beginning the effort of mining, leveraging, and analyzing the data. In effect, the body of case study material says don‚Äôt waste your time and resources and money on this effort unless the business is willing and able to answer these questions about purpose and anticipated value. Defining your expectations is just as important to the success of the effort.   What to do? Do some investigating into your current state of affairs ‚Äì specifically into the current state of your data and the amount of time personnel spend wrangling data throughout your enterprise. Ask yourself and your people if there is key data about an asset, investment, market or portfolio whose formal system of record in your organization is a spreadsheet? Ask around to see if a key bit of information (final acquisition price, for instance) is duplicated in two or more locations ‚Äì and if so, does everyone have and use the same number. Ask what work is required to summarize operational information across multiple funds or geographies. Finally, ask how much time is really spent doing work that is required (finding, assembling, testing, adjusting or fixing data) but doesn‚Äôt add value or improve asset performance. If you don‚Äôt like the answers, you‚Äôve got a dark data opportunity. Addressing this opportunity will have a payoff not only in the efficiency of your operations but also in your ability to scale your portfolio without hiring people in direct proportion to AUM. Chances are the benefits of efficiency pay for whatever effort it takes to realize the opportunity. The real payoff, however, is in the questions you can ask your data to help improve asset and portfolio performance and achieve greater returns. Think through these questions first, then ensure the data required to answer them is available and highly governed for consistency and accuracy. This will decrease risk and improve effectiveness of your firm. And you just might spot opportunities, threats, trends and risks that you hadn‚Äôt previously noticed.