Data play a vital role in informing decision-making in organizations across all sectors and sizes. For example, in government agencies, administrative data are often collected and used for accounting, reporting, and compliance purposes, and program administrators are also increasingly viewing data as valuable information that can be used to improve program performance. Many organizations, however, do not use their data to the data’s full potential, which can directly affect how they approach delivering on their missions. Meanwhile, technology has developed in such a way that organizations, and society at large, can now access more powerful data tools, including complex artificial intelligence (AI) methods, to conduct and automate a variety of data-analysis methods for themselves, often in real time and on very large data sets. However, organizations can only make good use of these methods if they have access to data from related databases that are stored and maintained separately, and the fact that many of them do not have the means to integrate such databases or methods of good quality control continues to make it difficult for organizations to execute data projects successfully. In fact, big data and AI data projects have a surprisingly high failure rate across public, private, and nonprofit organizations.

The Center for Data Insights at MDRC has partnered with organizations to develop and execute a variety of data projects, some with long-standing partners that have participated in MDRC’s long-term evaluations; others with newer partners that are just beginning to use data for program improvement. Important lessons about the essential ingredients for implementing successful data projects have emerged from this collaborative work. This brief discusses these ingredients in the context of the Center’s TANF Data Innovation (TDI) project.

The Center for Data Insights considers a data project a success when data are used routinely to inform policy and practice with the goal of improving the lives of the families served by programs. The Center has learned from these partnerships that barriers to success are not primarily about technical issues or analytic methods. Rather, data projects need three ingredients to be successful: people, perseverance, and project scoping that includes a clear understanding of a project’s needs and goals.

Read the full article about effective data projects by Edith Yang at MDRC.