Considerations and Approaches for Scaling DataOps

To fully harness data at the level that modern competitiveness requires, a business needs to become a cognitive enterprise that builds data-driven insight into all its operations. That transformation requires scalable, repeatable processes for managing data pipelines and their output, with quality controls built in. Put another way, organizations need a formal DataOps program instead of taking an ad hoc approach to running data management, analytics, big data and other BI programs. Unlike departmental initiatives or one-off data projects, DataOps requires changes that reach further across organizational culture and information architecture. Such changes are hard to make; 92 percent of companies say culture is their leading impediment to becoming a data-driven organization.

DataOps is becoming increasingly important to enterprise competitiveness, but it is hard to start and even harder to scale. There is a tendency to model DataOps efforts after DevOps, which most organizations now have some experience with. This approach is problematic, and the most serious problems tend to surface when organizations try to scale their DataOps efforts. And programs will need to scale – data and the demand for data-driven insights are both growing quickly. The average company was managing 5,000 datasets in 2020, up from 4,300 in 2018, a 16 percent increase

This thought paper will highlight the unique DataOps characteristics, explain the most important differences between DevOps and DataOps, identify the relevant DataOps challenges, and provide enterprise guidance for establishing and scaling DataOps programs.

Download the paper.