Why Data Governance Should Include Analytical Models

Our previous discussion focused on the data retention aspects of our data governance programs. Policies must be defined early, shared via data literacy programs and technical controls built to automate the retention, protection and purging of data. These policies are backed by processes that automate the storage and destruction of data, creating a strong audit trail for compliance teams to use for refining methods and tooling.

Our conversation to this point has focused entirely on the data within our environments, or the traditional scope of data governance programs. Many organizations are moving rapidly to deploy analytical models to enhance human decision making. This shift pushes us to define our programs as data and analytics governance, identifying and capturing the need to govern our analytical models, training sets, and outputs. This model governance ensures repeatability and elimination of bias from decision making while protecting organizational intellectual property. In this post, we’ll continue to reference data governance, but shifting our definition to include data and the analytical models that power our organizations.

Today’s analytical models can span a multitude of technologies including R, Python, Go, SQL-extensions, and SPSS. The deployment of these models can be via custom ML Ops frameworks or vendor supplied stacks including Google Vertex AI, AWS SageMaker, or Azure Machine Learning. While this technology will facilitate the implementation of governance policies, the organization will be responsible for setting conditions and boundaries of model deployment, testing, usage, and retention.

Model governance is still new as many organizations are early in their journey deploying models for production use. One industry leading the definition of standards is financial services. The use of analytical models for planning aspects like investments, cash reserves, stress testing, and modeling consumer behavior have stood out as early places where a high level of confidence is required. This process started as early as 2011 with the Federal Reserve issuing SR Letter 11-7 establishing standards for models used in various aspects of our financial systems.

Building a strong framework of policies, backed by systems automation for enforcement and auditing ensures consistency in model governance. The key elements of this framework include:

Each of these elements of model governance require specific technical features for implementation and automation. The greater use of automation will enhance reproducibility and minimize human error from model creation and deployment that could negatively affect an organization’s risk posture. Analytical models don’t stand on their own in today’s complex data landscapes; they have unique needs that must be captured in policy, automated through our ML Ops platforms and regularly reviewed to be updated to adjust for changing technology capabilities and market conditions.

The next post will explore the risks and rewards of data governance in legacy environments. Many organizations continue to rely on mainframe or UNIX technology from the 70s or 80s, creating added risk to data governance programs. Risks can manifest through skills attrition, lack of integration or gaps in automation of policy enforcement. We’ll explore these risks, mitigation techniques, and methods to modernize these platforms over time with data governance as the accelerating driver.

Make sure to sign up for updates so you don’t miss the next post.