Dec 2023: Colorado’s Quantitative Testing Protocol with Real World Data

In response to Colorado Division of Insurance’s DRAFT PROPOSED Algorithm and Predictive Model Quantitative Testing Regulation, released in October 2023, Bell Analytics conducted a case study, applying the draft protocol to real-world data. 

First, we inferred racial and ethnic classifications using BIFSG and compared to self-reported identifications, examining the applicability of the BIFSG approach.

We then followed the draft protocol. For certain model types and datasets, we found the test procedure:

• May miss clear examples of illegal and unfair discrimination

• May flag certain data points for remediation, without flagging others that are very closely related

• Does not fully account for reasonable correlations to the underlying insurance practice

• Omits an explicit measure of impact on protected classes or business objective specific to the data point in question

Find the full analysis here.