Building Credit Scoring Models That Satisfy the EU AI Act

Credit scoring sits squarely inside Annex III of the EU AI Act. Any AI system used to evaluate creditworthiness or establish credit scores of natural persons is classified high-risk by default, which means the obligations aren't a checklist to satisfy after a model is built — they are design constraints that need to shape the model, the data pipeline, and the surrounding process from the first architectural decision. Institutions that treat explainability as a report generated at the end of the project will find themselves re-engineering the model, not just the documentation. 

This piece lays out what "explainable by design" actually means for a credit scoring system under the AI Act, from a business analyst's perspective sitting between the regulatory text and the engineering team that has to build against it. 

What Annex III Actually Requires 

High-risk classification under Annex III triggers a specific set of obligations, and it's worth being precise about what they are, because vague compliance claims are exactly what regulators and auditors will probe first: 

Risk management system — a documented, continuous process for identifying and mitigating risks across the model's lifecycle, not a one-time assessment 

Data governance — training, validation, and testing datasets must be relevant, representative, and examined for bias, with documented data provenance 

Technical documentation — detailed enough that a third party (an auditor or regulator) can assess compliance without needing to interview the original development team 

Record-keeping and automatic logging — the system must log events throughout its operation, sufficient to trace decisions and identify circumstances that could lead to risk 

Transparency and provision of information to users — the credit institution deploying the model needs enough information to interpret its output and use it correctly, not just a score 

Human oversight — measures that allow a human to understand, monitor, and if necessary override or halt the system's output 

Accuracy, robustness, and cybersecurity — appropriate to the system's intended purpose 

None of these are satisfied by a well-written model card produced after training completes. Each one implies decisions made during model selection, feature engineering, and system architecture. 

Interpretability Starts at Model Selection 

The single highest-leverage decision in this entire process happens before a line of training code is written: what class of model are you going to use. 

There is a long-running tension in credit scoring between model performance and interpretability — gradient-boosted trees and neural networks often edge out logistic regression and scorecards on raw predictive accuracy, particularly on complex, high-dimensional data. But the AI Act doesn't require "no explainability," it requires that the system enable the deployer to interpret the output and the affected person to understand the reasoning behind a decision that affects them. A model that requires a post-hoc explainability layer (SHAP values, LIME, surrogate models) to approximate its own reasoning is a materially harder compliance position than a model that is interpretable natively. 

This doesn't mean defaulting to logistic regression for every use case. It means the model selection decision needs to explicitly weigh interpretability against the marginal accuracy gain of a more opaque model, and that weighing needs to be documented as a decision, with the trade-off made visible — not buried as an implementation detail nobody signed off on. In several engagements, we've found that a well-engineered feature set narrows the accuracy gap between an interpretable model and a black-box one enough that the compliance and audit-simplicity benefits clearly win. 

Where higher-complexity models are genuinely justified — say, incorporating alternative data sources with non-linear relationships — the design needs to budget for a robust post-hoc explainability layer from day one, not retrofit one when the compliance review flags it. 

Model Documentation as a Living Artifact 

Annex IV of the AI Act specifies what technical documentation needs to cover: system architecture, design specifications, training methodology, datasets used, and the risk management measures applied. In practice, the documentation that satisfies an auditor looks less like a report and more like a structured record that's updated as part of the development workflow itself. 

This has direct implications for how a BA structures the project: 

Documentation requirements get written into the definition of done for each model-affecting change, not assigned as a wrap-up task before go-live 

Every material feature engineering decision — inclusion, exclusion, transformation — gets a documented rationale tied to the business requirement or regulatory constraint driving it 

Version control on the model itself is paired with version control on its documentation, so an auditor reviewing a decision made in March can see the exact model version, feature set, and validation results in effect at that time 

Retrofitting this after a model has been through several iterations is materially more expensive than building it into the workflow from the start, because reconstructing the rationale behind a six-month-old feature engineering decision requires archaeology, not documentation. 

Audit Trails: Logging for Traceability, Not Just Debugging 

Standard application logging (errors, latency, throughput) is necessary but insufficient here. The AI Act's logging requirement is about enabling traceability of the system's functioning throughout its lifecycle — which means the log needs to capture, at minimum, the specific inputs used for a given credit decision, the model version that produced the output, the resulting score, and any human override applied. 

This has a concrete architectural consequence: the logging layer needs to be treated as a compliance-critical component, with its own retention policy, integrity guarantees, and access controls — not bolted on as an afterthought to application monitoring. We typically scope this as a dedicated audit log store, separate from operational logs, with append-only guarantees and retention aligned to both AI Act obligations and existing regulatory retention requirements the institution already operates under (which, for most EU banks, will overlap significantly with existing GDPR and EBA record-keeping obligations). 

Human Oversight Hooks: Designing for Intervention, Not Just Review 

Human oversight under the AI Act isn't satisfied by a human rubber-stamping model outputs after the fact. The requirement is for oversight measures that allow the human to genuinely understand the system's output, monitor its operation, and intervene or override when necessary — which means the system needs to be built with the hooks that make intervention actually possible. 

In practice, this shapes several design decisions: 

Confidence and reason codes, not just a score. The output presented to the credit officer needs to include the primary factors driving the score, in terms a non-technical reviewer can act on — not a raw probability and a black box. 

Override capability with a captured rationale. When a human overrides a model output, that override, and the reason for it, needs to be captured as part of the audit trail — both to satisfy the record-keeping requirement and because override patterns are themselves valuable signal for model risk monitoring. 

Threshold-based escalation. Scores near a decision boundary, or cases where the model's confidence is low, should route to mandatory human review rather than automated approval or denial — this is often where an oversight framework has the most practical bite, since it's the segment of decisions genuinely closest to being wrong either way. 

Monitoring for drift that would silently erode explainability. A model retrained on shifting data can drift toward relying more heavily on harder-to-explain feature interactions over time, even without a formal model change. Oversight needs to include monitoring for this kind of interpretability drift, not just accuracy drift. 

The BA's Role in Making This Work 

None of the above happens by accident, and it rarely happens if it's left entirely to the data science or engineering team, because the requirements originate in regulatory text that engineers aren't necessarily reading closely, and engineers are focused on model performance metrics that don't natively capture "was this decision made in a way a regulator can trace and a customer can understand." 

The BA's role here is translation in both directions: turning Annex III and Annex IV obligations into concrete, testable requirements the engineering team can build against, and turning the technical realities of the model (its feature set, its failure modes, its data lineage) into documentation and process that will hold up under audit. Done well, this isn't compliance overhead bolted onto a credit scoring project — it's a core part of how the system is specified from the outset, alongside accuracy targets and latency requirements. 

The Takeaway 

A credit scoring model that satisfies the EU AI Act isn't a compliant version of a model that would otherwise look different. Interpretability, documentation, audit trails, and human oversight are design constraints that shape model selection, data pipeline architecture, and logging infrastructure from the first specification. Institutions that build these in from the start end up with systems that are not only defensible to a regulator, but genuinely easier to monitor, debug, and trust — which was, in a sense, the point of the regulation to begin with.