NIST AI RMF, MeasureMEASURE 1.3

An internal models approved institution should have independent model vetting processes, as a second line of defense, to check whether models are sound and fit for their intended purpose, and a reviewer who does not direct or engage in model development.

You should have a model review process that is independent from model development and that assesses conceptual soundness and performance, using internal reviewers or objective third parties, with the extent and frequency of review commensurate with the model risk rating and prompted by new models, modifications, performance breaches, significant data changes and scheduled risk-based periodic reviews.

Submit audited financial statements for every financial year, and independent evaluation reports on your algorithms and software systems to the Commission at least once every two years from the date of your license and after any material change, seeking the Commission's prior approval before appointing the independent persons or experts (clause 13).

Validate all your models independently on a regular basis based on model type, with a validation that excludes the development team from the assessment of the model, that does not report to the business lines, and that covers both a qualitative and a quantitative validation (Sections 3.7.1, 3.7.5 and 3.7.7).

Have a validation function that gives an objective, unbiased and critical opinion on models and operates independently from model development and from model owners, demonstrated through separate reporting lines if you have approval to use internal models for regulatory capital; subject all models to independent review, monitor model performance against thresholds and revalidate periodically at a frequency set by model tier (Principle 4).

Have objective experts with appropriate expertise, sufficient independence to maintain objectivity, and the organizational standing and influence to effect change provide effective challenge of models throughout the model lifecycle.

If you are a Commission-designated very large online platform or very large online search engine, commission an independent audit, at your own expense and at least once a year, of your compliance with Chapter III's due diligence obligations, including the systemic-risk mitigation measures for generated and manipulated content, and with any codes of conduct or crisis protocols you have joined.

Select an auditing organisation that is independent of you and free of conflicts of interest: it must not have supplied you or a legal person connected to you non-audit services related to the matters audited in the twelve months before the audit began, must have committed not to supply them in the twelve months after it is completed, must not have audited you for more than ten consecutive years, and must not be paid on a result-contingent basis.

If you are a large frontier developer, obtain an annual independent third-party audit of your compliance, and meet the Act's access, retention, and publication requirements for the audit results. Within 30 days of receiving the independent audit report, publish a high-level summary of the audit findings and a redacted copy of the report on your website and transmit the redacted report to the Agency and the Attorney General.

It would make it unlawful to market, offer for sale, sell or deploy an artificial intelligence model that a third-party validator has not validated on task performance, determinism, latency and throughput, data provenance, disparate impact, lawful and secure data handling, and safety, and on any further topics the director of the Office of Cyber Command adds.

If your system is listed in point 1 of Annex III (biometrics) and, in demonstrating compliance with Section 2, you applied the harmonised standards of Article 40 or, where applicable, the common specifications of Article 41, choose between the procedure based on internal control in Annex VI and the procedure of Annex VII, which involves a notified body.

If your system is listed in point 1 of Annex III, follow the Annex VII procedure, rather than choosing, if harmonised standards do not exist and common specifications are not available, if you did not apply the harmonised standard or applied only part of it, if you did not apply existing common specifications, or, as to the restricted part, if a harmonised standard was published with a restriction.