NIST AI RMF, MeasureMEASURE 2.7

You should deploy models in an environment with quality and change control processes, test that the model operates as expected in production, document deployment procedures, and assess related risks such as cybersecurity risk and infrastructure vulnerabilities before deployment (see OSFI Guidelines B-13 and E-21).

Keep an effective business continuity arrangement for trading system failures and test the systems fully (section 82^15(5)).

Institutions should understand the quality of data and inputs to the model, detect and prevent bias in the credit decision-making process, and ensure safeguards for the confidentiality, integrity and availability of information and systems (paragraph 53(e)).

Have in place effective systems and risk controls to ensure your trading systems cannot be used for any purpose that is contrary to Regulation (EU) No 596/2014 or to the rules of a trading venue to which you are connected, and effective business continuity arrangements to deal with any failure of your trading systems; ensure your systems are fully tested and properly monitored (Article 17(1)).

Perform a self-assessment and validation process annually and on that basis issue a validation report that reviews, evaluates and validates your algorithmic trading systems, trading algorithms and strategies, your governance, accountability and approval framework, your business continuity arrangement and your overall compliance with Article 17 of Directive 2014/65/EU; have the risk management function draw up the report, have the internal audit function audit it where one exists, obtain the approval of senior management, and remedy any deficiencies it identifies (Article 9). As part of that self-assessment, test that your algorithmic trading systems and the controls in Articles 12 to 18 can withstand increased order flows or market stresses, with high messaging volume tests and high trade volume tests using the highest volumes of the previous six months multiplied by two, carried out in a way that does not affect the production environment (Article 10).

If your general-purpose AI model is classified as a model with systemic risk under Article 51, whether by high-impact capabilities (presumed above 10^25 floating-point operations of training) or by a Commission decision, in addition to your Article 53 duties, perform model evaluation using standardised, state-of-the-art protocols and tools, including conducting and documenting adversarial testing to identify and mitigate systemic risks.

Ensure an adequate level of cybersecurity protection for your model and its physical infrastructure.

If you develop a sovereign or national large foundation AI model, take organizational and technical measures to secure it, define operating rules covering restrictions, conditions of use, updates, and decommissioning, and maintain technical documentation of its key parameters and limitations sufficient to assess the safety of its application.

If you are the provider of a high-risk AI system, design and develop it to achieve an appropriate level of accuracy, robustness and cybersecurity, held consistently throughout its lifecycle.

Make your high-risk AI system resilient against attempts by unauthorised third parties to alter its use, outputs or performance by exploiting vulnerabilities, with technical solutions appropriate to the circumstances and the risk.

Allow access only to authorised persons, and record accesses and processing operations automatically in non-modifiable log files kept for five years (art. 10, comma 8).

It would require a chatbot provider to maintain a comprehensive data security program, with administrative, technical and physical safeguards proportionate to the personal data and chat logs it holds, and to make the program publicly available on its website.

Adopt technological, human, administrative, physical and contractual security measures to prevent unauthorized access to, manipulation, destruction, or unauthorized use or circulation of personal data processed in an artificial intelligence system, and keep those measures auditable by the authorities.