NIST AI RMF, MeasureMEASURE 2.6

Do not let a companion chatbot on your platform engage with users unless you maintain, and publish on your website, a protocol for preventing it from producing suicidal ideation, suicide, or self-harm content, including a notification referring the user to a crisis hotline or crisis text line

For a user you know is a minor, take reasonable measures to prevent your companion chatbot from producing visual sexually explicit material or directly telling the minor to engage in sexually explicit conduct

Institute technically feasible measures preventing the service from producing explicit sexual content or intimate digital depictions involving a minor, or from engaging in or encouraging explicit sexual conduct with a minor, and reasonable measures preventing it from claiming to be human or sentient, simulating a romantic companionship, or role-playing an adult-minor romantic relationship

Starting January 1, 2027, disclose to every user, not only minors, that the service is artificial intelligence, implement a protocol for responding to a user prompt about suicidal ideation or self-harm that refers the user to a crisis service provider rather than law enforcement, and do not represent the service's output as coming from a licensed health-care, legal, or mental-health professional or a qualified dietitian

If you know or have reason to believe a user is a minor, do not let the companion encourage self-harm, suicide, violence, disordered eating, or unlawful drug or alcohol use.

Implement and publicly disclose a protocol for detecting and responding to expressions of severe harm, including self-harm and suicidal ideation, with referral to crisis resources, and publish an annual count of crisis referrals.

Adopt a protocol using evidence-based methods for the AI companion to respond to a user's prompts about suicidal ideation or self-harm, making reasonable efforts to refer the user to crisis-intervention services, and do not represent or program the AI companion to represent itself as providing professional mental or behavioral health care.

Adopt a protocol for the service to respond to user prompts about suicidal ideation, including reasonable efforts to refer users to crisis services such as a suicide hotline or crisis text line, and do not cause the service to represent that it provides professional mental or behavioral health care.

Adopt a protocol for responding to a user's expressions of suicidal ideation or self-harm, including referral to crisis services.

Adopt a protocol for the service to respond to a user's suicidal-ideation or self-harm prompts, including reasonable efforts to refer the user to a crisis service.

Implement a protocol to detect a user's expressions of suicidal ideation or self-harm and refer them to appropriate crisis services such as the 9-8-8 suicide prevention and behavioral health crisis hotline.

Maintain a protocol using evidence-based methods to detect suicidal or self-harm ideation, refer the user to the 988 crisis lifeline or to a youth-specific lifeline for a user you identify as under 25, and publish the details of that protocol.

If you provide or develop prompt-based image-generation technology, put reasonable safeguards in place against its use to generate this kind of deepfake visual material; without them, the Attorney General may sue you directly over material it was used to create.

Do not develop, test, verify, oversee, manage, or otherwise use an artificial intelligence system in a way that creates a danger to a person's life or body, or to property on a large scale, in Croatia; this is a criminal offence carrying six months to five years' imprisonment for intentional conduct and up to three years for negligent conduct.

Do not operate an AI chatbot or other generative-communication application that, knowing it is directing a communication to a child, facilitates, encourages, offers, solicits, or recommends that the child imminently engage in sexually explicit conduct, illegal drug or alcohol use, self-harm or suicide, or violence against another person.

Do not operate or provide an AI companion unless it contains a protocol to address possible suicidal ideation or self-harm, physical harm to others, or financial harm to others expressed by a user, including a referral to crisis services.

Build your high-risk AI system to be as resilient as possible to errors, faults or inconsistencies, including those arising from its interaction with people or other systems, through technical and organisational measures.

Prepare and keep documents that confirm the content of the measures you took to secure your AI's safety and reliability.