关于生成式 AI 网络风险的通函 (TCRS/2024/05)

金融机构应关注生成式AI带来的多方面风险和陷阱。

FIs should also be mindful of the multifaceted risks and pitfalls that are associated with this development.

利用新技术检测深度伪造和AI生成内容

Leverage new technologies to detect deepfakes and AI-generated content

对高风险交易和高风险角色实施额外的用户验证，例如通过挑战问题、电子邮件等方式

Implement additional user verification for high-risk transactions and high-risk roles by different means such as challenge question, email etc.

实施深度伪造检测工具

Implement deepfake detection tools

对高风险交易实施额外验证

Implement additional verification for high-risk transactions

开展深度伪造意识宣传活动

Conduct deepfake awareness campaigns

使用另一因素重新确认身份

Reconfirm identity with another factor

整合人工智能并将威胁情报纳入日志监控，以更好地识别异常和可疑活动

Incorporate AI and integrate threat intelligence into log monitoring to better identify anomalies and suspicious activities

开展用户意识宣传活动

Conduct user awareness campaigns

采用多层网络防御策略

Adopt a multi-layered cyber defence strategy

采用利用机器学习和基于启发式的行为检测的解决方案，以阻止传统和新出现的恶意软件威胁

Incorporate solutions that leverage machine learning and heuristics-based behavioural detection to stop both legacy and new malware threats

金融机构应采取多层网络防御策略，以降低AI网络攻击风险。

FIs should adopt a multi-layered cyber defence strategy to mitigate risks from AI-enabled cyber attacks.

金融机构应保持基本的网络卫生，以应对AI网络攻击手法。

FIs should maintain basic cyber hygiene to counter AI-enabled cyber attack modus operandi.

保持基本的网络卫生

Maintain basic cyber hygiene

实施人工智能驱动的工具来检测多态恶意软件

Implement AI-powered tools to detect polymorphic malware

将AI和威胁情报整合到日志监控中，以更好地识别异常和可疑活动。

Incorporate AI and integrate threat intelligence into log monitoring to better identify anomalies and suspicious activities.

鼓励金融机构关注AI驱动的恶意软件检测和端点安全解决方案的发展，并在适当时采用。

FIs are encouraged to monitor developments in AI-enabled malware detection and endpoint security solutions, and adopt them where appropriate.

鼓励金融机构将AI解决方案与日志管理系统结合，通过收集网络设备数据点进行分析。

FIs are encouraged to incorporate AI solutions with log management systems, which work by ingesting data points from devices throughout the network.

鼓励金融机构将这些工具与威胁情报平台和服务集成，以跟上攻击者不断演变的战术、技术和程序，并利用信息增强日志分析，更好地识别可疑活动或潜在入侵。

FIs are also encouraged to integrate such tools with threat intelligence platforms and services to keep up with the evolving tactics, techniques and procedures of attackers, and use the information to enhance their log analysis to better identify suspicious activities or a potential breach.

监控并考虑采用AI工具检测企业终端上的多态恶意软件。

Monitor and consider incorporating AI tools to detect polymorphic malware on corporate endpoints.

允许员工使用公开GenAI工具的金融机构可能面临数据泄露风险，如果员工提交或上传敏感数据。

FIs that allow employees to use publicly accessible GenAI tools could be subject to potential data leaks if their employees submit or upload sensitive data while using those tools.

在使用第三方或开源GenAI模型时，金融机构需要管理相关的供应链和第三方风险。

While using third-party or open-source GenAI models, FIs will need to manage the attendant supply chain and third-party risks.

金融机构在使用GenAI解决方案时，也可能因未经授权的内部行为和不正当数据处理而面临数据泄露风险。

FIs could also be exposed to data leakage risks through unauthorised insider actions and improper data handling when using the GenAI solutions.

在开发和使用GenAI模型时引入控制措施。

Introduce controls while developing and using the GenAI models.

对第三方提供或开源的GenAI模型进行风险评估。

Conduct third-party provided or open-source GenAI model risk assessment.

对可输入GenAI模型的数据实施数据分类。

Implement data classification for data which can be entered into GenAI models.

开展员工向公共GenAI模型泄露数据的意识宣传活动。

Conduct awareness campaigns for data leaks by employees to public GenAI models.

制定用户政策并开展员工安全意识宣传活动。

Establish user policies and conduct employee awareness campaigns on security best practices.

对GenAI模型进行漏洞评估和安全测试。

Conduct vulnerability assessments and security testing on GenAI models.

为GenAI模型实施DLP工具和防火墙，以减轻机密数据泄露。

Implement DLP tools and firewalls for GenAI models to mitigate loss of confidential data to GenAI models.

在开发GenAI模型时采用安全最佳实践。

Adopt security best practices while developing GenAI models.

实施工具以记录和监控GenAI模型的输出

Implement tools to log and monitor output of GenAI models

实施人在回路中验证输出是否符合预期

Implement human-in-the-loop to verify that the output is as expected

确保对GenAI训练数据和基础模型实施稳健的访问控制

Ensure robust access controls to the GenAI training data and foundation model

鼓励金融机构实施适当的数据防泄漏控制，检查提示和响应中的敏感数据。

FIs are encouraged to implement appropriate DLP controls to check for any sensitive data provided in the prompts, as well as the responses generated by their GenAI solutions.

建立适当的GenAI模型和数据治理

Establish proper GenAI model and data governance

实施制衡机制以编辑基础模型中的数据

Implement maker-checker function to edit data in foundation models

可实施专为GenAI模型构建的防火墙，分析用户输入以检测数据提取或利用GenAI解决方案的企图。

GenAI firewalls that are purpose-built for GenAI models could also be implemented to analyse user inputs to detect any attempts to extract data or exploit the GenAI solutions.

将GenAI解决方案的应急措施纳入业务连续性计划

Include contingency measures for GenAI solutions into BCP

就GenAI模型部署中遇到的问题和挑战进行信息共享

Conduct information sharing on issues and challenges faced during GenAI model deployment

对高风险交易和高权限员工启用额外验证。

Enable additional verification for high-risk transactions and for staff in high privileged roles.

在人脸识别认证方案中实施活体检测技术以对抗深度伪造。

Implement liveness detection techniques in facial recognition authentication solutions to counter deepfakes.

开展活动提高用户对深度伪造和生成式AI钓鱼的认识。

Conduct campaigns to raise user awareness on deepfakes and GenAI-enabled phishing.

在事件响应中纳入深度伪造攻击场景。

Include deepfake attack scenarios in incident response.

金融机构应制定明确的数据分类和GenAI使用政策，指导员工安全使用GenAI以及可在公共GenAI解决方案上使用的数据类型。

FIs should have clear data classification and GenAI usage policies to guide employees on how to use GenAI safely, and the type of data that can be used on public GenAI solutions.

金融机构在使用第三方或开源GenAI解决方案之前，应进行风险评估、稳健测试和模型验证。

FIs should conduct risk assessments, robust testing and model validation on third-party or open-source GenAI solutions before using them.

金融机构应提高员工对安全使用公共GenAI模型的认识，例如对数据输入进行脱敏，不输入任何机密信息。

It is important for FIs to raise the awareness of their employees on how to use public GenAI models safely, for example to desensitise data inputs provided, and not input any confidential information.

鼓励金融机构对其GenAI解决方案进行漏洞评估、渗透测试和红队测试。

FIs are encouraged to perform vulnerability assessments, penetration testing and red teaming on their GenAI solutions.

这包括在需求收集和设计阶段进行威胁建模并纳入安全考虑，以及在开发阶段实施安全编码和代码审查。

This would include performing threat modelling and incorporating security considerations during the requirements gathering and design stages, as well as implementing secure coding and code reviews during the development stage.

鼓励金融机构在开发内部GenAI模型时采用安全设计方法和安全编码实践，以尽量减少引入的漏洞。

FIs are encouraged to adopt a security-by-design approach and secure coding practices while developing in-house GenAI models to minimize the vulnerabilities introduced.

应采取步骤检查并确保用于训练这些模型的数据未被污染。

Steps should be taken to check and ensure that the data used to train these models are not tainted.

为便于此，金融机构可使用模型卡记录模型技术细节，如模型能力、安全漏洞、训练数据信息和训练方法。

To facilitate this, FIs could use model cards to document model technical details, such as model capabilities, security vulnerabilities, information on the training data, and training methodology used to train the model.

鼓励金融机构针对常见类型的模型攻击（如OWASP所列）测试其GenAI模型。

FIs are encouraged to test their GenAI models against common types of model attacks, such as those listed by the Open Web Application Security Project ("OWASP").

金融机构还应实施最小权限原则，限制有权访问训练数据的人员，并实施日志记录以监控训练数据集的更改。

FIs should also implement the principle of least privilege to limit the personnel who have access to training data and implement logging to monitor changes to the training data set.

金融机构应实施严格的访问控制，限制谁可以访问和修改AI资源和训练数据。

FIs should implement strict access controls to limit who can access and modify the AI resources and training data.

金融机构还应实施稳健的数据治理流程，并进行数据质量检查，以确保用于GenAI模型的数据的准确性、一致性和完整性。

FIs should also implement robust data governance processes and perform data quality checks to ensure the accuracy, consistency, and completeness of the data used with the GenAI model.

金融机构可以采用制衡流程，例如人在回路方法，确保对训练数据或基础模型的更改由两个或更多人审查。

FIs could have a maker-checker process, such as human-in-the-loop approach, to ensure that changes to the training data or foundation models are vetted by two or more individuals.

金融机构应建立适当的模型治理，确保GenAI模型的端到端完整性、问责性和可审计性。

FIs should establish proper model governance to ensure the end-to-end integrity, accountability, and auditability of GenAI models.

鼓励金融机构将GenAI解决方案的应急措施纳入业务连续性计划。

FIs are encouraged to incorporate contingency measures for GenAI solutions in their business continuity plans.

对于使用GenAI模型的关键流程，鼓励金融机构确保制定应急措施，以防出现与GenAI模型相关的中断或安全事件。

For critical processes using GenAI models, FIs are encouraged to ensure that contingency measures are in place in case of disruptions or security incidents related to the GenAI model.

这可能包括保留训练数据的备份，以便在GenAI模型出现退化或操纵迹象时重新使用。

This could include keeping a backup of the training data which can be reused should there be signs of degradation or manipulation of the GenAI model.

鼓励金融机构通过信息共享平台分享部署GenAI模型时的有用经验教训和挑战。

FIs are encouraged to share information, such as useful learning points and challenges encountered while deploying GenAI models, to the wider FI community via information sharing platforms.

鼓励金融机构实施全面日志记录，并持续监控GenAI模型的性能下降、模型漂移或可能表明数据中毒的意外行为。

FIs are encouraged to implement comprehensive logging and continuously monitor GenAI models for performance degradation, model drift, or unexpected behaviour that could indicate a possibility of data poisoning.

模型应使用经过检查和验证的新鲜干净数据更新知识库。

Models should also update their knowledge bases with fresh and clean data that have been checked and validated.

金融机构应了解不断发展的GenAI技术及其风险影响，并紧跟相关行业最佳实践和风险缓解策略，以安全利用GenAI的优势。

FIs should be cognizant of the evolving GenAI developments and their risk implications, as well as keep abreast of the relevant industry best practices and risk mitigation strategies to safely harness the benefits of GenAI.