AgentSure
Tier 4frameworkIMDAv2024.05 · 2024-05-30

生成式 AI 治理示范框架

Model AI Governance Framework for Generative AI

别名: MGF GenAI · Model AI GF GenAI · IMDA GenAI Framework

用途 / Purpose

生成式 AI 自愿治理框架,覆盖问责、数据、可信开发、事件报告、测试、安全、内容溯源、内容审核、研究 9 个维度。

Voluntary governance framework for GenAI covering accountability, data, trusted development, incident reporting, testing, safety, content provenance, content moderation, research.

原子条款
40
AI 专属
37
跨文件关联
0
页数
28 pp

文件关系

原子条款(40

在搜索器中打开 →
shouldAI§c

行业应采用开发、评估及后续的“食品标签”式透明度和披露的最佳实践。

Industry should adopt best practices in development, evaluation, and thereafter 'food label'-type transparency and disclosure.

transparencydisclosure.customer
declarativeAI§d

建立支持事件监控和报告的结构和流程。

Establish structures and processes to enable incident monitoring and reporting.

cyber.incident-reporting
declarativeAI§e

制定围绕AI测试的通用标准以确保质量和一致性。

Develop common standards around AI testing to ensure quality and consistency.

model-risk.testing
shouldAI§e

公司应采用第三方测试和保证以向最终用户展示信任。

Companies should adopt third-party testing and assurance to demonstrate trust with their end-users.

model-risk.testing
mustAI§f

需要调整现有信息安全框架并开发新的测试工具以应对这些风险。

Existing frameworks for information security need to be adapted and new testing tools developed to address these risks.

cyber.access-controlcyber.pentest
declarativeAI§g

关于内容生成位置和方式的透明度使最终用户能够知情地决定如何消费在线内容。

Transparency about where and how content is generated enables end-users to determine how to consume online content in an informed manner.

genai.content-provenancetransparency
mustAI§h

需要加速研发投资以改善模型与人类意图和价值观的对齐。

Accelerated investment in R&D is required to improve model alignment with human intention and values.

lifecycle.training
declarative§Security

采用“安全设计”原则,在系统开发生命周期的每个阶段融入安全设计,以最小化漏洞并减少攻击面。

Adapt 'Security-by-Design' to minimise system vulnerabilities and reduce the attack surface through designing security into every phase of the systems development life cycle.

cyber.access-controllifecycle.deployment
shouldAIDesign

AI开发者应提供报告渠道,用于报告AI系统中的安全漏洞。

AI developers can apply this similar concept, by allowing reporting channels for uncovered safety vulnerabilities in their AI systems.

cyber.incident-reportingcyber.patching
shouldAIDesign

应更加重视制定通用基准和方法。

Greater emphasis should therefore be placed on setting common benchmarks and methodologies.

transparency
declarativeAIDevelopment — Baseline Safety Practices

检索增强生成和少样本学习等技术常用于减少幻觉并提高准确性。

Techniques like Retrieval-Augmented Generation (RAG) and few-shot learning are commonly used to reduce hallucinations and improve accuracy.

genai.hallucination
shouldAIDevelopment — Baseline Safety Practices

模型开发者和应用部署者应考虑用例背景并进行风险评估。

Model developers and application deployers should consider the context of the use case and conduct a risk assessment.

model-risk.identificationlifecycle.problem-formulation
shouldAIDisclosure — “Food Labels”

模型开发者应向下游用户提供相关信息,以便他们做出更明智的决策。

Model developers should provide relevant information to downstream users so they can make more informed decisions.

transparencydisclosure.customer
shouldAIp.10

政策制定者应促进所有相关利益方之间的公开对话,以了解快速发展的生成式AI技术的影响,并确保潜在解决方案平衡且符合市场现实。

Policymakers should foster open dialogue amongst all relevant stakeholders to understand the impact of the fast-evolving generative AI technology, and ensure that potential solutions are balanced and in line with market realities.

governance.policy
shouldAIp.10

AI开发者应采取数据质量控制措施,并采纳数据治理的一般最佳实践,包括一致且准确地标注训练数据集,以及使用数据分析工具促进数据清洗(例如,去偏和移除不当内容)。

AI developers should undertake data quality control measures and adopt general best practices in data governance, including annotating training datasets consistently and accurately, and using data analysis tools to facilitate data cleaning (e.g., debiasing and removing inappropriate content).

data.qualitydata.lineagedata.bias-check
is-expectedAIp.13

对于具有潜在高风险(如涉及国家安全或社会影响)的模型,需要向政府提供更高的透明度。

Greater transparency to government will also be needed for models that pose potentially high risks, such as advanced models that have national security or societal implications.

transparencydisclosure.regulator
considerAIp.13

定制或高级模型的开发者可以考虑披露额外信息。

Developers of customised or advanced models can consider disclosing additional information.

transparencydisclosure.customer
declarativeAIp.13

因此,政策制定者可以定义模型风险阈值,超过该阈值将适用额外的监督措施。

There is therefore space for policymakers to define the model risk thresholds, above which additional oversight measures would apply.

governance.policymodel-risk.monitoring
shouldAIp.14

需要致力于更全面和系统的安全评估方法。

There is a need to work towards a more comprehensive and systematic approach to safety evaluations.

model-risk.testingmodel-risk.monitoring
considerAIp.14

标准化方法可包括与政策制定者协商,定义一套基线安全测试并开发共享资源。

The standardised approach could also include defining a baseline set of required safety tests and developing shared resources, in consultation with policymakers.

model-risk.testinggovernance.policy
shouldp.17

报告应相称,即在全面报告和实用性之间取得平衡。

Reporting should be proportionate, which means striking a balance between comprehensive reporting and practicality.

cyber.incident-reporting
mustp.17

组织需要内部流程来报告事件,以便及时通知和补救。

Organisations need internal processes to report the incident for timely notification and remediation.

cyber.incident-reporting
shouldAIp.29

所有社会成员应以可信方式获得生成式AI。

All members of society should have access to generative AI, done in a trusted manner.

fairness.disparate-impact
mayAIp.29

政府和行业伙伴可提高认识并提供支持,推动中小企业的创新和AI使用。

Governments and industry partners can improve awareness and provide support to drive innovation and AI use among SMEs.

governance.training
mayAIp.29

政府可与公司和社区合作开展数字素养计划,鼓励安全负责地使用AI。

Governments can partner companies and communities on digital literacy initiatives to encourage safe and responsible AI use.

governance.training
shouldAIp.29

政府宜协调资源支持公共部门。

It is desirable for governments to coordinate resources to support public sector.

governance.accountability
shouldAIp.29

AI应以有影响力的方式服务公众。

AI should serve the public in impactful ways.

governance.accountability
mustAIp.3

需要确保数据质量,例如通过使用可信数据源。

There is a need to ensure data quality, such as through the use of trusted data sources.

data.quality
declarativeAIp.30

AI开发者和设备制造商更适合进行绿色计算技术的研发并采用节能硬件。

AI developers and equipment manufacturers are better placed to conduct R&D on green computing techniques and adopt energy-efficient hardware.

governance.policy
mustAIp.30

生成式AI生态系统中的利益相关者需要共同努力开发合适的技术(例如节能计算)以支持我们的气候责任。

Stakeholders in the generative AI ecosystem therefore need to work together to develop suitable technology (e.g., energy efficient compute) in support of our climate responsibilities.

governance.policy
mayAIp.30

AI工作负载可以托管在推动最佳节能实践并使用绿色能源或路径的数据中心。

AI workloads can be hosted in data centres that drive best-in-class energy-efficient practices, with green energy sources or pathways.

governance.policy
mustAIp.30

为了为这些计划提供信息,生成式AI的碳足迹(例如模型训练和推理)也需要被跟踪和测量。

To inform such plans, the carbon footprint of generative AI (e.g., for model training and inference) will also need to be tracked and measured.

lifecycle.monitoring
shouldAIp.30

鲁棒性和公平性测试应作为起点基线。

Testing for robustness and fairness should form a starting baseline.

model-risk.testingfairness.bias-mitigation
mayAIp.30

行业、政府和教育机构可以合作重新设计工作并为工人提供技能提升机会。

Industry, governments and educational institutions can work together to redesign jobs and provide upskilling opportunities for workers.

governance.training
mayAIp.30

随着组织采用企业级生成式AI解决方案,它们也可以为员工制定专门的培训计划。

As organisations adopt enterprise generative AI solutions, they can also develop dedicated training programmes for their employees.

governance.training
declarativeAIp.30

AI开发者通过帮助政府识别用例并提供AI解决方案来解决公民痛点,发挥贡献作用。

AI developers play a contributing role by helping governments identify use cases and providing AI solutions to address citizen pain points.

governance.accountability
shouldAIp.33

应探索新的取证工具,以增强识别和提取生成式AI模型中隐藏恶意代码的能力。

New forensics tools should be explored to help enhance the ability to identify and extract malicious codes that might be hidden within a generative AI model.

cyber.malwaregenai.content-provenance
mustAIp.33

必须开发新工具,包括输入过滤器,用于检测不安全提示,并需针对特定领域风险进行定制。

New tools have to be developed and may include: a) Input Filters: Input moderation tools detect unsafe prompts (e.g., blocking malicious code). The tools need to be tailored to understand domain-specific risks.

cyber.access-controlgenai.prompt-injection
shouldAIp.5

应考虑如何在开发过程中预先分配责任(事前)作为最佳实践,并提供事后补救的指导。

There should be consideration for how responsibility is allocated both upfront in the development process (ex-ante) as best practice, and guidance on how redress can be obtained if issues are discovered thereafter (ex-post).

governance.accountabilitygovernance.raci
shouldAIp.7

应用部署者应从可信平台下载模型,以降低模型被篡改的风险。

Application deployers should download models from reputable platforms to minimise the risk of tampered models.

third-party.due-diligencecyber.supply-chain
来源
https://aiverifyfoundation.sg/wp-content/uploads/2024/05/Model-AI-Governance-Framework-for-Generative-AI-May-2024-1-1.pdf
法定授权基础: IMDA / AI Verify Foundation — voluntary industry framework