当前位置: 首页 > 新闻中心 > 学院新闻 > 正文


来源:       发布时间:2023-11-06
2023年10月18-20日,图灵奖获得者Yoshua Bengio和姚期智、加州大学伯克利分校教授Stuart Russell以及清华大学智能产业研究院(AIR)院长张亚勤联合召集了来自中国、美国、英国、加拿大和其他欧洲国家的20多位顶尖AI科学家和治理专家,齐聚英国牛津郡迪奇利公园(Ditchley Park),进行了为期三天的首届“人工智能安全国际对话”(International Dialogue on AI Safety),部分与会者签署了一份联合声明:呼吁“在人工智能安全研究与治理上的全球协同行动,是避免不受控制的前沿人工智能发展为全人类带来不可容忍的风险的关键。”



  • 图灵奖获得者、“深度学习三巨头”之一(Yoshua Bengio)

  • 图灵奖获得者、中国科学院院士、清华大学交叉信息研究院院长(姚期智)

  • 中国工程院院士、清华大学智能产业研究院(AIR)院长(张亚勤)

  • 美国国家工程院外籍院士、北京智源人工智能研究院顾问、清华AIR卓越访问教授(张宏江)

  • 中国科学院自动化研究所类脑认知智能实验室负责人和人工智能伦理与治理研究中心主任、研究员(曾毅)

  • 人工智能标准教科书的作者(Stuart Russell)

  • 人工智能安全和隐私领域被引用次数最多的学者(Dawn Song)

  • 未来生命研究所创始人、MIT教授、《生命3.0》作者(Max Tegmark)

  • ……









Coordinated global action on AI safety research and governance is critical to prevent uncontrolled frontier AI development from posing unacceptable risks to humanity.

Global action, cooperation, and capacity building are key to managing risk from AI and enabling humanity to share in its benefits. AI safety is a global public good that should be supported by public and private investment, with advances in safety shared widely. Governments around the world — especially of leading AI nations — have a responsibility to develop measures to prevent worst-case outcomes from malicious or careless actors and to rein in reckless competition. The international community should work to create an international coordination process for advanced AI in this vein.

We face near-term risks from malicious actors misusing frontier AI systems, with current safety filters integrated by developers easily bypassed. Frontier AI systems produce compelling misinformation and may soon be capable enough to help terrorists develop weapons of mass destruction. Moreover, there is a serious risk that future AI systems may escape human control altogether. Even aligned AI systems could destabilize or disempower existing institutions. Taken together, we believe AI may pose an existential risk to humanity in the coming decades.

In domestic regulation, we recommend mandatory registration for the creation, sale or use of models above a certain capability threshold, including open-source copies and derivatives, to enable governments to acquire critical and currently missing visibility into emerging risks. Governments should monitor large-scale data centers and track AI incidents, and should require that AI developers of frontier models be subject to independent third-party audits evaluating their information security and model safety. AI developers should also be required to share comprehensive risk assessments, policies around risk management, and predictions about their systems’ behavior in third party evaluations and post-deployment with relevant authorities.

We also recommend defining clear red lines that, if crossed, mandate immediate termination of an AI system — including all copies — through rapid and safe shut-down procedures. Governments should cooperate to instantiate and preserve this capacity. Moreover, prior to deployment as well as during training for the most advanced models, developers should demonstrate to regulators’ satisfaction that their system(s) will not cross these red lines.

Reaching adequate safety levels for advanced AI will also require immense research progress. Advanced AI systems must be demonstrably aligned with their designer’s intent, as well as appropriate norms and values. They must also be robust against both malicious actors and rare failure modes. Sufficient human control needs to be ensured for these systems. Concerted effort by the global research community in both AI and other disciplines is essential; we need a global network of dedicated AI safety research and governance institutions. We call on leading AI developers to make a minimum spending commitment of one third of their AI R&D on AI safety and for government agencies to fund academic and non-profit AI safety and governance research in at least the same proportion.


上一条:2023乌镇峰会|张亚勤:大模型技术的六大发展趋势 下一条:【内附完整论文】AIR近期亮点论文解读







京ICP备15006448号  |   版权所有©清华大学智能产业研究院