Best Practices for AI and Automation in Trust and Safety

Artificial Intelligence (AI) and automation enable trust and safety operations to scale to address the speed and volume of abusive content and behavior online. Although AI and automation are not a panacea and come with several challenges and limitations, addressing content and conduct-related abuse at-scale is not possible without AI and automation.

This report describes how digital products and services use AI and automation in trust and safety, provides examples of best practices, and explores potential opportunities to use generative AI (genAI) for trust and safety. Key points include:

• AI and automation enrich and support the role of trust and safety professionals, blending machine and human processes to address abusive content and behavior.
• Companies offering very different digital products and services are aligned on both the challenges and opportunities for integrating AI and genAI into their trust and safety operations.
• When deployed responsibly, AI and automation can enhance all aspects of trust and safety, from product development, to governance, enforcement, improvement, and transparency. AI can also improve the wellbeing of trust and safety teams and reduce their exposure to traumatic content.
• Generative AI has the potential to significantly increase the capacity and accuracy of content moderation, however, further experimentation and risk assessment and mitigation is needed.
• Best practices that acknowledge the limitations of AI while encouraging collaboration across the trust and safety field provide a path forward on tough trust and safety challenges.