Showing 29 results
-
Viewpoint
Preserving Innocence in the Digital Age: A Fight against Online Child Sexual Abuse Material (CSAM)
May 05, 2025The global rise in digital content, driven by AI, has intensified online threats, including child sexual exploitation and abuse, and solicitation, creation, and distribution of Child Sexual Abuse Material (CSAM). CSAM moderation and prevention require cutting-edge detection tools, expert moderation workflows, and global threat intelligence. It calls for a united and proactive defense ecosystem comprising enterprises, regulatory bodies, T&S technology and service providers, and child welfare agencies worldwide. In this report, we examine emerging CSAM threats, including AI-generated CSAM, review evolving global regulations, and outline enterprise best practices. This report aims to guide organizations to strengthen CSAM detection and prevention efforts in today’s digital landscape. Scope All industries and geographies Contents In this report, we explore: The immediate and growing CSAM threats Global regulatory responses and their impact on enterprise policies Existing technologies for CSAM detection and their limitations Recommendations for enterprises to effectively moderate CSAM Memberships Trust and Safety Sourcing and Vendor Management -
March 28, 2025The Trust and Safety (T&S) industry experienced significant shifts last year. These shifts were due to a sharp rise in egregious content volumes, particularly during elections, new regulations requiring nuanced compliance strategies, and increased enterprise generative AI adoption. As a result, enterprise demand for specialized content moderation, data annotation and labeling, AI safety services, and other value-added solutions increased, but the demand for traditional moderation services matured. Enterprises now expect T&S service providers to meet evolving enterprise requirements and become strategic enablers by delivering comprehensive T&S solutions and supporting responsible AI deployment, content policy development, and regulatory compliance efforts. The current T&S service provider landscape includes IT/BPO firms, CXM providers, and niche specialists, each bringing their expertise to address enterprise T&S concerns. Providers are building proprietary solutions, expanding operations in talent-rich regions, and increasing investments in AI safety. Additionally, they leverage partnerships with technology firms to enhance T&S solutions across the value chain. In this report, we assess 28 T&S service providers’ capabilities. Each profile includes the provider’s key strengths and limitations. The report enables buyers to choose the right-fit provider based on sourcing considerations. Scope All industries and geographies This assessment is based on Everest Group’s annual RFI process for the calendar year 2025, interactions with leading T&S service providers, client reference checks, and an ongoing analysis of the T&S market Contents In this report, we: Evaluate 28 leading T&S service providers Assess the T&S landscape Compare providers’ key strengths and limitations
-
Thematic Report
Trust and Safety Chronicles: Unpacking 2024, Anticipating 2025
Feb. 27, 2025In 2024, the Trust and Safety (T&S) industry navigated complex challenges from global elections and evolving regulations on child safety, misinformation, AI, and data privacy. These challenges necessitated nuanced compliance strategies. Despite Safety by Design remaining a top priority, growth of enterprise spending on T&S, particularly legacy moderation, declined due to budget constraints and increased automation. The T&S BPS market is expected to mature further due to AI support services’ expansion and slower growth in legacy moderation. Key growth factors include complexity in moderation of AI-generated content, stricter regulations, and demand for specialized moderation and ethical AI practices. However, growth will be slow as businesses shift toward community-driven moderation and rely more on generative AI and agentic AI. This slowdown may drive market consolidation due to margin pressures, scalability challenges, talent needs, and technology disruptions. In this report, we highlight key T&S trends from 2024, analyzing key enterprise events and challenges. The report offers 2025 market predictions, including expected market size, key headwinds and tailwinds, and their potential impact. Scope All industries and geographies Contents In this report, we examine: Key events and happenings in the T&S industry in 2024 The T&S industry’s market size and outlook for 2025 and beyond Market themes for 2025 and beyond Membership(s) Trust and Safety Sourcing and Vendor Management -
Viewpoint
Key Tech Away 2.0
Dec. 19, 2024The third-party technology landscape, comprising established legacy companies and emerging start-ups, is rapidly evolving. The growing adoption and emergence of new solutions and increasing collaboration and co-innovation among stakeholders fuel this landscape’s evolution. Third-party Trust and Safety (T&S) technologies’ role will grow with innovation and accountability. Addressing data privacy and security, provider lock-in, and ethical concerns while partnering strategically with providers enables enterprises to scale AI technologies and optimize their T&S operations effectively. In this report, we evaluate the current state of third-party T&S technologies, key trends shaping third-party T&S technology adoption, and challenges enterprises encounter when implementing third-party T&S solutions. The report also includes some best practices for enterprises when partnering with technology providers for technology solutions. Scope All industries and geographies Contents In this report, we examine: Current T&S technology ecosystem’s status Key trends shaping the third-party T&S technology landscape Challenges while adopting third-party technology solutions Enterprise best practices while leveraging third-party T&S technology Membership(s) Trust and Safety Sourcing and Vendor Management -
Thematic Report
Content Moderation Technology Trailblazers: Top Technology Start-ups Creating Buzz in the Market
Nov. 28, 2024An effective technology-based moderation solution can reduce the need for moderators to engage directly with harmful content, thereby scaling and streamlining the process. As enterprises grapple with harmful content detection and moderation, they are increasingly investing in automated tools to prevent unpleasant user experiences. The current content moderation third-party technology landscape consists of established legacy companies and emerging startups that provide diverse solutions, including classifiers, APIs, and platforms, to help enterprises detect, moderate, and report harmful content. Beyond developing these solutions, these providers often collaborate with each other and regulatory bodies to foster co-innovation, enhance solution effectiveness, and streamline moderation and reporting processes. In this research, we assess 13 leading content moderation technology providers, primarily focusing on their core capabilities and market adoption and domain coverage. We present an assessment, short profiles, and Everest Group’s point of view of these content moderation technology providers. Scope All industries and geographies Contents In this report, we cover: Growth drivers and inhibitors of the third-party content moderation technology market Key enterprise expectations from third-party content moderation technology Enterprise sourcing criteria for content moderation technology Trailblazers, that is, 13 leading content moderation technology providers Membership(s) Trust and Safety Sourcing and Vendor Management -
Nov. 11, 2024With rising threats in the digital landscape and heightened user awareness, leading digital platforms have started publishing transparency reports to provide insights into the enforcement of their platform policies and actions taken to ensure user safety. Researchers, human rights experts, journalists, and advertisers, among others, can use these reports as they provide metrics and insights into government requests for content takedowns and categorization and platform enforcement actions. This viewpoint examines the intricacies of transparency reporting and discusses current challenges and emerging concerns. It also highlights the role that service and technology providers play and outlines key considerations to enhance transparency in reporting. Scope Industry: trust and safety Geography: global Contents This report examines the concept of transparency reports, their key features, current and future challenges, the role of providers, and essential considerations to enhance transparency reporting. Memberships Trust and Safety Sourcing and Vendor Management
-
State of the Market
User Safety Redefined in the Generative AI Era: Trust and Safety State of the Market 2024
Aug. 13, 2024In the rapidly evolving Trust and Safety (T&S) space challenges such as ensuring user safety in the gen AI era, adherence to regulatory requirements, and deploying AI while ensuring responsible and sustainable practices remain key enterprise challenges. Enterprises are facing these challenge head-on through innovative platform design and a synergistic use of AI and human intervention. They are further supported by third-party providers that are investing in capabilities to provide both scale and technology. As the enterprises further deploy gen AI for commercial applications, they will require additional T&S support to guardrail foundational models and ensure their responsible and safe deployment. Moreover, gen AI is anticipated to transform the nature of work, talent strategies, and pricing models for T&S, ultimately leading to a net increase in outsourced T&S spending. In this report, we offer a comprehensive coverage of the T&S market and analyse it across various dimensions, such as market overview, service provider landscape, and the evolving market situation. This report also provides actionable recommendations and innovative solutions, making it an essential resource for leveraging AI's potential while ensuring safety and security in customer-facing applications. Scope All industries and geographies Service: T&S Contents In this report, we examine: Enterprise T&S challenges and risk mitigation strategies Investments by third-party providers to support enterprises Impact of gen AI on T&S Future enterprise challenges T&S BPS outsourcing overview and competitive landscape Membership(s) Trust and Safety Sourcing and Vendor Management -
Viewpoint
Ordinance at the Gates
June 04, 2024AI offers numerous benefits but also presents unique challenges. Regulations are vital to promote responsible development and build trust in AI. While existing legislations struggle to keep pace with the rapid development of AI systems, the regulatory landscape is evolving toward implementing specific AI regulations. In this viewpoint, we examine the current and upcoming regulatory landscape, analyzing different governments’ regulatory approaches worldwide and their implications for enterprises. The report offers insights to help enterprises navigate the dynamic regulatory environment. Additionally, it provides an enterprise regulatory framework to assist enterprises in managing their AI compliance and better prepare for potential future regulations. Scope Industry: Trust and Safety (T&S) Geography: global Contents In this report, we: Examine the regulatory landscape worldwide Evaluate regulations’ implications on enterprises Assess the leading enterprises’ AI practices Propose an enterprise regulatory framework Recommend best practices for enterprises to prepare for future regulations Membership(s) Trust and Safety Sourcing and Vendor Management -
Thematic Report
Generative AI Impact and Adoption in Trust and Safety (T&S)
May 31, 2024In 2024, Trust and Safety (T&S) continues to be a top priority for online platforms due to the proliferation of harmful content and a tightening regulatory landscape. Gen AI, while contributing to the issue by enabling the creation of such content, also presents a promising solution as a robust T&S automation engine. In this report, we examine the transformative potential of gen AI within the T&S value chain, demonstrating enterprises’ transition from pilot projects to full-scale production deployments. This report explores gen AI use cases across various segments of T&S, including content curation, annotation, data analysis, review and compliance, platform safety, ad review and compliance, and content services. Each section offers detailed insights into specific T&S challenges, highlighting their risk mitigation strategies, the use of gen AI to enhance efficiency in existing and emerging T&S services, and the leading technology providers being leveraged by service providers to make significant strides in this domain. By offering actionable recommendations and highlighting innovative solutions, this report serves as an essential resource for those aiming to harness AI’s potential while ensuring safety and security in customer-facing applications.. Scope All industries and geographies Service: T&S Contents In this report, we examine: T&S services’ overview, value chain, key business issues, and challenges T&S use cases proliferating due to GenAI Market maturity for gen AI adoption High adoption use cases for gen AI in T&S based on the study Membership(s) Service Optimization Technologies (SOT) Trust and Safety Artificial Intelligence (AI) CX Excellence Sourcing and Vendor Management -
Provider Compendium
Trust and Safety Services – Provider Compendium 2024
May 17, 2024The Trust and Safety (T&S) industry has changed significantly, driven by the increasing volumes and diversity of content formats, the dynamic regulatory landscape online, and the need for nuanced regional content moderation services. As a result, the demand for data annotation and AI support services has grown as enterprises continue to explore using generative AI for content creation and moderation. This compendium provides accurate, comprehensive, and fact-based snapshots of 27 T&S service providers. Each profile offers a detailed overview of the provider’s scale of operations, recent developments and investments, T&S-specific technology solutions (including generative AI solutions), other technology capabilities, well-being initiatives, and delivery locations. The research enables buyers of T&S services to understand providers’ offerings and assess their capabilities. Scope All industries and geographies Contents This report features 27 T&S se