At Twitch, we have spent the last year developing a prevalence metric to track violative behavior in live chat. The new metric would help us measure the efficacy of our enforcement and safety work in new ways, identify areas of improvement, and create opportunities for greater transparency around our safety efforts.

We began with an adaptation of the industry standard metric, violative view rate (VVR). We quickly realized that VVR, which centers content views, is the wrong metric to track chat violations that could be harmful to creators broadcasting on a live streaming service. At Twitch, we constantly aim to meet the needs of our streamers. In this case, that meant developing a novel prevalence metric to capture the dynamic and ephemeral nature of live chat. We developed Violative Session Rate (VSR), which quantifies the fraction of chat sessions that violate Twitch’s guidelines for harassment, hate speech, and harm. This presentation will highlight the importance of prevalence metrics for services that host user generated content, describe our metric development process, and explain how VSR sampling differs from VVR.