Share This Article
Since AI took over the internet, we’ve been thinking about the different ways we could utilize this technology to improve online community engagement. Let’s be honest, there’s a lot of AI products out there varying in functionality and frankly, quality, and we set out from the beginning to create a product that brings value and has practical use for communities. This led us to create our AI moderation system as a way to support your growing community and let you focus on larger goals for your platform.
What is AI moderation and why would you want to use it?
So let’s talk about what we built. It’s an AI Moderation tool that checks the content that users post on your community for potential toxic communication. Toxic communication is any post that crosses a certain threshold (for example, 90% out of 100%) for categories that you choose (insults, profanity, threat, sexually explicit, flirtation, and more). And while each community is different (that’s why you choose which content to flag), these categories have been selected because they’re the most common spam content that communities would like to see less of throughout their threads.
What happens to the content?
Looks like the AI Moderator found some toxic content, what happens next? Well, just because our AI Moderator said your user was being toxic, that doesn’t mean that the content should immediately be deleted or the user should be banned. AI should be used to assist you in content moderation, not take over. There is so much nuance in human behavior and content creation, that we believe a human touch is critical in deciding what happens to flagged users and content. This means that our AI Moderator will do the heavy lifting of finding problematic content and bring it to the moderation queue, but you’ll still have the final say on what happens next.
On our forum we also use an AI moderation tool from Audentio that uses a model to detect certain toxic posts and sends them automatically to be manually approved by a human if the post is determined by AI to be something a forum wouldn’t want in public.
AI can be incredibly valuable because it helps automate and streamline tasks that would otherwise consume a lot of human time and effort. On our forum, for instance, I rely on AI to handle routine inquiries and support tasks, freeing me up to focus on more complex issues and engage in genuine human interactions where they’re most needed. It’s not about replacing human interaction but making sure we can do our jobs and help more people.
Lee Tongate, Trouble Free Pool
What problem are we trying to solve?
Moderation takes resources (specifically time and money). The bigger the community, the larger chance there is for toxic content to slip through the cracks while large amounts of content are being created daily or while there’s a lapse in moderator availability. Moderators can only moderate the content that they know about, so our goal with this add-on is to make moderators more efficient and more accurate, saving them time and making your community a happier place for everyone.
How did we create our AI Moderator?
It may seem like we used magic in the making of this add-on. We in fact, did not. We use Google Perspective, which is an AI model that is trained for this exact purpose. In its own words, “Perspective uses machine learning models to identify abusive comments. The models score a phrase based on the perceived impact the text may have in a conversation.”
Fine-tuning the model
Content isn’t all equal across all people in all communities and you know your users best, so we determined it was critical to give you granular control of the types of toxic content the AI Moderator is flagging. Here’s a quick list of things you can tweak to get your AI Moderator assistant just right:
- Type of toxic behavior (insults, profanity, threat, sexually explicit, flirtation
- Types of content (thread posts, profile posts, conversations)
- Negativity threshold (1/10 how aggressive it should be)
34 minutes
Saved per day in moderation
100%
Decrease in spam
51% of users
Saw an improvement in positivity
How do I get access?
This feature (and all of our premium tools) is part of our Community Resources Suite, which has several exclusive add-ons and themes like, achievements, leaderboards, and complete account, as well as support and updates for all ThemeHouse add-ons and themes.
Conclusion
We’re excited to hear about how communities we work with use this tool and how it improved moderation for their team. We’ve been continuing to explore the possibilities of AI and plan to continue developing new tools to help you build a successful community.
