.png){kind=small}
Online marketplace platforms that facilitate user-to-user transactions often come with in-app chat functions. Such a chat function allows users (buyers and sellers) to engage in necessary communication (e.g. negotiating prices, understanding product/service quality).
What’s the Issue?
It is crucial for users to keep such communication within the in-app chat to ensure traceability for recovery steps in the case of bad actors (e.g. Scam sellers not delivering goods). Furthermore, users sharing personal contact information such as phone numbers, email, or social media handles also poses serious personal privacy risks. In addition, bad actors could also engage in phishing attempts by sharing malicious links to target users on the platform.
To combat this, many platforms rely on message-blocking filters in their in-app chat function to prevent users from sharing such personal information or external links.
Current Approach and its Limitations
Current message-blocking filters usually rely on rule-based regex matching and traditional machine-learning techniques. These methods scan messages for patterns that resemble contact details or external links, blocking them to maintain the integrity of the platform.
Despite these efforts, users/bad actors often find clever ways to circumvent these filters. They might spell out phone numbers or use unconventional formats that slip through the cracks of standard detection methods. As a result, the current approach struggles to keep up with these workarounds.
Our Improved Solution with LLM
To address this gap, we turned to Large Language Models (LLMs). By leveraging few-shot prompting and combining it with reinforcement learning and human feedback, we created a dynamic solution. Our LLM-based model continuously learns and adapts to the evolving tactics users employ to share their contact details or external links. This advanced approach not only enhances the detection of contact information but also ensures effective moderation, even when users try inventive bypass techniques.
By implementing this solution, we significantly improved the in-app chat message-blocking filter for marketplace platforms, keeping their platforms secure and ensuring that users can safely conduct their transactions.
Conclusion
The fast-growing field of generative AI, especially with LLM models, offers great opportunities to improve current industry tools. In the case of message-blocking filters, the improved performance is evident through incorporating LLMs.
If you are interested in learning more about this implementation, please contact us at contactgentleai@gmail.com.
Are you starting your AI journey and unsure where to begin? Good news! You don't have to navigate this journey alone. Join the GentleAI community to kickstart your personal AI adventure. 🚀
GentleAI is a vibrant platform where non-technical AI enthusiasts come together to learn, share experiences, and grow. Whether you're just curious about AI or looking to dive deeper, you'll find like-minded people ready to support and inspire you.
Comments