Multi-modal Deep Fusion for Activity Detection from both the Virtual and Real-world.

Abstract

Illicit activity poses a significant threat in today's society in general and in the younger population in particular. Drug dealers post different drug images and contact information on social media. Tracking drug dealers among millions of social media users can be challenging for law enforcement agencies. Drug Trafficking is also related to various crimes in society. Therefore, it is crucial to automatically detect drug dealers (along with the type of drugs they sell and their contact information). In this thesis, We have presented a state-of-the-art social media analytic algorithm that does multi-modal analysis to detect drug-related posts and drug dealers from social media. We have proposed to detect different types of drugs from social media posts which include: pills, mushrooms, LSD, cannabis, cocaine, syrup, hookah, and cigars, using our drug type detection model. We have also proposed a novel AI-based illicit activity detection model which will detect illegal activities such as drug dealing, fighting, and gun violence from the street in real-time AI-based surveillance, inform nearby police officers, and help detect, disrupt, and ultimately dismantle these networks. Our approach is generalizable to detect illicit activities such as human trafficking, illegal gun sales, and money laundering.