Sum-up AI: Video Summarization Tool for Low-resource Language (Sinhala)

Abstract

Abstract Video content summarization focuses on extracting the relevant information from video data. It is very important to address the challenge of information overload, faced by users on platforms like YouTube and Vimeo. Different approaches are introduced for video content summarization using techniques such as Deep Neural Network (DNN), Natural Language Processing (NLP), Machine Learning (ML), Computer Vision, and Speech Recognition. But there exists a gap between summarizing the Sinhala language or mixed language (Sinhala + English) data. We proposed a solution to fill this gap by introducing a new platform for summarizing video content data called Sum-up AI, especially in the Sinhala language. In there, we experimented can we provide a contextually accurate summary of a video using the transcription of the video content, enabling users to grab the main idea of the entire video without wasting time to go through the whole content. For this experiment, we used three main modules: Google Transcriber API, fine-tuned sinmT5 model, and fine-tuned sinBERT model. Furthermore, our evaluation results show that the fine-tuned sinmT5 model generates summaries for a given video (only providing the input text) with 7.05 of average score out of 10, while the fine-tuned sinBERT model which is used for category classification, achieved 74% of accuracy. Findings of this research indicate that Sum-up AI provides moderately accurate summaries of Sinhala video content and serves as a good starting point for further research in the area of Sinhala video content summarization.