Automation of Constructing an eProfile from Web Contents

Abstract

The WWW is becoming more powerful in present information era. Extracting web content and summarizing them were became a di cult for Internet users. This thesis investigates the existing techniques for web data extraction and summarizing them. This research focus on creating an ePro le for scholars by extracting the web content, summarizing and classifying them. Therefore prototype application was built with a pipeline architecture. In the application, the extraction module was performed by using html parser and using XPath expressions. The summarization was done by calculating Sentence Importance Score and Semantic Similarity Score. In the classi cation module, a N-Gram based text classi cation technique was used. The proposed solution is intended to be usable for constructing the ePro le in Academia domain and also it could be expected to use to many other domains such as analyzing news articles, pro ling past activities of persons.