Skip to main content
WorldCist'18 - 6th World Conference on Information Systems and Technologies

Full Program »

Improvement of Implemented Infrastructure for Streaming Outlier Detection in Big Data with ELK Stack

Nowadays the usage of internet is constantly increasing the amount of data. As a result the need for analysing this data has recently emerged as we need to face a new phenomena known as the Big Data. This research is focused in finding appropriate architecture for real-time big data analytics and its main task is to detect anomalies in this real-time data. There are some tools that are used and analysed by us in order to find the best one, but in this paper we use Timeline and compare it with Fluentd which is the tool we used in previous research [12]. Here we are going to show the reasons why Timeline is better than Fluentd. Anomaly detection in real-time big data is a problem that faces many organizations and it is a challenge for researchers as well. Our research deals with developing infrastructure for monitoring e-dnevnik (education national system in Macedonia) application server and to detect errors in order to scale up the performance. In order to enable this infrastructure to detect anomalies in streaming data we implement different algorithms for anomaly detection in Timelion. Another important thing is to know how to visualize the results. In this paper, we show the visualization of an e-dnevnik log by using Kibana, and also how Timelion helps us to identify anomalies in real time.

Zirije Hasani
Faculty of Computer Science, University of Prizren "Ukshin Hoti", Prizren, Kosovo

Jakup Fondaj
Faculty of Computer Science and Technologies, South East European University, Tetovo


Powered by OpenConf®
Copyright ©2002-2017 Zakon Group LLC