International Scientific Journals
of Scientific Technical Union of Mechanical Engineering "Industry 4.0"

Social media, being used by billions of people around the globe have a significant impact on the field of big data. The proliferation of social platforms has resulted in a deluge of unstructured data in the form of user-generated content, including text, video, audio, and web pages. This data, while rich in information, poses significant challenges for traditional data processing tools, which are primarily designed to handle structured data. Processing unstructured data requires specialized tools and techniques, such as natural language processing (NLP) and artificial intelligence (AI). One solution for processing this type of data is the IDOL tool, which can not only collect data from social media, but can also process it with the help of artificial intelligence. With its support one can extract trends, dynamic patterns or relationships to improve business decision-making. In the current paper, a new approach for processing unstructured data from social media is proposed, which involves collaboration and integration of different software products.

Big data is large volume, heterogeneous, distributed data. Big data applications where data collection has grown continuously, it is expensive to manage, capture or extract and process data using existing software tools. With increasing size of data in data warehouse it is expensive to perform data analysis. In recent years, numbers of computation and data intensive scientific data analyses are established. To perform the large scale data mining analyses so as to meet the scalability and performance requirements of big data, several efficient parallel and concurrent algorithms got applied. For data processing, Big data processing framework relay on cluster computers and parallel execution framework provided by MapReduce. MapReduce is a parallel programming model and an associated implementation for processing and generating large data sets. In this paper, we are going to work around MapReduce, use a MapReduce solution for handling large data efficiently, its advantages, disadvantages and how it can be used in integration with other technology.