Indonesia Conference Directory

Big data is related to vast amount of data that could reach more than Gigabytes until Exabyte of data. Nowadays, there are many companies that already achieves exponential data growth that produced by their applications system. The technologies to maintain those dataset is not limited to single instance of database. Majority of those company have been used various tools that could help to maintain their big data system. This paper will be composed by conducting a document review and collect papers that related to big data solution and infrastructure. Afterwards, this paper might be a foundation to conduct another research for big data solution topic. The survey are trying to cover the phases for building a common big data pipelines by following these structures: enterprise vision and mission that affect the big data architecture, defining the data sources that become a center point of the business, building the backend services to collect the datapoint from the data sources, utilize the data pipeline to prevent data loss, consuming the data pipeline message and deliver it to the data storage, build the raw data storage, define business-driven data visualization and analytics, build the ETL (extract, transform, load), build business-oriented data storage, applying statistical and machine learning, perform the business-driven visualization and implementing the monitoring for the big data pipelines. By the end of the study review, The survey will emphasize that there are many big data component that could help the small-medium enterprise to tackle their big data operational issues. And the small-medium enterprise could gain a knowledge to utilize some products from open source and proprietary to help manage their large-scale dataset as a comprehensive pipeline.