Skip to content

Data & AI

NGOPI IT at TMA covers Big Data fundamentals: building the data engineering foundation for the digital era

Bandung, 6 July 2026 – In the strong current of digital transformation, data has become a strategic asset for almost every organization. But before talking about Artificial Intelligence, Machine Learning, or advanced analytics, every technology practitioner must understand one foundation: Big Data. With the spirit of building a culture of continuous learning, PT Tech Mayantara Asia (TMA) held the regular forum NGOPI IT (Ngobrol Pintar IT) with the theme "Big Data Fundamental", led by Akhmad Sofyan, known as Mang Made. The material invited participants to understand the basic concepts of Big Data, its characteristics, architecture, technology ecosystem, and real implementations in industry.

13 July 2026

Notes from the work

—views—shares
Share:
NGOPI IT Big Data Fundamental poster with speaker Akhmad Sofyan (Mang Made) and a Big Data ecosystem diagram.

Bandung, 6 July 2026 – In the strong current of digital transformation, data has become a strategic asset for almost every organization. But before talking about Artificial Intelligence, Machine Learning, or advanced analytics, every technology practitioner must understand one foundation: Big Data. With the spirit of building a culture of continuous learning, PT Tech Mayantara Asia (TMA) held the regular forum NGOPI IT (Ngobrol Pintar IT) with the theme "Big Data Fundamental", led by Akhmad Sofyan, known as Mang Made. The material invited participants to understand the basic concepts of Big Data, its characteristics, architecture, technology ecosystem, and real implementations in industry.

The event was held on Monday, 6 July 2026, starting at 16.00 WIB in hybrid format: offline at Kantor Magna Bandung and online through the Discord platform. As usual, the hybrid format is typical of TMA internal activities and supports the flexible work culture. Participants included software engineers, data engineers, data analysts, interns, and crew from different divisions who wanted to understand how Big Data became the main foundation of modern digital solution development.

From the start, the room was warm and interactive. Mang Made opened the presentation with a simple question: "Why did Big Data appear?" That question was the door to understanding the big change in the technology world. He explained that Big Data was born not just because data gets bigger, but because of the data explosion produced every second by smartphones, social media, the Internet of Things (IoT), industrial sensors, digital transactions, and cloud services. Platforms such as Google Search, TikTok, Shopee, Gojek, and other digital applications produce millions or even billions of records every day that can no longer be processed with a traditional database approach or a spreadsheet such as Microsoft Excel.

Mang Made then explained the definition of Big Data simply. Big Data does not only mean "big data". It means data that is technically no longer efficient to process with a conventional approach. So new technology is needed to store, process, and analyze very large amounts of data in a distributed way. This gave participants the picture that Big Data is more a change of paradigm than a simple increase in storage capacity.

The material then covered Big Data characteristics, known as the 5V concept, which has grown into 7V. Mang Made described each characteristic with examples close to daily life. Volume describes data size that now reaches terabyte to zettabyte scale. Velocity explains how data flows in real time at very high speed. Variety shows that modern data is not only tables but also text, images, video, audio, and semi-structured data. Then there is Veracity, which relates to data quality and accuracy, Value, which stresses that data only becomes valuable after it is processed into insight, Variability, which shows data patterns changing with context, and Visualization as the ability to present data so decision makers understand it easily.

The discussion became more interesting when Mang Made compared Big Data with traditional data systems. In conventional systems, data is usually gigabytes to a few terabytes, stored on one server using an RDBMS and processed with SQL. Big Data works at terabyte to exabyte scale, stores structured and unstructured data, uses many servers in a distributed way, and relies on technology such as Apache Hadoop and Apache Spark for parallel processing. This comparison helped participants understand why the old approach is no longer enough for modern analytics needs.

One part that drew the most attention was the discussion of the Big Data ecosystem. Mang Made explained that Big Data is not a single application but a set of technologies that complement each other. For data storage there is HDFS, Amazon S3, and Google Cloud Storage. On the processing side there is Apache Hadoop MapReduce, Apache Spark, and Apache Flink. For data ingestion there are tools such as Apache Kafka, Apache Flume, Sqoop, and Apache NiFi. Query and analytics needs can use Hive, Presto, Spark SQL, or Impala. Analysis results are then visualized with platforms such as Apache Superset, Tableau, Power BI, or Looker. This explanation opened participants' view of how each component has a different function but works as one unit in a modern Big Data platform.

As the company that develops TeMA BigData Platform, this discussion of the ecosystem felt very relevant. Many participants began connecting the material to TMA internal products built with open source technology such as Apache Spark, Apache Iceberg, Trino, Apache Superset, Apache Airflow, Apache NiFi, and OpenMetadata. The discussion grew toward how all these components can form a Modern Lakehouse Architecture that supports enterprise needs in managing large-scale data.

Next, Mang Made introduced the basic Big Data architecture, made of seven main stages. The process starts with Data Source: data sources such as databases, application logs, IoT sensors, and external APIs. Data then enters the Ingest stage using Apache Kafka, Flume, Sqoop, or Apache NiFi. After that, data is stored in a storage system such as HDFS or Amazon S3 before entering the Processing stage with Spark or Hive. Processed data is then analyzed with a query engine such as Apache Superset or Tableau, visualized as an interactive dashboard, and finally turned into actionable insight that can support business decisions. This simple diagram gave a complete picture of the data journey from when it is first produced to when it becomes valuable information.

To show that Big Data is not just theory, Mang Made brought real case studies from several industries. In e-commerce, Big Data is used to build product recommendation systems such as "Customers who bought this product also bought...". In transportation, real-time data is used to set dynamic fares based on demand. In healthcare, Big Data helps predict the spread of disease from social media data and internet search trends. In banking, the technology can detect fraud transactions in real time. In government, Big Data supports initiatives such as Satu Data Indonesia and analysis of social aid distribution. These examples helped participants see that Big Data is already part of daily life, often without notice.

Mang Made also invited participants to look at the challenges of Big Data implementation. On the technical side, building a Big Data platform needs complex infrastructure and solid engineering skills. On the non-technical side there are issues of security, data privacy, regulation such as Indonesia's Personal Data Protection Law (UU PDP), and data governance. The biggest challenge is in human resources, because organizations need Data Engineers, Data Scientists, Data Analysts, and data architects with special skills. According to Mang Made, investing in talent development matters as much as investing in technology itself.

As in earlier NGOPI sessions, the question and answer session was lively. Participants discussed their experience with data warehouse implementation, the use of Apache Spark, strategies for building a data lake, and the difference between Hadoop and modern Lakehouse architecture. The mood grew lighter with the typical engineer jokes that appear when talking about debugging data pipelines or handling billions of records. Laughter came now and then, but the discussion stayed full of useful technical insight.

The Discord platform again proved its effectiveness as a medium for technical collaboration at TMA. Participants joining online could discuss directly through the voice channel or chat without losing interaction with those at the office. The hybrid work culture the company applies lets forums like NGOPI reach the whole crew, wherever they work.

Through NGOPI IT: Big Data Fundamental, PT Tech Mayantara Asia again confirmed its commitment to building a culture of continuous learning. For TMA, mastering technology is not only about using the newest tools. It is about understanding the right conceptual foundation so the company can build solutions that fit customer needs. Big Data is one important pillar in the digital transformation journey and a foundation for other technology such as Artificial Intelligence, Machine Learning, and predictive analytics.

More than a knowledge-sharing session, NGOPI has grown into a collaborative space that strengthens the engineering culture at TMA. In this forum, engineers, developers, data specialists, product enthusiasts, and students learn together, exchange experience, and build cross-discipline insight. This spirit keeps pushing TMA to create innovation, develop data-based solutions, and prepare digital talent for future industry challenges.

Bring us the system that has to work.

Tell us what you are building, replacing, or keeping alive. We will tell you how we would approach it.