List: Data Engineer Things | Curated by Thin Nguyen Duy

May 19, 2022
25 stories
Data Engineer Things
In
TDS Archive
by
Hadi Fadlallah
Installing Hadoop 3.1.0 multi-node cluster on Ubuntu 16.04 Step by StepThere are many links on the web about install Hadoop 3, many of them are not working well or need improvements. This article is taken from…
Nov 5, 2018
6
Nov 5, 2018
6
João Torres
How To Set Up a Hadoop 3.2.1 Multi-Node Cluster on Ubuntu 18.04 (2 Nodes)To start: What is Hadoop?
Feb 3, 2020
19
Feb 3, 2020
19
karcuta
Apache Spark Fundamentals — Phần 4: Spark Catalyst Optimizer1. Spark Catalyst Optimizer
Jun 8, 2020
Jun 8, 2020
In
The Slalom Blog
by
Amarpreet Singh
Introduction to Schema Registry in KafkaApache Kafka has been gaining so much popularity these past few years due to its highly scalable, robust, and fault-tolerant…
Apr 15, 2020
3
Apr 15, 2020
3
In
Geek Culture
by
Nirav Bhatt
Why Every Developer Should Learn CloudEasiest way to stay relevant in ever-changing market
Jul 6, 2021
Jul 6, 2021
In
TDS Archive
by
Antony Henao
Building a Data Warehouse: Basic Architectural principlesAn approach towards simplifying data warehousing
Feb 10, 2020
3
Feb 10, 2020
3
Lewis Gavin
How to architect the perfect Data WarehouseKey warehousing techniques made simple
Jul 26, 2018
12
Jul 26, 2018
12
In
SoftwareMill Tech Blog
by
Andrzej Ludwikowski
The best serialization strategy for event sourcingA brief summary of the existing serialization solutions (not only) for events in Event Sourced systems.
Jun 19, 2019
6
Jun 19, 2019
6
In
WhatfixEngineeringBlog
by
Saravanan A R
Real-Time data processing architecture using Apache Spark, Apache Kafka, and ClickHouseThis blog will talk about Real-time data processing architecture using Apache Spark, Apache Kafka, and ClickHouse.
Jan 3, 2022
1
Jan 3, 2022
1
In
bakdata
by
Victor Künstler
Continuous NLP Pipelines with Python, Java, and Apache KafkaAdvancements in machine learning, data analytics, and IoT, and the business strategic shift towards real-time data-driven decision making…
Jul 6, 2020
Jul 6, 2020
In
Oceanize Lab Geeks
by
Faysal Ahmed
Concepts of Database ArchitectureWhat Is Database Architecture?
Aug 31, 2017
4
Aug 31, 2017
4
In
ZaloPay Engineering
by
AJ Pham
Tìm hiểu bài toán đồng thuận trong hệ thống phân tán (Phần 1)Khái niệm đồng thuận có nghĩa là nhiều node cùng đồng ý/hủy bỏ về một cái gì đó — nó có thể là một giá trị, một quá trình hành động hoặc…
Mar 2, 2019
Mar 2, 2019
In
ZaloPay Engineering
by
Alex Nguyen
SQL Planning: Parser & OptimizerBài viết giới thiệu về cách SQL Engine xử lý câu truy vấn như thế nào
Aug 13, 2019
1
Aug 13, 2019
1
In
Robinhood
by
Arpan Shah
Taming ELKThe ELK (Elasticsearch[ES], Logstash and Kibana) stack is an extremely powerful and useful tool for powering search and analytics. At large…
May 3, 2017
May 3, 2017
In
Robinhood
by
Vineet Goel
Adding Faust to your Existing ArchitectureQuick Links: GitHub | Documentation
Aug 22, 2018
4
Aug 22, 2018
4
In
TDS Archive
by
Kiruparan Balachandran
Connecting the Dots (Python, Spark, and Kafka)Python, Spark, and Kafka are vital frameworks in data scientists’ day to day activities. It is essential to enable them to integrate these…
Jul 8, 2019
1
Jul 8, 2019
1
In
Insight
by
Sriram Baskaran
Airflow 101: Start automating your batch workflows with easeIn this blog, I cover the main concepts behind pipeline automation with Airflow and go through the code (and a few gotchas) to create your…
Oct 17, 2018
5
Oct 17, 2018
5
In
TDS Archive
by
Mustavi Sarker
Using Kafka to optimize data flow of your Twitter StreamA guide to processing big data from Twitter efficiently
Jul 30, 2020
1
Jul 30, 2020
1
 This story is no longer available
Adnan Siddiqi
Create your first ETL Pipeline in Apache Spark and PythonIn this post, I am going to discuss Apache Spark and how you can create simple but robust ETL pipelines in it. You will learn how Spark…
Jun 9, 2019
6
Jun 9, 2019
6