The course Big Data Architectures: Hadoop and Spark It is your gateway to the world of big data analytics, a sector that is currently booming and seeing growing demand for skilled workers. As companies seek to harness the power of data, skills in Big Data have become essential. This course provides you with a comprehensive understanding of the Hadoop ecosystem and Scala programming, essential tools for any professional aiming to excel in the digital age. Furthermore, you will learn how to use RDD, dataframes and datasets, and you’ll explore machine learning, expanding your analytical skills. Not only will you gain theoretical knowledge, but you’ll also develop practical skills which will help you stand out in the job market. Make the most of this opportunity to become an expert in technologies that are redefining the way organisations make decisions.
Big Data Architectures: Hadoop and Spark
Big Data
45 hours
Introduction
Objectives
-
Understanding the fundamental concepts of the Big Data for use in real-world projects.
-
Analysing the ecosystem Hadoop 3.x and its components to optimise data processing.
-
Programming in Scala to develop efficient applications in Big Data environments.
-
Use RDD, dataframes y datasets to handle large volumes of data effectively.
-
Implement algorithms for machine learning to extract value from data using Spark.
-
To evaluate the efficiency of different data processing methods in Hadoop y Spark.
-
Apply knowledge of Big Data to design innovative solutions in complex problems.
Table of Contents
TEACHING UNIT 1. BIG DATA
1. Introduction to a big data ecosystem
2. Setting up the environment
3. Definition and characteristics
4. The life cycle of a big data project
5. Data-driven decision-making
TEACHING UNIT 2. THE HADOOP 3.X ECOSYSTEM
1. Introduction to Apache Hadoop
2. Apache Hadoop architecture
3. Hadoop Distributed File System
4. Hadoop YARN
5. Hadoop MapReduce (Hadoop MR)
TEACHING UNIT 3. PROGRAMMING IN SCALA
1. Introduction to Big Data Programming
2. Scala as a programming language
3. Scala and the object-oriented paradigm
4. Scala and the functional paradigm
5. Scala: A Multi-Paradigm Approach in Practice
TEACHING UNIT 4. RDD, DARAFRAMES, DARASETS AND MACHINE LEARNING
1. Spark: installation and set-up
2. Spark Core: RDD
3. Spark SQL: dataframes and datasets
4. Spark data streaming (real time)
5. Spark and machine learning