Pro Spark Streaming (e-book) (used book) | Zubair Nabi | bookbook.eu

Description

Chapter 1: Introduction to Spark

Chapter Goal: Introduce the reader to Spark in general. This book does not assume that the reader is already familiar with Spark.

Sub -Topics

1. Introduction to Spark and its key selling points
2. The programming model
3. Architecture
4. Introduction to other systems within the ecosystem, such as MLlib, GraphX, SparkSQL, and SparkR

Chapter Goal: Introduces Spark Streaming and the concept of micro batch processing (DStreams)

Sub - Topics

1. Introduction to Spark Streaming/DStreams
2. Comparison with traditional stream processing
3. How Spark Streaming works under the hood
4. Programming API and how it relates to the general Spark API
5. First sample application using FileInputDStream

Chapter Goal: To transfer best practices in terms of application development

Sub - Topics:

1. Maintaining state in an application
2. Data caching to reduce redundant work
3. Offloading RDD maintenance to Tachyon
4. Fault-tolerance and check-pointing

Chapter Goal: To enable the reader to understand the various data ingestion options, their pros and cons, and their integration with Spark Streaming

Sub - Topics:

1. Introduction to Receivers

2. Kafka

3. Twitter

4. Flume

5. Other sources

6. Writing your own connector. Example Apache Qpid

Chapter Goal: To help the user in optimizing an application and how it can be maintained in production

Sub - Topics

1. Different configuration parameters and how they affect the application
2. Parallelism
3. Serialization, memory, etc. enhancements
4. Various monitoring and instrumentation options

Chapter Goal: To illustrate how a SQL/Dataframe interface can simplify common transforms

1. Introduction to SparkSQL and SQLContext
2. Various SQL constructs
3. Integration with R
4. Design of a few real-world applications

Chapter Goal: Employ MLlib to implement streaming machine learning applications

1. Introduction to streaming algorithms in MLlib
2. Real-world applications using streaming MLlib

Chapter Goal: Blending data at rest with data in motion

1. Introduction to the Lambda Architecture
2. Design of Lambda Architecture using Spark

Chapter Goal: Introduction to Spark Streaming in Java and Python

1. Java API
2. Python API

Chapter Goal: Overview of some of the future plans for Spark Streaming from the ope

1. Project Tungsten and how its CPU and memory improvements can benefit streaming applications
2. Links to useful resources