目标

1. 整理RDD的概念&RDD的五大特性&源码体现
2. 整理SparkContext&SparkConf

RDD的概念&RDD的五大特性及源码体现

1. a list of partitions.

   protected def getPartitions: Array[Partition]

2. a function for computing each split/partition.

   def compute(split: Partition, context: TaskContext): Iterator[T]

3. a list of dependencies on other RDDs.
   RDDA => RDDB => RDDC => RDDD

   protected def getDependencies: Seq[Dependency[_]] = deps

4. optionally, a partitioner for key-value RDDs.

   @transient val partitioner: Option[Partitioner] = None

5. optionally, a list of preferred locations to compute each split on.
   preferred locations 最佳位置（ 数据本地性）

   protected def getPreferredLocations(split: Partition): Seq[String] = Nil

SparkContext&SparkConf详解