https://www.mayuan.site/tags/spark/ Recent content in spark on Prime's blog Hugo -- gohugo.io en Tue, 24 Dec 2019 10:22:52 +0800 https://www.mayuan.site/post/bigdata/spark%E5%9F%BA%E7%A1%80/ Tue, 24 Dec 2019 10:22:52 +0800 https://www.mayuan.site/post/bigdata/spark%E5%9F%BA%E7%A1%80/ spark初探 http://datastrophic.io/ 基本概念 Job: A piece of code which reads some input from HDFS or local, performs some computation on the data and writes some output data. Stages: Jobs are divided into stages. Stages are classified as a Map or reduce stages (Its easier to understand if you have worked on Hadoop and want to correlate). Stages are divided based on