Spark Standalone Installation – Install Spark to Local Cluster

 

Apache spark can easily be deployed in standalone mode, all you need is to Install Spark to Local Cluster. First download the pre-built spark and extract it. After that, open your terminal navigate to the extracted directory of spark from sbin start master.sh after that start slave.sh followed by master spark URL which will be obtained at localhost:8080. Now you have started a cluster manually.

After that, you can start the Spark-shell (for Scala) or Pyspark (for Python) or SparkR (for R) from bin.

  1. Download pre-built Spark.
  2. Extract the downloaded Spark built (you can extract spark in either way by terminal or manually).
  3. From your terminal navigate to the extracted folder, now you have to start master.sh from sbin command: sbin/start-master.sh
  4. After Master, you need to start slave.sh followed by master spark URL which you’ll get from browser by typing localhost:8080 command: sbin/start-slave.sh <URL>
  5. After performing step 3 & step 4, you have successfully started the cluster manually.
  6. Now you’ll be able to start your applications like Spark-shell, pySpark, SparkR for Scala, Python and R from bin. command: bin/spark-shell
  7. Start writing your code or application.

 

 Screenshots of Standalone Mode

Install Spark to Local Cluster at 2.53.05 PM

Install Spark to Local Cluster at 2.54.35 PM

Install Spark to Local Cluster at 2.55.22 PM

Install Spark to Local Cluster at 2.57.17 PM

Install Spark to Local Cluster at 2.58.33 PM

Install Spark to Local Cluster at 2.59.51 PM

Install Spark to Local Cluster at 3.01.03 PM

Install Spark to Local Cluster at 3.01.46 PM

Install Spark to Local Cluster at 3.02.47 PM

Install Spark to Local Cluster at 3.13.28 PM

 

Install Spark to Local Cluster at 3.16.49 PM

Install Spark to Local Cluster at 4.37.28 PM

 

 

 

for quick basic tutorial referred to official guide.

Comments

comments

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.