How to check spark version

How to check spark version

Hi,

How can I find which version of Apache Spark is running on my environment?

How to check spark version?

Thanks

View Answers

November 27, 2017 at 7:20 PM

Hi,

You can login to your box where apache spark is running then run the command:

spark-shell

It will display the version as shown below:

[root@sandbox ~]# spark-shell
SPARK_MAJOR_VERSION is set to 2, using Spark2
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://172.17.0.2:4041
Spark context available as 'sc' (master = local[*], app id = local-1511778019896).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.1.2.6.1.0-129
      /_/

Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_141)
Type in expressions to have them evaluated.
Type :help for more information.

scala>

scala>

Thanks









Related Tutorials/Questions & Answers:
How to check spark version
How to check spark version  Hi, How can I find which version of Apache Spark is running on my environment? How to check spark version? Thanks... the command: spark-shell It will display the version as shown below
How to check spark version
How to check spark version  Hi, How can I find which version of Apache Spark is running on my environment? How to check spark version? Thanks... the command: spark-shell It will display the version as shown below
Advertisements
how to check gremlin version
how to check gremlin version  Hi, I am inside gremlin console and want to check the version. How to check gremlin version? thanks   Hi...; Gremlin.version() ==>3.3.1 this will print the gremlin version. Thanks
How to check keras version in anaconda
How to check keras version in anaconda  Hi, I have installed anaconda python for machine learning in my computer. Now I want to know which version... of keras library installed?ADS_TO_REPLACE_1 How to check keras version
How to check Hadoop version in Ubuntu?
How to check Hadoop version in Ubuntu?  Hi, I am trying various... to Hadoop cluster through ssh terminal. How to check Hadoop version in Ubuntu? Thanks   Hi, Its easy to check the version of Hadoop in Ubuntu. type
How specify the version of python that spark-submit uses
How specify the version of python that spark-submit uses  Hi, I have installed python 3.7 and want to use with spark. How specify the version of python that spark-submit uses? Thanks   Hi, You have to modify
How to check CentOS version via command line?
How to check CentOS version via command line?  Hi, I have a linux machine and i don't know the version of the CentOS operating system installed... system installed on it. How to check CentOS version via command line? Thanks
how to insert check box
how to insert check box   how to insert check box into jtable row in swing
How to check a checkbox - Struts
How to check a checkbox  Hello Community, How can i check a checkbox defined with tags. with plain html, the tag checks the box. with the other tags i am getting an error that the property "checked" is not defined. Can someone
How to check if a pid is alive?
How to check if a pid is alive?  Hi, I have a process which is running in background and I want a simple command to find if the process id still running? How to check if a pid is alive? Thanks   HI, You can use
Check TensorFlow version
Check TensorFlow version - How to check TensorFlow version? In this tutorial we will learn how to check the version of TensorFlow machine learning library... program and want to check the version of TensorFlow installed on your computer
Check TensorFlow version
Check TensorFlow version - How to check TensorFlow version? In this tutorial we will learn how to check the version of TensorFlow machine learning library... program and want to check the version of TensorFlow installed on your computer
Check TensorFlow version
Check TensorFlow version - How to check TensorFlow version? In this tutorial we will learn how to check the version of TensorFlow machine learning library... program and want to check the version of TensorFlow installed on your computer
com.holdenkarau - spark-testing-base_2.11 version 0.10.0 Maven dependency. How to use spark-testing-base_2.11 version 0.10.0 in pom.xml?
com.holdenkarau  - Version 0.10.0 of spark-testing-base_2.11 Maven dependency? How to use  com.holdenkarau  - Version 0.10.0 of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version 0.10.0
com.crealytics - spark-google-analytics_2.11 version 0.8.2 Maven dependency. How to use spark-google-analytics_2.11 version 0.8.2 in pom.xml?
com.crealytics  - Version 0.8.2 of spark-google-analytics_2.11 Maven dependency? How to use  com.crealytics  - Version 0.8.2 of spark-google-analytics_2.11 in pom.xml? How to use spark-google-analytics_2.11 version 0.8.2
com.holdenkarau - spark-testing-base_2.11 version 1.5.1_0.9.0 Maven dependency. How to use spark-testing-base_2.11 version 1.5.1_0.9.0 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...? How to use  com.holdenkarau - spark-testing-base_2.11 version 1.5.1_0.9.0...com.holdenkarau  - Version 1.5.1_0.9.0 of spark-testing-base_2.11 Maven
com.databricks - spark-avro_2.10 version 0.2.0 Maven dependency. How to use spark-avro_2.10 version 0.2.0 in pom.xml?
com.databricks  - Version 0.2.0 of spark-avro_2.10 Maven dependency? How to use  com.databricks  - Version 0.2.0 of spark-avro_2.10 in pom.xml? How to use spark-avro_2.10 version 0.2.0 in pom.xml? Learn to use
com.crealytics - spark-excel_2.11 version 0.10.2 Maven dependency. How to use spark-excel_2.11 version 0.10.2 in pom.xml?
com.crealytics  - Version 0.10.2 of spark-excel_2.11 Maven dependency? How to use  com.crealytics  - Version 0.10.2 of spark-excel_2.11 in pom.xml? How to use spark-excel_2.11 version 0.10.2 in pom.xml? Learn to use
com.holdenkarau - spark-testing-base_2.10 version 1.6.1_0.6.0 Maven dependency. How to use spark-testing-base_2.10 version 1.6.1_0.6.0 in pom.xml?
of spark-testing-base_2.10 in pom.xml? How to use spark-testing-base_2.10 version...? How to use  com.holdenkarau - spark-testing-base_2.10 version 1.6.1_0.6.0...com.holdenkarau  - Version 1.6.1_0.6.0 of spark-testing-base_2.10 Maven
com.crealytics - spark-excel_2.11 version 0.8.4 Maven dependency. How to use spark-excel_2.11 version 0.8.4 in pom.xml?
com.crealytics  - Version 0.8.4 of spark-excel_2.11 Maven dependency? How to use  com.crealytics  - Version 0.8.4 of spark-excel_2.11 in pom.xml? How to use spark-excel_2.11 version 0.8.4 in pom.xml? Learn to use
com.spotify - spark-bigquery_2.10 version 0.1.2 Maven dependency. How to use spark-bigquery_2.10 version 0.1.2 in pom.xml?
com.spotify  - Version 0.1.2 of spark-bigquery_2.10 Maven dependency? How to use  com.spotify  - Version 0.1.2 of spark-bigquery_2.10 in pom.xml? How to use spark-bigquery_2.10 version 0.1.2 in pom.xml? Learn to use
com.holdenkarau - spark-testing-base_2.11 version 1.5.1_0.7.4 Maven dependency. How to use spark-testing-base_2.11 version 1.5.1_0.7.4 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...? How to use  com.holdenkarau - spark-testing-base_2.11 version 1.5.1_0.7.4...com.holdenkarau  - Version 1.5.1_0.7.4 of spark-testing-base_2.11 Maven
com.holdenkarau - spark-testing-base_2.11 version 2.0.2_0.6.0 Maven dependency. How to use spark-testing-base_2.11 version 2.0.2_0.6.0 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...? How to use  com.holdenkarau - spark-testing-base_2.11 version 2.0.2_0.6.0...com.holdenkarau  - Version 2.0.2_0.6.0 of spark-testing-base_2.11 Maven
com.springml - spark-salesforce_2.10 version 1.0.5 Maven dependency. How to use spark-salesforce_2.10 version 1.0.5 in pom.xml?
? How to use  com.springml  - Version 1.0.5 of spark-salesforce_2.10 in pom.xml? How to use spark-salesforce_2.10 version 1.0.5 in pom.xml? Learn...; dependency? How to use  com.springml - spark-salesforce_2.10 version
com.holdenkarau - spark-testing-base_2.11 version 2.2.1_0.11.0 Maven dependency. How to use spark-testing-base_2.11 version 2.2.1_0.11.0 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...; dependency? How to use  com.holdenkarau - spark-testing-base_2.11 version...com.holdenkarau  - Version 2.2.1_0.11.0 of spark-testing-base_2.11 Maven
com.holdenkarau - spark-testing-base_2.11 version 1.6.3_0.6.0 Maven dependency. How to use spark-testing-base_2.11 version 1.6.3_0.6.0 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...? How to use  com.holdenkarau - spark-testing-base_2.11 version 1.6.3_0.6.0...com.holdenkarau  - Version 1.6.3_0.6.0 of spark-testing-base_2.11 Maven
com.crealytics - spark-excel_2.11 version 0.9.18 Maven dependency. How to use spark-excel_2.11 version 0.9.18 in pom.xml?
com.crealytics  - Version 0.9.18 of spark-excel_2.11 Maven dependency? How to use  com.crealytics  - Version 0.9.18 of spark-excel_2.11 in pom.xml? How to use spark-excel_2.11 version 0.9.18 in pom.xml? Learn to use
com.holdenkarau - spark-testing-base_2.11 version 1.5.0_0.7.0 Maven dependency. How to use spark-testing-base_2.11 version 1.5.0_0.7.0 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...? How to use  com.holdenkarau - spark-testing-base_2.11 version 1.5.0_0.7.0...com.holdenkarau  - Version 1.5.0_0.7.0 of spark-testing-base_2.11 Maven
com.springml - spark-salesforce_2.10 version 1.0.1 Maven dependency. How to use spark-salesforce_2.10 version 1.0.1 in pom.xml?
? How to use  com.springml  - Version 1.0.1 of spark-salesforce_2.10 in pom.xml? How to use spark-salesforce_2.10 version 1.0.1 in pom.xml? Learn...; dependency? How to use  com.springml - spark-salesforce_2.10 version
com.holdenkarau - spark-testing-base_2.10 version 1.4.0_0.7.1 Maven dependency. How to use spark-testing-base_2.10 version 1.4.0_0.7.1 in pom.xml?
of spark-testing-base_2.10 in pom.xml? How to use spark-testing-base_2.10 version...? How to use  com.holdenkarau - spark-testing-base_2.10 version 1.4.0_0.7.1...com.holdenkarau  - Version 1.4.0_0.7.1 of spark-testing-base_2.10 Maven
com.holdenkarau - spark-testing-base_2.10 version 1.6.3_0.4.7 Maven dependency. How to use spark-testing-base_2.10 version 1.6.3_0.4.7 in pom.xml?
of spark-testing-base_2.10 in pom.xml? How to use spark-testing-base_2.10 version...? How to use  com.holdenkarau - spark-testing-base_2.10 version 1.6.3_0.4.7...com.holdenkarau  - Version 1.6.3_0.4.7 of spark-testing-base_2.10 Maven
com.holdenkarau - spark-testing-base_2.10 version 1.3.0_0.3.0 Maven dependency. How to use spark-testing-base_2.10 version 1.3.0_0.3.0 in pom.xml?
of spark-testing-base_2.10 in pom.xml? How to use spark-testing-base_2.10 version...? How to use  com.holdenkarau - spark-testing-base_2.10 version 1.3.0_0.3.0...com.holdenkarau  - Version 1.3.0_0.3.0 of spark-testing-base_2.10 Maven
com.crealytics - spark-google-analytics_2.11 version 1.1.1 Maven dependency. How to use spark-google-analytics_2.11 version 1.1.1 in pom.xml?
com.crealytics  - Version 1.1.1 of spark-google-analytics_2.11 Maven dependency? How to use  com.crealytics  - Version 1.1.1 of spark-google-analytics_2.11 in pom.xml? How to use spark-google-analytics_2.11 version 1.1.1
com.holdenkarau - spark-testing-base_2.11 version 2.3.3_0.12.0 Maven dependency. How to use spark-testing-base_2.11 version 2.3.3_0.12.0 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...; dependency? How to use  com.holdenkarau - spark-testing-base_2.11 version...com.holdenkarau  - Version 2.3.3_0.12.0 of spark-testing-base_2.11 Maven
com.holdenkarau - spark-testing-base_2.11 version 1.6.0_0.11.0 Maven dependency. How to use spark-testing-base_2.11 version 1.6.0_0.11.0 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...; dependency? How to use  com.holdenkarau - spark-testing-base_2.11 version...com.holdenkarau  - Version 1.6.0_0.11.0 of spark-testing-base_2.11 Maven
com.holdenkarau - spark-testing-base_2.10 version 1.6.0_0.3.3 Maven dependency. How to use spark-testing-base_2.10 version 1.6.0_0.3.3 in pom.xml?
of spark-testing-base_2.10 in pom.xml? How to use spark-testing-base_2.10 version...? How to use  com.holdenkarau - spark-testing-base_2.10 version 1.6.0_0.3.3...com.holdenkarau  - Version 1.6.0_0.3.3 of spark-testing-base_2.10 Maven
com.holdenkarau - spark-testing-base_2.10 version 2.1.0_0.9.0 Maven dependency. How to use spark-testing-base_2.10 version 2.1.0_0.9.0 in pom.xml?
of spark-testing-base_2.10 in pom.xml? How to use spark-testing-base_2.10 version...? How to use  com.holdenkarau - spark-testing-base_2.10 version 2.1.0_0.9.0...com.holdenkarau  - Version 2.1.0_0.9.0 of spark-testing-base_2.10 Maven
com.holdenkarau - spark-testing-base_2.11 version 1.5.0_0.7.2 Maven dependency. How to use spark-testing-base_2.11 version 1.5.0_0.7.2 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...? How to use  com.holdenkarau - spark-testing-base_2.11 version 1.5.0_0.7.2...com.holdenkarau  - Version 1.5.0_0.7.2 of spark-testing-base_2.11 Maven
com.holdenkarau - spark-testing-base_2.11 version 1.5.0_0.7.2 Maven dependency. How to use spark-testing-base_2.11 version 1.5.0_0.7.2 in pom.xml?
of spark-testing-base_2.11 in pom.xml? How to use spark-testing-base_2.11 version...? How to use  com.holdenkarau - spark-testing-base_2.11 version 1.5.0_0.7.2...com.holdenkarau  - Version 1.5.0_0.7.2 of spark-testing-base_2.11 Maven
Version of com.revelytix>spark dependency
List of Version of com.revelytix>spark dependency
How to check no records - JSP-Servlet
How to check no records  Hi all, I am developing an onlineshopping project, in that how i have 2 check no records. I saw an answer....... At first when a user registers means, his/her cart is empty na... how
how to check a remote applications validity
how to check a remote applications validity  i am developing a network application to avoid unauthorized access to a port .. for that i need to get the application id . how to get the appln id of an appln. or is there any way
How to check whether database exists?
How to check whether database exists?  Hi, I have to create... to creating the database, i want to check whether database exists or not? How can i do this check. can any one help me on this. Please find thesample codeADS
com.logimethods - nats-connector-spark-scala_2.11 version 0.3.0 Maven dependency. How to use nats-connector-spark-scala_2.11 version 0.3.0 in pom.xml?
com.logimethods  - Version 0.3.0 of nats-connector-spark-scala_2.11 Maven dependency? How to use  com.logimethods  - Version 0.3.0 of nats-connector-spark-scala_2.11 in pom.xml? How to use nats-connector-spark-scala_2.11
how to print spark dataframe data
how to print spark dataframe data  Hi, I have a dataframe in spark and i want to print all the data on console. How it can be done. how to print spark dataframe data? Thanks   Hi, Suppose your data frame
how to print spark dataframe data
how to print spark dataframe data  Hi, I have a dataframe in spark and i want to print all the data on console. How it can be done. how to print spark dataframe data? Thanks   Hi, Suppose your data frame
check
updated"); will the above code check if the user has entered value for empcode
Scala Shell in Spark - How do I run Scala in spark shell?
= 3.0.1 If you want to check the version of spark then you should run following... will show you how to run the Scala shell that comes with the Apache Spark... are using Apache Spark version 3.0.1 and our code is displaying the correct
How to check a file exists or not in java.
Description: This example demonstrate how to check a file exits at a given path or not.  The method isFile() of the File class check for its existence. As in the file path we see '\' (backslash) that is used to denote
How to check a file is hidden or not in java
Description: This example demonstrate how to check a file is hidden at a given path or not.  The method isHidden() of the File class check for its hidden property of a file. As in the file path we see '\' (backslash

Ads