How to check spark version

Ads
 

How to check spark version

Hi,

How can I find which version of Apache Spark is running on my environment?

How to check spark version?

Thanks

View Answers

November 27, 2017 at 7:20 PM

Hi,

You can login to your box where apache spark is running then run the command:

spark-shell

It will display the version as shown below:

[[email protected] ~]# spark-shell
SPARK_MAJOR_VERSION is set to 2, using Spark2
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://172.17.0.2:4041
Spark context available as 'sc' (master = local[*], app id = local-1511778019896).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.1.2.6.1.0-129
      /_/

Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_141)
Type in expressions to have them evaluated.
Type :help for more information.

scala>

scala>

Thanks

Ads









Related Tutorials/Questions & Answers:
How to check spark version
How to check spark version  Hi, How can I find which version of Apache Spark is running on my environment? How to check spark version? Thanks... the command: spark-shell It will display the version as shown below
How to check spark version
How to check spark version  Hi, How can I find which version of Apache Spark is running on my environment? How to check spark version? Thanks... the command: spark-shell It will display the version as shown below
Advertisements
how to check gremlin version
how to check gremlin version  Hi, I am inside gremlin console and want to check the version. How to check gremlin version? thanks   Hi...; Gremlin.version() ==>3.3.1 this will print the gremlin version. Thanks
How to check keras version in anaconda
How to check keras version in anaconda  Hi, I have installed anaconda python for machine learning in my computer. Now I want to know which version... of keras library installed? How to check keras version in anaconda? Thanks
How to check Hadoop version in Ubuntu?
How to check Hadoop version in Ubuntu?  Hi, I am trying various... to Hadoop cluster through ssh terminal. How to check Hadoop version in Ubuntu? Thanks   Hi, Its easy to check the version of Hadoop in Ubuntu. type
How specify the version of python that spark-submit uses
How specify the version of python that spark-submit uses  Hi, I have installed python 3.7 and want to use with spark. How specify the version of python that spark-submit uses? Thanks   Hi, You have to modify
How to check CentOS version via command line?
How to check CentOS version via command line?  Hi, I have a linux machine and i don't know the version of the CentOS operating system installed... system installed on it. How to check CentOS version via command line? Thanks
how to insert check box
how to insert check box   how to insert check box into jtable row in swing
How to check if a pid is alive?
How to check if a pid is alive?  Hi, I have a process which is running in background and I want a simple command to find if the process id still running? How to check if a pid is alive? Thanks   HI, You can use
Check TensorFlow version
Check TensorFlow version - How to check TensorFlow version? In this tutorial we will learn how to check the version of TensorFlow machine learning library... program and want to check the version of TensorFlow installed on your computer
Check TensorFlow version
Check TensorFlow version - How to check TensorFlow version? In this tutorial we will learn how to check the version of TensorFlow machine learning library... program and want to check the version of TensorFlow installed on your computer
Check TensorFlow version
Check TensorFlow version - How to check TensorFlow version? In this tutorial we will learn how to check the version of TensorFlow machine learning library... program and want to check the version of TensorFlow installed on your computer
Version of com.revelytix>spark dependency
List of Version of com.revelytix>spark dependency
how to check a remote applications validity
how to check a remote applications validity  i am developing a network application to avoid unauthorized access to a port .. for that i need to get the application id . how to get the appln id of an appln. or is there any way
Scala Shell in Spark - How do I run Scala in spark shell?
will show you how to run the Scala shell that comes with the Apache Spark... the version of spark then you should run following code: scala> spark.version... version 3.0.1 and our code is displaying the correct version of Apache Spark
how to print spark dataframe data
how to print spark dataframe data  Hi, I have a dataframe in spark and i want to print all the data on console. How it can be done. how to print spark dataframe data? Thanks   Hi, Suppose your data frame
how to print spark dataframe data
how to print spark dataframe data  Hi, I have a dataframe in spark and i want to print all the data on console. How it can be done. how to print spark dataframe data? Thanks   Hi, Suppose your data frame
Version of com.alpinenow>plugin-spark dependency
List of Version of com.alpinenow>plugin-spark dependency
Version of com.alta189>auto-spark dependency
List of Version of com.alta189>auto-spark dependency
Version of com.appsflyer>spark-bigquery_2.10 dependency
List of Version of com.appsflyer>spark-bigquery_2.10 dependency
Version of com.appsflyer>spark-bigquery_2.11 dependency
List of Version of com.appsflyer>spark-bigquery_2.11 dependency
Version of com.bizo>spark-plug_2.10 dependency
List of Version of com.bizo>spark-plug_2.10 dependency
Version of com.bizo>spark-plug_2.11 dependency
List of Version of com.bizo>spark-plug_2.11 dependency
Version of com.crealytics>spark-excel_2.10 dependency
List of Version of com.crealytics>spark-excel_2.10 dependency
Version of com.crealytics>spark-excel_2.11 dependency
List of Version of com.crealytics>spark-excel_2.11 dependency
Version of com.databricks>spark-avro_2.10 dependency
List of Version of com.databricks>spark-avro_2.10 dependency
Version of com.databricks>spark-avro_2.11 dependency
List of Version of com.databricks>spark-avro_2.11 dependency
Version of com.databricks>spark-csv_2.10 dependency
List of Version of com.databricks>spark-csv_2.10 dependency
Version of com.databricks>spark-csv_2.11 dependency
List of Version of com.databricks>spark-csv_2.11 dependency
Version of com.databricks>spark-redshift_2.10 dependency
List of Version of com.databricks>spark-redshift_2.10 dependency
Version of com.databricks>spark-redshift_2.11 dependency
List of Version of com.databricks>spark-redshift_2.11 dependency

Ads