PySpark is one of the popular API for writing Spark Jobs in Python programming language. Python is one of most popular programming language for developing machine learning and deep learning program. In this section we are discussing the questions of PySpark. We will try to provide answers of your questions. Ask questions related to PySpark programming language.

• python date parse

  Hi, I want to parse the string date "20220131" into Python date. How to do that? Thanks

  Posted on: October 26, 2022 at 3:05 AM
  View Post

• python date parse

  Hi, I want to parse the string date "20220131" into Python date. How to do that? Thanks

  Posted on: October 26, 2022 at 3:05 AM
  View Post

• python add 1 day to date

  Hi, In one of my project I am getting date from the some datasource and now I have to add one day to the date. How I can add 1 day to date in my Python program. Thanks

  Posted on: October 25, 2022 at 8:09 AM
  View Post

• node dependency mysql package.json

  Hi, How to add node dependency mysql package.json? Thanks

  Posted on: July 4, 2022 at 2:30 PM
  View Post

• How do you get the size of an array in Python

  Hi, I want to get the size of my array in Python. How do you get the size of an array in Python? How we are doing it in Python? Thanks

  Posted on: July 19, 2021 at 12:15 PM
  View Post

• Where is the lit() function in pyspark 2.2.0?

  Hi, I want to use the lit function in PySpark program. I am using Spark 2.2.0. Where is the lit() function in pyspark 2.2.0? Thanks

  Posted on: November 8, 2020 at 4:49 PM
  View Post

• pyspark lit function not found

  Hi, I want to use the lit function in PySpark Program. But unable to find it to use in program. How to use pyspark lit function and remove the pyspark lit function not found error? Thanks

  Posted on: November 8, 2020 at 4:39 PM
  View Post

• how to show dataframe in pyspark

  Hi, I want to display 5 records from dataframe in PySpark program. How to display specific no of records of Data Frame in Python? Thanks

  Posted on: November 4, 2020 at 7:17 AM
  View Post

• ImportError: No module named geog

  Hi, I am importing import geog and getting following error: ImportError: No module named geog How to solve this? Thanks

  Posted on: September 10, 2020 at 6:36 AM
  View Post

• ModuleNotFoundError: No module named 'IO'

  Hi, I am trying to import IO package of Python with following code: from IO import StringIO But program throws the error: ModuleNotFoundError: No module named 'IO'. How to resolve thi

  Posted on: July 16, 2020 at 1:05 PM
  View Post

• Steps to read text file in pyspark

  Hi, I am learning to write program in PySpark. I want to simply read a text file in Pyspark and then try some code. What are the Steps to read text file in pyspark? How much time it takes to learn PySpark Programming to get

  Posted on: April 22, 2020 at 9:48 AM
  View Post

• pip install requests

  Hi, What is the use of following command? pip install requests What is requests package in Python? I need some example of this package in Python 3. Thanks

  Posted on: April 11, 2020 at 8:08 AM
  View Post

• How to install the Six module in Python

  Hi, I want to install the six module in Python as my program is using this. I have checked it by importing this module in my program but by default it is not coming with the Python distribution. How to install the Six module in Pyt

  Posted on: April 11, 2020 at 1:03 AM
  View Post

• scala.collection.immutable.map$map1 cannot be cast to java.util.map

  Hi, I am trying to cast the Map in spark java but it throws following error: scala.collection.immutable.map$map1 cannot be cast to java.util.map How to solve this? Thanks

  Posted on: February 10, 2020 at 5:39 AM
  View Post

• ModuleNotFoundError: No module named 'pandas'

  Hi, I have been assigned work on python pandas and there is already written code. Which throws following error: ModuleNotFoundError: No module named 'pandas' More details about the error:

  Posted on: January 19, 2019 at 9:57 PM
  View Post

• pyspark : An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext

  Hi I am running pyspark code and it was running fine yesterday. Now its giving the error: pyspark : An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext How to resolve this? Here is the fu

  Posted on: December 27, 2018 at 12:57 AM
  View Post

• pip install beautifulsoup - install beautifulsoup on Ubuntu

  HI, I want to install beautifulsoup on my Ubuntu desktop operating system. What is the command to install beautifulsoup on Ubuntu 18.04? Thanks

  Posted on: December 24, 2018 at 10:27 AM
  View Post

• How should I start learning Python?

  Hi, I want to learn PySpark for machine learning projects. But for learning PySpark we have to learn Python first. How should I learn Python myself. I wish to learn Python programming as soon as possible. How should I start learning Python

  Posted on: December 16, 2018 at 9:36 PM
  View Post

• ImportError: No module named pandas - Error in sparkshell

  Hi, I am new to pyspark and learning it from beginning. I have successfully installed Spark in Ubuntu and now trying to write some program. I am trying to convert pandas array into dataframe but it giving error: ImportError

  Posted on: November 22, 2018 at 9:13 AM
  View Post

• importing pyspark in python shell

  Hi, I am learning to develop develop code for Spark in PySpark. Now I have installed Apache Spark but the issue is that its not coming in the python shell. How should I do for importing pyspark in python shell? I am experie

  Posted on: November 22, 2018 at 8:43 AM
  View Post

View all discussion for PySpark Questions and answers.