I want to use the lit function in PySpark program. I am using Spark 2.2.0.
Where is the lit() function in pyspark 2.2.0?
The lit function is frequently used function in Spark while developing the PySpark applications.
You can find it in the
pyspark.sql.functions package. To use this function in your program you add following code:
from pyspark.sql.functions import lit
After this lit function can be used in PySpark program. Here is the simple use of lit() function in Pyspark:
df2 = df.withColumn("SomeField",lit("1"))
Above code adds a new column named "SomeField" with value 1.
You can explore the use of lit() function at: