Support Questions
Find answers, ask questions, and share your expertise

pyspark question

pyspark question

%livy2.pyspark

datacontent = sc.textFile("file:///tmp/password.txt")

datacontent.collect()

When I run the scipt, I am getting the password, but how do I get only password without U, I need to assign it to another connection

[u'password']

1 REPLY 1

Re: pyspark question

@Anpan K,

u is the unicode representation of the string in python. To convert it into a normal string, you can use str() method. You can test it in python console

# a = u'teststring'
# a
>> u'teststring'
# b = str(a)
# b
>> 'teststring'

For your use case, you can use

datacontent = sc.textFile("file:///tmp/password.txt")
datacontent = datacontent.map(lambda x: str(x))
datacontent.collect()