Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can I configure Spark to connect to Cassandra in HDP 2.4?

How can I configure Spark to connect to Cassandra in HDP 2.4?

New Contributor

How can I configure Spark to connect to Cassandra in HDP 2.4?

https://community.hortonworks.com/questions/32957/how-can-i-install-cassandra-in-hdp-24-sandbox-virt...

I installed successfully Cassandra in hdp24, now I like to use pyspark to transform and save my Data back to Cassandra like shown here:

https://docs.datastax.com/en/datastax_enterprise/4.6/datastax_enterprise/spark/sparkPySpark.html

I could Create the Keyspace, Create the Tables and Insert the data.

Now I need the Spark Context with the connection settings to Cassandra..Someone got an Idea how I can do this in the Sandbox? (In the term of configuring Spark to be able to connect to Cassandra)

5 REPLIES 5

Re: How can I configure Spark to connect to Cassandra in HDP 2.4?

Hello Lukas:

I'm sorry to refer you to StackOverflow, but it appears that this is how you do it:

StackOverflow Connect to Cassandra over JDBC

JDBC Connection string sytax:

jdbc:cassandra://host1--host2--host3:9160/keyspace1?primarydc=DC1&backupdc=DC2&consistency=QUORUM"

Or directly via Java code:

String serverIP = "127.0.0.1";
String keyspace = "system";

Cluster cluster = Cluster.builder()
.addContactPoints(serverIP)
.build();

Session session = cluster.connect(keyspace);

Re: How can I configure Spark to connect to Cassandra in HDP 2.4?

New Contributor

I am new to JDBC Connectors, so where do I put the .jar file ? What are the configuration steps I need to do to get the JDBC connector running?

(Maven? in Cassandra POM or in Spark?)

Re: How can I configure Spark to connect to Cassandra in HDP 2.4?

Guru

Re: How can I configure Spark to connect to Cassandra in HDP 2.4?

New Contributor

@Lukas Ott hello sir

Am new in casandra and HDP !

Can you help me out please

I wonna install the cassandra with HDP 2.6.4 using this link !

https://github.com/Symantec/ambari-cassandra-service

can you help me out please !

Highlighted

Re: How can I configure Spark to connect to Cassandra in HDP 2.4?

New Contributor

Did you follow the READ.ME on the github link? - Please be more precise.