Created 05-13-2016 11:02 AM
How can I configure Spark to connect to Cassandra in HDP 2.4?
I installed successfully Cassandra in hdp24, now I like to use pyspark to transform and save my Data back to Cassandra like shown here:
https://docs.datastax.com/en/datastax_enterprise/4.6/datastax_enterprise/spark/sparkPySpark.html
I could Create the Keyspace, Create the Tables and Insert the data.
Now I need the Spark Context with the connection settings to Cassandra..Someone got an Idea how I can do this in the Sandbox? (In the term of configuring Spark to be able to connect to Cassandra)
Created 05-13-2016 08:13 PM
Hello Lukas:
I'm sorry to refer you to StackOverflow, but it appears that this is how you do it:
StackOverflow Connect to Cassandra over JDBC
JDBC Connection string sytax:
jdbc:cassandra://host1--host2--host3:9160/keyspace1?primarydc=DC1&backupdc=DC2&consistency=QUORUM"
Or directly via Java code:
String serverIP = "127.0.0.1"; String keyspace = "system"; Cluster cluster = Cluster.builder() .addContactPoints(serverIP) .build(); Session session = cluster.connect(keyspace);
Created 05-18-2016 07:23 AM
I am new to JDBC Connectors, so where do I put the .jar file ? What are the configuration steps I need to do to get the JDBC connector running?
(Maven? in Cassandra POM or in Spark?)
Created 05-13-2016 09:17 PM
The instructions at https://docs.datastax.com/en/datastax_enterprise/4.6/datastax_enterprise/spark/sparkPySpark.html are for dse pyspark. You can either install that version or try https://github.com/TargetHolding/pyspark-cassandra or https://github.com/Parsely/pyspark-cassandra
Created 04-02-2018 04:17 PM
@Lukas Ott hello sir
Am new in casandra and HDP !
Can you help me out please
I wonna install the cassandra with HDP 2.6.4 using this link !
https://github.com/Symantec/ambari-cassandra-service
can you help me out please !
Created 04-03-2018 06:54 AM
Did you follow the READ.ME on the github link? - Please be more precise.