Member since
02-01-2019
650
Posts
143
Kudos Received
117
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2612 | 04-01-2019 09:53 AM | |
1376 | 04-01-2019 09:34 AM | |
6473 | 01-28-2019 03:50 PM | |
1484 | 11-08-2018 09:26 AM | |
3609 | 11-08-2018 08:55 AM |
01-04-2017
07:39 PM
@Sami Ahmad Below is the simple wordcount example from Spark docs and this should work with both Spark 1.6.x and Spark 2.x. >>> textFile = sc.textFile("file:///etc/passwd")
>>> wordCounts = textFile.flatMap(lambda line: line.split()).map(lambda word: (word, 1)).reduceByKey(lambda a, b: a+b)
>>> wordCounts.collect() Ref: http://spark.apache.org/docs/latest/quick-start.html#more-on-rdd-operations Hope this helps.
... View more
01-04-2017
08:53 AM
@Maciej Bogucki Zookeeper will have these values as null and -1, But the storm topology will parse the listener : SASL_PLAINTEXT://hdw2dn2.infovidematrix.pl:6667 to get broker hostname and port and this is expected. The bug i mentioned above as the spout was unable to parse the listener and get the hostname and port. This is fixed in the HDP version you are using. (HDP 2.5.3)
... View more
01-03-2017
02:33 PM
@sathish jeganathan I am not sure about the connector, Request you to ask a new question for the same.
... View more
01-03-2017
08:38 AM
@sathish jeganathan I'm not sure about the mysql connector. But the class you are using belongs to https://github.com/confluentinc/kafka-connect-jdbc which is not shipped with HDP.
... View more
01-02-2017
12:06 PM
1 Kudo
@sathish jeganathan You need to add below properties in /etc/kafka/conf/connect-standalone.properties if you are using SASL. producer.security.protocol=SASL_PLAINTEXT
producer.sasl.kerberos.service.name=kafka
consumer.security.protocol=SASL_PLAINTEXT
consumer.sasl.kerberos.service.name=kafka
... View more
01-02-2017
10:00 AM
Here is the content of connect-log4j.properties log4j.rootLogger=DEBUG, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.logger.org.apache.zookeeper=DEBUG
log4j.logger.org.I0Itec.zkclient=DEBUG
... View more
01-02-2017
09:19 AM
Can you change the log4j properties to DEBUG mode in connect-log4j.properties and see what is the error it is throwing ?
... View more
01-02-2017
09:10 AM
@sathish jeganathan Please replace localhost with the broker hostname and re-run the connect command.
... View more
01-02-2017
08:58 AM
1 Kudo
@sathish jeganathan Check connect-standalone.properties file and see if you have the valid broker hostname and port (bootstrap.servers).
... View more
12-30-2016
10:02 AM
thanks @nyadav i don't see the above two properties in spark-thrift-sparkconf. Adding them helped in resolving the issue.
... View more