Member since
04-04-2016
41
Posts
6
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
748 | 04-28-2016 01:28 AM | |
845 | 04-05-2016 05:06 PM |
09-29-2016
09:50 AM
Hi @Predrag Minovic Thanks a lot for the response. this works!! However, this is working only when i am having 2 columns in the input... but not working when there are more columns !!
... View more
09-17-2016
06:40 AM
I have built a storm topology, that consumes data from kafka and writes into hdfs. When using storm & kafka dependencies like below, <storm.version>0.10.0.2.3.4.0-3485</storm.version>
<kafka.version>0.8.2.1</kafka.version> topology is failing with below stated exception: **java.lang.NoSuchMethodError: kafka.javaapi.consumer.SimpleConsumer.<init>(Ljava/lang/String;IIILjava/lang/String;Ljava/lang/String;)V at storm.kafka.DynamicPartitionConnections.register(DynamicPartitionConnections.java:60) at storm.kafka.PartitionManager.<init>(PartitionManager.java:66) at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:138) at backtype.storm.daemon.executor$fn__7098$fn__7113$fn__7142.invoke(executor.clj:596) at backtype.storm.util$async_loop$fn__543.invoke(util.clj:475) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745)**
But, When using storm & kafka dependencies like below topology is running fine and downloading messages and writing to hdfs , without any issues!! <storm.version>0.9.3.2.2.4.0-2633</storm.version>
<kafka.version>0.8.2.1</kafka.version> Can someone please help what is the causing the issue and help me fix the issue !
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Storm
08-01-2016
09:05 PM
Thanks for your response and time @mqureshi I tried for ^^... its still not working...!! And, sorry, my above comment had a typo !! Below is what i am trying !! create external table <table_name> ( col1 string, col2 string )
ROW FORMAT SERDE "org.apache.hadoop.hive.contrib.serde2.RegexSerDe"
WITH SERDEPROPERTIES (
"input.regex" = "^^"
)
STORED AS TEXTFILE
LOCATION "<hdfs_path>";
... View more
08-01-2016
06:52 PM
Thanks for the response @mqureshi I Tried below two ways, but that did not work... "input.regex" = "\\^\^\\^\\^\\^\\^\\^\\^\\^\\^" "input.regex" = "(\\^\^\\^\\^\\^\\^\\^\\^\\^\\^)" Any other thoughts i can try of ?
... View more
08-01-2016
05:35 PM
Can someone pls help me creating regex pattern to use for creating a hive table with RegEx Serde.... I want hive regex table to be created with the pattern ^^^^^^^^^^ [10 anchor characters] as a delimiter! I am not sure what would be the regex pattern of hive table for this!! please help. Thanks a lot in advance
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
05-16-2016
08:16 PM
@pshah okay will check by logging in. Anyways i decompiled in editor and verified the code, and i am good! Do we have any sample application that does the editing of stored offsets in zookeer ?
... View more
05-16-2016
05:54 PM
I have a github account, but not really active on itt!! please let me know what information you are looking for ? @pshah
... View more
05-16-2016
05:52 PM
with the help of the classes you mentioned i am able to check the committed kafka offsets!! I am trying to write an application now to be able to edit the stored offsets!! thanks a lot for your help!! @pshah
... View more
05-12-2016
06:39 PM
Thanks very much @pshah, i will check those classes!!
But, unfortunately, that links you shared navigating me to "page not found"! But i can still check the classes by decompiling my storm version jars....!!
... View more
05-12-2016
04:00 PM
Thanks for the response @pshah i am using the storm version, 0.9.3.2.2.4.0-2633 can you pls help me with what classes in this version are doing this fetching offsets/editing them into zookeeper. Thanks very much. My whole intension is to see the committed offsets and edit them as necessary. Also, in which version of Storm i can say to storm to not use the zookeeper for offsets ?
... View more