Member since
09-24-2015
178
Posts
113
Kudos Received
28
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3433 | 05-25-2016 02:39 AM | |
3664 | 05-03-2016 01:27 PM | |
857 | 04-26-2016 07:59 PM | |
14589 | 03-24-2016 04:10 PM | |
2109 | 02-02-2016 11:50 PM |
10-21-2015
02:01 PM
1 Kudo
What is the best way to clean / purge the messages in a Kafka queue (irrespective of whether they have been read or not)? Scenario is that in a dev environment a dummy publisher is pushing messages to a Kafka queue from which a storm topology is reading the messages. If a change has to be made in the publisher such that the current messages in the queue (read or not) are all invalid, so we need to purge those messages and start again.
... View more
Labels:
- Labels:
-
Apache Kafka
10-21-2015
01:56 PM
Based on the error it does seem that the principal already exists. I would have the AD admin look in that OU to confirm that the user is not there.
... View more
10-19-2015
03:41 PM
That database exists. User is able to use that database and also manually verified that in the metastore.
... View more
10-18-2015
03:08 PM
2 Kudos
Terry - Are you using Ambari Wizard for kerberizing the cluster? If so, passwords are randomly generated. Here are the pointers to code that does it - Password is generated here - https://github.com/hortonworks/ambari/blob/d4edf4619c1c0bb309920ba86e66012a2a2e7090/ambari-server/src/main/java/org/apache/ambari/server/security/SecurePasswordHelper.java#L123-L190 The above function is called from here - https://github.com/hortonworks/ambari/blob/d4edf4619c1c0bb309920ba86e66012a2a2e7090/ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java#L150 And then passed to this method for creating principal in KDC / AD - https://github.com/hortonworks/ambari/blob/8967ed9bc8967f6f6783c16f6403a3de0a0b2792/ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandler.java#L241
... View more
10-18-2015
02:56 PM
Can you tell if both these env using same KDC server? DistCP (which is what is used by Falcon for data replication) supports copying data between two kerberos environments so its a matter of making sure that the clusters can communicate correctly.
... View more
10-17-2015
04:00 PM
Hey Chakra - You need to provide the Java class name for "Database Driver Class NameInfo" and not the jar file path. The jar file path goes in Database Driver Jar UrlInfo.
... View more
10-17-2015
01:03 PM
Thanks David, for some reason the 3rd statement to alter the table to rename it fails with following error message - metastore.RetryingHMSHandler (RetryingHMSHandler.java:invoke(159)) - InvalidOperationException(message:Unable to change partition or table. Database th_ti_hit_src does not exist Check metastore logs for detailed stack.junk_th)
at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTable(HiveAlterHandler.java:253)
Looks like we will have to clean the table up manually from metastore and then delete the partitions manually.
... View more
10-15-2015
02:15 PM
How about - ConvertCSVToAvro - & then - ConvertAvroToJSON ? Alternatively, You can use ExtractText with regex to convert CSV to JSON.
... View more
10-14-2015
12:40 PM
Thanks David, will try that and let you know how it goes.
... View more
10-14-2015
05:42 AM
Also, Since the DROP table command just hangs, what will be a good work around to completely removing this table and partitions?
... View more