I recommend to trigger the manual rebalancing via kafka-reassign-partition.sh, not by bringing down one node. In most scenarios, when you bring down one node and bring it back on afterwards, it is also not considered for balancing. And you might cause other issues as well. https://community.hortonworks.com/content/supportkb/150063/how-to-add-new-kafka-brokers-to-a-cluster-and-move.html ... View more

and no user connected to the system to start jobs in your cluster? ... View more

This is typically the case when the resources are exceeded. This could be the memory of the node, but also the queue itself. Can you check if the jobs getting stuck are all submitted in the same queue? https://community.hortonworks.com/questions/96750/yarn-application-stuck-in-accepted-state.html ... View more

did you check this: https://community.hortonworks.com/questions/2349/tip-when-you-get-a-message-in-job-log-user-dr-who.html The solution there was: RESOLUTION Customer changed the following property in core-site.xml to resolve the issue. Other values such as hdfs or mapred also resolve the issue. If the cluster is managed by Ambari, this should be added in Ambari > HDFS > Configurations>Advanced core-site > Add Property hadoop.http.staticuser.user=yarn ... View more

so it has the format of basically providing the sql statements? Like this example from Oracle? begin insert into ODI_STAGE.SRC_SALES_PERSON1 values (11,'Andrew','Andersen','22/02/1999',sysdate); insert into ODI_STAGE.SRC_SALES_PERSON1 values (12,'John','Galagers','20/04/2000',sysdate); end; If that's the case, you can process and transform the file with Nifi. And here is a nice tutorial on how to achieve it using GetFile and SplitText processors: https://community.hortonworks.com/articles/66861/nifi-etl-removing-columns-filtering-rows-changing.html ... View more

I guess you need to first give some details on the format or origin of your .DAT file. There is more than just one format associated to this extension. ... View more

sorry, my response contained a typo, should have been: scan 'PUR_ACCT_PHX',{VERSIONS=>40} ... View more

did you just left out the timestamp from the log entry, or is a log line exactly as you provided? In your example you only have 1 number preceding the IP address, but you try to read two numbers (timestamp and duration)? ... View more

i am not sure if this is your issue, but i found the query syntax to ask for more versions to be: scan 'PUR_ACCT_PHX',{VERSIONS >=40} you might want to try it. ... View more

good to hear it worked. it would be great if you accept the answer so that this thread is marked as closed. I submitted the hint on the documentation as an idea. ... View more