I am using https://github.com/rcongiu/Hive-JSON-Serde to query the JSON data via hive. As part of testing, I am using an external table to query the JSON plain text file in HDFS. i am able to query the data from hive using select, However when i do select * from JSON_EXTERNAL_TABLE limit 1, the output is an Invalid JSON though the message in HDFS is a valid JSON. Is this an expected one ? ... View more

I am working Chef blueprint deployment where in blueprint json memory values gets adjusted based on the system memory available. the target system system memory can be from 8 GB to 145GB. HDP will be used for running OpenSOC.(Storm,Kafka,HBase,HDFS) For Example Kafka,KAFKA Heap > 5GB would not give any benefit whatever the source memory is. So looking for automic sizing of the componetns(Hbase,Storm,AMS,HDFS) where more memory would help the system to perform better . This is for HDP 2.2.8 and Ambari 2.1.2. ... View more

Kafka error logs are getting filled bringing the kafka down.Looking for options to purge the old kafka errors logs. Logs that are getting fillled are server.log.**. kafka.out. ... View more