In Nifi, There is a bootstrap.conf file inside the conf folder.  You can update the value of the following properties, # JVM memory settings java.arg.2=-Xms2g  java.arg.3=-Xmx8g https://community.cloudera.com/t5/Community-Articles/How-to-address-JVM-OutOfMemory-errors-in-NiFi/ta-p/244431  ... View more

Please note that the file upload size is limited by the /var/lib/ambary-server/resources and you'll definitely need USE permissions on 'default' as the procedure relies on 'create table as select' from an intermediate table stored in. ... View more

If the input is text data, its better to write a java utility which reads line by line, applies all the rules & re-writes it as four column as per conversion required. Then load that new file into hive to query it. It will be cumbursome to do it in sql, as it needs another case statement inside case statement to validate the length of the split. ... View more

I think you can use regex to load data .     ... View more

@Harsh J is there a way to avoid giving the column names manually...   beacuse I have 150 columns per table and more than 200 tables which is a huge number. ... View more

  I tried using below command but getting error   hive> LOAD DATA LOCAL INPATH 'hdfs location of .avro' OVERWRITE INTO TABLE hive_from_avro_schema; FAILED: SemanticException [Error 10028]: Line 1:23 Path is not legal '' hdfs location of .avro '': Source file system should be "file" if "local" is specified hive>   ... View more

I got the solution...we need to provide fsUri after hdfs to acces hdfs location ... View more

No, STORED AS AVRO will dynamically convert your table column metadata information to a schema at runtime. In general if you have avro converted files under the table directory, you can use 'hadoop jar /opt/cloudera/parcels/CDH/lib/avro/avro-tools.jar getschema hdfs://path/to/file.avro to fetch its generated schema. ... View more