Hello @kunal_agarwal  If you are using Knox Gateway, it may be the bug, presented here To fix it you could apply this changes  to file: ${KNOX_GATEWAY_HOME}/data/services/yarnui/2.7.0/rewrite.xml to rewrite rules of yarnui service ... View more

Run msck command for the table you want to truncate in hive shell. hive> use <database-name>; hive> msck repair table <table-name>; If it will show any error then rectify it as we got one of our partition was missing.   So we create that partition directory on hdfs location and re-run msck repair command. Now it would not show any issue.   Now running truncate command will run successfully.  hive> truncate table <table-name>;   [NOTE: Please update database and table name as per the requirement] ... View more

Run Hive Jobs with Oozie Specify the hive-site.xml in the job-xml parameter. Specify the name of the script (for example, script.q ) that contains the hive query in the script parameter. Optionally, add properties used by the Oozie launcher job. Add the prefix oozie.launcher to the property names.   see here Dgcustomefirst ... View more

Hi Amit, what is that sparkUber.jar i copied spark-assembly jar to /oozie/share/lib/spark folder please let me know about that sparkUber.jar ... View more

It looks like tempTable is not sticking around: scala> case class rowschema(id:Int, record:String){ | val df = sqlContext.createDataFrame(Seq(rowschema(1,"record1"), rowschema(2,"record2"), rowschema(3,"record3")));{ | df.registerTempTable("tempTable") | } | } defined class rowschema scala> sqlContext.sql("create table newHiveTable as select * from tempTable") org.apache.spark.sql.AnalysisException: Table not found: tempTable; line 1 pos 43 ... View more

@Hari Rongali This is a current limitation with ACID transactions supported only in ORC format. ... View more

Hi @milind pandit, @AnjiReddy Anumolu, I am facing the same issue. Would you please share how did you resolve this issue? I am confused about the GetFile and PutHDFS processor configuration. let me explain a little more: I just want to setup a data flow with nifi for now to test whether i can transfer files from HDF cluster to HDFS of HDP cluster.For that I am just using two processor "GetFile" and "PutHDFS". I deployed 2 clusters with 2 nodes of each- HDF 3.1(2 nodes-Nifi and services) and HDP 2.6.4(2 nodes- Master and worker). Now I want to transfer files through Nifi and write those data in HDFS . How can I do that? Please share your experiences how you resolved this. Thank you. ... View more

@AnjiReddy Anumolu Just to add a little more detail to the above response from @zblanco. When NiFi ingest data, that data is turned in to NiFi FlowFiles. A NiFi FlowFile consists of Attributes (Metadata) about the actual data and the physical data. The FlowFile metadata is stored in the FlowFile repository as well as JVM heap memory for faster performance. The FlowFile Attributes includes things like filename, ingest time, lineage age, filesize, what connection the FlowFile currently resides in dataflow, any user defined metadata, or processor added metadata, etc....). The physical bytes that make up the actual data content is written to claims within the NiFi content repository. A claim can contain the bytes for 1 to many ingest data files. For more info on the content repository and how claims work, see the following link: https://community.hortonworks.com/articles/82308/understanding-how-nifis-content-repository-archivi.html Thanks, Matt ... View more