@Anishkumar Valsalam Once you have developed a custom NAR file, you simply have to drop it in /usr/hdf/current/nifi/lib/ folder and restart Nifi. Once Nifi is restarted, you should be able to see your new custom processor along with other processors. ... View more

@PPR Reddy When you say schema was dropped, you mean, you dropped the table which means the metadata is gone. Your data still exists, right? You just have to run a create table again. Don't you have this saved somewhere already? Why not just create the table. If not, then, assuming your metadata table is MySQL, you can use the following method: https://twindb.com/recover-after-drop-table-innodb_file_per_table-is-off/ What is a little surprising is you have not lost the data so writing a create table statement again even if its 100 lines is not as bad as if your data was in mysql and you had dropped it without a backup. that would have been a much bigger issue. ... View more

@Tech Gig Which version of Hadoop are you using? In an HA cluster between Namenode and Standby namenode, there is Quorum journal manager (usually three nodes - one disk each). Assume everything is up to date. Now if a namespace change occurs, namenode is going to write to Quorum journal manager the same change. Standby Namenode is also watching this Quorum journal manager and promptly apply the changes to its own copy of namespace. To ensure fast failover, datanodes are also configured with the location of both namenode and standby namenode and they send block information and heartbeats to both namenodes (although only one is active and other is standby). For additional prevention of corruption of data, administrators use some fencing mechanism to prevent what is called a "split-brain-scenario". One way to achieve this is having journal nodes allow write operation by only one namenode. When your active namenode goes down, your standby will become the writer of journal nodes. Now if standby is down, when it comes back up, it reads the journal nodes to bring itself up to date. ... View more

@ed day If you followed the process in that link then you shouldn't be running into this issue. You don't really need to download Ambari jar separately. Can you please confirm that you have the repo file for your operating system? If you are on CentOS 7 then is your file for CentOS 7? http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.0.1/bk_ambari-upgrade/content/upgrade_ambari.html ... View more

@bhavik shah Are you willing to write a map reduce job? You can use mapreduce.input.fileinputformat.split.minsize to reduce the split size of your ORC file. The split is calculated using max(mapreduce.input.fileinputformat.split.minsize, min(mapreduce.input.fileinputformat.split.maxsize , dfs.blocksize)) Set your mapreduce.input.fileinputformat.split.minsize to 50 MB and then send the output of each mapper to S3. Writing a mapreduce will be right way to do it. If you don't want to write a map reduce and instead use hive, then you will have to create a new table with same data and use Insert Select to populate new table: set hive.merge.mapredfiles=true; set hive.merge.mapfiles=true; set hive.merge.smallfiles.avgsize=51200000; set hive.merge.size.per.task=51200000; ... View more

@dominic lodento This is coming from CentOS. How much memory do you have? For Ambari, can you please check Ambari logs under /var/log? ... View more

@John T I am a little confused since you say file name is "file123.txt" but then say you don't know how the file name starts or what the extension is (seems like extension is .txt). Remote File Name supports expression language so you should be able to use "contains" for the file. ... View more