@hud  When you run nifi as a microservice then you can configure PVC [Persistent Volume Claims] using helm in AKS or Kubernetes which will ensure that evenif the nifi pods restarts it will always have the same volume mounted.   Under the persitence configuration the parameter persistence.enabled should be set to true see Helm Chart for Apache Nifi    HTH ... View more

@saivenkatg55    For the screenshots, the 2 notebooks you create are Untitled Note 1 and Untitled Note 2  which should appear in the drop-down list under Notebook on the top menu. Below I will create a Spark interpreter notebook named saivenkatg55   from step 2 above This should appear on the under the notebook Launched a test     I can see the job was accepted and running in RM UI So where exactly are you encountering issues? Happy hadooping !             ... View more

@saivenkatg55  I have tried to analyze your situation but with access to the Linux box it rather difficult,but I think there is a workaround.   The chattr linux command makes important files IMMUTABLE (Unchangeable).   The immutable bit [ +i  ] can only be set by superuser (i.e root) user or a user with sudo privileges can be able to set. This will prevent the file from being deleted forcefully, renamed or change the permissions, but it won’t be allowed says 'Operation not permitted“' # ls -al /var/run/hadoop-yarn/yarn/ total 8 . .. -rw-r--r-- 1 yarn hadoop 0 Dec 24 09:34 hadoop-yarn-nodemanager.pid   Set immutable bit # chattr +i hadoop-yarn-nodemanager.pid   Verify the attribute with command the below command # lsattr ----i--------e-- ./hadoop-yarn-nodemanager.pid   The normal ls command shows no difference # ls -al /var/run/hadoop-yarn/yarn/ total 8 drwxr-xr-x 2 root root 4096 Dec 24 09:34 . drwxr-xr-x 3 root root 4096 Dec 24 09:34 .. -rw-r--r-- 1 yarn hadoop 0 Dec 24 09:34 hadoop-yarn-nodemanager.pid   Deletion protection # rm -rf /var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid rm: cannot remove ‘/var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid’: Operation not permitted   Permission change  protected # chmod 755  /var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid chmod: changing permissions of ‘/var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid’: Operation not permitted How to unset attribute on Files # chattr -i /var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid After resetting permissions, verify the immutable status of files using lsattr command # lsattr ---------------- ./var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid   Please do that and revert      ... View more

@Prakashcit  That by design: A NOVALIDATE constraint is basically a constraint that can be enabled but for which hive will not check the existing data to determine whether there might be data that currently violate the constraint. This is useful if we know there’s data that violates the constraint but we want to quickly put on a constraint to prevent further violations, with the intention to clean up any possible violations at some future point in time. It’s also potentially useful if we know the data is clean and so want to prevent the potentially significant overheads of hive having to check all the data to ensure there are indeed no violations. ... View more

@Uppal    Best way to duplicate a partitioned table in Hive Create the new target table with the schema from the old table  the describe formatted could help with the SQL Use hadoop fs -cp to copy all the partitions from source to the target table Run MSCK REPAIR TABLE table_name; on the target table HTH ... View more

@GrahamB    No you don't need to wait for 24 hours to destroy a kerberos ticket you will need to run on the Kerberos server  as the user   Check valid ticketTo list all of the entries in the default credentials cache $ klist  You should have some out here To delete the default credentials cache for the user $  kdestroy Then to obtain a ticket-granting ticket with a lifetime of 10 hours, which is renewable for five days, type: $ kinit -l 10h -r 5d your_principal   HTH   ... View more

@saivenkatg55    The file permission should be 644 not 444    # chmod 644 /var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid   Do that and revert please ... View more

@saivenkatg55    This "Exiting with status 1: java.io.IOException: Problem starting http server" error should be linked to your other question I just have  responded to  https://community.cloudera.com/t5/Support-Questions/Unable-to-start-the-node-manager/td-p/286013 If this is resolved then the java.io.IOException shouldn't occur   HTH ... View more

@saivenkatg55    I think there is a permission issue with the pid file  Can you check the permissions, if for any reason the are not as shown in the screenshot please run the chown as root to rectify that # chown yarn:hadoop /var/run/hadoop-yarn/yarn/hadoop-yarn-nodemanager.pid Do that for all files in the directory whose permissions are not correct. HTH     ... View more