Scenario: Trying to add new columns to an already partitioned Hive table. Problem: The newly added columns will show up as null values on the data present in existing partitions. Solution: One of the workaround can be copying/moving the data in a temporary location,dropping the partition, adding back the data and then adding back the partition. It works and the new column picks up the values. But for big tables this is not a viable solution. Best approach: We need to construct the alter statement to add columns with CASCADE option as follows: ALTER TABLE default.test_table ADD columns (column1 string,column2 string) CASCADE; From the Hive documentation: “ALTER TABLE CHANGE COLUMN with CASCADE command changes the columns of a table's metadata, and cascades the same change to all the partition metadata. RESTRICT is the default, limiting column change only to table metadata.” I found out that this option is not at all in wide use and can help who face this situation. Thanks ... View more

Problem: Trying to use sudo to run commands(as root) in a cluster gives error " Stderr: Pseudo-terminal will not be allocated because stdin is not a terminal. sudo: sorry, you must have a tty to run sudo" This is a common situation where you are login as some other user in a terminal and then sudo to root to perform the task. And you can only ssh from the user you logged in and not from root. Solution: Note: You have to install pssh if it is not already installed. in your cluster. Keep all host names in a file say all_hosts This kind of execution will fail: pssh -i -h all_hosts "sudo whoami;hostname -f" Will fail to execute and give error: Stderr: sudo: sorry, you must have a tty to run sudo OR pssh -i -h all_hosts -x "-t" "sudo whoami;hostname -f" Will fail to execute and give error: Stderr: Pseudo-terminal will not be allocated because stdin is not a terminal. sudo: sorry, you must have a tty to run sudo The correct command should be run like this: pssh -i -h all_hosts -x "-t -t" "sudo whoami;hostname -f" And you can safely ignore this error: "Stderr: tcgetattr: Invalid argument" Your command will run in the way you intent it and you will be able to manage your cluster from one terminal Hope this helps. Thanks ... View more

PROBLEM: Hive queries were getting stuck/hang on both mr and tez engines while selecting from a table containing few csv files. While the query was working fine for few csv files, for others it just hangs. Nothing in the logs also. I was using Hive 1.2.1 on HDP 2.5.3.0 After some investigation, I found out that those files have some empty values '' in the fields where an rpad function was getting used. You can easily reproduce the issue by firing: select rpad('',1,''); You will see that the query just hangs. The reason is it goes to an infinite loop. More details here: HIVE-15792 RESOLUTION: nvl will not work in this case. That is select nvl('','D'); --will return '' I resolved using a query like this: SELECT rpad(CASE WHEN LENGTH(nvl(COLUMN_NAME,null)) > 0 THEN COLUMN_NAME ELSE null END, 1, ''); In this case the query will return null for both null and empty string values occurring in COLUMN_NAME. Hope this helps. Thanks, Rajdeep ... View more

Found in version Ambari 2.4.2.0 Generation of SmartSense bundle fails with following error: ERROR 2017-02-24 10:52:53,284 shell.py:95 - Execution of command returned 1. Exception in thread "main" java.lang.IllegalArgumentException: Illegal group reference at java.util.regex.Matcher.appendReplacement(Matcher.java:857) at java.util.regex.Matcher.replaceAll(Matcher.java:955) at java.lang.String.replaceAll(String.java:2223) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.handleApplyGroupPatternKeyValue(BundleAnonymizer.java:690) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.applyGroupPattern(BundleAnonymizer.java:673) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.applyPropertyRule(BundleAnonymizer.java:612) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.applyRules(BundleAnonymizer.java:393) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymizeFolder(BundleAnonymizer.java:291) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymizeFolder(BundleAnonymizer.java:259) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymizeFolder(BundleAnonymizer.java:224) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymize(BundleAnonymizer.java:160) at com.hortonworks.smartsense.anonymization.Main.run(Main.java:82) at com.hortonworks.smartsense.anonymization.Main.start(Main.java:210) at com.hortonworks.smartsense.anonymization.Main.main(Main.java:294) ERROR 2017-02-24 10:52:53,284 anonymize.py:67 - Execution of script /usr/java/default/bin/java -Xmx2048m -Xms1024m -Dlog.file.name=anonymization.log -Djava.io.tmpdir=/hadoop/smartsense/hst-agent/data/tmp -cp :/etc/hst/conf/:/usr/hdp/share/hst/hst-common/lib/* com.hortonworks.smartsense.anonymization.Main -m /hadoop/smartsense/hst-agent/data/tmp/master001.dev.company.com-a-00027129-c-00065260_comhdpdev_0_2017-02-24_10-52-04 -c /etc/hst/conf/hst-agent.ini failed ERROR 2017-02-24 10:52:53,284 anonymize.py:68 - Execution of command returned 1. Exception in thread "main" java.lang.IllegalArgumentException: Illegal group reference at java.util.regex.Matcher.appendReplacement(Matcher.java:857) at java.util.regex.Matcher.replaceAll(Matcher.java:955) at java.lang.String.replaceAll(String.java:2223) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.handleApplyGroupPatternKeyValue(BundleAnonymizer.java:690) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.applyGroupPattern(BundleAnonymizer.java:673) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.applyPropertyRule(BundleAnonymizer.java:612) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.applyRules(BundleAnonymizer.java:393) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymizeFolder(BundleAnonymizer.java:291) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymizeFolder(BundleAnonymizer.java:259) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymizeFolder(BundleAnonymizer.java:224) at com.hortonworks.smartsense.anonymization.BundleAnonymizer.anonymize(BundleAnonymizer.java:160) at com.hortonworks.smartsense.anonymization.Main.run(Main.java:82) at com.hortonworks.smartsense.anonymization.Main.start(Main.java:210) at com.hortonworks.smartsense.anonymization.Main.main(Main.java:294) ERROR 2017-02-24 10:52:53,285 AnonymizeBundleCommand.py:62 - Anonymization failed. Please check logs. Traceback (most recent call last): File "/usr/hdp/share/hst/hst-agent/lib/hst_agent/command/AnonymizeBundleCommand.py", line 58, in execute context['bundle_dir'] = anonymizer.anonymize(bundle_dir) File "/usr/hdp/share/hst/hst-agent/lib/hst_agent/anonymize.py", line 69, in anonymize raise Exception("Anonymization failed.") Exception: Anonymization failed. Cause: A group Regex exception is happening when the constant REPLACE_PROPERTY_VALUE_PATTERN regex pattern is not able to properly group search the parameter patternStr; Resolution for the above error: Option1 (preferred): Preferred way to upgrade is upgrade Ambari and follow post upgrade procedures as per hortonworks docs NOTE: Make sure that current Ambari version is lower than 2.4.2.8-2 Option2: 1. Stop Smartsense from Ambari 2. Uninstall smartsense-hst rpm on all nodes rpm -e smartsense-hst 3. Install smartsense-hst rpm on all nodes For Centos/Redhat 7: rpm -ivh http://private-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.8-2/smartsense/smartsense-hst-1.3.1.0-2.x86_64.rpm For Centos/Redhat 6: rpm -ivh http://private-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.2.8-2/smartsense/smartsense-hst-1.3.1.0-2.x86_64.rpm 4. On Ambari Server host run hst add-to-ambari 5. Restart Ambari Server 6. Delete SmartSense service from Ambari if already there 7. Add SmartSense service through Ambari's add service wizard A great many thanks to @sheetal for providing the solution. Thanks. ... View more