@jayes I had another look into the problem description and I see queries run with 2 categories Column names enclosed in backquotes Column names not enclosed in backquotes. below are the root cause for both the categories, please find the solution at the end of this comment. Case 1: This one fails because you are running SQL with -e option and and column values are enclosed in backticks "`", the value inside backticks is considered as a Linux command as opposed to be a part of SQL leading to the compilation error. Command : sudo -u hive beeline -u "jdbc:hive2://cdp1:2181,cdp2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test" -n hive --showHeader=false --outputformat=tsv2 -e "SET hive.support.quoted.identifiers=column;use jayesh; create external table testkeyword1( `transform` string);"; error : Error: Error while compiling statement: FAILED: ParseException line 1:45 cannot recognize input near ')' '<EOF>' '<EOF>' in column type (state=42000,code=40000) Command : sudo -u hive beeline -u "jdbc:hive2://cdp1:2181,cdp2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test" -n hive --showHeader=false --outputformat=tsv2 -e "set hive.support.sql11.reserved.keywords=false; set hive.support.quoted.identifiers=column;use jayesh; create external table testunderscore( `_name` string, `_id` string)"; error : -bash: _name: command not found -bash: _id: command not found Error: Error while compiling statement: FAILED: ParseException line 1:45 cannot recognize input near ',' 'string' ')' in column type (state=42000,code=40000) Case 2: This one fails because you are using reserved keyword transform which is not escaped properly. Command : sudo -u hive beeline -u "jdbc:hive2://cdp1:2181,cdp2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test" -n hive --showHeader=false --outputformat=tsv2 -e "set hive.support.sql11.reserved.keywords=false; set hive.support.quoted.identifiers=column;use jayesh; create external table testkeyword3( transform string)"; error : Error: Error while compiling statement: FAILED: ParseException line 1:38 cannot recognize input near 'transform' 'string' ')' in column name or constraint (state=42000,code=40000) Command : sudo -u hive beeline -u "jdbc:hive2://cdp1.:2181,cdp2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test" -n hive --showHeader=false --outputformat=tsv2 -e "SET hive.support.sql11.reserved.keywords=false;use jayesh; create external table testkeyowrd2( transform string)"; error : Error while compiling statement: FAILED: ParseException line 1:38 cannot recognize input near 'transform' 'string' ')' in column name or constraint (state=42000,code=40000) Solution: You can escape backticks while passing SQL from command line eg:- sudo -u hive beeline -u "jdbc:hive2://cdp1:2181,cdp2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test" -n hive --showHeader=false --outputformat=tsv2 -e "SET use jayesh; create external table testkeyword1( \`transform\` string);"; You can also pass the SQL to beeline command as a file. eg;- transform.sql create external table testkeyword1( `transform` string); Beeline command: sudo -u hive beeline -u "jdbc:hive2://cdp1:2181,cdp2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test" -n hive --showHeader=false --outputformat=tsv2 -f transform.sql Let me know if this helps fixing the issue.
... View more
