@Sreejith Madhavan I see the comment "running a SELECT statement in LLAP" but using Hive1.2 env, Try using Hive2 code base. Connect to HSI (Hiveserver2Interactive) url via beeline, which will make use of llap containers ... View more

@Santosh Kum Use --as-avrodatafile or --as-parquetfie in the sqoop commnad line options for respective formats. Also check if you SQOOP_HOME/lib contains parqust and avro jar files. ... View more

@zkfs hope this article might help you for correct settings https://community.hortonworks.com/articles/591/using-hive-with-pam-authentication.html ... View more

@Harold Allen Badilla There is no disadvantage in importing a table from Sql Server directly to Hive. In fact its a single command which internally does importing data into HDFS loaction(you can specify via --warehouse-dir) creates hive table schema and Loads the data to Hive table. This create Hive table name/schema similar to the source database table. sqoop import --connect "jdbc:sqlserver://11.11.111.11;databaseName=dswFICO" \ --username sqoop\ --password sqoop \ --driver com.microsoft.sqlserver.jdbc.SQLServerDriver \ --table KNA1 \ --warehouse-dir <HDFS path> --hive-import -> additionally you can specify --hive-overwrite if you want to overwrite any exiting data on the Hive table(if exists) -> If you want to load data into a table(hive) of your choice you can use --create-hive-table --hive-table <table name> ... View more

@elliot gimple seems this doc might help you. https://forums.databricks.com/questions/7599/create-a-in-memory-table-in-spark-and-insert-data.html ... View more

@Tin Huynh Since the db dump is huge MR is getting timed out, try using this option in the Sqoop job to customize timeout. -D mapred.task.timeout=1500 ... View more

@Pavani N Check the following params core-site.xml <property> <name>hadoop.proxyuser.<loginuser>.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.<loginuser>.groups</name> <value>*</value> </property> hdfs-site.xml <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> ... View more

@Nilesh Shrimant Try to create table in parquet format , and set this config set hive.fetch.task.conversion=more; https://issues.apache.org/jira/browse/HIVE-11785 hive> create table repo (lvalue int, charstring string) stored as parquet; OK Time taken: 0.34 seconds hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo; Loading data to table default.repo chgrp: changing ownership of 'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not belong to hive Table default.repo stats: [numFiles=1, numRows=0, totalSize=610, rawDataSize=0] OK Time taken: 0.732 seconds hive> set hive.fetch.task.conversion=more; hive> select * from repo; Option 2: There is some info here: http://stackoverflow.com/questions/26339564/handling-newline-character-in-hive Records in Hive are hard-coded to be terminated by the newline character (even though there is a LINES TERMINATED BY clause, it is not implemented). Write a custom InputFormat that uses a RecordReader that understands non-newline delimited records. Look at the code for LineReader / LineRecordReader and TextInputFormat . Use a format other than text/ASCII, like Parquet. I would recommend this regardless, as text is probably the worst format you can store data in anyway. ... View more