Hi,   NOTE:- Parquet is hard coded to write the temporary data in /tmp even though the target directory is different.   Kindly check /tmp for intermediate data, you will see it there.   Regards ... View more

Hi,   Can you please share the sqoop command that you are running?   Regards Nitish ... View more

Hi,   To solve these kind of issue there are 2 options.   1. Either clean the special characters at the DB level. 2. Kindly use any binary format import in sqoop like avro and parquet.   In these format the line won't be splitted like it does in text format. Regards Nitish ... View more

Hi,   Can you please check if your hive gateway has been configured on all the Node manager because this sqoop command can be run from any of the NM and your hive gateway should be there.   Regards Nitish ... View more

HI,   I see that you are passing -m3 but there is no split by in the sqoop command.   Can you please add that or if not then can you please try to run with -m1   It if fails again then please upload the full application logs. Regards Nitish ... View more

Hi,   You can use this tag " --temporary-rootdir" to make sure that the temporary data goes into this folder.   Example:- [example] sqoop import --target-dir /<hdfs path>/<import_data_dir> --temporary-rootdir /<dfs path>/<tmp_dir> ...   Regards Nitish ... View more

Hi,   Can you please share the exact error that you are seeing?   Also kindly share the exact line so we can test the issue in house.   Regards Nitish ... View more

Hi, Can you please share the oozie launcher logs?   I am worried the --hadoop-classpath you are passing is not correct.   Regrads Nitish ... View more

Hi, Can you please confirm how you are running the oozie job?   Is it from command line or HUE?   If it is command line then kindly do the kinit prior running the oozie job so that the oozie will use its own keytab and provide user the delegation token.   Regards Nitish ... View more

Hi,   Sqoop is a client and not a Service.   You can check HDFS audit logs which will tell what all files have been used.       Regards Nitish ... View more

Hi,   Spark is a tool used for processing the data and Sqoop is used to import/export.   NOTE:- Both the tools have different behaviours and are used accordingly.   Hope able helps.   Regards Nitish ... View more

Hi,   You can try like below example.   Data in source ###########   [root@host-10-17-103-77 ~]# sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "create table test_partition(id int,name varchar(30),par varchar(20))" [root@host-10-17-103-77 ~]# sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "insert into test_partition values(1,'nitish','par1')" [root@host-10-17-103-77 ~]# sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "insert into test_partition values(2,'mohit','par2')" [root@host-10-17-103-77 ~]# sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "insert into test_partition values(3,'mohit123','par3')"   Table in Hive:- ############ 0: jdbc:hive2://host-10-17-103-79.coe.clouder> create external table test_sqoop_par(id int,name string) partitioned by (par string) row format delimited fields terminated by '\t' location '/user/systest/test_sqoop_par';   NOTE:- partition column as string is only supported.   Sqoop command:- ############## [root@host-10-17-103-77 ~]# sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --hcatalog-database default --hcatalog-table test_sqoop_par --table TEST_PARTITION -m1 --hive-partition-value par   Data in Hive:- ########## 0: jdbc:hive2://host-10-17-103-79.coe.clouder> show partitions test_sqoop_par; +------------+ | partition   | +------------+ | par=par1   | | par=par2   | | par=par3   | +------------+   [root@host-10-17-103-77 ~]# hadoop fs -ls /user/systest/test_sqoop_par Found 3 items drwxr-xr-x   - systest supergroup           0 2019-09-30 18:49 /user/systest/test_sqoop_par/par=par1 drwxr-xr-x   - systest supergroup           0 2019-09-30 18:49 /user/systest/test_sqoop_par/par=par2 drwxr-xr-x   - systest supergroup           0 2019-09-30 18:49 /user/systest/test_sqoop_par/par=par3   Hope above helps.   Regards Nitish   ... View more

Hi,   Not sure at this point what is the issue on that host as I am not able to debug the host related issue.   Have you configured Sqoop gateway on both the hosts?   Regards Nitish ... View more

HI,   Means only hive import is causing the issue right?   Can you share the output of below command.   ## show create table <tablename> from hive.   NOTE:- I am expecting the earlier sqoop import(first time) you ran, you are importing with the same delimiter.   Regards Nitish ... View more

This means that from the master node the job is failed to be created but when you are running the job from slave node then it is getting created.   Am I right?   If yes then I would request you to check the difference b/w the hosts. Also check if the sqoop gateway has been deployed? If yes then on what node.   There is some setup issue in your cluster which is causing this.   Regards Nitish ... View more

Hi,   I would request you to configure Sqoop gateways on that hosts and install metastore.   KB:-  https://my.cloudera.com/knowledge/Creating-and-Executing-Sqoop-Jobs-Remotely-|-Sqoop1-Shared-Metastore-Database?id=70942   Regards Nitish ... View more

Also, If you run the same job from other node then it fails with the same message? Can you please confirm?   Regards Nitish ... View more

Hi, Did you install the sqoop metastore? Any configuration changes you made in Sqoop configurations via CM?   Regards Nitish ... View more

Hi,   Can you please install Sqoop gateway on that node.   We always recommend to install sqoop gateways on that host on which you are running the Sqoop commands.   Link:- https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_mc_sqoop1_client.html   Also can you please confirm if you are able to run the normal sqoop import commands from that host?   Can you run :- sqoop job --list and share the output of this?   You have enough disk space on that host ? Because Sqoop needs to create the local directories as a metastore?   Regards Nitish   ... View more

Hi Akhila,   This is the output of what command?   Can you please run the below commands from the host and share the output.   ## ls -ltr /var/lib/sqoop ## ls -ltra /home/<username>   Regards Nitish ... View more

Hi,   Can you please share the full stderr log file so that can give more light on the error?     Is this the first time you are installing oozie server?   Regards Nitish ... View more

Hi,   You will see the captured output in "Stdout" logs.   Please let us know if you are not able to see in that section as well.   Regards Nitish ... View more

Hi,   Generally we recommend to use spark action as it is available.   If there is something that spark action can't do then you can have a java action which will run your spark job.   Regards Nitish ... View more

Hi,   For this I would request you to place the keytab file in HDFS and just reference the name in the <spark-opts>   Example   <spark-opts> --principal <abc> --keytab <abc.keytab> </spark-opts> <file> <path of HDFS keytab></file>   NOTE:- Do add the <file> tag which will be pointing to the location of keytab on HDFS.   This will localize the keytab file and will use in the oozie spark action.   Kindly try the above and let us know how it goes.   Regards Nitish ... View more

Hi,   Hope you are doing good !!!   Can you please confirm how you are creating the workflows? Is it from HUE?   Can you share the screenshot with us so that we can check what steps you are performing?   Regards Nitish ... View more

Hi,   It looks like the spark job that is started is getting failed.   Can you please share the logs for the Spark job. The application ID is mentioned in the snapshot for the spark as it gets initiated but after that it fails.   ##  yarn logs -applicationId <application ID> -appOwner <owner name>   Kindly copy the logs in notepad and attach to the case.   This will give more leads on this.   Regards Nitish   Regards Nitish ... View more

Hi,   Can you share the sqoop command?   Also does this same sqoop command used to work fine in C5?   Regards Nitish ... View more

Hi, Can you share the output of below command from the host on which you are running the Sqoop job?   ## ls -ltr /var/lib/sqoop ## ll -aR /home/<username>   This will help us to check the permission on these folders.   Regards Nitish ... View more

Hi,   It looks like you are trying to import the data and it is failing with the delimiter.    Let's break the testing in 2 jobs. Let's try to import the data first in HDFS and once this works then we will move to next step.   Can you please try to run the sqoop command with incremental and import just to HDFS.    Option 1:- If this works fine then we will go with next step. Option 2:- If this doesn't work fine then can you please share the sqoop command and the console output so we can debug further.   Regards Nitish ... View more