Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

parquet files sqoop import

Highlighted

parquet files sqoop import

Super Collaborator

Hi:

i cant import from sqoop as parquet file, here is the error:

16/09/16 09:01:49 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:244)


and here the script

sqoop import -D oraoop.disabled=true --verbose \
        --connect xxxx \
        --username=xxxxxx \
        --password=xxxxxxx \
        --query "select id_interno_pe,cod_nrbe_en,mi_nom_cliente, TO_CHAR(fec_ncto_const_pe, 'YYYY-MM-DD') as edad, TO_CHAR(fecha_prim_rl_cl, 'YYYY-MM-DD')  as ant, sexo_in,cod_est_civil_indv,cod_est_lab_indv,num_hijos_in,ind_autnmo_in,cod_ofcna_corr,cod_cpcdad_lgl_in from xxxxxxxx.tabla where \$CONDITIONS AND cod_nrbe_en = 'xxxxxxxxxx'" \
        --num-mappers 5 \
        --split-by id_interno_pe \
        --target-dir /tmp/CHEMAPPER \
        --fetch-size 50000 \
        --as-parquetfile \
        --outdir /home/hdfs/scripts/sqoop/desercion/output \
        --bindir /home/hdfs/scripts/sqoop/desercion/output \


3 REPLIES 3
Highlighted

Re: parquet files sqoop import

You may have to perform a two step import. See here for an explanation of a similar issue with ORC format: https://community.hortonworks.com/articles/9828/import-rdbms-into-hive-table-stored-as-orc-with-sq.h....

What is your HDP version and your Sqoop version?

Highlighted

Re: parquet files sqoop import

Super Collaborator

2.4.0 the version I have thanks

Re: parquet files sqoop import

Cloudera Employee

Seems like you are hitting https://issues.apache.org/jira/browse/SQOOP-2571

Try using --table and --where arguments in the command instead of --query

Don't have an account?
Coming from Hortonworks? Activate your account here