Cloudera Community
Daco
user rank
Explorer
My Accepted Solutions
Show All

Re: Sqoop export to Postgresql with geometry colum...

by Explorer in Support Questions
‎04-24-2023 12:54 AM
‎04-24-2023 12:54 AM
Hi,   Unfortunatly --query is not an Sqoop export argument...   I'm still facing this issue   ... View more

Re: Sqoop export to Postgresql with geometry colum...

by Explorer in Support Questions
‎04-18-2023 02:00 AM
‎04-18-2023 02:00 AM
Hi,   Do I need specific JDBC driver in Sqoop librairies to manage postgresl/postgis database ?   tkx ... View more

Re: Sqoop export to Postgresql with geometry colum...

by Explorer in Support Questions
‎04-17-2023 05:12 AM
‎04-17-2023 05:12 AM
Thanks for hint. I customed it has following to make it running but facing new issue... Here with right syntaxe : --map-column-java geom=String  (need uppercase for first datatype)   Looks better but facing this output error : Error: column « geom » has type geometry but expression type is character varying Hint: You should rewrite expression or make type transformation.   Tried few queries but not working... 1) --map-column-java geom=Geometry \ Had output :  ERROR orm.ClassWriter: No ResultSet method for Java type Geometry ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.IllegalArgumentException: No ResultSet method for Java type Geometry java.lang.IllegalArgumentException: No ResultSet method for Java type Geometry   2) --map-column-java geom=String,value=Geometry \ ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.IllegalArgumentException: No column by the name valuefound while importing data; expecting one of [Y_REEL, INCERTITUDE_Z, DATE_POSE, geom, COTE_FIL_EAU, INCERTITUDE_XY, ANTI_ODEUR, OBSERVATION_RES, USER_VALID, DATE_GEOLOC, PRESTATAIRE, WM_VERSION, USER_CRE, USER_MAJ, LTT_FEATURE_ID, id, WM_USERNAME, id_metier, WM_ROWID, MARQUE_MATERIEL, X_REEL, Z_REEL, SOURCE_MAJ, FORMAT_ORIGINE_XY, WM_CREATEWORKSPACEID, LTT_STATUS, DATE_CRE, DATE_MAJ, COMMUNE, FORMAT_ORIG_ZRAD, NUM_SERIE_MATERIEL, WM_CREATETIME, TYPE_AVALOIR, ENTREPRISE_RELEVE, WM_RETIRETIME, COTE_TN, QUALITE_GEOLOC_XY, WM_WORKSPACE, FORMAT_ORIG_ZTN, RESEAU, MAITRE_OUVRAGE, QUALITE_GEOLOC_Z, QUAL_ANNEE, niveau, WM_OPTYPE, LTT_CURRENT_REVSET_ID, ID1, COTE_RADIER, WM_RETIREWORKSPACEID, EXPLOITANT] java.lang.IllegalArgumentException: No column by the name valuefound while importing data; expecting one of [Y_REEL, INCERTITUDE_Z, DATE_POSE, geom, COTE_FIL_EAU, INCERTITUDE_XY, ANTI_ODEUR, OBSERVATION_RES, USER_VALID, DATE_GEOLOC, PRESTATAIRE, WM_VERSION, USER_CRE, USER_MAJ, LTT_FEATURE_ID, id, WM_USERNAME, id_metier, WM_ROWID, MARQUE_MATERIEL, X_REEL, Z_REEL, SOURCE_MAJ, FORMAT_ORIGINE_XY, WM_CREATEWORKSPACEID, LTT_STATUS, DATE_CRE, DATE_MAJ, COMMUNE, FORMAT_ORIG_ZRAD, NUM_SERIE_MATERIEL, WM_CREATETIME, TYPE_AVALOIR, ENTREPRISE_RELEVE, WM_RETIRETIME, COTE_TN, QUALITE_GEOLOC_XY, WM_WORKSPACE, FORMAT_ORIG_ZTN, RESEAU, MAITRE_OUVRAGE, QUALITE_GEOLOC_Z, QUAL_ANNEE, niveau, WM_OPTYPE, LTT_CURRENT_REVSET_ID, ID1, COTE_RADIER, WM_RETIREWORKSPACEID, EXPLOITANT] ... View more

Sqoop export to Postgresql with geometry column

by Explorer in Support Questions
‎04-17-2023 01:45 AM
‎04-17-2023 01:45 AM
Using Sqoop export on CDP cluster (hdfs), I'm trying to export data geometry to Postgresql table but I'm facing a Sqoop ERROR orm.ClassWriter: No Java type for SQL type 1111 for column 'geom'. The 'geom' column definition on PostGis is : geometry (Point,3945). Here is my Sqoop export query : ``` sqoop export --connect "jdbc:postgresql://$database_server/$database_source_name" \ --username $username --password $password_alias \ --table $database_table_name \ --export-dir "${DIR_OUT_HDFS}" \ --fields-terminated-by '|' --lines-terminated-by "\n" \ -- --schema $database_table_name 1>>${LOG_FILE} 2>>${LOG_FILE} ``` Here is Sqoop output error : ``` ERROR orm.ClassWriter: No Java type for SQL type 1111 for column geom ERROR orm.ClassWriter: No Java type for SQL type 1111 for column geom ERROR orm.ClassWriter: No Java type for SQL type 1111 for column geom ERROR orm.ClassWriter: No Java type for SQL type 1111 for column geom ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException java.lang.NullPointerException at org.apache.sqoop.orm.ClassWriter.parseColumn(ClassWriter.java:1417) at org.apache.sqoop.orm.ClassWriter.myGenerateParser(ClassWriter.java:1555) at org.apache.sqoop.orm.ClassWriter.generateParser(ClassWriter.java:1518) at org.apache.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:1979) at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1764) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:77) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:113) at org.apache.sqoop.Sqoop.run(Sqoop.java:151) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:187) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:241) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:250) at org.apache.sqoop.Sqoop.main(Sqoop.java:259) ``` ... View more
Labels:

Re: Impala-shell is it possible to pass a where cl...

by Explorer in Support Questions
‎09-02-2020 07:02 AM
‎09-02-2020 07:02 AM
Hi Ian, Thanks for taking time to have a look on it.   I actually tried to add backslash ( \' ) but did not worked. I also tried to add 2 quotes (/!\ not double quotes) same, it did not worked.   The only way it worked is to put the query behind double quotes : -q "my_query where col1 IN ('arg1','arg2')"   PS: As a reminder my script is on a Shell script file .sh   Thanks again Alex ... View more

Re: Impala-shell is it possible to pass a where cl...

by Explorer in Support Questions
‎09-02-2020 05:47 AM
‎09-02-2020 05:47 AM
Fixed by using double quote for the query "the query with where clause" (vs 'the query with where clause')   Full cmd : impala-shell -B -q "select col1,col2,Col3 from MyDB.MyTable where col1 IN ('argument1','argument2','argument3','argument4','argument5') order by col1,col2" --delimited --output_delimiter='|' -o ${DIR_OUT}/MyTable_$(date +"%Y%m%d-%H%M%S").csv --print_header 2>${DIR_LOG}/MyTable_$(date +"%Y%m%d-%H%M%S").log;   That's all folks ! ... View more

Impala-shell is it possible to pass a where clause...

by Explorer in Support Questions
‎09-02-2020 03:07 AM
‎09-02-2020 03:07 AM
Hi guys,   This is basic question but not able to solve it... I created a scheduled workflow in Oozie to run a Shell script witch runs some impala-shell cmd. Everything worked fine since I added a where clause for one of them. I'm wondering if it's possible to pass a where clause on a query impala-shell ?   Here is my basic cmd in shell script : impala-shell -B -q 'select col1,col2,Col3 from MyDB.MyTable where col1 IN ('argument1','argument2','argument3','argument4','argument5') order by col1,col2' --delimited --output_delimiter='|' -o ${DIR_OUT}/MyTable_$(date +"%Y%m%d-%H%M%S").csv --print_header 2>${DIR_LOG}/MyTable_$(date +"%Y%m%d-%H%M%S").log;   It works fine without "where col1 IN ('argument1','argument2','argument3','argument4','argument5')"   Is there a way to do it without using a file ? (with -f option ???)   Thanks for your advise Alex ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎09-09-2024 11:57 AM
Community Statistics
Member Since ‎08-11-2020 07:12 AM
Last Visited ‎09-09-2024 11:57 AM
Posts 7
Kudos received 1