Support Questions
Find answers, ask questions, and share your expertise

Sqoop not creating entities in Atlas

Sqoop not creating entities in Atlas

Rising Star

Using sqoop on hdp 3.1 to import Oracle DB data into HDFS as parquet and expecting it to appear in atlas as an entity (I assume it should be the default sqoop_process type). However, after running the sqoop job, nothing appears in atlas. Sqoop job looks like...

sqoop import     -Dmapreduce.reduce.log.level=DEBUG"Ora import table $tablename"     -Doraoop.timestamp.string=false     $oracle_cnxn_str     --as-parquetfile     --target-dir $importdir     -query "select a.*, current_date as etl_date from $tablename a where 1=1 AND \$CONDITIONS"     --split-by $splitby     --where "1=1"     --num-mappers 12     --delete-target-dir

Looking in the hdp ambari interface for atlas, I can see that the sqoop plugin for atlas is configured110272-1565211369078.png(did not initially do any manual tweaking to it after installing atlas according to the docs). Adding the property


based on another Hortonworks post and attempting the sqoop job again still shows nothing in the atlas UI. Anyone with more experience with hdp and atlas have any debugging suggestions or fixes?

Don't have an account?