Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop not creating entities in Atlas

Highlighted

Sqoop not creating entities in Atlas

Contributor

Using sqoop on hdp 3.1 to import Oracle DB data into HDFS as parquet and expecting it to appear in atlas as an entity (I assume it should be the default sqoop_process type). However, after running the sqoop job, nothing appears in atlas. Sqoop job looks like...

sqoop import     -Dmapreduce.map.memory.mb=3144 -Dmapreduce.map.java.opts=-Xmx1048m     -Dyarn.app.mapreduce.am.log.level=DEBUG     -Dmapreduce.map.log.level=DEBUG     -Dmapreduce.reduce.log.level=DEBUG     -Dmapred.job.name="Ora import table $tablename"     -Djava.security.egd=file:///dev/urandom     -Djava.security.egd=file:///dev/urandom     -Doraoop.timestamp.string=false     -Dmapreduce.map.max.attempts=10     $oracle_cnxn_str     --as-parquetfile     --target-dir $importdir     -query "select a.*, current_date as etl_date from $tablename a where 1=1 AND \$CONDITIONS"     --split-by $splitby     --where "1=1"     --num-mappers 12     --delete-target-dir

Looking in the hdp ambari interface for atlas, I can see that the sqoop plugin for atlas is configured110272-1565211369078.png(did not initially do any manual tweaking to it after installing atlas according to the docs). Adding the property

110281-1565211459573.png

based on another Hortonworks post and attempting the sqoop job again still shows nothing in the atlas UI. Anyone with more experience with hdp and atlas have any debugging suggestions or fixes?

Don't have an account?
Coming from Hortonworks? Activate your account here