Support Questions

Find answers, ask questions, and share your expertise

ETL Tool using Yarn Distributed Shell jar

avatar
Rising Star

Can I execute scripts involving 3rd part ETL Tool using Yarn Distributed Shell?

Currently I am able to execute DTL scripts inside shell script (.sh file). But whenever I am trying execute same using yarn distributed shell, it is throwing error as "command not found". Any suggestions to resolve this!

1 ACCEPTED SOLUTION

avatar
Guru

Syncsort DMX-h is an excellent ETL tool that is YARN-enabled

http://www.syncsort.com/en/Products/BigData/DMXh

View solution in original post

7 REPLIES 7

avatar
Guru

Syncsort DMX-h is an excellent ETL tool that is YARN-enabled

http://www.syncsort.com/en/Products/BigData/DMXh

avatar
Rising Star

Thanks @gkeys, I am using DMX ETL tool only :). But I am getting " dmexpress: command not found " error while running DMX script using yarn distributed shell. Below observations, I have noted

1. I have DMX installed in all the nodes of the my cluster and able to run scripts in all nodes locally.

2. Although, Same DMX Script is running fine inside shell script, It is throwing above error while running using yarn distributed shell.

Kindly suggest which all areas I need to check to resolve this.

avatar
Guru

@Gaurab D Just to be sure ... are you using DMX-H or simply DMX. The H in the first on means Hadoop. This is the one the integrates with Hadoop and Yarn.

avatar
Rising Star

I am using DMX-H only. But, I am sorting a local file for testing and not connecting to hadoop.

avatar
Guru

Syncsort is a 3rd party partner of ours. We work very closely with them but we do not support them. I suggest you head over to their support site http://bigdatakb.syncsort.com/Default.htm#KB/FAQ/297_Creating_DMX-h_Job_Tutorial.htm

avatar
Master Guru

Doing a search you can find a number of certified on YARN ETL Tools including informatica, Microstrategy, and talend.

http://hortonworks.com/partners/certified/yarn-ready/

avatar
Rising Star