I had the same issue and I am able to resolve it by Download and add the repository, then update. wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm yum update I assume you have taken care the below. yum install -y mysql-connector-java ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar ... View more

@kkawamura, Thanks for your response. In my case, the FlowFiles are not xml files. It will be txt or gz files. So I will be using LookupAttributes (which will be using XMLFileLookupService) to read the XML config file (shown above) for to-path to put FlowFiles (*.txt or *.gz) into destination path (to-path: ex: /d_salary/1.0/csv or /d_pension/2.2/csv). So I need to know the way to loop my XML config file. I have missed add one more attribute in XML config file. I will be having something like below for each dataFlow. <filePattern>salary_*.gz</filePattern> <filePattern>pension_*.gz</filePattern> I will correct my above XML file. So if I have salary_*.gz, the file should be moved to /d_salary/1.0/csv. If I have pension_*.gz then it should be moved to /d_pension/2.2/csv. Any help is much appreciated. ... View more

@jfrazee, How do I get value of country & datasource in LookupAttribute? I have created new attribute as dataFlows.dataFlow.to.path for path. Not sure how I can get value of country. Is that dataFlows.dataFlow.properties.property[@name="country"].value? I couldn't get this correctly. Also do you know how to see the output of attributes? Is that logAttribute which can help me to debug the values of attribute? ... View more

@Pierre Villard Sorry for the delay in reply. I was focusing on to solve the other bits and took time to come to you. I think I jump too much. Let me take one at a time. For now, I will focus on a single file. I have the below xml file. <configuration verbose="false" debugMode="false"> <dataFlows> <dataFlow configurationId="7c888be1-04ac-11e7-8ab3-3c970ee9aa0b" name="dept" enabled="true"> <properties> <property name="country" value="it" /> <property name="datasource" value="dept" /> </properties> <from> <filePattern>department_*.csv.gz</filePattern> </from> <to> <path>/var/tmp/${country}/${datasource}/csv</path> </to> </dataFlow> </dataFlows> </configuration> When I try to access path from LookupAttribute, putFile is reading the path as /var/tmp/${country}/${datasource}/csv instead of /var/tmp/it/dept/csv. Do you know the easy way to solve this before I ask further support? Many thanks. ... View more

@Pierre Villard, I can combine all 3 config files into a single xml file as below. <configuration verbose="false" debugMode="false"> <dataFlows> <dataFlow configurationId="7c888be1-04ac-11e7-8ab3-3c970ee9aa0b" name="dept" enabled="true"> <properties> <property name="country" value="it" /> <property name="datasource" value="dept" /> </properties> <to> <path>/var/tmp/${country}/${datasource}/csv</path> </to> </dataFlow> <dataFlow configurationId="8a888be1-04ac-11e7-8ab3-3c970ee9aa0b" name="emp" enabled="true"> <properties> <property name="country" value="it" /> <property name="datasource" value="emp" /> </properties> <to> <path>/var/tmp/${country}/${datasource}/csv</path> </to> </dataFlow> <dataFlow configurationId="9b888be1-04ac-11e7-8ab3-3c970ee9aa0b" name="sal" enabled="true"> <properties> <property name="country" value="it" /> <property name="datasource" value="sal" /> </properties> <to> <path>/var/tmp/${country}/${datasource}/csv</path> </to> </dataFlow> </dataFlows> </configuration> Could you advise how I can lookup for all 3 feeds attributes (path) and ingest them to local / hdfs path (with putFile)? ... View more

@Pierre Villard, Many thanks. I am able to use the given xmllookup.xml to complete my use case. I understand that I should give the config file path at Controller Service to extract the path and putFile. To extend my current use case, for example, I will be having 3 input files (emp.txt, dept.txt, sal.txt) at 3 different locations (like /tmp/emp/emp.txt, /tmp/sal/sal.txt, etc ...) and I will have 3 config files (emp-config.xml, sal-config.xml, etc) and the putFile paths will be defined in config files. What would be the recommended approach to achieve this? In my real use case, I will be having 100s of input feeds (with different file name patterns) and 100s of xml files which defines where the input files should be stored (on HDFS). Many thanks in advance. ... View more

@Pierre Villard, Many thanks for the quick response. I have tied the below without any luck. Not sure what is wrong in the config but I am getting NullPointerException at PutFile process. Any thoughts? 1. GetFile 2. FetchFile (for config file) 3. EvaluateXPath 4. putFile ... View more