NiFi LookupAttribute and UpdateAttributes

[#1]

I need to getfiles and put into output location. The output location will be defined in external xml file. As per the response from other thread, I have listFile, FetchFile, LookupAttribute (with XMLFileLookupService) and PutFile.

The below is the xml file and when I try these sequences, the files were moved to /var/tmp/${country}/${datasource}/csv instead of /var/tmp/it/emp/csv

Any suggestions???

<configuration>
    <dataFlows>
        <dataFlow>
            <properties>
                <property name="country" value="it" />
                <property name="datasource" value="emp" />
            </properties>
            <to>
              <path>/var/tmp/${country}/${datasource}/csv</path>
            </to>
        </dataFlow>
    </dataFlows>
</configuration>

[#2]

@Pavan Challa LookupAttribute is not going to interpolate the values being returned, just the lookup keys themselves. So, the way you should do this is do the lookup on country and datasource and then in the PutFile directory property have /var/tmp/${country}/${datasource}/csv as the value (instead of in the XML file). If you must have the full to-path being pulled out the config, you might be able to do a follow on UpdateAttribute or ExecuteScript to get the country and datasource interpolated.

[#3]

@jfrazee, How do I get value of country & datasource in LookupAttribute? I have created new attribute as dataFlows.dataFlow.to.path for path. Not sure how I can get value of country. Is that dataFlows.dataFlow.properties.property[@name="country"].value? I couldn't get this correctly. Also do you know how to see the output of attributes? Is that logAttribute which can help me to debug the values of attribute?

[#4 · Accepted solution]

@Pavan Challa I've created a template [1] that sketches out how to satisfy your use case. In it I used an ExecuteScript [2] to evaluate the attribute expression from the to.path in the XML file. I do think it'd be better/easier/more perspicuous to just set the directory in PutFile as `/var/tmp/${country}/${datasource}/csv` but figured it was worth showing you how you could do this anyway.

The (default) expression engine for the lookups [3] is rather simple so you can't do conditional lookups like you're suggesting, but you can, however, do what you want using the index of the properties. There is a way to enable XPath in the underlying libraries but that'll require some internal code changes and won't help you immediately.

1. https://gist.github.com/jfrazee/26deffba3a7d50e991495e223a020b93#file-lookupattribute_xml_putfile_ex...

2. https://gist.github.com/jfrazee/26deffba3a7d50e991495e223a020b93#file-evalexp-groovy

3. http://commons.apache.org/proper/commons-configuration/apidocs/org/apache/commons/configuration2/tre...