I'm trying to setup a nifi processor to run once daily, using the 'cron' option under scheduling. Under "run schedule", I put "01 18 * * * ?" , which should be 6:01pm (I couldn't get nifi to accept it without the question mark). But this causes the scheduler to run on the hour, every hour. Can someone please help me with my syntax here? ... View more

I am trying to use the CsvBulkUploadTool to get data from Hive to Phoenix/HBase. As I typically do, I created a Hive table w/ the copy of the data that I care about, and with the properties: “row format delimited fields terminated by ‘|’ null defined as ‘null’ stored as textfile location ‘my location’ “ This correctly outputs a series of CSV files in HDFS with rows that look like this: 96|9|116|183|[-6, -81, 96, 43, 108, 12, 0, 116, 30, 88, -29, 87, -73, -106, 0, 9, 27, 1, 71, 3, 0, 2, 13, 118, 119]|13|Some_String|3180|1474517022732|0|150 The 5 th column needs to be stored as a string. I can manually enter this into my HBase table: upsert into my_table values (96,9,116,183,’[-6, -81, 96, 43, 108, 12, 0, 116, 30, 88, -29, 87, -73, -106, 0, 9, 27, 1, 71, 3, 0, 2, 13, 118, 119]’,13,’Some_String’,3180,1474517022732,0,150) However, the CsvBulkUploadTool fails. I’m passing the “ -d '|' “ parameter (and I’ve also tried with double quotes), but I still get errors like the one below. Can anyone tell me how to accomplish my objective here? 16/10/07 14:33:16 INFO mapreduce.Job: Task Id : attempt_1475193681552_0605_m_000376_0, Status : FAILED Error: java.lang.RuntimeException: java.lang.RuntimeException: Error on record, java.sql.SQLException: ERROR 201 (22000): Illegal data., record =[96|9|116|183|[-6, -81, 96, 43, 108, 12, 0, 116, 30, 88, -29, 87, -73, -106, 0, 9, 27, 1, 71, 3, 0, 2, 13, 118, 119]|13|Some_String|3180|1474517022732|0|150] at org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:176) at org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) ... View more

I have a flow that uses an ExecuteStreamCommand processor to populate a folder with some data. The actual flow files are essentially just arguments to a script that eventually generates some output. When the script is done, I want to go pick up the output using GetFile or FetchFile, and then continue some further processing. However, I can't seem to connect my ExecuteStreamCommand processor to a GetFile or FetchFile processor. The GUI simply won't allow it. Can anyone straighten me out here? Is there some reason that what I'm trying to do is invalid? Is there a better way to accomplish this? ... View more

(I'm still getting familiar w/ NiFi, so this might have a really obvious answer) I have a flow where I take a text file that contains a per-line list of files. I grab this file with a FetchFile processor, and then feed the output to a SplitText processor, giving me one flow file per row. So my flow file is literally just a string with a file path "x", like '/home/zack/my_folder/my_file'. Then, I want to add either a FetchFile processor or a ExecuteStream processor to get the file represented by "x" and copy it to a different folder. However, I can't figure out how to reference "x". Is there a way, using the markup language I would guess, to somehow get a handle on that file path string so I can use it? thanks! ... View more

Suppose I have a SplitJSON processor that splits based on Customer ID, and that eventually flows to a RouteOnAttribute processor, which flows to 3 different processors "A", "B", and "C" based on attributes "IsA", "IsB", and "IsC". And suppose these are all true for a given Customer ID. Is there a processor that I can connect them all to that will not begin until A, B, and C are done for a given customer? In other words, for customer X, I want to run processors A, B, and C simultaneously, and when the are all done, I want to run processor D. Can someone point me in the right direction there? ... View more