Member since
04-09-2019
24
Posts
1
Kudos Received
0
Solutions
12-12-2019
01:43 PM
Hey @Matt, I'm curious about the MonitorActivity how would the scheduling take on the changing of days in month? Like I want it to run on the 20th of each month, how would I configure that?
... View more
12-12-2019
10:05 AM
Hey, so I want to create a flow that creates a putemail trigger when NO files are present in a hdfs path for a certain date range:
my current flow is
GETHDFS->
ROUTEONATTRIBUTE(has the proper date matching criteria)
-->MATCHED(should be getting files here)
-->UNMATCHED(when this outputs zero I want to receive an email alert)
RouteOnAttribute Logic:
${Path:substring(0,6):equals(${now():toNumber():minus(86400000):format('yyyyMM')})}
Pretty much if I find no files within the current month(last 20 days) then I need to receive an email alert.
How do I get an alert when there are no files that fit the criteria?
@Matt
Thank you matt sorry for yet another question.
... View more
- Tags:
- NiFi
Labels:
- Labels:
-
Apache NiFi
12-11-2019
09:00 AM
Hello all, I am trying to get a rowcount from hdfs location, then depending on the number, send out an email to flag that directory.
So here is my flow: GETHDFSFILEINFO -> COUNTTEXT -> EXTRACTTEXT -> ROUTEONCONTENT -> PUTEMAIL
I need help with the EXTRACTTEXT configuration for this flow, here is the contents of the flowfile coming from the extracttext: { "objectName" : "**************", "path" : "************", "type" : "directory", "owner" : "***********", "group" : "hdfs", "lastModified" : "************", "length" : "5501268", "replication" : "0", "countFiles" : "38", "countDirs" : "6", "permissions" : "***********" }
So I am using the "countFiles" and want to just extract this value with the EXTRACTTEXT processor. Can someone please help me with the proper configurations?
@Matt
... View more
Labels:
- Labels:
-
Apache NiFi
12-10-2019
01:12 PM
I'm okay with writing over the content of the rest of the file. I'm trying to output the rowcount results in a putemail so I'm okay with overwriting the rest of the flowfile. Can you still help with this?
... View more
12-10-2019
10:45 AM
So it's part of the flowfile content. I attached the flow and the configs. Sorry for the confusion if I caused any I'm using a count
... View more
12-10-2019
09:14 AM
Hey Matt, thank you so much I've set up that flow but I'm having trouble on the actual updateattribute function I'm applying: ${COUNTFILES:minus(31)} I'm using this and it's not modifying the value. Could you please help?
... View more
12-09-2019
11:36 AM
Hello, I want to perform a simple mathematical operation on a json attribute. I simply want to subtract 30 from the value.
Any help would be appreciated
... View more
Labels:
- Labels:
-
Apache NiFi
10-29-2019
11:19 AM
Hello, I am curious if it's capable through scripting or another measure to start NiFi Processors/process groups in a specific manner. Any help or leads will be appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
10-21-2019
07:55 AM
Good Morning,
Is it possible to simultaneously clear the state of multiple processors? I have 108 different process groups and I want to see if there is a way to clear the state of the QueryDatabaseTable in each process group without manually going into each one and clearing it.
Any ideas/suggestions would be appreciated.
... View more
Labels:
- Labels:
-
Apache Hadoop
05-24-2019
03:18 PM
Hey, so I want to use a flow(open to change) that goes like this: QueryDatabaseTable(Oracle source) => ExecuteSQL=> RouteonAttribute => [ PutDatabaseRecord(insert) => PutdatabaseRecord(update)](IBM/DB2) I want to use the Route on Attribute to flow directly to the PutDatabaseRecord based on the Primary Key of the Tables. i.e if the PK exists, route to Update and if it doesn't route to Insert. Any Help would be greatly appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Ranger
05-09-2019
09:03 PM
I want to configure the mergecontent processor so that when it flows to the putemail processor, it lists all the individual flowfiles within the email that it sends out. As of now I am getting just one line of text from the putemail and it has one line of text " Greetings, 172927055__________ has failed to load in into ______________________ at but succesfully loaded into HDFS" I would like to receive a list of files that failed to load, not just one new flowfile number. Please help.
... View more
Labels:
- Labels:
-
Apache NiFi
05-06-2019
05:18 PM
Good Afternoon, I am creating an apache nifi flow in which I want to flow files from Oracle DB to IBM/DB2, I have completed that part of the flow. I need help with configuring a way to reconcile failed files, I want to know exactly which files failed and the error which caused it. I'm considering using a PUTHDFS to put the failed files into a HIVE TABLE that way I can capture the flow file error. I would then re-enter the failure files back into the flow via GETHDFS. So, I'm curious is this possible? Also is there a mechanism in NiFi to identify the singular file that fails in a batch?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi
04-19-2019
03:58 PM
so how would I do the 2nd one? I tried the first one but it put "Null" into all the rows in the destination table.
... View more
04-19-2019
01:28 PM
Hey, I'm receiving a special character error in Apache Nifi, I have 4 columns that could be causing it: UCR_COST_IN_$, ACTUAL_COST_IN_$ or REQUESTED#, ASSIGNED# any help or solutions would be greatly appreciated!
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache NiFi
04-17-2019
06:46 PM
Two things with this DB, 1) The same notes column as above 2) The "Requested#" and the "Assigned#" both end in "#" which show up in the color red when I'm typing which columns to return.
... View more
04-17-2019
06:09 PM
Good afternoon from your inept Jr. Developer, I am trying to create a NiFi flow from ORACLE to IBM DB2 z/OS. I am encountering issues when it's coming into the PutDataBaseRecord form the QueryDataBase Table. I have a column named "notes" that is of data type CLOB. I have read this can cause serious issues. I could be incorrect on that being the issue, I have the error message posted below. Any help is appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
04-12-2019
12:47 PM
Thank you very much Mr. Burgess!
... View more
04-12-2019
12:22 PM
Hello, so I am stuck on this NiFi flow, everything will flow correctly until I get to the PutSQL, when the flowfiles are queued between the CONVERTJSONTOSQL AND PUTSQL, 'view' content looks like: "INSERT INTO [new_table_name] (Column_name1,Column_name2,Column_name3,Column_name4,Column_name5,Column_name6,Column_name7,Column_name8,Column_name9,Column_name10,Column_name11,Column_name12,Column_name13,Column_name14 ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) The ??????? are where I believe my error is and it's probably due to my SQL Statement in the 2nd picture below, any advice or help would be greatly appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
04-09-2019
02:45 PM
1 Kudo
Good Morning, I have a problem with conversions in NiFi. I have a large amount of tables in Oracle that I need to transfer via NiFi into one of ten set tables. I am very new to NiFi and am running into a big problem in which I have a column name in Nifi i.e "Birthday_Date" but the column name for that type of value is "B_Day_Date" in the new preset tables. How would I go about converting this in Nifi?
... View more
Labels:
- Labels:
-
Apache NiFi