Member since
10-24-2016
47
Posts
16
Kudos Received
0
Solutions
12-06-2020
10:24 PM
@Vin900, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.
... View more
09-04-2019
05:02 AM
writes attribute of some processors (SplitRecord, ReplaceText ) does not contains the error it writes during execution. How and where to identify the error in that case?
... View more
09-30-2017
04:24 AM
@Jorge moyano I tried the below code as you suggested but still facing the same problem File fileObj = new File(pathname);
FileBody fileBody = new FileBody(fileObj, ContentType.MULTIPART_FORM_DATA);
HttpEntity multiPart = MultipartEntityBuilder.create().addPart("template", fileBody).build();
HttpPost httpPost = new HttpPost("http://localhost:8080/nifi-api/process-groups/root/templates/upload");
httpPost.addHeader("Content-type", "multipart/form-data");
httpPost.setEntity(multiPart);
HttpClient httpClient = HttpClientBuilder.create().build();
HttpResponse response = httpClient.execute(httpPost); Response HttpResponseProxy{HTTP/1.1 500 Internal Server Error [Date: Sat, 30 Sep 2017 04:24:37 GMT, X-Frame-Options: SAMEORIGIN, Content-Type: text/plain, Transfer-Encoding: chunked, Server: Jetty(9.4.3.v20170317)] ResponseEntityProxy{[Content-Type: text/plain,Chunked: true]}}
... View more
11-02-2017
01:13 AM
@sri chaturvedi if you are having d_last_updt attribute value as 2017-03-12 06:07:12:01 then in Route On Attribute processor add new property as ${d_last_updt:toDate('yyyy-MM-dd'):format('yyyyMMdd'):gt('20170731')} //todate extracts year month date from your lastupdatedate attribute and formats as yyyyMMdd then give your value in gt function. This property checks all d_last_updt attribute value if the value is greater than 20170731 if attribute value satisfies the condition then it routes to property name relation , if not it routes to unmatched relation. Route on Attribute processor Configs:-
... View more
01-24-2017
03:15 PM
@Balakrishnan Ramasamy
The NiFi expression language allows users to dynamically retrieve values from FlowFile Attributes, environment variables, JVM properties, or a variable registry file. Only properties that are marked as supporting expression language can be passed value dynamically. Floating your cursor over the little question mark next to any property will tell you if it support expression language or not. For sensitive property fields, the value entered is encrypted upon hitting OK. It is then stored in the encrypted format in the flow.xml.gz file. It does not support the expression language.
Please accept my answer if you feel i have addressed your question. Thank you, Matt
... View more
05-14-2018
06:38 PM
@Tarek Elgamal Assuming you are referring to settings for "Max Timer Driven Thread count"? That setting controls the max number of threads that can execute at one time. Does not guarantee any order to the execution of threads. NiFi's controller in the back ground does not operate under this thread pool. Both processors will be scheduled to run based on their configured run schedule. Those concurrent tasks then get stacked in a request queue waiting on one of the threads from that pool to service them. This way, every processor is eventually going to get a chance to run thier code. Also keep in mind that some processors work on batches of FlowFiles while others process one FlowFile per task. Also hard to say that each processed FlowFile will take same amount of time to complete an operation. Really depends on processor and what it is designed to do. Thanks, Matt
... View more
11-15-2016
05:09 PM
Hi @bala krishnan , Not a solution but just to let you know that with the next version of NiFi (coming soon) you will be able to use ValidateCSV processor to achieve what you are looking for. In the meantime, I think that splitting the file is not going to help. Maybe trying something custom with ExecuteScript processor but probably not ideal. Hope this helps.
... View more
11-15-2016
01:26 PM
3 Kudos
What does your current schema look like? If you have a field with a type of something like ["null","int"] then it is being declared as a "nullable union", meaning the value can be null or a valid integer. If instead you use simply "int" for the type, then it should enforce non-null values for that field. If it does not, then the CSV reader from the Kite SDK (used to parse the CSV in the ConvertCSVtoAvro processor) likely treats missing values as empty or default rather than null. If this is the behavior you're seeing, please feel free to file a Jira to improve the handling of missing CSV values.
... View more
11-14-2016
02:58 PM
2 Kudos
ConvertCsvToAvro attempts to convert each record... if at least one record was converted successfully then it transfers that flow file to "success", if any records could not be converted then it also transfers a flow file to "incompatible" which has all of the original CSV in the content and a summary of the incompatible records in an attribute called "errors". I think you need to implement a custom version of this processor that only transfers the flow file to success when the error count equals 0.
... View more
11-10-2016
05:44 PM
1 Kudo
You can use ExecuteScript with Groovy and the following script (assuming your input is newline-delimited):
def flowFile = session.get()
if(!flowFile) return
def header = ''
session.read(flowFile, { inStream ->
header = new BufferedReader(new InputStreamReader(inStream)).readLine()
} as InputStreamCallback)
flowFile = session.putAttribute(flowFile, 'header', header)
session.transfer(flowFile, REL_SUCCESS)
This puts the first line in an attribute called 'header', which you can use with RouteOnAttribute to decide where to send the flow. Note that this script doesn't do error handling, but you could put a try/catch around the session.read to session.transfer, the catch could route the flow file to REL_FAILURE.
... View more