Hi Vijay, Thanks for the detailed output. Looks like both Hue and Beeline have the same issue, that both of they were not able to retrieve results from HS2. A few more questions: 1. how many HS2 hosts do you have? 2. Can you check the HS2 log on the host that Hue and Beeline connects to and see what error you can get from there? It might tell more story on why it failed 3. Can you also try below queries: select * from tdb1.t1 where bs1_dt=2017-06-23; select * from tdb1.t1 where bs1_dt=2017-06-26; And see which partition might be causing the issue, or is it both? Thanks Eric ... View more

I need to zip 3 files from S3, so will the flow be like  3  FetchS3Object  processors (  for 3 files )  → Merge Content → PutS3Object ?  ... View more

Please send me the screen shots of the flow. I am working with the same use case  writing csv content to DB. help me I am unable define the properties in the processors.   ... View more

@Rohit1981  Try using ValidateRecord processor as described in this article with  Strict Type Checking is set to "true" and to parse fixed width file refer to this link for detailed explanation for parsing.   ... View more

@ANMAR  Try with this regex in   ExtractText   processor. (?:"x":.\w+?)(\d+) This regex will extract only the digit in "x" key and adds that value for "y" key in   ReplaceText   processor. ... View more

Hi,   Exit code 143 is related to Memory/GC issues. Your default Mapper/reducer memory setting may not be sufficient to run the large data set. Thus, try setting up higher AM, MAP and REDUCER memory when a large yarn job is invoked.  For more please refer to this link. https://stackoverflow.com/questions/42972908/container-killed-by-the-applicationmaster-exit-code-is-143   Thanks AKR ... View more

I think that QueryDatabaseTable processor doesn't support fetching updated data, you can use CaptureDataChange Processor instead! ... View more

Is there a way so that new queries are executed through this previous TEZ application rather than generating new ones stucked on `ACCEPTED`? (for it is very annoying to have to kill the previous each time I want to run something new) ... View more

@Raymond Cui Try with adding new attribute in UpdateAttribute processor as epochtime ${file.creationTime:toDate("yyyy-MM-dd'T'HH:mm:ss+0000"):toNumber()} Then nifi will match the format and convert to epoch time. ... View more

@Satish Karuturi This is an expected behaviour from ExecuteStreamCommand processor and best practice is to place shell script file on all nodes of NiFi cluster. As you are having 2 node nifi cluster and we are not able to control which will be primary node and ExecuteStreamCommand processor will run only on primary node. In case of primary node changes nifi will pick the shell script from the active primary node and continue to execute the script without any issues..! In addition you can also use ExecuteProcess processor to execute Shell script in NiFi. ... View more

Thanks for helping, @Shu! I now get this alert: ... View more

This is the error if I process the queue with the above content in PuthiiveQL PutHiveQL[id=89b230de-8dc5-18e0-9cac-668cfe8085b0] Failed to update Hive for StandardFlowFileRecord[uuid=1bb63fbe-29af-4422-b8a3-a282063514e8,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1564127716134-1, container=default, section=1], offset=186312, length=69],offset=0,name=9d11a5b0-78e8-43f1-9514-9e4c4509855c,size=69] due to Error while compiling statement: FAILED: ParseException line 1:58 cannot recognize input near '?' ',' '?' in value row constructor; routing to failure: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: ParseException line 1:58 cannot recognize input near '?' ',' '?' in value row constructor ... View more

Splits a JSON File into multiple, separate FlowFiles for an array element specified by a JsonPath expression. Each generated FlowFile is comprised of an element of the specified array and transferred to relationship 'split,' with the original file transferred to the 'original' relationship. If the specified JsonPath is not found or does not evaluate to an array element, the original file is routed to 'failure' and no files are generated. regards, titaniumtv download ... View more

@Shu Thank you and you are right. Now it can process and faster than before. Just wondering about combination of load balancing and concurrent task. If we assign concurrent tasks to some CPU-intensive processor like ExecuteScript and ExecuteStreamCommand, which run on all nodes, will load balancing the queued up data before the processor give better result than simply running it with concurrent task? Because I thought it takes some time to distribute relatively large data (~5 MB) across the cluster. And why not PutHDFS with some concurrent tasks and load balance together working? ... View more

@shahid rehman If the answer is helpful to resolve the issue, Login and Click on Accept button below to close this thread.This will help other community users to find answers quickly 🙂 ... View more

The above question and the entire response thread below was originally posted in the Community Help track. On Sun Jun 30 21:03 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about Nifi processors. ... View more

@Juhyeon Yun Configure directory as /user/paxata/job and keep recursive sub directories to true will list all the files in the sub directories. Similar question reported in this HCC thread. ... View more

hi @Shu thanks for answering, I already create an internal table and I got an error like that: ... View more

The above question and the reply thread below was originally posted in the Community Help Track. On Tue Jun 18 03:04 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself. ... View more

@jingyong zou The issue is with the flowfile format that is passed through the processor. As ConvertAvroToJson processor accepts only Avro format but i think you are passing Json format to the processor which is causing java.io.IOException: Not a data file error. ... View more

@James Willson I just downloaded NiFi-1.9.2 from this link. GenerateFlowFile processor is included in NiFi-1.9.2 version. Not sure why you are not able to have GenerateFlowFile processor in 1.9.2. If i try to search for generate then i was to see GenerateFlowfile processor and GenerateTableFetch Processors. ... View more

@3nomis ConvertRecord processor is still running as we can see running threads as one at top right corner of the processor. If it is running for ever (or) you are not convinced with performance then you can try splitting the 2GB file into smaller chunks by using SplitRecord processor then use MergeRecord processor to merge them back by using Defragment as merge strategy. ... View more