Member since
07-19-2018
613
Posts
101
Kudos Received
117
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 5686 | 01-11-2021 05:54 AM | |
| 3811 | 01-11-2021 05:52 AM | |
| 9485 | 01-08-2021 05:23 AM | |
| 9285 | 01-04-2021 04:08 AM | |
| 38600 | 12-18-2020 05:42 AM |
01-25-2020
06:08 AM
I’m the schema remove the second row in fields. So it’s just id1. Now do a test: use generateflowfile with the schema in contents. Then evaluatejson. In this processor click +, and make id1. The value for id1 is then $.value.id1 (this is the code to get the real value. Then route evaluatejson routes to an output port. Run. Inspect the item in the queue and confirm you see the attribute id1. This is a simple example to help you understand how json works, how to get the data from the value object, how to confirm schema is correct, and how to unit test by looking at flowfile in the queue.
... View more
01-24-2020
04:11 AM
1 Kudo
The FlowFile is not valid json. Try something like a basic json example found on the internet: {"results":[{"term":"term1"},{"term":"term2"}..]} Notice the object results is the array of values and everything is "quoted". Something like this based on your sample image: { "world_rank": "1", "country": "China", "population": "1388232694", "World": "0.185" } If you have to use what you have in the sample image, then you will need to extracText (link here for examples of regex to match values) to get the values for world_rank, country, population, world. Each values attribute regex like this: .*world_rank :(.*?) .*
.*country: (.*?) .*
.*population: (.*?) .*
.*world: (.*?) .* Then use attributesToJson to build the Json object of the attributes you defined above which you can then send to the reader and ultimately to MongoDb. If my reply helps you solve your Use Case, please accept it as a solution to close the topic.
... View more
01-23-2020
01:14 PM
1 Kudo
@MattWho best news I heard all day.... after I just finished my POC with new parquet reader/writer 1.10 in 1.9 nifi... was able to get 1.3 billion rows out of a single parquet file on a very small azure cluster!!
... View more
01-21-2020
06:11 AM
@Manus This is a perfect Use Case for Wait / Notify. You just need to be aware of the pitfalls, and recognize the behavior when your wait/notify condition doesn't act and of course test that the timeout condition works as needed. The beauty of NiFi is that you should be able to account for every possible scenario, and part of the fun in NiFi for me is being creative, being able to capture exceptions, and either further educate my existing flow to handle them, or by creating new parts of my flow specifically for the exceptions.
... View more
01-21-2020
06:06 AM
You should be able to accept your own reply as solution marking it closed/solved. One thing I do notice, which I HIGHLY recommend against is using "localhost"...
... View more
01-13-2020
04:23 AM
I believe the best thing to do is jump right in. Start working with Cloudera Products. Complete some demos, join in online training, study documentation, and start creating experience. As an example here are a few things that Solution Architects work on at Cloudera: Install CDH or previous products: CDM, HDP, & HDF. Hadoop Yarn & Map Reduce Spark Kafka Hive & Hive LLAP NiFi, NiFi Registry, & Schema Registry Security (SSL, Encryption, etc) Active Directory, LDAP, and Ranger Other Components: Hbase, Solar, Impala, Zeppelin Cloudera Course List: https://ondemand.cloudera.com/courses Free CDP Essentials: https://www.cloudera.com/about/training/courses/cloudera-essentials-for-cdp.html (free)
... View more
01-10-2020
05:33 AM
2 Kudos
@VijaySankar can you accept my answer (remove localhost) as the solution?
... View more
01-10-2020
05:05 AM
Ahh I did not pay attention to that in the original screen shot, just tried to offer the syntax for the json parsing. Glad you got it to work! Isn't learning NiFi fun? I love it.
... View more
01-09-2020
01:37 PM
In the routeonattribute proc you use just the ${attribute:equals(‘value’)} that you parsed from evaluatejson proc. Evaluatejson: attribute = $.HR.finance.name RouteOnAttribute ${attribute:equals(‘Murali’)}
... View more
01-09-2020
11:07 AM
1 Kudo
Based on your content, the values are accessed as follows: $.HR.finance.name $.HR.finance.age
... View more