Member since
07-19-2018
613
Posts
101
Kudos Received
117
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 5097 | 01-11-2021 05:54 AM | |
| 3422 | 01-11-2021 05:52 AM | |
| 8790 | 01-08-2021 05:23 AM | |
| 8385 | 01-04-2021 04:08 AM | |
| 36691 | 12-18-2020 05:42 AM |
05-04-2020
07:25 AM
@varun_rathinam Accessing json in an array object via EvaluateJsonPath can be quite confusing. I also notice the structure of your json is kind of confusing with same values in both. I have adjusted id2 for cc and dd for testing so that I can tell id1 and id2 values apart. The solution you want is (see template for exact string values): Notice we use the normal tree for each json object ( $.object ) then access the array ( 0, 1 ) then access the array's objects. Also notice it is possible to access the json object array with or without a . before the [. Reference: https://community.cloudera.com/t5/Support-Questions/how-to-extract-fields-in-flow-file-which-are-surrounded-by/m-p/208635 You can also find my template during testing of your issue on my GitHub: https://github.com/steven-dfheinz/NiFi-Templates/blob/master/NiFI_EvaluateJsonPath_Demo.xml If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven @ DFHZ
... View more
04-30-2020
07:35 AM
@bhara did you follow all the steps to use Oracle?
... View more
04-29-2020
01:22 PM
Yah it seems that error is usually related to the stock db. If you can't see any tables in the oracle, then after configure for oracle and restart it must not have installed the metadata tables. You will need to get them installed and I suspect the lock error will go away. Check here for some oracle steps: https://docs.gethue.com/administrator/administration/database/#oracle
... View more
04-29-2020
01:17 PM
@mzidy Have you tried to append the data you want to pass in the Remote Url? Some Apis will accept the values in the $_GET request. For example: DELETE: https://api.cloudera.com/api/service/?value1=value1&value2=value2 Some apis could be: DELETE: https://api.cloudera.com/api/service/1/2/ In both these examples the $_GET variables are used not a $_POST. I believe this may be typical for most DELETE calls and may explain why DELETE doesn't send a $_POST (body).
... View more
04-29-2020
01:08 PM
@bhara The error is very specific "OperationalError: database is locked" Have you checked the database.table you are querying is locked or not? If the error is indeed from the oracle metastore, is that database or table locked? You could test another query on another table as well, to see if it is just that table locked. After removing the lock, does the query work again? Sometimes hive can lock tables, and never unlock them. You would need to query the hive metastore directly, and manually remove these locks if they persist.
... View more
04-29-2020
06:27 AM
@abhinav_joshi In NiFi you can have all kinds of different processors from different versions. The preferred procedure is to put them in a custom-libs folder location available to nifi(s), add a value to nifi config, delete the work directory in nifi(s), and restart nifi(s). You can see more info about how I did this on the following post: https://community.cloudera.com/t5/Support-Questions/Can-I-put-the-NiFi-1-10-Parquet-Record-Reader-in-NiFi-1-9/m-p/289111#M214056
... View more
04-28-2020
05:15 AM
@Damian_S You cannot delete and recreate a hadoop cluster in this manner. The data in the metastore is very specific to the previous installation. A proper migration will back up the required artifacts from the original cluster, and migrate them to a new existing ambari cluster that has already completed the initial cluster installation.
... View more
04-28-2020
05:09 AM
@san_re The configuration of Remote Url in InvokeHTTP should be as follows: /j_security_check?j_username=${j_username}&j_password=${j_password} Notice the credential values are passed to the url with the ?. This is the only change in the processor config. Here is a demo flow: GenerateFlowFile - starts flow for testing UpdateAttribute - sets user, pass,and mime type attributes needed in InvokeHttp InvokeHttp - sends POST You can find this template on my GitHub to inspect the entire flow: https://github.com/steven-dfheinz/NiFi-Templates/blob/master/InvokeHttp_Demo.xml
... View more
04-28-2020
04:49 AM
1 Kudo
@arunnalpet This sounds like a perfect job for ReplaceText: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ReplaceText/ I did notice that you have set as $set. In order to maintain a real json object, I have used just "set". The set object is available as $.set if you use EvalateJsonPath. If you still need $set, please explain... Below is the configuration you need for ReplaceText. It regex matches the entire flow file then adds the prepend before $1 and adds the append after. { "set" : $1 } (this is the only change in the processors from default) And the output: { "set" : { "first_name": "alpha", "age": "30" } }
... View more
04-24-2020
06:16 AM
@Former Member I think you misunderstood the Variable Scoping. All sensitive values in templates and registry controlled process groups do not migrate to the new instance. These sensitive values should be set at the Root NiFi Canvas level per environment. Then when you import a template, or in your case, a version controlled process group, the values are already set per environment. For example: Let's say we have 3 environments: dev, staging, and prod. Each host has its own "database" host with different database tables, users and passwords. I want all 3 environments to run a single flow with NiFi Registry Version Control. I create the flows database controller services using ${host}, ${user}, ${pass}, ${database}, etc. Now in each environment, at the root level, I define each variable. Since the sensitive value is defined at a higher Variable Scoping level (root canvas level or other higher level) it will be available to any template or version controlled process group at any lower Variable Scoping level.
... View more