Member since
06-26-2015
346
Posts
84
Kudos Received
66
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
204 | 05-19-2022 06:31 PM | |
61 | 05-17-2022 05:36 PM | |
95 | 04-21-2022 02:51 PM | |
210 | 04-21-2022 02:46 PM | |
69 | 04-21-2022 12:20 AM |
04-21-2022
02:46 PM
1 Kudo
@J0sh , If you use a ReplaceText with the following settings it will clean up the garbage in your flowfiles: Search Value: (?s)(^|(\})<br>).*?(\n).*?(<br>(\{)|$)
Replacement Value: $2$3$5 This file, though, has not 1 JSON, but 8 different JSON fragments. So the resulting flowfile will still have 8 lines, each with a valid JSON. You must use a SplitText process to separate those into 8 different flowfiles before applying the EvaluateJSONPath. Cheers, André
... View more
04-21-2022
02:04 AM
You can copy and paste the content into a Word file (.docx) and attach it to the post.
... View more
04-21-2022
01:26 AM
@J0sh , Can you provide me with the entire contents of your flow file? Cheers, André
... View more
04-21-2022
12:44 AM
@ken_zz , Please run the following command to list the principals contained in your keytab: klist -kt <keytab_file> The full principal name is the long form: either username@REALM or username/fqdn@REALM. Try specifying the full principal name when starting impala-shell. Cheers, André
... View more
04-21-2022
12:41 AM
@sahil0915 , Please try the JoltTransformJSON processor with the following Chain specification: [
{
"operation": "default",
"spec": {
"json_data": {
"items[]": {
"*": {
"quality": "good"
}
}
}
}
}
] Cheers, André
... View more
04-21-2022
12:24 AM
@Amber , Please check https://medium.com/javarevisited/10-best-spring-framework-books-for-java-developers-360284c37036 André
... View more
04-21-2022
12:20 AM
1 Kudo
@r_rachuri , Your expression works for me on NiFi 1.15.2. It's too complicated, though, and it still leaves a pair of curly brackets behind. Try using this one instead: \{[ \n]*\}[ \n]*,?[ \n]* Cheers, André
... View more
04-20-2022
11:56 PM
@J0sh You can use a ReplaceText processor with the following properties: Search Value: (?s)^.*?(\{".*\}).*$
Replacement Value: $1 Cheers, André
... View more
04-20-2022
11:33 PM
@ramks , You can try using the latest Cloudera Hive JDBC driver from here: https://www.cloudera.com/downloads/connectors/hive/jdbc/2-6-18.html Otherwise, you need to upgrade to CDP 7.x or, at least, to CDH 6.3.3 or later. Cheers, André
... View more
04-20-2022
10:27 PM
@Seaport , Please try the below: import json
def jsonize(k, v):
ret = json.loads(v)
ret.update({'key': k})
return ret
...
rdd = reader.map(lambda x: jsonize(*x))
... You need to make sure your schema includes the added key column. Cheers, André
... View more
04-08-2022
05:03 AM
@modususer , After installation, log on to Cloudera Manager and select Administration > License to go to the license page. Upload your license file on that page to activate it. If you want/need to apply the license from the command line, you can do so using the REST API. The curl command below is an example of how you can do that: curl \
-X POST \
-u admin:admin \
"http://cm-home.example.com:7180/api/v43/cm/license" \
-F license=@/path/to/license.txt Cheers, André
... View more
04-08-2022
04:43 AM
@ep_gunner , You can download the NAR from here: https://repo1.maven.org/maven2/org/apache/nifi/nifi-sql-reporting-nar/1.16.0/ Make sure you get the one that matches your NiFi version. Cheers, André
... View more
04-08-2022
04:38 AM
1 Kudo
@furkanaykut , THat's an indication that the resources configured for YARN are already in use somewhere else. The containers are pending because there's not enough resources to run all the containers at once and they have to wait for others to finish. Cheers, André
... View more
04-08-2022
04:33 AM
@Jerry_Molan , Please check this matrix: https://www.ibm.com/docs/en/spectrum-scale-bda?topic=requirements-support-matrix Cheers, André
... View more
04-08-2022
04:17 AM
@ravi_tadepally , Is your RabbitMQ configured to require TLS Mutual Authentication (or Client Authentication)? If so, you must also provide a keystore in the SSL Context in NiFi, besides the truststore. Cheers, André
... View more
04-08-2022
04:13 AM
1 Kudo
@Ibrmca , CDSW 1.6.1 does not support RHEL 7.9. Please see the supported OS's here. CDSW 1.9 and above do support RHEL 7.9. Please check the documentation here for upgrade details. Cheers, André The
... View more
04-02-2022
06:09 AM
@templarian , Your file has the characters "ú" and "í", which are not valid in an UTF-8 encoded file. Try replacing them with either "u" and "i", or with the proper UTF-8 encoding sequences "\xFA" and "\xED", respectively. For example: Extraer \xFAltimas 10 l\xEDneas del log Cheers, André
... View more
04-01-2022
07:28 PM
@Rohan44 , Unfortunately, I don't think there's a way to do that. What are you using to write the files to HDFS? Cheers, André
... View more
04-01-2022
07:26 PM
@gizo , In your processor you set "Ranger Key Value" to "${dynamodb.item.range.key.value}". This would reference to an attribute in your flowfile, but from the details you provided, there's no such attribute there. This may be the cause of the error. You must ensure that the attribute exists in the flowfile so that you can reference it in the processor configuration. Cheers, André
... View more
04-01-2022
07:20 PM
@templarian , If you could provide the converted YAML configuration, it could help understanding what the problem is. Cheers, André
... View more
04-01-2022
03:13 AM
@Rohan44 , Try creating your datafile with each JSON record in a single line, like so: {"buyer":{"legalBusinessName":"test1 Company","organisationIdentifications":[{"type":"abcd","identification":"test.bb@tesr"},{"type":"TXID","identification":"12345678"}]},"supplier":{"legalBusinessName":"test Company","organisationIdentifications":[{"type":"abcd","identification":"test28@test"}]},"paymentRecommendationId":"1234-5678-9876-2212-123456","excludedRemittanceInformation":[],"recommendedPaymentInstructions":[{"executionDate":"2022-06-12","paymentMethod":"aaaa","remittanceInformation":{"structured":[{"referredDocumentInformation":[{"type":"xxx","number":"12341234","relatedDate":"2022-06-12","paymentDueDate":"2022-06-12","referredDocumentAmount":{"remittedAmount":2600.5,"duePayableAmount":3000}}]}]}}]} Cheers, André
... View more
03-31-2022
09:54 PM
Which log file are those errors coming from? André
... View more
03-31-2022
06:39 PM
Also this: namei -l /run/cloudera-scm-agent/process/9295-yarn-RESOURCEMANAGER/creds.localjceks
... View more
03-31-2022
06:38 PM
@iamfromsky , Could you please share the output of this command: ls -ln /run/cloudera-scm-agent/process/9295-yarn-RESOURCEMANAGER/creds.localjceks Cheers, André
... View more
03-31-2022
02:46 PM
@gizo , The JSON you shared is the "content" of the flowfile. The flowfile also have "attributes" metadata. Could you please share a screenshot of that? André
... View more
03-31-2022
02:40 PM
@iamfromsky , What are the default permissions of the creds.localjceks file? Cheers, André
... View more
03-31-2022
02:25 AM
@dmharshit , Could you please share the output of this command? namei -l /usr/share/java/mysql-connector-java.jar Cheers, André
... View more
03-30-2022
10:47 PM
@dmharshit , Did you execute these instructions? https://docs.cloudera.com/cdp-private-cloud-base/7.1.3/installation/topics/cdpdc-install-config-postgresql-for-cdp.html Cheers, André
... View more
03-30-2022
10:43 PM
@gizo , Is "dynamodb.item.hash.key.value" a property of your flowfile? Could you show a screenshot of the flowfile properties para with that property? Cheers, André
... View more
- Tags:
- error
- NiFi
- putdynamodb
03-30-2022
10:40 PM
@gizo Could you provide the full configuration for both processors? The screenshots are cropped. Cheers, André
... View more