1973
Posts
1225
Kudos Received
124
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2453 | 04-03-2024 06:39 AM | |
| 3802 | 01-12-2024 08:19 AM | |
| 2049 | 12-07-2023 01:49 PM | |
| 3033 | 08-02-2023 07:30 AM | |
| 4153 | 03-29-2023 01:22 PM |
10-22-2020
11:12 AM
https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html Maybe shrink your provenance repo and send everything you need via: https://www.datainmotion.dev/2020/04/sql-reporting-task-for-cloudera-flow.html Or via Atlas
... View more
10-22-2020
09:47 AM
provenance is for every step of every flowfile in every flow. I recommend rolling those logs
... View more
10-22-2020
09:23 AM
you need to restart your servers after making the setting changes. do you have space?
... View more
10-09-2020
07:16 AM
1 Kudo
https://www.datainmotion.dev/2020/09/devops-working-with-parameter-contexts.html download the flow/backup up/store in git copy a flow to archive remove from production https://www.datainmotion.dev/2019/11/nifi-toolkit-cli-for-nifi-110.html
... View more
10-06-2020
07:01 PM
https://docs.cloudera.com/runtime/7.2.1/integrating-hive-and-bi/topics/hive-kafka-integration.html
... View more
10-06-2020
06:59 PM
1 Kudo
I believe you need to be running HDP 3.1 or CDP for the Hive-Kafka integration since it's part of Hive 3.x and CDH has Hive 1 or 2.
... View more
08-19-2020
10:01 AM
https://www.datainmotion.dev/2020/08/deleting-schemas-from-cloudera-schema.html
... View more
08-13-2020
07:35 AM
looks like some Hadoop interfacing issues, open a ticket with Cloudera support. https://community.cloudera.com/t5/Support-Questions/DataXceiver-error-processing-unknown-operation-java-io/td-p/164048
... View more
08-12-2020
06:45 AM
That is really odd, I recommend you open a ticket with your Cloudera support. I think having an engineer walk through your environment may be the only way to find such a complex issue. This one is really odd. I haven't seen this occur unless there was an issue with security or HDFS. Or missing / broken file path or file permissions https://community.cloudera.com/t5/Support-Questions/putHDFS-processor-errors-in-Nifi/td-p/191407 You could try PutOrc or PutFile. Can that NiFi reach any other Kerberos protected resources like HBase or Hive? Dual NIC issue (networking) https://community.cloudera.com/t5/Support-Questions/Nifi-putHDFS-throwing-replication-error-for-single-node/td-p/199784 Invalid Private Address https://community.cloudera.com/t5/Support-Questions/Nifi-Processor-PutHDFS-not-working-to-write-files-on-HDFS/td-p/237546 bootstrap.conf issue java.arg.101=-Djavax.security.auth.useSubjectCredsOnly=true https://community.cloudera.com/t5/Support-Questions/Nifi-PutHDFS-Processor-stuck-and-not-able-to-release-item/td-p/229780 Keypath https://community.cloudera.com/t5/Support-Questions/How-to-connect-to-Kerberized-HDP-cluster-from-separate/td-p/152730 https://risdenk.github.io/2018/03/15/hdf-apache-nifi-kerberos-errors-usesubjectcredsonly.html Login time https://community.cloudera.com/t5/Support-Questions/PutHDFS-Impersonation-error/td-p/111628 https://community.cloudera.com/t5/Support-Questions/NiFi-PutHDFS-Login-Failure-using-kerberos/td-p/195595/page/2 https://community.cloudera.com/t5/Support-Questions/how-to-use-apache-Nifi-on-kerberized-HDP-Cluster-nodes/td-p/116503
... View more
08-11-2020
01:49 PM
Most people doing give root accounts HDFS accounts or kerberos accounts. Depends on the environment. Just trying to see what is different between environments. If it's not user, permissions, firewall, version of NiFi, that's weird. I have seen some odd setups of Linux where security intercepts somethings or prevents root users from doing certain things. i have seen devops tools change permissions automatically in the background. You can change PutHDFS to Primary node to have just one write to it to narrow down which machine in the cluster may have an issue.
... View more