Member since
02-01-2022
274
Posts
97
Kudos Received
60
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
439 | 05-15-2025 05:45 AM | |
3470 | 06-12-2024 06:43 AM | |
6105 | 04-12-2024 06:05 AM | |
4174 | 12-07-2023 04:50 AM | |
2243 | 12-05-2023 06:22 AM |
03-16-2023
08:16 AM
1 Kudo
Awesome news, +2 solutions here.
... View more
03-02-2023
06:44 AM
1 Kudo
@fahed What you see with the CDP Public Cloud Data Hubs using GCS (or object store) is a modernization of the platform around object storage. This removes differences across aws, azure, and on-prem (when Ozone is used). It is a change by customer demand so that workloads are able to be built and deployed with minimal changes from on prem to cloud or cloud to cloud. Unfortunately that creates a difference you describe above, but those are risks we are willing to take ourselves in favor of modern data architecture. If you are looking for performance, you should take a look at some of the newer options for databases: impala and kudu (this one uses local disk). Also we have Iceberg coming into this space too.
... View more
03-01-2023
06:33 AM
1 Kudo
I increased the size of the ListenUDP processor and turned to false the content.repository.archive property and everything works again. Thanks for your reply
... View more
03-01-2023
04:14 AM
Nice and Quick! Excellent!
... View more
02-24-2023
05:55 AM
1 Kudo
@kishan1 In order to restart a specific processor group you will need to use some command line magic against the Nifi API. For example, this could be done by using a command to stop the processor group, then the restart nifi command, then start processor group. You can certainly be creative in how you handle that approache once you have experimented with the API. https://nifi.apache.org/docs/nifi-docs/rest-api/index.html
... View more
02-23-2023
05:09 AM
1 Kudo
@fahed That size is to be able to grow and serve in production manner. At first that disk usage could be low. For DataHubs, My recommendation is to start small and grow as needed. Most of your work load data should be in object store(s) for the data hubs, so dont think of that "hdfs" disk as being size constrained to initial creations of the hub.
... View more
02-22-2023
08:28 AM
1 Kudo
@merlioncurry Lacking a bit of deatils, so making some assumptions that you used an Ambari UI to upload to HDFS. So those files are going to be in hdfs://users/maria_dev, not on the actual machine location for the same users. You will need use hdfs commands to view them. If they do not work, then the path you uploaded may be different. From the sandbox prompt: hdfs dfs -ls /users/ hdfs dfs -ls /users/maria_dev
... View more
02-09-2023
06:22 AM
1 Kudo
@Techie123 Well, like i said, you have to learn the aws side of providing access to a bucket. A public bucket starting point will show you what you have to do, inside of the bucket config, to allow other systems to access that bucket. For example starting from public open bucket, to whatever access control level you ultimately need to have. Getting lost in that space is not necessarily a "nifi" thing.... so my recommendation is to build nifi with public bucket, THEN when it works, start testing the deeper access requirements. The controller service configuration provides multiple ways to access a bucket and a bunch of settings. Make sure you have a working access/key credentials tested directly in the processor before moving to the Controller Service.
... View more
02-07-2023
05:02 AM
@Abdulrahmants if you need to talk to someone about getting those added, please reach out in direct message. Another approach could be to create an API input endpoint on nifi (handleHttpRequest/handleHttpResponse), and make a scripted (python,java,etc) process to send the file to the nifi endpoint.
... View more