Member since
06-26-2015
505
Posts
129
Kudos Received
114
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
837 | 09-20-2022 03:33 PM | |
2644 | 09-19-2022 04:47 PM | |
1515 | 09-11-2022 05:01 PM | |
1641 | 09-06-2022 02:23 PM | |
2484 | 09-06-2022 04:30 AM |
06-30-2023
02:31 AM
We are excited to announce the general availability of Cloudera Streaming Analytics (CSA) 1.10.0 on CDP Private Cloud Base. This release includes a massive set of improvements to SQL Stream Builder (SSB), including the addition of built-in widgets for data visualization, as well as a rebase to Flink 1.16. Some of the key features of this release are: Rebase to Apache Flink 1.16 - Apache Flink 1.16 is now supported in CSA 1.10. PyFlink Support - The Python API for Flink is now supported in CSA. Customers can now create Flink DataStream applications using Python, besides Java and Scala, to build scalable batch and streaming workloads like real-time data processing pipelines, large-scale exploratory data analysis, Machine Learning (ML) pipelines and ETL processes Built-in Widgets for Data Visualization - Built-in data visualization widgets have been added to the SQL Stream Builder (SSB) UI to provide a quick and simple way to visualize data from streaming jobs and materialized views in real-time. Built-in Support for Confluent Schema Registry - New catalog type in SSB to make it very easy to read and write data from Confluent Cloud clusters using their Schema Registry service. Flexible Schema Handling for Schema Registry catalogs - Cloudera Schema Registry catalog can now handle separate schemas for message key and payload. Useful Links Documentation Release notes NEW BLOG: Building a Stateful Intrusion Detection System with SSB Cloudera Stream Processing (CSP) Community Edition - Try SSB for free!
... View more
Labels:
06-07-2023
01:58 AM
This worked. Thanks.
... View more
04-17-2023
01:03 PM
@Manika The single-user-provider used by NiFi out of the box does not support multiple users. It simply sets up a single user to access the secured NiFi suing the out of the box NiFi setup. Only the very first time NiFi is launched with this setup will the generated username and password be output in the nifi-app.log. You can use the following command to change that username and password: $ ./bin/nifi.sh set-single-user-credentials <username> <password> This single user username and password has full access to everything, so NiFi does not expose "policies" as setting up additional users is not possible. NiFi provides no other provider for local users. NiFi historically has left user management as an external dependency. In order tp support multiple users with and various level of authorization, you will need to switch to using a different authorizer and authentication method. Switching to another authentication method will require you establish an external means for user authentication. NiFi has numerous supported methods (User certificate, LDAP, Kerberos, etc.) https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Once you have selected the method you will use to authenticate your users that is not the "single-user-provider", you need to change your authorizer from "single-user-authorizer" to "standard-managed-authorizer": https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#standardmanagedauthorizer If you found that any of the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
03-21-2023
11:39 AM
@udayAle @ep_gunner When NiFi is brought down, the current state (stopped, started, enabled, disabled) of all components is retained and on startup that same state is set on the components. Only time this is not true is when the property "nifi.flowcontroller.autoResumeState" is set to false in the nifi.properties file. When set to false a restart of NiFi would result in all components in a stopped state. In a production environment, this property should be set to true. Perhaps you can share more details on the maintenance process you are using as I am not clear on how your maintenance is impacting the last known state of some components. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
03-09-2023
04:58 PM
The Cloudera Data in Motion (DiM) team is pleased to announce the general availability Cloudera Streaming Analytics (CSA) 1.9.0 on CDP Private Cloud Base 7.1.7 SP2 and 7.1.8. This release includes a massive set of improvements to SQL Stream Builder (SSB) as well as updates to Flink 1.15.1. These changes are focused on enhancing the user experience and removing objections and blockers in the sales cycle. All the features described below are already available in the Cloudera Stream Processing - Community Edition release, which is the fasted way for you to try them out for free. Links: Documentation Release notes CSP Community Edition Download and Install Blog - A UI That Makes You Want To Stream Blog - SQL Stream Builder Data Transformations Blog - Job Notifications in SQL Stream Builder Key features for this release: Reworked Streaming SQL Console: The User Interface (UI) of SQL Stream Builder (SSB), the Streaming SQL Console has been reworked with new design elements. Software Development Lifecycle (SDLC) support (Tech Preview): Projects are introduced as an organizational element for SQL Stream Builder that allows you to create and collaborate on SQL jobs throughout the SDLC stages with source control. For more information, see the Project structure and development documentation. Confluent Schema Registry support. Confluent Schema Registry can be used as a catalog in SQL Stream Builder and Flink. This unblocks the onboarding of customers that are using Confluent Kafka with Confluent Schema Registry. Improved REST API for SSB. Several new endpoints have been added to the API, making it easier to automate deployments to SSB and to integrate it with other applications. Updated CSP Community Edition. Community edition has been refreshed to include all these features including the revamped UI and SSB Projects and offers the fastest way for you to try out these new features. And, as usual, bug fixes, security patches, performance improvements, etc.
... View more
Labels:
02-01-2023
09:19 AM
I agree with Smarak, the error code typically means that there were not enough resources available to start the job. You could probably use Grafana dashboard (available in the Admin page) to look at the cluster resources and load around the time you had this issue. Is it happening consistently? For Jobs, I usually see this at the start or end of the month, when a lot of people schedule periodic jobs and they all trigger at the same time.
... View more
11-16-2022
06:27 AM
I am facing similar issues. Please help me out if anyone has any solution to this.
... View more
11-15-2022
11:19 PM
I'm able to fix this .. In JSON Tree READER the configuration for `Schema Access Strategy` should be "Infer Schema" Input date format will be "yyyy-MM-dd HH:mm:ss.SSS'Z'" avro schema for the same will be {
"name": "createdAt",
"type": {
"type": "long",
"logicalType": "timestamp-millis"
}
}
... View more
10-25-2022
03:28 AM
Hello bro @SAMSAL , were you successfully control the nifi component using postman? If yes, would it be okay to share your countermeasure. Currently encountering the same issue that you had. Thanks in advance.
... View more
10-21-2022
02:09 AM
The issue was not related to the size of the queue, but instead to the fact that when to destination that we are sending those logs to had connectivity issues (which was causing quite a lot of timeout errors), NiFi was using a lot of a significant number of threads to retry sending the logs to the InvokeHTTP destination. This high usage of threads was causing ListenSyslog to not function properly - it was dropping the logs that it was supposed to receive.
... View more