Member since
02-01-2022
223
Posts
71
Kudos Received
48
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
141 | 08-17-2023 06:07 AM | |
177 | 08-14-2023 05:54 AM | |
236 | 08-07-2023 06:51 AM | |
194 | 07-17-2023 08:43 AM | |
311 | 07-17-2023 08:38 AM |
12-16-2022
05:19 AM
The hive libraries are quite large and are not included in the base binary distribution. You can add them manually, or use github to clone and build the full project. Reference: https://github.com/apache/nifi/blob/rel/nifi-1.19.0/nifi-assembly/pom.xml#L1054
... View more
12-16-2022
04:43 AM
Ahh my confusion, it is hard to assume without a lot more detail. Reference: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#gt ${fileSize:gt( 1024 )} You need to use the expression "gt( some value )" against a parameter or variable. In the case above, "file size greater than 1024".
... View more
12-15-2022
06:31 AM
@wallacei There are roles attached to the main control plane and roles attached to specific environment. Work with you environmentAdmin to make sure you have all the correct roles at the control plane level and then deeper at the environment level if necessary. Below is the EnvironmentAdmin [
{
"crn": "crn:altus:iam:us-west-1:altus:policy:EnvironmentAdminPolicy",
"policyStatements": [
{
"rights": [
"environments/getFreeipaOperationStatus",
"environments/repairFreeIPA",
"environments/upgradeFreeIPA",
"environments/createDatahub",
"datahub/read",
"datahub/write",
"datalake/read",
"datalake/write",
"environments/read",
"environments/write"
],
"resources": [
"*"
]
}
]
}
] You may need more specific roles to access additional services around the environment.
... View more
12-15-2022
06:16 AM
@saicharan You can find the hive jdbc driver for mac here which allows the operating system drop down: https://www.cloudera.com/downloads/connectors/hive/jdbc/2-5-4.html Also be sure to share your connection string and specific error message for better responses.
... View more
12-15-2022
06:12 AM
@Jaimin7 assuming "gt" is your parameter, that would be referenced as #{gt} and i think would be: {"updatedAt" : {#{gt} : 2022-12-10}} If "gt" is your variable, that would be represented as ${gt} and that would then be: {"updatedAt" : {${gt} : 2022-12-10}} Preference going forward is to use Paramaters as Variables will be going away soon.
... View more
12-12-2022
05:42 AM
1 Kudo
@hegdemahendra Awesome to see you making custom processors. A few things that might help: Make sure this controller service exists in your nifi, some nifi builds do not include all nars Make sure this controller service exists in your project dependencies That said, you may find more advanced nifi help engaging with the nifi developer community on slack or the mailing list. You can find the slack invite link at the bottom of that page.
... View more
12-09-2022
06:03 AM
@Althotta I do not think this is possible for the create to reference schema registry, only insert/update. To do this table creation in NiFi you will need to do some flowfile content manipulation work to fabricate the CREATE TABLE statement. For example if the flowfile content is the avro schema (you could get this form the schema registry API) then you can parse out the columns and data types to get the middle of the create statement: (col_name data_type [COMMENT 'col_comment'],, ...) then a ReplaceText processor to add the top lines: CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name and bottom lines: [COMMENT 'table_comment'] [ROW FORMAT row_format] [FIELDS TERMINATED BY char] [STORED AS file_format]; This would give you a full create statement which you can then execute against Hive.
... View more
12-09-2022
05:54 AM
@quangbilly79 The top screen shot is for CDP Public Cloud and the bottom screen shot is for CDP Private Cloud Base. As such, the CDE (Cloudera Data Engineering) "Schedule Job" is not available in the Cloudera Manager UI. You could use something like Oozie: https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/configuring-oozie/topics/oozie-introduction.html
... View more
12-09-2022
05:39 AM
@abdebja Have you completed HUE configuration for SSL of each services host(s)? The docs are as follows: https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/securing-hue/topics/hue-configuring-tls-ssl.html
... View more
12-09-2022
05:33 AM
@KPG1 Here is a good post i found that describes the required steps to move postgres to mysql including a php to do the postgres to mysql conversion. https://cloudera.ericlin.me/2016/10/hive-metastore-migration-from-embedded-postgres-to-mysql/ You may need to find a more modern tool for that conversion, but the process to dump, convert, and move metastores should be very similar.
... View more