Solved

When I start restart Master is OK.but after 10s Master is shutdown.I don't see error in log. I only find  CDH 6.3.2 enable kerberos

Hello i have the following problems  when i use hbase hbck 10 inconsistencies detected.Status: INCONSISTENT2023-02-01 13:51:01,346 INFO [main] zookeeper.ZooKeeper: Session: 0x1860d1d2ec00091 closed2023-02-01 13:51:01,346 INFO [main] client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService2023-02-01 13:51:01,346 INFO [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x1860d1d2ec000922023-02-01 13:51:01,346 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down2023-02-01 13:51:01,349 INFO [main] zookeeper.ZooKeeper: Session: 0x1860d1d2ec00092 closed2023-02-01 13:51:01,349 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down Also am unable to create a new table and this is the message that is being displayed  when i use create 'users','userinfo'ERROR: java.io.IOException: Table Namespace Manager not ready yet, try again later 

Hi all, I am new to Cloudera. Recently I am studying on CDSW and would like to have a try on it. May I ask is there any way that I can try CDSW for free? I don't have a paid account. I saw that there is 60 days trial on CDP Public/Private Cloud, is CDSW included? Hope someone can help me out. Thank you very much.

Hi, I have a workflow that is picking up an Excel file, that contains 3 sheets, and is attempting to run it through a ConvertExcelToCSVProcessor, but it is failing with the error below:  Failed to process incoming Excel document. Tried to allocate an array of length 328,219,733, but the maximum length for this record type is 100,000,000. If the file is not corrupt or large, please open an issue on bugzilla to request increasing the maximum allowable size for this record type. As a temporary workaround, consider setting a higher override value with IOUtils.setByteArrayMaxOverride(): org.apache.poi.util.RecordFormatException: Tried to allocate an array of length 328,219,733, but the maximum length for this record type is 100,000,000. If the file is not corrupt or large, please open an issue on bugzilla to request increasing the maximum allowable size for this record type. As a temporary workaround, consider setting a higher override value with IOUtils.setByteArrayMaxOverri

Hello, I have a question about the performance of the local index in Apache Phoenix  For example, I have a table TBL_X having 6 columns (A , B ,C ,D E, F).1. Primary Key is (A, B) 2. One of the query patterns is   "Select * from  TBL_X where A = 'val1' and C = 'val3' "  For the better performance of the above query, I try to build the local index. Does   [ Local index on (C, A)  ]  perform better than   [ Local index on (C) ]  ?? or Will there be no performance difference since A is already in the first part of the PK ??   

table is `HiveFileFormat`. It doesn't match the specified format `ParquetFileFormat` I have this issue when I try to write using pyspark with the following command:df.write.mode("append").format("parquet").saveAsTable("schema.table") Before you say change from parquet to hive i know it works. But the thing is the table is partitioned in parquet and I really don't know why not its not working any more. It worked fine until now. The same command ran correctly for 1 month and 5 times so far. But today it does not want to write like this any more. If i check the metadata it also points to everything being in parquet:``103SerDe Library:      org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeNULL104InputFormat:        org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatNULL105OutputFormat:       org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutput

How to run selected processor group in apache nifi while restarting nifi?

How to set the partition life cycle of the hive table？ Error: Error while compiling statement: FAILED: ParseException line 1:270 missing EOF at 'lifecycle' near ')' (state=42000,code=40000)

To get started with CDP public cloud we have to create a env that is named as datalake and if we wanna run any jobs/workload on top of it we have to create datahubs based on the services we need.  Inside the datahub cluster, there is an hdfs service that serves as intermediate place to store data while working on to be eventually stored/moved to cloud object storage. But there is another hdfs service inside the dataleke? what is usage of it? it's total capacity of 1.4TB SSD, how can we utilize it?

is there anyway to change username and credential for Nifi in Windows ? there is a command in Documentation to change it but I guess it works only in Linux

Beginner here...after installing the Hortonworks Sandbox HDP 3.0 on VirtualBox (Mac), I uploaded couple of csv files through ambari under the user name maria_dev by following the tutorial, was expecting to find in /users/maria_dev but logging in as maria_dev into the sandbox, i could not locate these and get an error "No such file or directory".  I managed to upload files via scp into the sandbox and complete the tutorial, however still confused where the upload to ambari UI might have gone. Not sure where the files went or i am looking at the wrong place. Any help will be appreciated.  

My use-case is: 1) Have API credentials2) Use UpdateAttribute to update (1) schema, (2) s3 bucket/location (my list of reports)3) Query API endpoint for report4) API endpoint paginates and gets more records5) Call MergeRecord6) Save to s3 Since 3, 4, 5, 6 are all the same, I'm re-using the processors like below (screenshot).  My problem is (5) MergeRecord will try to merge different schemas together, which is obviously a problem. How can I restructure this?  I'd like to re-use processors as much as possible, but still be able to add more schemas as my needs evolve. 

Hi,we would like install Cloudera CDP 7.1.7 SP1 on linux host with Linux Centos Stream 8.x.The information in Cloudera's support matrix is ​​generically "Centos". Can we consider them valid for Linux Centos Stream 8.x? Thank you.   

Error while processing statement: FAILED: Error in acquiring locks: Error communicating with the metastore (state=42000,code=10) Unable to create Hive Table, while creating a table getting error in acquiring locks: Error communicating with the metastore, ERROR : FAILED: Error in acquiring locks: Error communicating with the metastoreorg.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastoreat org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:178)at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:447)at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocksWithHeartbeatDelay(DbTxnManager.java:463)at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:278)at org.apache.hadoop.hive.ql.lockmgr.HiveTxnManagerImpl.acquireLocks(HiveTxnManagerImpl.java:76)at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:95)at org.apache.hadoop.hive.ql.Dr

Please help design data flow in nifi, I need to insert the incoming JSON data into table which having single column. The condition are I need to concat all the keys and values and insert that data. 

Hi Team, I'm new to CDP and am currently looking at myaccount page "CDP Public Cloud Usage summary" staring to get a feel for the CDP product, but wanted to know if anyone has had experience in Billing and Metering of CDP subscriptions.  I'd like to know if it is at all possible to create rules around subscription spend eg. If estimated cost went above $20, can an alert generated advising me of this.Also is there any other info that might help with managing cost that this team might know about, that might be useful?  Thanks

I have an avro schema that I can get from a registry:  { "name": "bar_tbl", "fields": [ { "name": "foo1", "type": "string" }, { "name": "foo2", "type": "string" } ]}  I know how to use `EvaluateJsonPath` and get `name` and `fields`.I'd like to set an attribute with `UpdateAttribute` to equal:  select foo1, foo2 from bar_tbl  to use it later downstream in another processor.How can I achieve this? Or, more directly, I have a REST api that takes an url-encoded sql statement.  How can I feed the sql statement to `InvokeHttp`?

We are encountering some issues when starting Nifi 1.16.3 after calling encrypt-config.shFor background, Nifi is deployed and managed using an in-house deployment product, and is used as a component within our application stack. In general everything works fine when using Nifi 1.15.1, however, we are encountering issues when using Nifi 1.16.3.  See the following in nifi-bootstrap.log  2022-11-13 06:32:06,488 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 828732022-11-13 06:32:14,269 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 329082022-11-13 06:34:06,217 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Decryption Failed with Algorithm [PBEWITHMD5AND256BITAES-CBC-OPENSSL]2022-11-13 06:34:06,218 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...2022-11-13 06:34:06,945 INFO [main] org.apac

It seems this question gets asked a lot in this forum, but there aren't any good responses.  Hopefully somebody can help me out here.I am attempting to use the InvokeHTTP processor to POST a query to an REST endpoint.  I currently use the InvokeHTTP processor to access data on this data service using the GET method, and that always works fine.  However, there is a specific service endpoint that only accepts POST requests.  I've verified this service is functional using curl.  But when I attempt to use the NiFi InvokeHTTP processor, I get no response.  Hopefully somebody here knows what's going on.  Below are some screen shots of my processor configuration. If I understand the documentation correctly, I need to specify in the "Attributes to Send" field the flowfile attributes I want to send as parameters.  Since this processor is the first processor in the flow, it does't have any attributes yet, so I add a dynamic attribute named "data" (sho

I have a apache/nifi:latest instance spun inside an Amazon Linux 2 EC2. For reference, see this guide: hereI have a QuerySalesforceObject ver. 1.18.0 that makes use of StandardOauth2AccessTokenProvider.  The oauth2 provider url is configured at https://test.salesforce.com/services/oauth2/tokenI can curl this url from the box and from inside the docker container just fine (I don’t get a timeout).[root@ip-10-229-18-107 \~\]# docker exec -it nifi_container_persistent /bin/shprintenv | grep -i proxyHTTPS_PROXY=http://proxy.MY_DOMAIN.com:3128no_proxy=localhost,127.0.0.1,MY_DOMAIN.com,.amazonaws.comNO_PROXY=localhost,127.0.0.1, MY_DOMAIN.com,.amazonaws.comhttps_proxy=http://proxy.MY_DOMAIN.com:3128http_proxy=http://proxy.MY_DOMAIN.com:3128HTTP_PROXY=http://proxy.MY_DOMAIN.com:3128curl https://test.salesforce.com/services/oauth2/token{"error":"unsupported_grant_type","error_description":"grant type not supported"}#But when I run the task, oauth2 fails with an errorjava.io.UncheckedI

Hi,Currently we are using CDP 7.1.7 with phoenix version 5.0.We were told by platform admins that cloudera is dropping support for phoenix from the next cdp version onwards. Is this true? If so, what are the alternative to phoenix are suggested?What is the strategy on embodying phoenix with CDP package? ThanksSagar

Other API calls work such as to /nifi-api/tenants/user-groups. I do not see an entry for /resources in authorizations.xml nor does there seem to be an entry VIA the gui policy editing. NIFI 19.1...

I want to transfer the HR system of our company to BigData. My archive has 500 GB of data on SAP. I want to install Apache Sqoop, Impala and Hadoop HDFS. How much of a resource do I need? What would my Cloudera license cost be? How can I make 2 nodes

Hi, I am using Cloudera Quickstart VM 5.13.0 to write code using pyspark. Trying to import SparkSession using below command:from pyspark.sql import SparkSession It's throwing an error saying Importerrir: cannot import name SparkSession I need help to fix this. Please suggest me if anything missed

hello  I learn sqoop and mysql fist timei use hdp 2.65 and use puttymy user name is root@localhost I want to sqoop movie file when I input order  sqoop import --connect jdbc:mysql://localhost/movielens --driver com.mysql.jdbc.Driver --table movies -m 1 and the following warning message appears. Warning: /usr/hdp/2.6.5.0-292/accumulo does not exist! Accumulo imports will fail.Please set $ACCUMULO_HOME to the root of your Accumulo installation.23/01/31 19:47:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.5.0-29223/01/31 19:47:12 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.23/01/31 19:47:12 INFO manager.SqlManager: Using default fetchSize of 100023/01/31 19:47:12 INFO tool.CodeGenToo