Solved

Hi, I have a hadoop environment that has 2 nifi nodes running. I am not able to connect to the nifi UI since yesterday and i try do restart one of the node without any success....   In in nifi-app.log, file seeing this as only Error level error: 2020-03-17 13:42:37,974 ERROR [main] o.a.n.c.c.node.NodeClusterCoordinator Event Reported for ***-nf02.****.ca:9091 -- Node disconnected from cluster due to org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow.   Could please help?

Hi all,I'm trying to create a function to use in imapla.my function is simply re-using  hive's sha2() function.the creation of the function goes smoothly :create function to_sha2(string,int) returns string location 'user/hive/hive.jar' symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2' ;  but when I try to use it doesn't work launching this warning  : select to_sha2('test',256);   Query State: EXCEPTION Query Status: ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2     I have tried to search for UDFSha2 that doesn't contain the Generic word in the hive's jar but I couldn't find it.the original built-in function in hive :sha2(string/binary, len) - Calculates the SHA-2 family of hash functions (SHA-224, SHA-256, SHA-384, and SHA-512). |the other functions are working normally in impala (for example  I have tried to create UDF MD5 function from hive's jar and it was working ).so my question

Restart Spark2 ClientTask Logstderr: /var/lib/ambari-agent/data/errors-8648.txt 2020-03-20 16:05:33,015 - The 'spark2-client' component did not advertise a version. This may indicate a problem with the component packaging.Traceback (most recent call last):File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_client.py", line 55, in <module>SparkClient().execute()File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in executemethod(env)File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 966, in restartself.install(env)File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_client.py", line 35, in installself.configure(env)File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_client.py", line 41, in configuresetup_spark(env, 'client', upgrade_type=upgrade_type, action = 'config')File "/var/lib/ambari-agent/cache/sta

I would like to know what is the difference between the HDP Sandbox VM and CDH.   I am looking to setup CDH on GCP,can anyone let me know would i still be able to use the services after the 60 days trial phase if not what is the cost like?This is for personal use.   Is it also possible with HDP Sandbox on GCP?Again for own use and not for any business account.  

Hi, Can i install cloudera manger offline?  Thanks

i want to install Ambari 2.7.5, refer to page  https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/access_ambari_paywall.html .   after log in to cloudera and navigate to https://my.cloudera.com/ . I'm can't file a non-technical case within the support portal (https://my.cloudera.com) to obtain credentials. because i haven't Permission.   what should i do.   Thank everyone.

Hi all,    We've recently upgraded our cluster from Cloudera Enterprise 5.13.1 to 5.16.1, and it seems the value of the Impala Query Monitoring Failures Threshold cannot be changed anymore.    By default, the Warning and Critical fields are set to "Never" and "Any" respectively, but when we try to switch them to "Specify" and set a value, we get an error that says "<value> is less than the minimum allowed value 0". We've tried typing in all sorts of values, including zero, and negatives...none are accepted. We know for sure this error didn't exist in the previous version.                 To us, it appears to be some sort of logical error in the value check, but maybe there's some other reason for this error we haven't thought of...   Does anyone have the same issue, or can think of a root cause or fix ?   cbfr

We recently upgraded Kafka to 2.3.0 after that one of our connector is failing with the below error.  org.apache.kafka.connect.errors.ConnectException: org.apache.kafka.connect.errors.SchemaProjectorException: Switch between schema-based and schema-less data is not Any ideas anyone ?I have already looked and implemented this https://issues.redhat.com/browse/DBZ-235 but no luck. Nothing has changed in the schema. We dont think its a tombstone issue. I was able to check where the connector stopped and looked at the events and its not a tombstone. 

Hello community,   I'm writing a Python script, and connect through the 64-bit odbc driver to Hive and Impala. I'm interested in creating an external table using the Hive connection, and then run some faster-than-hive queries using an Impala connection. After table creation, I am able to see and query the external tables in both hive and impala editors in HUE. However, these newly created tables are not visible nor available through the Impala connection.   Tried the    invalidate metadata <table_name>   but it's not working either.   Any pointers would be greatly appreciated.   Luis

Hi All, I have implement almost 300 lines of hive query. The query template is as below. Insert into targetTableSelect col1,col2...........cols....(Almost 300 lines in select query with lots of case when statemnet)From source table My observation is that it works without giving any error in apache zeppeline. But when i trying to execute it through beeline it's failing with  transport exception response code 500. Other observation,when i remove few number of columns from select query in that  also it works fine. I do not understand why query is failing ? I need to know the reason behind failure? Thanks in advance  

Hi All, I have a flow setup where I am fetching some data from an Oracle DB. Some of the columns of the table are in Oracles Number type, and I need to put them into a Postgres DB as floats. If I look at the Schema that is being created on read from Oracle it looks like it's the data that's being defined as a type of "Bytes" with a logical type of "decimal" that I am talking about. Currently I am using the following flow as I need to add some additional update onConflict logic to the Postgres insert to handle updates. Oracle -> Convert JSON -> Split Record -> Convert JSON to SQL -> Replace Text -> Execute SQL But at the moment this throws an error when trying to do the insert into Postgres, which seems to be related to the number not being coerced into an actual number and rather been empty in the SQL. Anyone got any ideas/suggestion on how I can force the type of the numerical types to be a float in the Avro and then a number type once

I am using spark 2.3.2 and i am trying to read tables from database. I established spark connection. But i am unable to read database tables from HUE cloudera and unable to query them in pyspark as well.   Here is my code, import findsparkfindspark.init('C:\spark-2.3.2-bin-hadoop2.7\spark-2.3.2-bin-hadoop2.7')import pysparkfrom pyspark.sql import SparkSessionspark = SparkSession.builder.config("hive.metastore.uris", "thrift://10.1.1.70:8888").enableHiveSupport().getOrCreate()#spark.catalog.listTables("tp_policy_operation")import pandas as pdsc = spark.sparkContextsc from pyspark import SparkContextfrom pyspark.sql import SQLContextsql_sc = SQLContext(sc)SparkContext.setSystemProperty("hive.metastore.uris", "thrift://10.1.1.70:8888")spark.sql("SELECT * FROM tp_policy_operation")   ######The error i am getting Traceback (most recent call last): File "<ipython-input-4-8f0aa5852b01>", line 16, in <module>spark.sql("SELECT * FROM tp_policy_operation")## Database ? File

Hi Team, We are unable to start Hbase.   We are using Hadoop 3.0.1 on Isilon One FS Error Message :   Creating: Resource [source=null, target=/apps/hbase/data, type=directory, action=create, owner=hbase-xxxxx, group=null, mode=null, recursiveChown=false, recursiveChmod=false, changePermissionforParents=false, manageIfExists=true] in default filesystemException occurred, Reason: Unexpected error: status: STATUS_MEDIA_WRITE_PROTECTED = 0xC00000A2 with path="/apps/hbase/data", username=hbase-xxxxx, groupname=org.apache.hadoop.ipc.RemoteException(java.io.IOException): Unexpected error: status: STATUS_MEDIA_WRITE_PROTECTED = 0xC00000A2 with path="/apps/hbase/data", username=hbase-xxxxx, groupname= at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1497)

Hi,Do we need to stop or put all hadoop services in maintenance, during our DB patching activity or just putting Cloudera Manager in maintenance mode will suffice? ThanksAnm

Hello guys,I am extracting attributes from the flow file and putting it to the PostgreSQL.2017-04-27 09:15:04.794761 10.69.1.138 -> 10.69.1.100 <4>Apr 27 03:44:58 kernel: [ 542.579606] TS|242268|00:14:5a:03:29:72|RSSI_BCN|56 2017-04-27 09:47:59.351047 10.69.1.138 -> 10.69.1.100 <4>Apr 27 04:17:43 kernel: [ 2517.159922] TS|2216852|00:14:5a:03:23:01|RSSI_BCN|53 I am trying to extract integer value 56,53... end of the line and put it to the column<col> of PostgreSQL table<tab> where the <col>datatype is integer.  I used GetFile->SplitText->ExtractText ->putSQL in my process flow. I used (\d*)\z regular expression in ExtractText processor and its seems working well for extracting the value 56.  But after a certain time the putSQL processor shows error ERROR: invalid input syntx for integer:"". If I change the datatype from integer -> text in the <col> of my PostgreSQL table then I don't face this problem but I need to store

I have installed Hue 3.8.1 from tarball and it's running on centos 7. I am not sure how to start hue.    Below is the directory structure.   [root@ds hue-3.8.1]# ls -ltotal 56drwxrwxr-x. 21 sitadmin sitadmin 4096 May 4 2015 appsdrwxr-xr-x. 3 root root 17 Mar 10 23:14 builddrwxrwxr-x. 5 sitadmin sitadmin 58 May 4 2015 desktopdrwxrwxr-x. 6 sitadmin sitadmin 142 May 4 2015 docsdrwxrwxr-x. 3 sitadmin sitadmin 24 May 4 2015 ext-rw-rw-r--. 1 sitadmin sitadmin 11358 May 4 2015 LICENSE.txt-rw-rw-r--. 1 sitadmin sitadmin 4715 May 4 2015 Makefile-rw-rw-r--. 1 sitadmin sitadmin 8505 May 4 2015 Makefile.sdk-rw-rw-r--. 1 sitadmin sitadmin 3498 May 4 2015 Makefile.vars-rw-rw-r--. 1 sitadmin sitadmin 2192 May 4 2015 Makefile.vars.privdrwxrwxr-x. 2 sitadmin sitadmin 21 May 4 2015 maven-rw-rw-r--. 1 sitadmin sitadmin 801 May 4 2015 NOTICE.txt-rw-rw-r--. 1 sitadmin sitadmin 1562 May 4 2015 READMEdrwxrwxr-x. 5 sitadmin sita

How to solved the recurring Heap size issue? we just add size on the heap size but do we have any other solution? is it manageable now in 6.3 version of CDH?

Hi May I know please if what are the versions of the components like Spark, Hbase etc under the new 6.3 cloudera enterprise?

Used case :  Need to transfer entire folder of files based on date to S3. Ex :  Source Path (Linux) : /users/abc/20200312/gtry/gyyy.csv             S3    : /Users/datastore/20200312/gyyy.csv   Since dates keep changing everyday, and i need to build dataflow which would pick files from date folder

Hi,   I'm running Cloudera 5.16.1 on CentOS 7 and OpenJDK8 I enabled TLS/SSL on the Cloudera Manager (level 1 - level3) with Self-Signed certificate and then moved to enable Kerberos with AD. I followed the docs but enabling Kerberos is failing when trying to import credentials.   /usr/share/cmf/bin/import_credentials.sh failed with exit code 1 and output of <<+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin+ KEYTAB_OUT=/var/run/cloudera-scm-server/cmf1018223695564634823.keytab+ USER=cdhadmin@<XXXXX>.COM+ PASSWD=REDACTED+ KVNO=1+ SLEEP=0+ RHEL_FILE=/etc/redhat-release+ '[' -f /etc/redhat-release ']'+ set +e+ grep Tikanga /etc/redhat-release+ '[' 1 -eq 0 ']'+ '[' 0 -eq 0 ']'+ grep 'CentOS release 5' /etc/redhat-release+ '[' 1 -eq 0 ']'+ '[' 0 -eq 0 ']'+ gr

Hi, I am trying to obtain data ingestion lineage of Nifi via Cloudera Navigator. I have not come across any solution or documentation for the same. Has anyone experimented with this successfully?  Is there an existing solution for this?

Hi, I tried using the Data Warehouse service in my CDP account. What i came across is that Data Warehouse in the backend creates a EKS cluster in your AWS environment. But i was unable to access this EKS cluster using kubectl. Is this use case blocked or I am missing something. I tried accessing this EKS cluster with admin credentials of my AWS account but was still unable to access it.  Can anyone help me with this.

Small and empty files are recurring on our current version of CDH Cluster. Does is still exist on 6.3.3 version?

I have NAS mounted on Linux server and have to copy files from location to S3, i am using ListFile→FetchFile-→PutS3.  While reading files from NAS mount location, i am getting error saying file not found but i see all required files are present at location. Is there any special configuration which needs to be set while reading files from NAS drive.

Good night people Is there an easier way to install the hadoop ecosystem using the cloudera solution? , I am trying to install version 6.3 for a proof of concept, but I am having difficulty, I remember that before I had a VM that made installation a lot easier, but no longer, could someone help me?