Solved

I have installed cdsw trail version before purchasing the product but stuck with errors. 500 gig block device  and using default /var/lib/ for application.  CDSW disks provisioned via ESX/VMware.  CDSW starts fine in CM but failed to launch containers. Here is the status.    Failed events are below for web-5dddb59597-pclf2  (redacted host name for security reasons). Not sure why tcp-ingress-controller-7d65df6d8f-qqxzn  is starting , i am not using TLS for CDSW Any help is highly appreciated.    NAME READY STATUS RESTARTS AGEcdsw-compute-pod-evaluator-566f7664df-rqm2j 1/1 Running 0 9m38scron-6dd99c7b78-kh4rw 1/1 Running 0 9m40sdb-86bbb69b54-nxz4z 1/1 Running 0 9m40sdb-migrate-46715e4-nf5dh 0/1 Completed 0 9m40sds-cdh-client-578fc69b9-hcqv7 1/1 Running 0 9m38sds-operator-66ff9c4b78-r6bk7 1/1 Running 0 9m38sds-reconciler-64bccdd574-vhhpz 0/1 CrashLoopBackOff 4 9m38sds-vfs-648f9bddf-7n6hg 1/1 Running 0 9m38simage-puller-xwbcs 1/1 Running 0

I just started using cm_client python for swagger v32. I wonder if I missed something because I found some simple task seemed quite hard to accomplish as a new API user.   For example, I am trying to use the API to set a Yarn configuration mapreduce.job.split.metainfo.maxsize. In CM GUI I can go to YARN -   Configuration - search for such a setting and then update it, quite simple. However, in order to do it in API, I was not sure which "update_config" I should use because there are many of them. I tried "serviceResourceAPi.update_service_config( )" for yarn service but the specific setting was not there. then I thought I should try update_role_config or update_role_config_group. Then I realized I had to inpect the many "read_config" output manually to find out which API may be relavent to the parameter. I could not do it programly because I was not sure  how the  setting/parameter would be  called in API - maybe  mapreduce.job.split.metainfo.maxsize

How to resolve this issue. please tell me step by step process   File /var/log/ambari-server/ambari-server.log: Spoiler (Highlight to read) 2019-09-15 05:21:14,273 WARN [C3P0PooledConnectionPoolManager[identityToken->1hgeumca5eo2un51maxtev|d554c5f]-HelperThread-#0] BasicResourcePool:220 - Having failed to acquire a resource, com.mchange.v2.resourcepool.BasicResourcePool@a21c74 is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.2019-09-15 05:21:14,298 ERROR [main] AmbariServer:1114 - Failed to run the Ambari Serverjavax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DatabaseExceptionInternal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!Error Code: 0at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:815)at or

I tried logging out and then logging back in several times but could not find the button to ask a question. Does this happen to anyone else?

Hi, folks, You can tell me if it's possible to use Ambari with Two Active Directories.   For example   dc01.test.comdc02.test.com   With the Ambari server, there is some way to use this option in High availability.   Greetings

Is there a way to store intermediate query results in Beeline - Hivesomething like below in pseudo code. within a beeline sessionbeeline -u  -e" set var1 = select max(date) from tableA; set var2 = select count(emps) from tableB;set var3 = select min(date) from tableC;Insert into table TableD select var1, var2, var2" --...

Below article shows how to restart clients on Specific host. https://cwiki.apache.org/confluence/display/AMBARI/Restarting+host+components+via+the+APIIs there any way we can perform the same on all the hosts in the cluster at one time.Thank you in advance.

I am running spark 2.4 on Azure HDI version 4.0, which includes HDP 3.0.After the upgrade, we are unable to use spark-hbase connector version 1.1.1-2.1-s_2.11, it fails with below errorjava.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/JsonInput;Z)Lorg/json4s/JsonAST$JValue; at org.apache.spark.sql.execution.datasources.hbase.HBaseTableCatalog$.apply(HBaseTableCatalog.scala:257) at org.apache.spark.sql.execution.datasources.hbase.HBaseRelation.<init>(HBaseRelation.scala:80) at org.apache.spark.sql.execution.datasources.hbase.DefaultSource.createRelation(HBaseRelation.scala:51) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318) at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167) at withCatalog(<console>:42) ... 56 elidedAs per

The non-interactive setup of Ambari seems to be broken. It looks like the /etc/ambari-server/conf/ambari.properties file is not updated when using the ambari-server setup CLI flags (non-interactive setup).   Has anyone ever had the same issue?   Environment CentOS Linux release 7.6.1810 (Core) http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0 repo for ambari-server Steps to reproduce sudo yum install -y postgresql-jdbc ambari-serversudo ambari-server setup --help Using python /usr/bin/pythonSetup ambari-serverUsage: ambari-server.py action [options]Options:-h, --help show this help message and exit-v, --verbose Print verbose status messages-s, --silent Silently accepts default prompt values. For db-cleanupcommand, silent mode will stop ambari server.Database options (command need to include all options):--database=DBMS Database to useembedded|oracle|mysql|mssql|postgres|sqlanywhere--databasehost=DATABASE_HOSTHostname of database server--

Hello Team, All of sudden service monitor was went down and when i checked the logs i can see connection refused error in logs.I have tried to restart all the CMS services and this time all went down.What i was did is i have deleted CMS and trying to re-add the CMS to cloudera manager,This time i am not able to add the CMS and when i am trying to adding it active monitor and reports manager cant able to find databases and DB's in test connection stage.Can some one please respond ASAP and it would be greatfull. Thanks,Vinod  

We have a SolrCloud service in Kerberos security enabled CDH 6 cluster. There a lot of collection in SolrCloud and Metric monitoring configuration is enabled, but there are NO DATA in all charts and collection name is not displaying under Collections dropdown in Collection Statistics tab.   How do we fix this situation?    

Hi All,   I've been using CM API V32 to update various service configuration and I noticed it works as long as we set service level attributes. However, while trying to update attributes at roleType level, the API responds with error code 400, stating "'Role type configuration is supported only up to API v2' in CM API V32.   Interestingly, the model schema provided by Cloudera recommends using the same attribute for the service config update.   I am using PUT `/clusters/{clusterName}/services/{serviceName}/config` API.   Anyone else facing a similar issue or knows how to deal with this issue?   Thanks.

Followed Hive metadata importing to atlas from the following link: https://atlas.apache.org/Hook-Hive.html Eventhough hive-site.xml contains  <property><name>datanucleus.schema.autoCreateTables</name><value>true</value></property> .    and I ran ./schematool -initSchema -dbType derby --verbose .  to initialize the db schemas, but still during importing the hive-dbs in atlas (./import-hive.sh) , I am getting this following error. But hive-db creation , deletion on its own works good, no issues. More so schematool info gives:Booting Derby (version The Apache Software Foundation - Apache Derby - 10.14.1.0 - (1808820)) instance a816c00e-016d-1feb-fbec-00000aae9110on database directory /opt/hive/bin/metastore_sql in READ ONLY mode with class loader sun.misc.Launcher$AppClassLoader@574caa3f.Loaded from file:/opt/hive/lib/derby-10.14.1.0.jar.java.vendor=Oracle Corporationjava.runtime.version=1.8.0_222-b

I am trying to install agent using Cloudera Express edition on different host. But getting following error   [12/Sep/2019 09:53:43 +0000] 20745 MainThread agent        ERROR    Heartbeating to omhs61p383.nbo.com:7182 failed.Traceback (most recent call last):  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/agent.py", line 1387, in _send_heartbeat    self.cfg.max_cert_depth)  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 139, in __init__    self.conn.connect()  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/httpslib.py", line 69, in connect    sock.connect((self.host, self.port))  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 309, in connect    ret = self.connect_ssl()  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL

Hi, I'm trying to get all entries from a redis cache. When i try to remove, getbykey etc that work but when i call myCache.keySet, i face an issue with following exception: UnsupportedOperationEception on null. I check it in git source repo and keySet method is implemented. May be there is a mistake in my deserializer ? I would like to know the point of view of Nifi community . I'm currently stuck as i need to list all entries that are in redis in order to split them in other processor for a specific action in Nifi context. BElow some piece of code : - The deserializer   public static class StringDeserializer implements Deserializer<String>{ @Override public String deserialize(byte[] input) throws DeserializationException, IOException { if (input == null || input.length == 0) { return null; } return input.toString(); } }    and piece of code of my custom processor code:   final DistributedMapCacheClient cache = cont

Hello Everyone,We have a Nifi production cluster made of 4 nodes from HDF 3.1.2.0, using our HDP Zookeeper. We had a major breakdown and had to restart all cluster and since the restart one of the node is unable to join the cluster again. Here is the log I have on it :2019-08-06 17:56:32,385 ERROR [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl Background retry gave uporg.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss at org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:838) at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.ut

NiFi processor updateAttributes runs too many tasks, but I just have only one flowfile transfers to it. I wonder why so many tasks are running.

Hello,   I've installed the sandbox-hdp docker-container on my ubuntu-server using Kitematic.The deploy process was quick and easy. The ports 22, 4200 and 8080 are forwarded to localhost on different ports (32785-32783).When trying to access these with my browser the page can't be reached.   Any tips? Do I need to use some other network adapter? Are other ports needed?   regards muffex

Using Cloudera Altus Director 6.3 to deploy CDH 6.3 to AWS. I submitted a similar post back in April, thinking that it had since been resolved - but apparently not.  We are ramping up CDH6 deployments again since our initial testing in April and again are seeing Hue fail to start during first run of CDH6 deployments due to Python lib folders being too restrictive.  Interestingly we do not see this issue when using the same CentOS 7.6 based AMI to deploy CDH 5.16.2 clusters.  The Python lib permissions differ on between the CDH5 and CDH6 deployments: CDH5:ls -l /usr/lib/python2.7/site-packages/six*-rw-r--r--. 1 root root 29664 Jan 2 2015 /usr/lib/python2.7/site-packages/six.py-rw-r--r--. 1 root root 29708 Nov 20 2015 /usr/lib/python2.7/site-packages/six.pyc-rw-r--r--. 1 root root 29708 Nov 20 2015 /usr/lib/python2.7/site-packages/six.pyo/usr/lib/python2.7/site-packages/six-1.9.0-py2.7.egg-info:total 16-rw-r--r--. 1 root root 1 Nov 20 2015 dependency_links.txt-rw

Can I access HDP Atlas without Ranger?I am new to HDP and so far I can use Hive, HBase, and Sqoop without Ranger installed. Now I am trying Atlas and following this link.https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.0/configuring-atlas/content/configure_atlas_security.htmlIt looks like I have to use Ranger foraccessing Atlas, and communication between Ranger and other services like Hive and HBase.Is this correct?Thanks.

while running a normal spark job SparkContext is not getting initialized giving below error: [Thread-6] ERROR org.apache.spark.SparkContext - Error initializing SparkContext.org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. I believe it is pointing to standby NN.This error is coming after we enabled Namenode HA in our cloudera's cluster.I have checked hive-site.xml, core-site.xml and spark-defaults.xml.These all are havinf correct configs pointing to nameservice id. 

I have the following configuration in hive-site.xml <configuration><property><!-- this should eventually be deprecated since the metastore should supply this --><name>hive.metastore.warehouse.dir</name><value>${WAREHOUSE_DIR}</value><description></description></property><property><name>hive.exec.scratchdir</name><value>${TMP_DIR}/scratchdir</value><description>Scratch space for Hive jobs</description></property><property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://localhost:3306/metastore_db?createDatabaseIfNotExist=true</value></property><property><name>javax.jdo.option.ConnectionDriver</name><value>com.mysql.jdbc.Driver</value></property><property><name>javax.jdo.option.ConnectionUserName</name><value>root</value></property><property><name

I have an event  {"a":"avc123.abc.com","b":"24 - ny","c" : "21321kodi"}i would like to use extracttext to filter/extract only the "b" however, it is picking up the first match. How can I specify to extract text from field "b" only not "a" neither "c"

how can I extracttext specific key and value from json data ?{"key1" : "value1","key2" : "value2","key3" : "value3",}I would like to apply the extraxttext on Key2 and its value only. 

Does anyone know why the "database/table filter" and the "search saved documents" fields automatically populate with my username every time the page is reloaded? It takes me 3 button clicks to get rid of it every time just to be able to browse databases and tables, making my work incredibly frustrating.