Solved

Hello,tdWhenever i add hive service into my cluster it always fails to start.following is the stderr and stdout.Can someone help me resolve this issue?Thank you in advance.stderr: Traceback (most recent call last):File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 995, in restartself.status(env)File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_metastore.py", line 87, in statuscheck_process_status(status_params.hive_metastore_pid)File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/check_process_status.py", line 43, in check_process_statusraise ComponentIsNotRunning()ComponentIsNotRunningThe above exception was the cause of the following exception:Traceback (most recent call last):File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_metastore.py", line 201, in <module>HiveMetastore().execute()File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/scr

1、Config values: 2、Actual Values: 3、Appreciate any advice  🙂

Hi Friends,I have mongo query which running perfectly fine from mongo shell.b.test650.aggregate([{$group: {"_id": {X: "$X",Y_DT: "$Y_DT",Z: "$Z"},adj: {$sum: "$adj" },bjc: {$sum: "$bjc" },jbc: {$sum: "$jbc" },mnk: {$sum: "$mnk"}}}]) The same query when in ran from Nifi , RunMangoAggregation  throwing out error, though the mongo aggregation query changed to json type query ...{"$group": {"_id": {"X": "$X","Y_DT": "$Y_DT","Z": "$Z"},"adj": {"$sum": "$adj" },"bjc": {"$sum": "$bjc" },"jbc": {"$sum": "$jbc" },"mnk": {"$sum": "$mnk"}}} Getting following error ..error run mongodb aggregation query.: com.fasterxml.jackson.databind.exc.MismatchedInputException:canot deserialize instace of java.util.ArayList' out of START_OBJECT token   Nifi workflow.. Processor(runMangoAggregation) configuration What is the change I need to do in json query which supposed to executed @ runMongoAggregation  processor?

Hello everybody,the title explains pretty much the annoying issue I'm facing. I have to install the JCE policy files, but when I click on the button there is no actual response from Cloudera Manager. It keeps loading the wizard page but it doesn't go on. I know I should be redirected on the wizard page. I tried to restart the cm-server process, the cms, the cm-agent with no result. Also I can't find anything inside the cm-server and cm-agent logs. I'm using Cloudera Express 6.3.0. I faced this problem another time in the past, but in that case it magically autosolved: I was tired of this annoying issue and I started doing some other work and left the tab open. At a certain point it loaded the wizard page. But this time I can't wait too much since I  have some deadlines to deal with and honestly more than three hours have passed now. Any help would be appreciated, thanks.

 I need a Help i have two question please 1-how I can transform the data from apache kafka to hdfs????2-how I can transform the data from apache kafka to SparkStreming???? thank you

Is the namespace field in an AVRO schema required?  Does the namespace first have to be created if you provide it in the schema?  Not sure how this works. Thanks, johndcal

Hi Community members, We have some maintenance on a host. And want to bring down a host for an hour or so.Got the below document:https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/cm_mc_host_maint.html So, I want to ask, what is the proper way to proceed with it:- Select the host --> Stop all roles on the host and then bringing down cloudera-scm-agent on the host and after maintenance bring those upor as the document mentions:- Select the host -->Decommission Host(s)-->Take DataNode Offline and after maintenance Recommission Host(s).-->Bring hosts online and start all roles Please suggest what is the difference in these two approaches and which one is best in which scenario?  

Dear Cloudera Community, Before installing the trial version of CDP Data Center,we must make sure :"The Cloudera Manager Server host must have uniform SSH access on the same port to all hosts " What does it means and how to do that ? Regards,BSST       

Hello Team,We are using Impala to query data stored as parquet on s3. This has been an awesome feature. Recently Amazon S3 has announced a new feature called S3 Select  which helps in speeding up the column projection when querying data stored on S3. As of now Hive and Presto support  S3 select push down.Is Impala going to support S3 select push downs? Parth

Hello All, We are running Spark jobs via yarn and its failing with the below error, any help / pointer to fix is much appericated. Shell output: main : command provided 1main : run as user is TEST1main : requested yarn user is TEST1Writing to tmp file /data/8/yarn/nm/nmPrivate/application_1587389136999_0013/container_e56_1587389136999_0013_01_000477/container_e56_1587389136999_0013_01_000477.pid.tmpWriting to cgroup task files...Container exited with a non-zero exit code 1org.apache.spark.rpc.RpcTimeoutException: Cannot receive any reply in 120 seconds. This timeout is controlled by spark.rpc.askTimeout at org.apache.spark.rpc.RpcTimeout.org$apache$spark$rpc$RpcTimeout$$createRpcTimeoutException(RpcTimeout.scala:48) at org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:63) at org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59) at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunc

Hi Cloudera Community,   I have just completed installation of CDSW 1.6.1 in my DEV cluster. I just want to know the initial "admin" account credentials.   I have tried below listed username/password combinations so far, but no luck: 1. admin/admin 2. cloudera/cloudera 3.admin/cloudera   Regards, Nanda    

Hi , From Ambari 2.7.5 documentation, one of the prerequisites mentioned is to set HOSTNAME in /etc/sysconfig/network , but as per RHEL 7, /etc/sysconfig/network file is deprecated. What is your recommendation on this? Thanks,Satish.

Dear Cloudera Community , I want to learn about cloudera data warehouse. I try to install to trial version, I have some questions : How to Install the CDP Trial Version ?What is the Hardware Requirement including the requiment of space, cpu dan memory ? What is the Operating System Requirement ?  Regards,  BSST

Hi,I'm trying to update the PostgreSQL from 9.2.24 to 9.6. I tried the followingTook backup of the DB with following command.pg_dumpall > outputfileOnce the backup is done I stopped the service and Rename the pgsql directory under /var/lib/pgsql/Updated the postgres with the following command :yum install postgresql96-server postgresql96.After the installation the new directory got created /var/lib/pgsql/9.6/. I ran the initdb command to create the cluster. Restored the old  pg_hba.conf and postgresql.conf files from the backup directory. Started the postgres service with the following command:/usr/pgsql-9.6/bin/pg_ctl start -D /var/lib/pgsql/9.6/dataOnce the service was up I restored the DB with the following command:/usr/pgsql-9.6/bin/psql -d postgres -f outputfileThe DB restoration done but still while starting the ambari server  I'm getting the error  Ident authentication failed for user "ambari" I have the doubt that is the upgrade properly done I mean the a

Hi All,My services are not starting and it failed after sometimes. CDH : 6.2.0Error:AttributeError: 'NoneType' object has no attribute 'get'[16/Jul/2019 10:31:20 +0000] 6528 MainThread agent ERROR Caught unexpected exception in main loop.Traceback (most recent call last): File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/agent.py", line 752, in __issue_heartbeat heartbeat = self.prepare_heartbeat() File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/agent.py", line 1295, in prepare_heartbeat process.update_supervisor_data(supervisor_data.get(process.name, None)) File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/process.py", line 1282, in update_supervisor_data self.deactivate_complete() File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/process.py", line 974, in deactivate_complete LOG.info("[%s] Unregistered supervisor process %s", self, self.fixed_supervisor_state()) File "/opt/cloudera/cm-agent/lib/python2.7/si

I had question to write hive script to produce report. But location/file_name details were missing for the report.I wrote query to export report on HDFS location. In test result I got Incorrect solution SQL comment.Please let me know what is the expected result for this type of question.Thanks. 

-- I adapted from this example www.cloudera.com/documentation-- impala sideCREATE TABLE flat_struct_array (continent STRING, country STRING, city STRING);INSERT INTO flat_struct_array VALUES ('North America', 'Canada', 'Toronto') , ('North America', 'Canada', 'Vancouver') , ('North America', 'Canada', "St. John\'s") , ('North America', 'Canada', 'Saint John') , ('North America', 'Canada', 'Montreal') , ('North America', 'Canada', 'Halifax') , ('North America', 'Canada', 'Winnipeg') , ('North America', 'Canada', 'Calgary') , ('North America', 'Canada', 'Saskatoon') , ('North America', 'Canada', 'Ottawa') , ('North America', 'Canada', 'Yellowknife') , ('Europe', 'France', 'Paris') , ('Europe', 'France', 'Nice') , ('Europe', 'France', 'Marseilles') , ('Europe', 'France', 'Cannes') , ('Europe', 'Greece', 'Athens') , ('Europe', 'Greece', 'Piraeus') , ('Europe', 'Greece', 'Hania') , ('Europe', 'Greece', 'Heraklion') , ('Europe', 'Greece', 'Rethymnon') , ('Europe', 'Greece', 'Fir

Does Sentry have data in rest and data in motion encryption? or only HDFS and Cloudera Navigator have these features?

I am using CM 5.14.2, and from hive's configuration page, i can see the value for javax.jdo.option.ConnectionUserName, which is used to access the underneath metastore db like mysql, but i can't see the value of javax.jdo.option.ConnectionPassword (which is display as **** in the page). i have forgotten the password used to access hive metastore db. Is there any way to find it back?

Hi there, I added Atlas in my CDP cluster, and find can't connect the web UI. In the application.log, I found the error: Can not find the specified config set: vertex_index.I followed the instruction in the following links, but still couldn't get it to work. Any suggestions?https://issues.apache.org/jira/browse/AMBARI-18368https://community.cloudera.com/t5/Support-Questions/Atlas-UI-not-available-Service-Unavailable/td-p/132435 Here is the application.log:                              Atlas Server (STARTUP) project.name: apache-atlasproject.description: Metadata Management and Data Governance Platform over Hadoopbuild.user: jenkinsbuild.epoch: 1574150696506project.version: 2.0.0.7.0.3.0-79build.version: 2.0.0.7.0.3.0-79vc.revision: cbb309de67ee0215efa0763f03c2f32f40f1d8f3vc.source.url: scm:git:git://git.apache.org/atlas.git/atlas-webapp####################################################

Greetings, After enabling "Kerberos Authentication for HTTP Web-Consoles" for YARN the Resource Manager WebUI and the HistoryServer Web UI become inaccessible with a valid Kerberos ticket (without a ticket the UI correctly gives the "Authentication required" HTTP 401 error message). Navigating to either of the interfaces returns the following error:       HTTP ERROR 500Problem accessing /jobhistory. Reason: Server ErrorCaused by:java.lang.IllegalArgumentException: Empty key at javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:96) at org.apache.hadoop.security.authentication.util.Signer.computeSignature(Signer.java:93) at org.apache.hadoop.security.authentication.util.Signer.sign(Signer.java:59) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:587) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767) at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doF

Hello All,     I've installed CM manager 6.3.0 and agents in my 7 node cluster. While setting up the CDH 6.3.1 installation, i'm stuck at setup database step, where the test connection is running for more than 2 hours, with only info found in Cloudera manager log given below. There is no error or any other information. I can successfully connect to my databases from command line. But CM is unable to verify the connections. Any help will be appreciated.   2019-12-11 21:48:41,016 INFO scm-web-93:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/dbTestConn/testConnection2019-12-11 21:48:41,036 INFO scm-web-93:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/dbTestConn/testConnection, Status:2002019-12-11 21:48:43,071 INFO scm-web-93:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/dbTestConn/checkConnectionResult2019-12-11 21:48:43,081 INFO scm-web-93:com.cloudera.en

I'm using CDH 6.3.2. When trying to create directories in hdfs, I'm having permission denied like this one:[devuser@master01 ~]$ hdfs dfs -mkdir /devmkdir: Permission denied: user=devuser, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-xHow can I add my user on hdfs? and how can my user run jobs as well?

Hi,  New to Cloudera search.   How do I get the Solr.xml from zookeeper using solrctl tool or any other tools Basically want to see the entire configuration for Solr service on the cluster   migrated cluster from 5.14 to 6.2 Cluster Info: CDH 6.2 Lucene 7.4.0        

I had the cloudera manager(cms) , management service (mgmt) and the database(mariadb) running on a single VM. In order to enable HA for cms, I separated the mgmt and database to independent servers. I shutdown scm-agent on the cms server and have it running on the mgmt server. I also deployed the HAproxy (v1.8) for load balancing as advised in the documentation. I migrated the databases from the current cms server to the new DB server. Ref# https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_cm_ha_hosts.html   The DB and NFS mounts for the management service directories are served by a single host. After following exactly as per the doc, management roles applied starts up fine in the UI without errors, however the status is always "Unknown Health". To the top it shows this message: The status of the services are "?" The logs on the management service VM has the below errors: cloudera-scm-agent LOG Errors:   Spoiler (Highlight to read)[30/Mar/2020 16:1