Q&A

Hi, I have just downloaded the quickstart Cloudera VM and when I try to start all services it fails telling me that Service mgmt has 0 Reports Managers. Cloudera Management Service requires at least 1 Reports Manager. I tried to configure Reports Manager by adding a Role instance for Reports Manager, however it requires me to give a databasename and a tablename. My question is what columns or table structure should I have for reports manager? Thanks!

Hello,   I am new to HDP and going thought the tutorials, I got stuck when I tried to launch the DAS UI, I get an error message "cannot connect to server at sandbox-hdp.hortonworks.com, FYI I use Ubuntu"   Why am i getting this error?   Is this OS issue?

Does Phoenix query server support https protocol? My Hadoop system is kerberized and ssl enabled, HTTPS encryption for wire encryption. I did not find any document for PQS request over HTTPS. If it is supported then please point me to the correct configurations parameters.

Hi Experts,we setup Cloudera cluster on  to verify the performanceCloudera was installed with Auto-TLS enabled and we are facing issues to connect to hive. [root@slave1 agent-cert]#  beeline -u jdbc:hive2://slave1:10000;ssl=true;sslTrustStore=/var/run/cloudera-scm-agent/process/72-hive-HIVESERVER2/cm-auto-host_keystore.jks;trustStorePassword=yeap4IhJzRvK5gBGVMeTahoL21BNmBF2TSi46pbQTP6WARNING: Use "yarn jar" to launch YARN applications.SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]Connecting to jdb

How I can want development scheduling algorithm and implement in yarn  Please any help  

I have newly imported HDP 2.6.5 Sandbox after removing the 2.6.4,  and tried to connect via ssh. My host machine Windows 10. My command:   ssh root@sandbox-hdp.hortonworks.com -p 2222   The error output:   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!Someone could be eavesdropping on you right now (man-in-the-middle attack)!It is also possible that a host key has just been changed.The fingerprint for the RSA key sent by the remote host isSHA256:8F0VOvcJIA2FD3Ld90kv5FiD3270onl/+uDeDBNffQ0.Please contact your system administrator.Add correct host key in /c/Users/user/.ssh/known_hosts to get rid of this message.Offending RSA key in /c/Users/user/.ssh/known_hosts:3RSA host key for [sandbox-hdp.hortonworks.com]:2222 has changed and you have requested strict checking.Host key verification failed.

I am trying to install ambari on ubuntu 16.04 LTS on server with zero internet access. I am following the steps mentioned in: https://docs.cloudera.com/HDPDocuments/Ambari-2.7.4.0/bk_ambari-installation/content/ I hosted a local repository using apache http server. I configured to use local repository in /etc/apt/sources.list.d/ambari.list. The file contains #VERSION_NUMBER=2.7.4.0-118#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.jsondeb [trusted=yes] http://my.repository/ambari/ubuntu16/2.7.4.0-118/ Ambari main   when I run sudo apt-get update it is not fetching the binaries. I am attaching log of the process Get:1 http://my.repository/ambari/ubuntu16/2.7.4.0-118 Ambari InRelease [7379 B]Ign:1 http://my.repository/ambari/ubuntu16/2.7.4.0-118 Ambari InRelease Get:2 http://my.repository/ambari/ubuntu16/2.7.4.0-118 Ambari/main amd64 Packages [1513 B] 0% [Connecting to archive.ubuntu.com (91.189.88.31)] [Connecting

Hello,I have a scenario with a Hadoop cluster installed with HDP2.6.5 and a Kafka cluster installed with HDF 3.3.0 with Ranger Service configured.I want to store the Ranger Audit logs in HDFS so I setup in kafka the property xasecure.audit.destination.hdfs.dir pointing to the HDFS directory.Case one: when using the namenode in the URI the logs are stored in HDFS successfully (xasecure.audit.destination.hdfs.dir=hdfs://namenode_FQDN>:8020/ranger/audit)Case two: Using a haproxy, since i have namenode HA enabled and want to point always to the active NN, i get the following error2019-04-02 12:00:13,841 ERROR [kafka.async.summary.multi_dest.batch_kafka.async.summary.multi_dest.batch.hdfs_destWriter] org.apache.ranger.audit.provider.BaseAuditHandler (BaseAuditHandler.java:329) - Error writing to log file.java.io.IOException: DestHost:destPort <ha_proxy_hostname>:8085 , LocalHost:localPort <kafka_broker_hostname>/10.212.164.50:0. Failed on local exception: java.io.IOException:

Hello Friends   You could help me to secure my Kafka servers for both Broker and client. As additional data I am using a KDC. As I see I currently have the security.inter.broker.protocol SASL_PLAINTEXT I don't know if it is correct or how I should secure this service.   Thank you

Hi ,Myself Ravikiran Sharma, I have purchased HDP Certified Spark Developer (HDPCSD2019) Exam on Sunday 22/oct 9:05 pm Indian time. But I did not get any confirmation mail from Hortonworks.I contacted cloudera(Natalie), she told my record is not found in the system. My account got deducted with 250$. its more than one week issue is not fixed. I want the issue to fixed as soon as possible.MY TICKET NUMBER is : 20203Please help.Thanks and RegardsRavikiran Sharma

Is the next-gen cloud-native Cloudera Data Platform available now? Which cloud providers? Is it available for on-prem kubernetes deployments (such as IBM ICP and/or RedHat Openshift)?   Thanks,   Julius

Where do I get a repo or tar file of Cloudera for on premise installation / Upgrade?   Also is CDP sandbox available?

I have a complex flow where I created 3 attributes, [name, date, json_content] extracted from other flow file data that need to go into a database. How can I take these 3 attributes convert them to a new flow file with those columns? The schema I will use has those names.  Schema:{"type": "record","name": "mytable","fields": [{"name": "name","type": [ "string" ]},{"name": "date","type": "type" : [ "null", { "type" : "long", "logicalType" : "timestamp-millis" } ], "default" : null }},{"name": "json_content","type": [ "string" ] }]}

Hello, I saw couple of blogs talks about using Superset with Kudu throw Impala.http://www.bigdatareflections.net/blog/?p=52http://www.bigdatareflections.net/blog/?cat=5 Beside the fact that the blog is from 2017, do I need to install Impala to get it work, or there is another way to plug Kudu with Superset like in the case of Druid (Superset has a native connector to Druid and a majority of Druid’s features are accessible through Superset)? RegardsAnis

Hi, I am looking for the way to convert the CSV to JSON I have a csv file with json object as value,   "name","item""david","{""currency"":""USD"",""content"":""Beauty Lips""}"     I used ConvertRecord processor with CSVReader and JsonRecordSetWriter And the output JSON is   { "name":"david", "item":"{\"currency\":\"USD\",\"content\":\"Beauty Lips\"}"}   But what I expected is   { "name":"david", "item":{"currency":"USD","name":"Beauty Lips"}}     How can I archive this? Thank you   David

Hi! I am trying to restart my Ambari Metrics Monitor on an HDP 3.0 cluster, but the component start is failing with an execution error 255.

Need to know how to differentiate headless and service keytab.. any specific commands

 if there is postgresql install then also can i install MySQL 2. If i want to add Service Nifi & Spark in existing ambari server then which i have to go first.

Hi, We are using CDH 5.14. Webhdfs is enabled by default on cloudera manger and i am able to list HDFS data through webhdfs protocol.   [root@hadoop-slave-1 ~]# curl -i -L "http://172.16.0.10:50070/webhdfs/v1/?op=LISTSTATUS"HTTP/1.1 200 OKCache-Control: no-cacheExpires: Fri, 27 Sep 2019 13:52:56 GMTDate: Fri, 27 Sep 2019 13:52:56 GMTPragma: no-cacheExpires: Fri, 27 Sep 2019 13:52:56 GMTDate: Fri, 27 Sep 2019 13:52:56 GMTPragma: no-cacheContent-Type: application/jsonX-FRAME-OPTIONS: SAMEORIGINTransfer-Encoding: chunked {"FileStatuses":{"FileStatus":[{"accessTime":0,"blockSize":0,"childrenNum":2,"fileId":33381,"group":"supergroup","length":0,"modificationTime":1569326622576,"owner":"hdfs","pathSuffix":"cli","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":10,"fileId":17365,"group":"supergroup","length":0,"modificationTime":1569589022824,"owner":"hdfs","pathSuffix":"hadoop","permission":"755","replication":0,"storage

Hi Team,   I am using cloudera manager.In this i am trying to give an access for particular services to the new users. Like i am a new user and i needed only spark and pig so once i added in cloudera as a user so i can read and write for these two services only not all the services.   For that i have tried to sentry option but it is not full fill my requirement. Can you give me or provide me any solution for this.   Thanks, Vikas,.

Couldn't connect to the cluster { "status" : 404, "message" : "The requested resource doesn't exist: Cluster not found, clusterName=

Where would someone increase the database connection pool for Director? Would this be in application.properties or in /etc/default/cloudera-director-server?   I've noticed after tuning the amount of threads for tasks and the amount of instances in parallel in the application.properties that if I try to provision 4 clusters of 100 nodes each at around the same time or with a small delay that Director has issues with not being able to get database connections causing failures.   To pre-empt this I have migrated to a serverless aurora with a minimum of 8 units which from the cloudwatch charts shows no spikes in connections yet Director still fails pipeline errors with the following:     java.util.concurrent.ExecutionException: org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection   I guess what I'm try

Recently upgraded Director from 6.1 to 6.2.1 and noticed a lot more spam in the logs with the following trace:   [2019-07-22 17:17:27.632 +0000] ERROR [task-thread-10] - - - - - com.cloudera.api.ext.ClouderaManagerClientProxy: API call to Cloudera Manager failed, gathering informationjavax.ws.rs.NotFoundException: HTTP 404 Not Found at sun.reflect.GeneratedConstructorAccessor246.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.cxf.jaxrs.client.AbstractClient.convertToWebApplicationException(AbstractClient.java:544) at org.apache.cxf.jaxrs.client.ClientProxyImpl.checkResponse(ClientProxyImpl.java:326) at org.apache.cxf.jaxrs.client.ClientProxyImpl.handleResponse(ClientProxyImpl.java:891) at org.apache.cxf.jaxrs.client.ClientProxyImpl.doChainedInvocation(ClientProxyImpl.java:797) at org.apache.cxf.jaxrs.client.ClientProxy

Hey, folks, I have an error when executing kafka when executing a message for the topic. kafka-console-producer.sh --broker-list w01.s03.hortonweb.com:6667 -topic TestNYC error. WARN [Producer clientId=console-producer] Bootstrap broker w01.s03.hortonweb.com:6667 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient) I have 3 Brokers, which are working and is configured according to the parameters. just a topic that I just realized. I have 2 network cards one internal and external to netstat I see that port 6667 is listening to the internal. but still executing the command with the internal IP kafka-console-producer.sh --broker-list 192.168.0.9:6667 -topic TestNYC I still get the same mistake. Someone could help me. Greetings  

We have a Cluster running CDH 5.15.1 on 150+ nodes. Cluster nodes storage capacity varies (Mix of various node sizes, 10 TB, 13 TB, 25 TB, 48 TB).  Nodes are connected with a 10G network and presently running with a physical topology of Rack awareness.  Issue is, while few racks have 18 servers others are just having 1 or 2 servers on them. we have been advised to move away from this nonuniform rack awareness topology to a Logical Rack awareness topology which will help us put together UNIFORM no. of nodes on each rack.   I would like to understand, if the rack awareness topology is changed, would HDFS kick-off immediate Block movement or the new topology will only be effective for NEW blocks written on the cluster.