Q&A

I use chmod to give others the read permission on the home/maria_dev directory. However, the user hive still cannot read the directory.[maria_dev@sandbox-hdp ~]$ ls -lddrwx---r-- 1 maria_dev maria_dev 4096 Mar  5 05:35 .[maria_dev@sandbox-hdp ~]$ sudo su hive[hive@sandbox-hdp maria_dev]$ lsls: cannot open directory .: Permission denied

Hi,Can you provide the procedure for backing up ranger kms? I've backed up the ranger and ranger audit databases using the postgres dump command (pg_dump), but I'm running into an issue when I try the same procedure for the rangerkms database. I tried the following command and got the error shown below. To access the ranger kms I have to login using the keyadmin user. I was thinking that this is my issue, that I'm trying to create a backup of this database using the wrong user? Any thoughts are suggestions are appreciated as I haven't been able to find any support articles on this. Thanks in advance...pg_dump -U rangeradmin rangerkms > rangerkms.sqlPassword:pg_dump: [archiver (db)] query failed: ERROR: permission denied for relation ranger_masterkeypg_dump: [archiver (db)] query was: LOCK TABLE public.ranger_masterkey IN ACCESS SHARE MODE

In my cluster Event server is faild to start.When I restart it it shows the below error message : ---------Process Status:This role's process failed to start.Cloudera Manager Descriptor AgeNot enough data to test: Test of whether the Cloudera Manager descriptor is up to dateUnknown : Not enough data to test: Test of whether the Cloudera Manager descriptor is up to date.--------------- Someone sugesated to increase Java heap size of Event server 1 GB. I can see its already 1 GB. Can you guys help me why its failed to start , what other things I can check?? 

Hi!How can I configure InvokeHTTP to return comma separated flow file? At the moment it is returning file in below format (tab separated):RELM Satieva_microloans 6 3369 2019-06-12 11:42:00 772103971RELM Satieva_microloans 6 3369 2019-06-12 11:42:00 772104117RELM Satieva_microloans 6 3369 2019-06-12 11:42:00 772101000Thanks in advance!

Hi,I wanted to know whether it is possible to use HDP and HDF simultaneously on same virtual machine. Actually wanted to use HDF Nifi/Kafka (producer) services to fetch data from source and wanted to trigger Kafka(consumer) at HDP.Facing issue when both HDF/HDP are running not able to use localhost:1080

Hi,Need help here:When I am doing "ip addr show" in HDP 3.0 shell in box; it is showing only eth0 and lo. Though in NETWORK options I have two network adapters 1) NAT2) NETNETWORKIn my VirtualBox NET NETWORK setting I have given NETNETWORK 11.0.2.0/24. My concern is when I am doing ip addr show my NETNETWORK ip 11.0.2.# should be displayed, which is not displaying currently.Do I need to have some setting changes in HDP 3.0. Or HDP 3.0 doesn't sense network adapter other than NAT?

Hello,what is the purpose of Spark2_Jobhistoryserver_Process? . Under what circumstances we get alert for 'Spark2_Jobhistoryserver_Process' apart from stopping the service.

Hello, where I can find updated information upon how to install a small cluster in Azure? Preferably a video.I understand that using cloudbreak looks like the most promising way. After I have my cluster, is it possible to store the parameters and be able to launch the cluster on Azure at will, without having to spend a lot of time configuring the same things?Or even better do you know if there is a way to launch already preconfigured clusters from a blueprint? Say for example that I have a script which automatically provides 5 or 6 VMs on Azure and then launce HDP automatically?Thank you

0: jdbc:hive2://localhost:21050/defalt> select 1 * (NULL);null+-------------+--+| 1 * (null) |+-------------+--++-------------+--+ vs. 0: jdbc:hive2://localhost:21050/defalt> select 1 * (CASE WHEN '1' IS NULL THEN NULL ELSE NULL END);Error: AnalysisException: Arithmetic operation requires numeric operands: 1 * (CASE WHEN '1' IS NULL THEN NULL ELSE NULL END) (state=HY000,code=0) Is this a bug in Impala? Impala version is 2.11 (CDH 5.14.4)   Same statement works with Hive.

Hello, im currently an intern and was given a task to automate a hadoop cluster. It should auto start the component once its down and alert the user (email or similar) that the node was autostarted. How would i go about doing this? Thanks in advance!

Hello everyone,   I’m newer to Hive so I’m assuming that I’m doing something wrong. I’ve configured Hive to require LDAP BasedDN authentication (not AD) and configured TLS for HiveServer2.   With beeline I’m able to authenticate with an interactive user prompt. For example: beeline> !connect jdbc:hive2://<serverName>:10000/default;ssl=true;sslTrustStore=<trustStorePath>.jks;trustStorePassword=<trustStorePassword> Login with username and password: Enter username for jdbc:hive2://…Enter password for jdbc:hive2://… What I would like to do is connect with username/password in the connection string. I see example to configure the connection string with TLS or username/password, but haven’t been able find an example of both. I’ve attempted combining settings and I’m doing something wrong.   This is an example of what I've attempted: beeline -u jdbc:hive2://<serverName>:10000/default;ssl=true;sslTrustStore=<truststorePath>.jks;trustStorePassword

HI All,I have deployed zeppelin in my Ubuntu instance (My private IP: 20.20.20.213) using the tgz file. I have changed the zeppelin-en.sh.template to zeppelin-en.sh and zeppelin-site.xml.template to zeppelin-site.xml. Also, I have configured my port in the site-xml file as 8090 and the server addrs as 0.0.0.0 default only. On checking the log file i get the following code:I even tried to change the server addrs to 20.20.30.213 but still the same error I am getting. Please help!ubuntu@ip-20-20-30-213:~/zeppelin/logs$ cat zeppelin-root-ip-20-20-30-213.log INFO [2019-06-12 17:24:41,675] ({main} ZeppelinConfiguration.java[create]:121)                                                                              - Load configuration from file:/home/ubuntu/zeppelin/conf/zeppelin-site.xml INF

I run a sqoop import command  sqoop import --connect jdbc:oracle:thin:.... --username ....  --password ...--query "SELECT $CHOOSE_COLUMN_NAMES FROM $OTABLE where $OWHERE and \$CONDITIONS"  ... through unix shell. If the command takes a long time  how can i kill through shell ? Thanks in advance. 

I am trying to install components using a python script. Based loosely upon the community example I found here:https://github.com/datagulp/hdp_utilities/blob/master/autoScale.shAt the step where I get a list of components from another compute node, then try to replicate, I can add each component onto a node with a POST successfully. I can see the components being added. But when I try to then do a PUT to set the state of each one to INSTALLED, it fails. curl -u user1 -i -H "X-Requested-By:ambari" -X PUT -d '{"HostRoles": {"state":"INSTALLED"}}' http://my-ambari-host:8080/api/v1/clusters/my-cluster/hosts/my-host/host_components/ATLAS_CLIENT{ "status" : 400, "message" : "java.lang.IllegalArgumentException: Missing KDC administrator credentials.\nThe KDC administrator credentials must be set as a persisted or temporary credential resource.This may be done by issuing a POST to the /api/v1/clusters/:clusterName/credentials/kdc.admin.credential API entry point with the following payload:\n

Hello! My problem is to make hive queries. Using Impala works fine. Both in HUE The query I tried to make is:e.g.load data local inpath '/tmp/expectancy_country.csv' overwrite into table expectancy_country; After 2:45 i get the message time out and the log shows:INFO : Compiling command(queryId=hive_20190612043030_466f94db-6fb9-41c4-ad84-098d8f8decbe): load data local inpath '/tmp/expectancy_country.csv' overwrite into table expectancy_countryINFO : Semantic Analysis CompletedINFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)INFO : Completed compiling command(queryId=hive_20190612043030_466f94db-6fb9-41c4-ad84-098d8f8decbe); Time taken: 0.061 secondsOn the left side is still the red quader indicating that the query is running. I also tried:create table expectancy_country (expectancy int, country string) row format delimited fields terminated by ','; With the same result. Butshow tables works fine. The quickstart does not show

Hey Cloudera family, just a quick nooby question :am working on cloudera Single node and i want to add some securitymy 1st question is it possible to add Kerberos security/authentification ?my 2nd question is should i go for Kerberos or Sentry ?thanks.

HelloI have a flow which reads certain parameters and decide if it is a database ingestion or file ingestion. DB ingestions are working fine but for file ingestions i am not able to connect the flow with FetchFile/GetFile/List file processors Looks like these processors does not allow input port to connect. Is there any way or processor to get this done...? My File Ingestion templates is designed to take the input from different folders/files based on input parameters from outerflow.

Despite adding the following, --conf spark.driver.extraClassPath=/opt/cloudera/parcels/CDH/jars/htrace-core-3.2.0-incubating.jar:/opt/cloudera/parcels/CDH/lib/hive/conf:/opt/cloudera/parcels/CDH/lib/hive/lib/*.jar \--conf spark.executor.extraClassPath=/opt/cloudera/parcels/CDH/jars/htrace-core-3.2.0-incubating.jar:/opt/cloudera/parcels/CDH/lib/hive/conf:/opt/cloudera/parcels/CDH/lib/hive/lib/*.jar \ I get the following exception:Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace I have checked the file permission and directory permission and they have the right permission (directories have and read and execute perms while the files have 755 perms). Please let me know what needs to be changed. 

I want to hit a specific JMX port for the DataNode (which is port 50070), but I don't want to hardcode the port number. How can I retrieve this port number dynamically, and where exactly is it stored?

Hi All,The configured HDP 2.6.3 cluster with Ambari 2.6 is unable to show contents inside the Alerts page.Can anyone help me, how to find the RCA for the same?Attaching screenshot of the Ambari Alerts Page:How to monitor log ? for this Alert page.

I cant solve the problem of connections between hosts caused by Errno:111 !!My :hostname is right,iptables is stoped,selinux is stoped.safemode is off on the namenodeHere is the datanode log file :root@node4:~# cat /var/log/hadoop/hdfs/hadoop-hdfs-datanode-*.log2019-06-11 12:30:42,269 INFO  datanode.DataNode (LogAdapter.java:info(51)) - STARTUP_MSG:/************************************************************STARTUP_MSG: Starting DataNodeSTARTUP_MSG:   host = node4.rh.bigdata.cluster/172.16.138.113STARTUP_MSG:   args = []STARTUP_MSG:   version = 3.1.1.3.1.0.0-78STARTUP_MSG:   classpath = /usr/hdp/3.1.0.0-78/hadoop/conf:/usr/hdp/3.1.0.0-78/hadoop/lib/snappy-java-1.0.5.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/token-provider-1.0.1.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/jersey-server-1.19.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/jersey-servlet-1.19.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/stax2-api-3.1.4.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/netty-3.10.5.Final.jar:/usr/hdp/3.1.0.0-78/hadoo

@Geoffrey Shelton OkotHello Community,I cant solve the problem of connections between hosts caused by Errno:111 !!My :hostname is right,iptables is stoped,selinux is stoped.Here is the log of starting the name node of my cluster.Mysafemode: Call From node4.rh.bigdata.cluster/172.16.138.113 to node4.rh.bigdata.cluster:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused 2019-06-07 10:07:14,531 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://node4.rh.bigdata.cluster:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. safemode: Call From node4.rh.bigdata.cluster/172.16.138.113 to node4.rh.bigdata.cluster:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused safemode: Call From node4.rh.bigdata.cluster/172.16.138.11

Hi,    I am not able to produce the msg on cloudera. I am running below command get the error.    COMMAND: kafka-console-producer.sh \--broker-list localhost:9092 \--topic kafkademo1   ERRPR MSG:    WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)       Need support in this regard.               

Hi,I have 3 flows files(CSV) which are generating from ExecuteSQL -> ConvertRecord processor. Flow file 2 and 3 have single column let's say ID field and flow file 1 is a csv with 'n' columns. I want to create a one flow file which is a summary from the 3 flow files i.e, count of rows in flow files 1,2,3 which i can get it from executesql.row.count attribute and content of flow files 2 and 3 which is ID. Later i will send this flow file as an attachment via PutEmail processor. How can i achieve this?Thanks&Regards,P Vinay.

How to enable debug logs for Application Master ?