Solved

Hello! My problem is to make hive queries. Using Impala works fine. Both in HUE The query I tried to make is:e.g.load data local inpath '/tmp/expectancy_country.csv' overwrite into table expectancy_country; After 2:45 i get the message time out and the log shows:INFO : Compiling command(queryId=hive_20190612043030_466f94db-6fb9-41c4-ad84-098d8f8decbe): load data local inpath '/tmp/expectancy_country.csv' overwrite into table expectancy_countryINFO : Semantic Analysis CompletedINFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)INFO : Completed compiling command(queryId=hive_20190612043030_466f94db-6fb9-41c4-ad84-098d8f8decbe); Time taken: 0.061 secondsOn the left side is still the red quader indicating that the query is running. I also tried:create table expectancy_country (expectancy int, country string) row format delimited fields terminated by ','; With the same result. Butshow tables works fine. The quickstart does not show

Hey Cloudera family, just a quick nooby question :am working on cloudera Single node and i want to add some securitymy 1st question is it possible to add Kerberos security/authentification ?my 2nd question is should i go for Kerberos or Sentry ?thanks.

HelloI have a flow which reads certain parameters and decide if it is a database ingestion or file ingestion. DB ingestions are working fine but for file ingestions i am not able to connect the flow with FetchFile/GetFile/List file processors Looks like these processors does not allow input port to connect. Is there any way or processor to get this done...? My File Ingestion templates is designed to take the input from different folders/files based on input parameters from outerflow.

Despite adding the following, --conf spark.driver.extraClassPath=/opt/cloudera/parcels/CDH/jars/htrace-core-3.2.0-incubating.jar:/opt/cloudera/parcels/CDH/lib/hive/conf:/opt/cloudera/parcels/CDH/lib/hive/lib/*.jar \--conf spark.executor.extraClassPath=/opt/cloudera/parcels/CDH/jars/htrace-core-3.2.0-incubating.jar:/opt/cloudera/parcels/CDH/lib/hive/conf:/opt/cloudera/parcels/CDH/lib/hive/lib/*.jar \ I get the following exception:Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace I have checked the file permission and directory permission and they have the right permission (directories have and read and execute perms while the files have 755 perms). Please let me know what needs to be changed. 

I want to hit a specific JMX port for the DataNode (which is port 50070), but I don't want to hardcode the port number. How can I retrieve this port number dynamically, and where exactly is it stored?

Hi All,The configured HDP 2.6.3 cluster with Ambari 2.6 is unable to show contents inside the Alerts page.Can anyone help me, how to find the RCA for the same?Attaching screenshot of the Ambari Alerts Page:How to monitor log ? for this Alert page.

I cant solve the problem of connections between hosts caused by Errno:111 !!My :hostname is right,iptables is stoped,selinux is stoped.safemode is off on the namenodeHere is the datanode log file :root@node4:~# cat /var/log/hadoop/hdfs/hadoop-hdfs-datanode-*.log2019-06-11 12:30:42,269 INFO  datanode.DataNode (LogAdapter.java:info(51)) - STARTUP_MSG:/************************************************************STARTUP_MSG: Starting DataNodeSTARTUP_MSG:   host = node4.rh.bigdata.cluster/172.16.138.113STARTUP_MSG:   args = []STARTUP_MSG:   version = 3.1.1.3.1.0.0-78STARTUP_MSG:   classpath = /usr/hdp/3.1.0.0-78/hadoop/conf:/usr/hdp/3.1.0.0-78/hadoop/lib/snappy-java-1.0.5.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/token-provider-1.0.1.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/jersey-server-1.19.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/jersey-servlet-1.19.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/stax2-api-3.1.4.jar:/usr/hdp/3.1.0.0-78/hadoop/lib/netty-3.10.5.Final.jar:/usr/hdp/3.1.0.0-78/hadoo

@Geoffrey Shelton OkotHello Community,I cant solve the problem of connections between hosts caused by Errno:111 !!My :hostname is right,iptables is stoped,selinux is stoped.Here is the log of starting the name node of my cluster.Mysafemode: Call From node4.rh.bigdata.cluster/172.16.138.113 to node4.rh.bigdata.cluster:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused 2019-06-07 10:07:14,531 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://node4.rh.bigdata.cluster:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. safemode: Call From node4.rh.bigdata.cluster/172.16.138.113 to node4.rh.bigdata.cluster:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused safemode: Call From node4.rh.bigdata.cluster/172.16.138.11

Hi,    I am not able to produce the msg on cloudera. I am running below command get the error.    COMMAND: kafka-console-producer.sh \--broker-list localhost:9092 \--topic kafkademo1   ERRPR MSG:    WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)       Need support in this regard.               

Hi,I have 3 flows files(CSV) which are generating from ExecuteSQL -> ConvertRecord processor. Flow file 2 and 3 have single column let's say ID field and flow file 1 is a csv with 'n' columns. I want to create a one flow file which is a summary from the 3 flow files i.e, count of rows in flow files 1,2,3 which i can get it from executesql.row.count attribute and content of flow files 2 and 3 which is ID. Later i will send this flow file as an attachment via PutEmail processor. How can i achieve this?Thanks&Regards,P Vinay.

How to enable debug logs for Application Master ?

while starting oozie service getting below error:oozie server status:Execution of 'source /usr/hdp/current/oozie-server/conf/oozie-env.sh ; oozie admin -oozie http://master1:11000/oozie -status' returned 255. Connection exception has occurred [ java.net.ConnectException Connection refused (Connection refused) ]. Trying after 1 sec. Retry count = 1 Connection exception has occurred [ java.net.ConnectException Connection refused (Connection refused) ]. oozie server web ui:Connection failed to master1:11000/oozie/?user.name=oozie (<urlopen error [Errno 111] Connection refused>) oozie log:Jun 03, 2019 8:34:40 PM org.apache.coyote.http11.Http11Protocol pauseINFO: Pausing Coyote HTTP/1.1 on http-11000Jun 03, 2019 8:34:41 PM org.apache.catalina.core.StandardService stopINFO: Stopping service CatalinaJun 03, 2019 8:34:41 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbcSEVERE: The web application [/oozie] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to

Hi All,when we start spark2 thrift server , its start for ashort time - 30 sec and then fail back.I have attched the spark2 logs.19/06/07 11:22:16 INFO HiveThriftServer2: HiveThriftServer2 started19/06/07 11:22:16 INFO UserGroupInformation: Login successful for user hive/lhdcsi02v.production.local@production.local using keytab file /etc/security/keytabs/hive.service.keytab19/06/07 11:22:16 ERROR ThriftCLIService: Error starting HiveServer2: could not start ThriftBinaryCLIServicejava.lang.NoSuchMethodError: org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server.startDelegationTokenSecretManager(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/Object;Lorg/apache/hadoop/hive/thrift/HadoopThriftAuthBridge$Server$ServerMode;)Vat org.apache.hive.service.auth.HiveAuthFactory.<init>(HiveAuthFactory.java:125)at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:57)at java.lang.Thread.run(Thread.java:748)19/06/07 11:22:16 INFO HiveServer2: Shutti

I'm trying to execute the "Refresh yarn queue's" command through ambari API. Unable to find a solution for this. If anybody has any idea please let me know.

Dear Community,I am not able to insert data in my oracle database.I am getting below issue :-Cannot create JDBC driver of class 'oracle.jdbc.driver.OracleDriver' for connect URL 'jdbc:oracle:thin@//hostname:1521/Databasename'; Driver name : oracle.jdbc.driver.OracleDriverDriver location : /opt/jdbc/ojdbc6.jarI have also given the user name and password correctly.May be the url syntax is wrong. I am not able to correct it. COuld you please provide me a pictorial image to get the correct syatnx of the DBCP connection pool for OracledbCould you please help me here.My host name is a remote hostname. and Database name is the schema (owner of the table) in which i am going to insert the data,Also do i need to mention the service name anywhere here,Thanks,Ankit SIngh7428344123

i really love hdp especially ambari . could somebody answer my question since i noticed no more new hdp release for a while.

Hi for all ! Is anybody know why processor's thread info (up right corner) turn in red and have an info about - "0 active threads (1 terminated) "? The processor works fine but this red info makes me to warn about. How it is critical and how can I fix this? Thanks!Screenshot in an attach.

I haven't enabled SSL for Ranger—only for the Ambari UI—but my Ranger UI keeps opening with HTTPS instead of HTTP.

Hi, I am trying to use the custom log4j to gather Spark driver logs( submitting jobs under CLUSTER mode), but unable to achieve it.Here is my custom log4j.properties file content:log4j.rootCategory=ALL,FILElog4j.appender.FILE=org.apache.log4j.RollingFileAppender#Below is the unix server path from where job is getting submittedlog4j.appender.FILE.File=/some/path/to/edgeNode/SparkDriver.loglog4j.appender.FILE.Append=falselog4j.appender.FILE.layout=org.apache.log4j.PatternLayoutlog4j.appender.FILE.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n And cmnd to submit job:spark2-submit --files /apps/test/config/driver_log4j.properties --conf "spark.driver.extraJavaOptions=-Dlog4j.configuration=driver_log4j.properties" --master yarn --deploy-mode cluster --num-executors 2 --executor-cores 4 --driver-memory 1g --executor-memory 16g --keytab XXXXX.keytab --principal XXXXX --class com.test.spark.par_1_submit par_submit.jar Error I'm getting:java.io.FileNotFoundExceptio

 1. Webhdfs: curl with Kerberos enabled  with port  50070 I am able to list status  2. HTTPFS: curl with kerberos enabled with port 14000, I am getting 404 not found.   Both curl commands file path location is the same.  Can anyone have this issue?

I have a scenario where i'm trying to create a table which points to an HDFS location which has a directory name starting with an "_" in the HDFS path. Now table creation goes through but If I try to read data out of the table it throws error, below is what i get: create external table `ingest.workgroup__views2` row format serde 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' location 'hdfs://nameservice1/user/data/ingest/mdm/workgroup_i/workgroup/_views' tblproperties ('avro.schema.url'='hdfs://nameservice1/user/data/ingest/mdm/workgroup_i/workgroup/_views/_gen/_views.avsc');No rows affected (0.232 seconds)0: jdbc:hive2://t-hive.sys.cigna.com:25006/de> select * from ingest.workgroup__views2;Error: java.io.IOException: org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://nameservice1/user/data/i

I am using the Python client API (cm_client) to create batch processes for replicaiton schedules.  What I would like to do is list the command details for a specific replication schedule, specifically the subcommands (as described in the documentation - https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_dg_view_running_recent_commands.html#concept_ktn_xsf_4l__section_s5d_kqf_4l)  Is there any such API GET call that retrieves the Command Details in the Running Commands dialog box?  Thank you for any help you can provide.

Is there a maximum amount of nodes a Nifi cluster can contain? If so what and where are the bottlenecks? Are there figures for max data per second or max events per second?

It looks like there is a problem with http://repo.hortonworks.com/, it is returning empty responses. Any ETA on when it will be fixed?

I am new to Hortonwork. I have installed Hortonworks. Clicking on "Files View" in Ambari. On Service check complete it is showing me the below-mentioned error. How to solve this issueService 'hdfs' check failed:java.lang.NullPointerException at org.apache.ambari.server.view.ClusterImpl.getConfigByType(ClusterImpl.java:72) at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.copyPropertiesBySite(ConfigurationBuilder.java:223) at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.buildConfig(ConfigurationBuilder.java:358) at org.apache.ambari.view.utils.hdfs.HdfsApi.<init>(HdfsApi.java:75) at org.apache.ambari.view.utils.hdfs.HdfsUtil.getHdfsApi(HdfsUtil.java:157) at org.apache.ambari.view.utils.hdfs.HdfsUtil.connectToHDFSApi(HdfsUtil.java:145) at org.apache.ambari.view.commons.hdfs.HdfsService.hdfsSmokeTest(HdfsService.java:145) at org.apache.ambari.view.filebrowser.HelpService.hdfsStatus(HelpService.java:95) at sun.reflect.NativeMethodAccessor