Q&A

while starting oozie service getting below error:oozie server status:Execution of 'source /usr/hdp/current/oozie-server/conf/oozie-env.sh ; oozie admin -oozie http://master1:11000/oozie -status' returned 255. Connection exception has occurred [ java.net.ConnectException Connection refused (Connection refused) ]. Trying after 1 sec. Retry count = 1 Connection exception has occurred [ java.net.ConnectException Connection refused (Connection refused) ]. oozie server web ui:Connection failed to master1:11000/oozie/?user.name=oozie (<urlopen error [Errno 111] Connection refused>) oozie log:Jun 03, 2019 8:34:40 PM org.apache.coyote.http11.Http11Protocol pauseINFO: Pausing Coyote HTTP/1.1 on http-11000Jun 03, 2019 8:34:41 PM org.apache.catalina.core.StandardService stopINFO: Stopping service CatalinaJun 03, 2019 8:34:41 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbcSEVERE: The web application [/oozie] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to

Hi All,when we start spark2 thrift server , its start for ashort time - 30 sec and then fail back.I have attched the spark2 logs.19/06/07 11:22:16 INFO HiveThriftServer2: HiveThriftServer2 started19/06/07 11:22:16 INFO UserGroupInformation: Login successful for user hive/lhdcsi02v.production.local@production.local using keytab file /etc/security/keytabs/hive.service.keytab19/06/07 11:22:16 ERROR ThriftCLIService: Error starting HiveServer2: could not start ThriftBinaryCLIServicejava.lang.NoSuchMethodError: org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server.startDelegationTokenSecretManager(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/Object;Lorg/apache/hadoop/hive/thrift/HadoopThriftAuthBridge$Server$ServerMode;)Vat org.apache.hive.service.auth.HiveAuthFactory.<init>(HiveAuthFactory.java:125)at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:57)at java.lang.Thread.run(Thread.java:748)19/06/07 11:22:16 INFO HiveServer2: Shutti

I'm trying to execute the "Refresh yarn queue's" command through ambari API. Unable to find a solution for this. If anybody has any idea please let me know.

Dear Community,I am not able to insert data in my oracle database.I am getting below issue :-Cannot create JDBC driver of class 'oracle.jdbc.driver.OracleDriver' for connect URL 'jdbc:oracle:thin@//hostname:1521/Databasename'; Driver name : oracle.jdbc.driver.OracleDriverDriver location : /opt/jdbc/ojdbc6.jarI have also given the user name and password correctly.May be the url syntax is wrong. I am not able to correct it. COuld you please provide me a pictorial image to get the correct syatnx of the DBCP connection pool for OracledbCould you please help me here.My host name is a remote hostname. and Database name is the schema (owner of the table) in which i am going to insert the data,Also do i need to mention the service name anywhere here,Thanks,Ankit SIngh7428344123

i really love hdp especially ambari . could somebody answer my question since i noticed no more new hdp release for a while.

Hi for all ! Is anybody know why processor's thread info (up right corner) turn in red and have an info about - "0 active threads (1 terminated) "? The processor works fine but this red info makes me to warn about. How it is critical and how can I fix this? Thanks!Screenshot in an attach.

I haven't enabled SSL for Ranger—only for the Ambari UI—but my Ranger UI keeps opening with HTTPS instead of HTTP.

Hi, I am trying to use the custom log4j to gather Spark driver logs( submitting jobs under CLUSTER mode), but unable to achieve it.Here is my custom log4j.properties file content:log4j.rootCategory=ALL,FILElog4j.appender.FILE=org.apache.log4j.RollingFileAppender#Below is the unix server path from where job is getting submittedlog4j.appender.FILE.File=/some/path/to/edgeNode/SparkDriver.loglog4j.appender.FILE.Append=falselog4j.appender.FILE.layout=org.apache.log4j.PatternLayoutlog4j.appender.FILE.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n And cmnd to submit job:spark2-submit --files /apps/test/config/driver_log4j.properties --conf "spark.driver.extraJavaOptions=-Dlog4j.configuration=driver_log4j.properties" --master yarn --deploy-mode cluster --num-executors 2 --executor-cores 4 --driver-memory 1g --executor-memory 16g --keytab XXXXX.keytab --principal XXXXX --class com.test.spark.par_1_submit par_submit.jar Error I'm getting:java.io.FileNotFoundExceptio

 1. Webhdfs: curl with Kerberos enabled  with port  50070 I am able to list status  2. HTTPFS: curl with kerberos enabled with port 14000, I am getting 404 not found.   Both curl commands file path location is the same.  Can anyone have this issue?

I have a scenario where i'm trying to create a table which points to an HDFS location which has a directory name starting with an "_" in the HDFS path. Now table creation goes through but If I try to read data out of the table it throws error, below is what i get: create external table `ingest.workgroup__views2` row format serde 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' location 'hdfs://nameservice1/user/data/ingest/mdm/workgroup_i/workgroup/_views' tblproperties ('avro.schema.url'='hdfs://nameservice1/user/data/ingest/mdm/workgroup_i/workgroup/_views/_gen/_views.avsc');No rows affected (0.232 seconds)0: jdbc:hive2://t-hive.sys.cigna.com:25006/de> select * from ingest.workgroup__views2;Error: java.io.IOException: org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://nameservice1/user/data/i

I am using the Python client API (cm_client) to create batch processes for replicaiton schedules.  What I would like to do is list the command details for a specific replication schedule, specifically the subcommands (as described in the documentation - https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_dg_view_running_recent_commands.html#concept_ktn_xsf_4l__section_s5d_kqf_4l)  Is there any such API GET call that retrieves the Command Details in the Running Commands dialog box?  Thank you for any help you can provide.

Is there a maximum amount of nodes a Nifi cluster can contain? If so what and where are the bottlenecks? Are there figures for max data per second or max events per second?

It looks like there is a problem with http://repo.hortonworks.com/, it is returning empty responses. Any ETA on when it will be fixed?

I am new to Hortonwork. I have installed Hortonworks. Clicking on "Files View" in Ambari. On Service check complete it is showing me the below-mentioned error. How to solve this issueService 'hdfs' check failed:java.lang.NullPointerException at org.apache.ambari.server.view.ClusterImpl.getConfigByType(ClusterImpl.java:72) at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.copyPropertiesBySite(ConfigurationBuilder.java:223) at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.buildConfig(ConfigurationBuilder.java:358) at org.apache.ambari.view.utils.hdfs.HdfsApi.<init>(HdfsApi.java:75) at org.apache.ambari.view.utils.hdfs.HdfsUtil.getHdfsApi(HdfsUtil.java:157) at org.apache.ambari.view.utils.hdfs.HdfsUtil.connectToHDFSApi(HdfsUtil.java:145) at org.apache.ambari.view.commons.hdfs.HdfsService.hdfsSmokeTest(HdfsService.java:145) at org.apache.ambari.view.filebrowser.HelpService.hdfsStatus(HelpService.java:95) at sun.reflect.NativeMethodAccessor

I have a table with 38M rows which is being updated daily. The table is in orc format and it is managed table.I want to create partitions on that table but I couldn't find a way to alter existing non-partitioned table to create partitions in that table. I tried searching all over the google but the only option I was able to find was Create a new partitioned table and insert data into it from the old table. I was wondering if there was any way to create partitions on an existing table. My HIVE version is 3.1.Thanks & Regards,Kartik

I ran the the command ambari-server setup, after which I select accordingly. I am prompt to select the java and selected custom JDK and enter the JAVA_HOME but I was shown the error below.

I've a Hbase mapped Hive table which retrieves rows when I run select * from my_table;But when I execute select count(*) from my_table; the output is 0.What must be the issue in this case ?

I get Error 404. Please see the attached pic

Basically, I am trying to analyze SQL server log files using Hive.The layout is the log file ischar(23) - for timestampchar(1) - spacechar(12) - sourcethe rest of the row, and the length varies. The row delimiter is CR+LF. Below are some entries in the log.2019-05-28 07:29:55.03 Server      UTC adjustment: -7:00 2019-05-28 07:29:55.03 Server      (c) Microsoft Corporation. 2019-05-28 07:29:55.03 Server      All rights reserved. 2019-05-28 07:29:55.03 Server      Server process ID is 3368.There are several posts here regarding fixed-width column layout. But in my case, the last column is identified not by the width but by the row delimiter.

Due to changes in our networking and firewall requirements we had to switch from the hortonworks public repositories to the local repositories (artifactory). After making this change we are seeing the following exception when we try to "Start All" (from Ambari) Which triggers the installation of the client packages and they are failing with the following exception.Essentially similar error is thrown when we try to install any new package after this change.OS: Ubuntu 16.04No package found for hadoop-${stack_version}-client(hadoop-(\d|-)+-client$)2019-05-29 15:47:30,684 - The repository with version 2.6.5.0-292 for this command has been marked as resolved. It will be used to report the version of the component which was installedCommand failed after 1 tries

Hello. I'm a new guy using Coludera.I have very simple question about a log. I need to check every single minute's resource's log such as cpu usage, mem usage, disk I/O etc,but it's hard to find that kinds of data in log files.To be more specific, I want to utilize resource logs as data frame shapefor recognizing whether the server is going well or not. so, can I obtain resource data ? Aplogies for this stupid question. Appreciate your response.  

I have a json flowfile with below content{"name":"abc"}I want the output should be {"name":"abc","attributes":{"id":"12233","map":"Y"}}How to achieve the same?

I've successfully cloned and built the NIFI system. It built cleanly. Though initially I experienced a few random unit test failures. That was solved (it was actually a network configuration issue on my end that caused this).The instructions I followed are here:https://nifi.apache.org/quickstart.html The built system is in nifi-assembly/target. I want to run the primary NIFI backend application in the debugger (not the web app, but rather the backend). I have the nifi set of maven projects loaded into eclipse.Can someone provide me information that would assist in getting the app up and running in the debugger?For example:What project contains the main entry point?Aside from running the main application, how do I launch the web app? I'm assuming it is launched separately from the main application.

Hi, I was about to give my exam on CCA175, but the Cloudera team canceled my exam schedule hours before my exam and they sent me an email stating some technical issues they are facing. It's been 3 days I am waiting to give the exam and looks like the technical issues are not resolved yet. The technical team is not giving me any ETA and I don't see my exam voucher in my examlocal account. I never expected this from a reputed company like Cloudera. Who can help me getting back my exam voucher and letting me schedule my exam?

Hello All, I am looking for some best practices or recommendations to set a best possible value for rlimit_fds (Maximum Process File Descriptors) property. Currently, it is set to default i.e. 32768 and we are getting File Descriptor Threshold alerts. We would first like to look for a best possible value for rlimit_fds. Is there a formulae or a practice or few checks that can be performed to determine a best value? Thankssnm1523