Solved

helloWhen i start nifi with ambari i get this error Java Heap Space OutofMemory but when i start it with console by script nifi.sh i get no error

Hi Team,   Can anyone share if they have updated resource pool configurations in YARN using CM API, what end points were used if so. Thanks in advance.

Hello All,   I am trying to add sentry service to the cloudera CDH 5.12 cluster.   As part of that I created database and user and granted all privileges to the user for the sentry database.   But I am getting below error.   Logon denied for user/password. Able to find the database server and database, but logon request was rejected.   Please suggest.   Thanks,

@Geoffrey Shelton Okot, @mthiele,I want to install Kerberos on Hadoop cluster, i have seen the articleConfiguring Ambari and Hadoop for Kerberos using AD as the KDC - Videohttps://community.hortonworks.com/questions/103945/kerberos-setup-on-hdp-26.html and this is really helpful.I have couple of questions on this,1.Do I need to install Kerberos via command line before starting this task?2. Because i see krb5.conf file exists in my cluster and kerberos is not enabled, do I need to delete the conf files in all nodes and run"yum install -y krb5-server krb5-libs krb5-workstation" command then carry on the steps provided in the aboe article or i don't need to bother to about and straightaway start from Ambari for enabling process.3. As per the official doc. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_security/content/configuring_amb_hdp_for_kerberos.html we need to enable Kerberos from Ambari,Now i need a clear idea 1st do I need to install kerberos using yum command and the

Using CDH 5.3.1 (without CM), I have a DataNode that seems to not start it's block report. The particular DN has 100x more RBW files than other DNs (some RBW files are a year old). The driving symptom is blocks reported missing, but the particular blocks are indeed under /finalized/ directory of the DN.  A few thousand files have missing blocks that are in this state and no alternative blocks/replicas are on the cluster, so we would like to recover these files. The missing blocks are NOT under /rbw/ dir., hence the concern over the "RemoteException in offerService" error.  Classpath and VERSION files look good compared to known-good DNs. See point-in-time logs entries below.  * Question: How can a /finalized/ block (replica) be considered missing after DN has been up for many hours?  * Question: what if I manually copy the finalized blk_* files in question to another DN? would that DN pick them up upon restart?  * Questio

Good day! We actively use software configuration management (SCM, for example saltstack, ansible, puppet/etc), and we decide add new host to cluster Cloudera hadoop through this system.Our steps:1. Add repo cat /etc/apt/sources.list.d/cloudera-manager.listdeb https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.13.0 contri  2. Install soft (cloudera-manager-daemons, cloudera-manager-agent) 3. Edit /etc/cloudera-scm-agent/config.ini [General]# Hostname of the CM server.server_host=main.test.com4. Start agent  After that we see that new host in cluster.Also I use this doc - https://www.cloudera.com/documentation/enterprise/5-13-x/topics/cm_mc_adding_hosts.htmlfor checking our steps in SCM.But we run into some trouble A. http://community.cloudera.com/t5/Cloudera-Manager-Installation/Bad-The-hostname-and-canonical-name-for-this-host-are-not/m-p/86905#M16122?eid=1&aid=1  - there I wrote some description about our error B.

I am looking for a way to migrate a Hive table from one Hadoop cluster to another cluster that is running on a different Kerberos REALM.

Hi, I'm trying to register for CCA175 exam but I have got a doubt. While doing the Checkout, I have to enter the personal information and there is one text field called "Account", do I have to enter my Community Nickname in the Account field or is it something else? Can anyone please provide me a confirmation on this as soon as possible? Thank you. Regards,Varun Chawla

I have a table stuck in 'DISABLING' state and one of the four table regions is stuck in 'OPENING' state. Hbase 2.0 hbck doesn't allow fixing tables../hbase zkcli, ls /hbase-unsecure/table/<table name> doesn't show anythingrestarting the hbase master, regionservers, and zookeeper doesn't resolveHow do I fix this issue.

I got this error while trying to install Cloudera manager on my Rhel7 workstation Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-: managerResolving Dependencies--> Running transaction check---> Package oracle-j2sdk1.8.x86_64 0:1.8.0+update141-1 will be installed--> Finished Dependency ResolutionDependencies Resolved================================================================================Package Arch Version Repository Size================================================================================Installing:oracle-j2sdk1.8 x86_64 1.8.0+update141-1 cloudera-manager 176 MTransaction Summary================================================================================Install 1 PackageTotal download size: 176 MInstalled size: 359 MDownloading packages:No Presto metadata available for cloudera-manager[Errno 14] curl#56 - "SSL received a record with an incorrect Message Authentication Code."Trying other mirror. 

 Hello all, I seem to have run into a Kudu 1.4 bug, and I am wondering if there is any work around other than upgrading.  I lost a tablet in my dev environment and wasn't aware of it for a while (24hours or so) which I just assumed I could just wipe the directories and rebuild it thinking all the replicas had already been replicated to other tablets well that wasn't the case. After getting the tablet backup I ran a ksck and noted I have two tables that are under-replicated, and I am seeing a lot of alerts like the one below.  Now I did see this https://issues.apache.org/jira/browse/KUDU-1613 and it says it's fixed in 1.7 but upgrading isn't an option at this time. T acc6adcd6b554b5cabfed268184591a9 P fa6625fcbb204fba86ac6d4007c3f695 -> Peer 13950e716d21404188b733521dbf4d97 (1.2.3.4.domain.com:7050): Couldn't send request to peer 13950e716d21404188b733521dbf4d97 for tablet acc6adcd6b554b5cabfed268184591a9. Error code: WRONG_SERVER_UUI

Hi,   We have a Kudu cluster with replication factor 3. All Kudu machines have time synchronization via NTP. Our application performs a SELECT from a Kudu table via Impala JDBC driver. The Impala Daemon is configured with the command-line option: --kudu_read_mode=READ_AT_SNAPSHOT. This operation fails:   [Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: Unable to open scanner: Timed out: unable to retry before timeout: Remote error: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: Tablet is lagging too much to be able to serve snapshot scan. Lagging by: 61148 ms, (max is 30000 ms):: Remote error: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: Tablet is lagging too much to be able to serve snapshot scan. Lagging by: 30821 ms, (max is 30000 ms):, Query: SELECT COUNT(*) as `expr_0` FROM `ci_pdo_PDO_Func_Tests`.`alarm_relationship_rules`

Hi, I just installed the CDH 6.1 on my VMs, and I've tried to access to oozie web ui. But I failed with the message below. { "errorMessage": "org.apache.jasper.JasperException: PWC6033: Error in Javac compilation for JSP\n\n PWC6199: Generated servlet error:\npackage javax.servlet does not exist\n\n PWC6199: Generated servlet error:\npackage javax.servlet.http does not exist\n\n PWC6199: Generated servlet error:\npackage javax.servlet.jsp does not exist\n\n PWC6199: Generated servlet error:\npackage org.apache.oozie.sla.service does not exist\n\n PWC6199: Generated servlet error:\npackage org.apache.oozie.service does not exist\n\n PWC6199: Generated servlet error:\npackage org.apache.oozie.service does not exist\n\n PWC6199: Generated servlet error:\npackage org.apache.oozie.service does not exist\n\n PWC6199: Generated servlet error:\npackage org.apache.oozie.service does not exist\n\n PWC6199: Generated servlet error:\npackage org.apache.

Using Hue™ 4.1 on CDH 5.16 I only see the Delete Forever button in File Browser but not the Move to Trash even though Use Trash is enabled in HDFS configuration. How to enable this in HUE?

Hi,After I have upgraded to CM/CDH 6.1 from 5.16.1, my hosts randomly and periodically having "Unknown Health" for about a few seconds and then go back to green. I have not seen/found any WARNING nor any ERROR in any logs from hosts or any services.The entire cluster works without any issue, I have run host inspection and network inspection without any problem. Also, synced time/date a few times just in case but still I can watch my hosts (also services because of the hosts) going grey with "Unknown Health" and back to green randomly for few seconds. Cloudera Management Service is on one server with 14 cores and 28G memory. I have checked this server activity, it is pretty idle, so the cluster is not a busy cluster. Either way, this is the heap size for the monitorings:Java Heap Size of Activity Monitor in Bytes: 2GBJava Heap Size of Alert Publisher in Bytes: 256MBJava Heap Size of EventServer in Bytes: 1GBJava Heap Size of Host Monitor in Bytes: 4GBJava Heap Size of Service Monit

We upgrade our Ambari from 2.6 to 2.7.3, after the upgrading, our dashboard metrics keep showing the loading animation without any data. However, Ambari metrics service is ok and metrics page on each components also works.I have checked log file on host of Ambari server, but no exception found. And i also tried to clear Hbase root and tmp dir, not work neither.Any suggestion is appreciate, thanks in advance!

i following bellow steps to setup sso:Install Knox.Configure Ambari Authentication for LDAP/AD.Configure an LDAP/AD Identity Provider (IdP).Enable Knox SSO using the Ambari CLI.after this step sso worked me for redirect to ambari ui.in the next step when i add the bellow property to core-site.xml, restarting hdfs failed:"hadoop.http.authentication.type": "org.apache.hadoop.security.authentication.server.JWTRedirectAuthenticationHandler”"hadoop.http.authentication.public.key.pem": “$SSOPUBLICKEY""hadoop.http.authentication.authentication.provider.url": "$SSOPROVIDERURL"when add this property (hadoop.http.authentication.type) to hdfs happening error.hdfs log file:2019-02-26 16:28:06,366 ERROR datanode.DataNode (DataNode.java:secureMain(2870)) - Exception in secureMainjava.io.IOException: Problem starting http server Caused by:         at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1165)        at org.apache.hadoop.hdfs.server.da

Hello Friends:   On a relatively new installation of CDH6.1 (parcels) with one node for CDH manager and a second node for Master and Slave services (combined), I'm getting this error:   org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "hdfs"' after running this:   user$ /opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/bin/parquet-tools \ cat hdfs://tmp/1.parquet Here is the output of hadoop classpath:   /etc/hadoop/conf:/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/hadoop/libexec/../../hadoop/lib/*:/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/hadoop/libexec/../../hadoop/.//*:/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/hadoop/libexec/../../hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/hadoop/libexec/../../hadoop-hdfs/.//*:/opt/cloudera/parcels/CDH/lib/hadoop

Hi,I need to handle an HTTP request and I need to respond an HTML file with NiFi.My flow is likeHandleHttpRequest -> ExecuteSQL -> HandleHttpResponse.I need to query our database to show proper HTML result.My problem is HandleHttpRequest runs twice for each request when triggered by browsers such as Chrome, Firefox. HandleHttpRequest runs once if I use Postman or SoapUI.I've tried to eliminate the second call by using HashContent and DetectDuplicate but it makes some of our requests fail. I've investigated the issue and I've found that Chrome and Firefox creates an additional request for displaying favicon and sometimes some other reasons. (Link)I need to find real request and I need to eliminate other requests. The real request can be the second one depending on the browser status. Browser decides when to create new requests and their orders.So, my question is how can I run my flow exactly one time when user creates request from their browser? Because I do some operations on the

./import-hive.sh -Dsun.security.jgss.debug=true -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.krb5.conf=/etc/krb5.cont f -Djava.security.auth.login.config=/etc/atlas/conf/atlas_jaas.confUsing Hive configuration directory [/etc/hive/conf]Log file for import is /var/log/atlas/import-hive.loglog4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.Exception in thread "main" org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed. at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:650)Caused by: java.lang.IllegalArgumentException: Can't get Kerberos realm at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:306) at org.apache.hadoop.security.UserGroupInformation.ensureIni

I have a GetFile processor followed by a PutSFTP. I simulate SFTP connection issue, and then would like to replay the same flow. However, I can't replay at GetFile as it shows "Cannot replay data from Provenance Event because the event does not contain the required Content Claim". PutSFTP has no data provenance event as well since the SFTP failed. Please advise.

I searched in google but not find it,is it possible to create link between HDFS folder to local folder?examplewe want to create link between folder_1 in HDFS to /home/hdfs_mirror local folderHDFS folder:su hdfs $ hdfs dfs -ls /hdfs_home/folder_1Linux local folder:ls /home/hdfs_mirror

Hello, I have been working on upgrading our cluster from CDH5.15 to CDH6.1.0 version. We have CentOS 6.10 operating system and Python version 2.6.6 on our boxes. According to the documentation the python version is compatable for the OS vesion (https://www.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_os_requirements.html#os_requirements) "Cloudera Enterprise 6 requires Python 2.6 or higher on RHEL 6 compatible operating systems".But, when trying to upgrade and launch services (CDH 6.1), HUE service fails with the following error message, 2019-02-25 22:20:31,504 INFO CommandPusher:com.cloudera.cmf.service.AbstractDbConnectionTestCommand: ++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/hadoop-kms ++ export CDH_PARQUET_HOME=/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/parquet ++ CDH_PARQUET_HOME=/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702/lib/parquet ++ export CDH_AVRO_HOME=/opt/cloudera/parcels/CDH-6.1.0-1.cdh6.1.

recently I have upgraded to HDP 3. If I run the stale config, it is not working(failing) to one of the master server(NN runnnig) all other nodes are working fine if I apply stale config.Whatever the services running in this host are failing.even i have restarted ambari server and ambari agent but no luck.Please helpThanks in advance..

I am encountering the error org.apache.hive.service.cli.HiveSQLException: Invalid SessionHandle: SessionHandle while running the %jdbc(hive) interpreter in Apache Zeppelin version 0.8 on HDP (Hortonworks Data Platform) version 3.0.1.