Solved

Hi.I am trying to install Ambari server on a three node cluster with one namenode and other two as data nodes.Ambari Version: 2.7.3.0HDP Version: 3.1.0Linux Version: RedHat 7  I am able to connect to the internet through each server and hence using public repository to install ambari server. I have also the necessary authentications from cloudera for ambari and hdp repos. I have followed instructions as mentioned in the following urlhttps://docs.cloudera.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/download_the_ambari_repo_lnx7.html. My repository base urls are as followsAmbari: https:///LOGIN:PASSWORD@https://archive.cloudera.com/p/ambari/2.x/2.7.3.0/centos7/HDP: http:///LOGIN:PASSWORD@archive.cloudera.com/p/HDP/3.x/3.1.0.0/centos7HDP Utils: http://LOGIN:PASSWORD@archive.cloudera.com/p/HDP-UTILS/1.1.0.22/repos/centos7I am using the same above urls in Select Version step of installaton wizard and checked on Public Repository as below

I installed NIFI in my system but when I tried to check the status using nifi.sh status under my bin directory it gives the below error:Request you to please suggest a solution for this.Thanks!

There are user directories under hdfs:///user/ When a user `foo` retires, I need to delete a root directory for the user: hdfs:///user/foo However, it occasionally fails because of the snapshottable directories under the user root directory. Checking all the subdirectories whether it's snapshottable, disallowing snapshottable subdirectories , and then deleting the user's root directory doesn't seem to be the best way.(Or if this is the best way, I cannot come up with a simple code..) Is there a command to disallow snapshots for all the subdirectories? How can I effectively delete a directory that might have snapshots?  

Is there a way to test MIT kerberos high availability functionality?Any approaches?thanks in advance.

I would like to run Hbase major compaction on particular yarn queue. so that way the compaction activity will not impact the other resources in the cluster. Do we have option to so.     

Facing these errors:  PLEASE SOMEONE HELP IT OUT.THANK YOU

I am having a CDH 6.3.2 cluster and would like to backup Hbase table in an incremental fashion.Also need the backup to be done on AWS S3. Please suggest. Thanks in advance. -Aman

hello. I'm using on apache ambari 2.7.5.0 and hdp 3.1.4.0. After installing kerberos on ambari, hdfs, hive, and hbase were successfully authenticated using keytab and then used services. However, if I use yarn to run wordcount, I get an error. In the case of the file permission executing wordcount, the same was confirmed, and in the case of the user name authenticated using keytab, the same was confirmed. When running wordcount using yarn, the same error message as when yarn node -list -all is output is displayed. Below is the error message printed by 'yarn node -list -all.'   21/07/21 14:24:03 INFO retry.RetryInvocationHandler: java.io.IOException: DestHost:destPort hadoop1:8050 , LocalHost:localPort hadoop1/192.168.22.168:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS], while invoking ApplicationClientProtocolPBClientImpl.getClusterNodes over null after 24 failover attempts. Tr

I have to create one attribute with the following details:1. There are two attributes are flowing from EvaluateJson Proecessor :   Location1 = $.place1 and Location2 = $.place22. There is one string like folder1 is to be append in these attributes as mentioned below:Place : /folder1/Location1/Location2 I am trying with the below expression:path: ${path: append("folder1/"):append(${Location1}):append('/'):append(${Location2})}Please let me know where I am doing wrong. Early reply will be highly appreciated. thanks & regards 

Hi! I have an array with dates and I want to insert it into my flowFile through a groovy script but I can't find a way to do it, could you help me?that's my script with which I get dates.and this is my groovy script but it always goes to the queue of failure.

recently ambari server logs are showing some warnings /var/log/ambari-server/ambari-server.log 02 Jul 2021 18:43:52,514 INFO [ambari-client-thread-792188] AmbariAuthToLocalUserDetailsService:109 - Translated knox/<knox_gateway>@<REALM> to knox using auth-to-local rules during Kerberos authentication.02 Jul 2021 18:43:52,515 WARN [ambari-client-thread-792188] AmbariAuthToLocalUserDetailsService:143 - Failed find user account for user with username of knox during Kerberos authentication.02 Jul 2021 18:43:52,516 WARN [ambari-client-thread-792188] AmbariKerberosAuthenticationFilter:149 - Negotiate Header was invalid: Negotiate YIIDlAYGKwYBBQUCoIIDiDCCA4SgDTALBgkqhkiG9xIBAgKhBAMCAXaiggNrBIIDZ2CCA2MGCSqGSIb3EgECAgEAboIDUjCCA06gAwIBBaEDAgEOogcDBQAgAAAAo4IB/WGCAfkwggH1oAMCAQWhMxsxMjZGNURFMDEtNUU0MC00RDhBLTk4QkQtQTQzNTNCN0JGNUUzLkRBVEFMQUtFLk9WSKJPME2gAwIBAKFGMEQbBEhUVFAbPG92aC1lbm9kZTYuMjZmNWRlMDEtNWU0MC00ZDhhLTk4YmQtYTQzNTNiN2JmNWUzLmRhdGFsYWtlLm92aKOCAWYwggFioAMCARKh

Hi All, We have Hue (version 4.2.0) in our prod environment and it is coupled with our LDAP. We manually onboard new users by going to "Manage Users" -> "Add/Sync Manage Users". Here, we add the LDAP ID and later when the user gets added. Once the user is added, we again search the newly added LDAP user, click on that and go to "Profiles and Group". Here, we select "temporary-access" so that User can able to access the UI. Without Temporary-access selection, user will not be able to see the UI. We want to get this automated by writing a Py program and we are looking for the right REST API for adding users to Hue and give him/her "temp-access" in profiles and group. The following link wasn't helpful - http://cloudera.github.io/hue/latest/developer/api/ Please assist us. I would highly appreciate any form of assistance or help. Regards,Shesh Kumar

Flow: QueryDatabaseTable >> PutDatabaseRecord (using the same MS SQL table) Original Date: '2020-01-01 01:00:00.000'Saved Date: '2019-12-31 22:00:00.000' Another:Original Date: '1753-01-01 00:00:00.000' (SQL Server minimum date)Saved Date: Routing to failure.: The conversion of a datetime2 data type to a datetime data type resulted in an out-of-range value. My timezone: (GMT-3) 

Hi We are periodically getting errors in our resource manager logs, which then cause no Spark jobs to be accepted until we restart the YARN ResourceManager.We are on HDP 3.1.4 and we have Kerberos enabled.  The stack trace is below. Does anyone have ideas, please?    2021-05-08 04:15:59,407 ERROR yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(68)) - Thread Thread[Thread-20,5,main] threw an Exception.java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.ResourceCommitRequest.<init>(ResourceCommitRequest.java:101) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.createResourceCommitRequest(CapacityScheduler.java:2848) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.submitResourceCommitRequest(CapacityScheduler.java:2685) at org.apache.hadoop.yarn.server.resourceman

I have a json like this coming from an input port: {"url": "blablabla","keyword": "foo"} then I have to generate a new json to pass to a post call. The new json is something like this: { "requests": [ { "source": "blablabla", "params": { "keywords": [ "something" ], "sub-field1": "1" } } ], "field1": "1", "field2": "2", "field3": false} where the array keywords should be replaced with a new array with the value of the previous Json ("foo"). The resulting is:   { "requests": [ { "source": "blablabla", "params": { "keywords": [ "foo" ], "sub-field1": "1" } } ], "field1": "1", "field2": "2", "field3": false} then i pass this json to a invokehttp to call the post.  Could you please explain me ho

I used to have some tables in HIVE but after an incident HIVE went down and some new tables erased and old tables went back to a previous schema state. What I want, is to ask if it is there any option to backup tables' schemas into MySQL which stores HIVE metastore in order to retrieve them if there occurred any incident again.

"The conversion from UNKNOWN to CHAR is unsupported."I'm using only 2 processors:QueryDatabaseTable (reading UniqueIdentifier Column) >> PutDatabaseRecord (writing in the same table and column)Table:CREATE TABLE TableTest(OID [uniqueidentifier] NULL,Bit1 [bit] NULL)QueryDatabaseTable:PutDatabaseRecord:Error:The conversion from UNKNOWN to CHAR is unsupported.Supposedly, Nifi's internal array is inverting the columns, putting the bit column in place of the UniqueIdentifier column, producing this "Unknown to char" error. Bug.I tried to install Nifi 1.14 but, it keeps asking for username and password and there is no ".bat" to generate username and password, only the shell ".sh". That is, it only works on Linux.

I'm quite new in Apache Nifi but I'm encountering some issues trying to connect a kafka microservice (with a producer) with Apache nifi consumer.Basically, I have a docker-compose like this: zookeeper: container_name: zookeeper_test image: wurstmeister/zookeeper #zookeeper:3.5.7 ports: - 2181:2181kafka: container_name: kafka_test image: wurstmeister/kafka #:2.13-2.6.0 ports: - 9092:9092 environment: KAFKA_ADVERTISED_HOST_NAME: kafka KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true' KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 depends_on: - zookeeper - kafkauikafkaui: container_name: kafka-ui_test image: provectuslabs/kafka-ui:latest environment: - KAFKA_CLUSTERS_0_NAME=kafka - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092 - KAFKA_CLUSTERS_0_ZOOKEEPER= zookeeper:2181 ports: - 6789:8080test: container_name: test build: context: ./test dockerfile: Dockerfile depends_on: - kafka command: python test.pytest is my producer:

Hello, Is there a way to find which kudu table is stored in which tablet server ? Thanks Amn

Please help me out in this scenario:1. I have to create a new project. The details are accessed using GetFileProcessor.2. I have passed two input to Invokehttp Processor(one Authorization token using CustomField and other as data to be used for POST action). However, this process is not working as expected. Please find the attached screenshot for reference. Data To be used for POST:  Early Response much appreciated!Thanks!

Hello Team, can you please advise how to replace nulls by a default value? select concat(cast(glb.budget_version_id as string)," ", cast(glc.segment1 as string)) as balidfrom oracle_financial.gl_balances glb,oracle_financial.gl_code_combinations glc,oracle_financial.gl_ledgers glg ; If I add a null to a value, null will be returned. I tried NVL it is not working. Thanks, Roshan 

Hi, We are processing ZIP file contains multiple timestamp files (.hiveql,.csv) in distributed manner .We check the file extension whether it is .hql or .csv then we route the file to execute it PutHiveQL and PutHDFS processor respectively.The files(timestamp order starts with for example t1 or system timestamp)  below contains in ZIP file to be extracted and processed in order.table_info.zip  table_info_t1.hqltable_info_t1_1.csvtable_info_t1_2.csvtable_info_t2.hqltable_info_t2_1.csvtable_info_t2_2.csvtable_info_latest.hqltable_info_latest.csv   Please find the below NiFi flow and RouteonAttribute property  Is there any way to make us to wait first puthivesql executes first and give indication to putHDFS execution next for each timestamp file one by one order. Can we group each timestamp files into group and process the .hql file and the put .csv file into HDFS?  @Nifi

I want to download http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo.But got ERROR 403: Forbidden.Any help is appreciated!

Hello! Can you kindly help me in an NiFi upgrade error? I followed this tutorial: https://cwiki.apache.org/confluence/display/NIFI/1.x.0+to+1.x.0+Upgrade Following it, i stopped my processors, waited until my flows had no queued flowfiles, stopped old nifi services, changed property names to my new nifi version, moved the flow.xml.gz to my new installation and started the services in the new version. After this, i get the error shown bellow:  Is this a regular mistake? Do you know any alternatives in this case? Thank you,Lucas  

use script ：  /opt/module/cm/cm-5.16.2/share/cmf/schema/scm_prepare_database.sh  Error：  Mon Jul 12 17:30:10 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.[ main] DbProvisioner ERROR Exception when creating/dropping database with user 'temp' and jdbc url 'jdbc:mysql://localhost/?useUnicode=true&characterEncoding=UTF-8'com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 287 milliseco