Solved

Can anyone please share with me the Ambari installation documentation on Ubuntu 16.04?

Dear All, I am using this sqoop command for importing Oracle Table into Hive Table sqoop import --connect jdbc:oracle:thin:@123.123.123.112:1500:COL --username admin -P --table db.table1 --hive-import --hive-overwrite --create-hive-table --hive-database hive_db --hive-table table1 --split-by primary_Key After Importing from oracle to Hive Some rows and columns are entirely Showing 'NULL' . ORACLE TABLE HIVE TABLE -ISSUE 1. PRIMARY_KEY (NOT NULL COLUMN)2. There is 0 'NULL' Value 1. SOME ROWS AND COLUMNS ARE NULL2. 35 NULL (ROWS and COLUMS) How to clear it?

step 1 get rsa key from umbari host # cat /root/.ssh/id_rsa step 2 Amabri web UI choose Hosts...Action Add Host. enter ambari.example.com in Target Hosts Field Paste the key from step 1 into where it asks for 'ssh prv key' step 3 click button Register and confirm.... returns Failed ==========================Creating target directory...==========================Command start time 2019-02-05 14:23:08root@ambari.example.com: Permission denied (publickey,password). SSH command execution finishedhost=ambari.example.com, exitcode=255 Command end time 2019-02-05 14:23:08ERROR: Bootstrap of host ambari.example.com fails because previous action finished with non-zero exit code (255) ERROR MESSAGE: root@ambari.example.com: Permission denied (publickey,password).STDOUT: root@ambari.example.com: Permission denied (publickey,password). OK

I found it necessary to troubleshoot a problem enabling kerberos to modify the ambari server and cluster boxes /etc/hostname from name.example.com to name.corp.example.comI also edited the /etc/hosts on all to reflect accordingly.I also ran ambari-server setup-security and option 1 to DISABLE the https.Enter choice, (1-5): 1 Do you want to disable HTTPS [y/n] (n)? y Ambari server URL changed. To make use of the Tez View in Ambari please update the property tez.tez-ui.history-url.base in tez-site Adjusting ambari-server permissions and ownership... NOTE: Restart Ambari Server to apply changes ("ambari-server restart|stop+start")restarted the ambari service #ambari-server restart|Now I get the login page on port 8- butif I try to login as admin the screen just freezes and mouse cursor becomes circle with line through it.Any suggestions welcome to get back to admin able to log in.

I have a NiFi job starting with a GetFile processor configured to run on primary node to avoid duplicate flowfiles. After a NiFi node restart, causing a Primary node re-election, the GetFile processor has created two flowfiles (one expected and one duplicate). I suppose that the GetFile was configured to run with the pre-election primary node and post-election one too, causing those two flowfiles (both flowfiles were processed by different nodes according to NiFi data provenance). Is there a way to avoid this behavior, and is it a NiFi bug ? NB: While the job was running, coordinator and primary nodes were the same. Thanks a lot.

The support matrix states PostgreSQL 10.2 or 9.6 are supported..... Is this a hard and fast rule or can we use 11.1 or 10.2 ???We are getting ready to stand up a new cluster and I wanted to stay on a fairly current version.

Hi,  is it supported to submit a new cluster template to Cloudera Altus Director and use an existing Cloudera Manager installation?  I had a Director template for Cluster A and CM A and it was deployed sucessfully. As the Director server was not needed any more, it was deleted (so the operations is managed by CM, but no changes were made in cluster config). Now I would like to create Cluster B also with Director, but to point to an existing Cloudera Manager A (and thus using the same KDC)    Is this supported?   Thanks    

Dear all!   I have kerberized cluster on Cloudera Express 5.9.3, Java Version: 1.7.0_67. Since I turned on HDFS Data At Rest Encryption using Java Keystore KMS I recieve an error in Pig job    Failed to renew token: Kind: kms-dt, Service: 10.6.1.44:16000, Ident: (kms-dt owner=hdfs, renewer=yarn, realUser=, issueDate=1547451323413, maxDate=1548056123413, sequenceNumber=2, masterKeyId=2)   Yarn log:   Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect Caused by: java.security.UnrecoverableKeyException: Password verification failed   I have checked the pessword by doing keytool -list, the password is correct.   Please, help me to resolve this problem.  

Hello,I have enable kerberos on cluster. I can successfully connect to consumer through kafka-konsole-consumer.But when i try to connect to topic through NiFi ConsumeKafka processor i have error:WARN [Timer-Driven Process Thread-8] o.a.n.p.kafka.pubsub.ConsumeKafka_1_0 ConsumeKafka_1_0[id=504e5811-0168-1000-0000-000024c83cc5] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@51911c24. Will roll back session and discard any partially received data. WARN [kafka-kerberos-refresh-thread-nifi/*@*] o.a.k.c.security.kerberos.KerberosLogin [Principal=nifi/*@*]: TGT renewal thread has been interrupted and will exit.

hi all we are trying to delete service from ambari and we get 500 ERRORwhat is needed in order to fix this issue ?we have ambari cluster with HDP version - 2.6.4 and ambatri - 2.6.1

Is there a way to let Atlas discover existing Hive tables (in HDP 2.6.5)? I've got a couple of Hive external tables that existed before Atlas was enabled. It didn't pick them up, but Atlas does find new tables.I'm trying to get one Hive table in Atlas, but it's not exactly easy to do that for tables, columns, storagedesc and references. There are very few (simple) examples out there. I'm getting one ObjectId is not valid AtlasObjectId error after the other.

Greetings, We are using Cloudera Director, with databases stored on AWS RDS.   Director was using the IP address of the RDS database, as opposed to the hostname. I recently enabled Multi-AZ on the RDS database, and now it has multiple IP addresses.   How can I update the config for the external database in Director so that new clusters are created against the RDS database's hostname instead of the IP address?I tried the unsupported database update mechanism via the director API at `PUT /api/v12/environments/{environment}/databaseServers/{externalDatabaseServer}` (filled in obviously) with a bunch of different json body contents. However, director just responds with a `UnsupportedExternalDatabaseServerUpdateException` regardless of the format I give.  I've tried enabling debug logging to see if there is additional information in application.log, but I can't seem to find the good stuff.   I tried opening director's database, but the tables con

Hi all,I have to import a mysql table into hive environment.The table has a "notes" column with free text, including the possibility for the user to insert CRLF. when I try to sqoop import that table, it seems the sqoop command interprets this CRLF as a end of row.... Then everything, from that point on, is not correctly imported: I mean that the CRLF is the end of the row and the first column of the next row in filled in with the "rest" of free text and so on I use this command: sqoop import --connect jdbc:mysql://myserver:3306/mydb --username myusername --password mypwd--table mytable --hive-import --hive-table mytable --hive-database mydb --as-textfile--warehouse-dir /user/mydb --fields-terminated-by '|' --direct -m 1 --driver com.mysql.jdbc.Driver

Hello all,We are installing 5 nodes HDF cluster using local repository and facing below issue .its seems look like not pointing to right local repo. Can someone please help me .Thanks in advance .-VickyCannot find a valid baseurl for repo: HDF-3.3-repo-152Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stack-hooks/before-INSTALL/scripts/hook.py", line 37, in <module> BeforeInstallHook().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute method(env) File "/var/lib/ambari-agent/cache/stack-hooks/before-INSTALL/scripts/hook.py", line 33, in hook install_packages() File "/var/lib/ambari-agent/cache/stack-hooks/before-INSTALL/scripts/shared_initialization.py", line 37, in install_packages retry_count=params.agent_stack_retry_count) File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/ambari-agent/lib/resource_managemen

Hi All,I need help . I am in the process of installing HDF 3.3.1 cluster and the Ambari version is 2.7.3 My Installation fails while configuring the Ranger with Mysql DBshell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory Error occurred during initialization of VM java.lang.Error: Properties init: Could not determine current working directory. at java.lang.System.initProperties(Native Method) at java.lang.System.initializeSystemClass(System.java:1119)Thanks in advance.-Vicky

I am looking for new auto start function in which I wan't to start kafka when it fails or stale for sometime and it gets started after some interval. Currently, auto start is giving functionality to start service or reboot which I don't wan't. In my case kafka service get stop's after midnight and there is no one to start it hence it affects the business.any suggestions? I have already tried shell script but due to security issues I couldn't deploy shell scripts.

If it is possible, Could you please share steps that how I can do it.Thanks.Versions:Sqoop1.4.7 NiFi1.8.0

Hi ,While connect power bi to cloudera db getting 1107 error . Can I have the solution.

Hello,  I'm trying to run a bash script located at a HDFS folder via oozie workflow. This script has a lot o configurations to be done by the user, and they are passed to the script via a file the script has to read and is put at the same HDFS folder as the script. The script cannot read it, though, and I wonder if I should comply with some specific path configurations for this to work on HDFS.  When this code here is reached:  if [ ! -e `pwd`/vars.sh ] ; then echo "Unable to continue: file vars.sh doesn't exist!" exit 1fithe script exits. Is the "pwd" a problem? I know that it pressuposes that the oozie workflow would respect the path of execution of my script. Is this a mistake? Thanks! 

Hi,  Executing "hdfs dfs -ls /", we are getting aKrbException: Fail to create credential. (63) - No service credseven with a valid kerberos ticket on a particular Kerberos realm. We've been looking into the source code of the JVM and we think we might find the cause with DEBUG level. According to http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html, there is a loglevel option to the hdfs command. So we launch it like this: hdfs dfs -ls / --loglevel DEBUG But that seems to have no effect (only WARN is getting printed). How can we control the loglevel? Thanks in advance

Hello,We are using Cloudbreak rc-2.9 with LDAP integration, and I'd like to know if it's possible to 1) create workspace for other users in automated way2) assigning these users to Workspace (without them logging in yet)3) setting default workspace for userI wasn't able to found anything like that in API docs, maybe some hacking through SQL against CB repository should help?Thanks!

I have requirement to modify the email subject from Ambari for certain alerts. For example, anything related to HDFS, I need to have "CRITICAL HDFS" in the subject line. I went through this link : https://community.hortonworks.com/questions/198815/ambari-email-alert-format.htmlbut it does not talk about customizing the subject for certain alerts.Has anyone has this setup in your environment ?

We have tried several attempts to get the hortonworks platform to work with kerberos using the ambari automated install. the errors are below. any advice to offer is appreciated. Tried to install manually the krb5-user but seems I am getting broken package errors, not sure of that was a good approach afer the automated failed.2019-01-28 22:35:23,105 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}2019-01-28 22:35:23,142 - Skipping installation of existing package unzip2019-01-28 22:35:23,143 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}2019-01-28 22:35:23,177 - Skipping installation of existing package curl2019-01-28 22:35:23,177 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}2019-01-28 22:35:23,211 - Skipping installation of existing package hdp-select2019-01-28 22:35:23,214 - The repository with version 3.1.0.0-78 for this command has been marked as resolved. It will be used to report the v

I've installed CDH-5.16.1-1.cdh5.16.1.p0.3 on SLES 12.3 in "single user mode". I have some services runningcloucera-scm-servercloudera-scm-agentCloudera Management Service (Alert Publisher, Event Server, Host Monitor, Service Monitor)ZookeeperWhen I try to start HDFS, Secondary Name Node and Data Node seems ok, but Name Node fails with this error:  2019-01-25 17:21:52,016 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at org.apache.hadoop.ipc.Server.start(Server.java:2696) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.start(NameNodeRpcServer.java:448) at org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:713) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:692) at org.

Hello, I am trying to totally remove Kerberos and re-enable it. I have followed all the rollback steps as described in several posts: https://community.cloudera.com/t5/Cloudera-Manager-Installation/Disabling-Kerberos/td-p/19654https://stackoverflow.com/questions/29744821/how-to-disable-hadoop-kerberoshttp://bigdata-tips.blogspot.com/2017/03/how-to-disable-kerberos-in-cloudera-cdh.html I seems that kerberos is indeed disabled BUT I cannot get the Enable Kerberos button back:  What should I do to re-enable it? Thank you,Gerasimos