Hello. There is correct way to moving NiFi (config, repositories and certificates) to another directories at same server? I installed the HDF over the HDP through the Ambari and I want moving NiFi folder /var/lib/nifi to /hadoop/nifi (at different disk). I changed paths into the NiFi configs and restart NiFi services, but NiFi Certificate Authority not started. How can I tell nifi where his certificate is now? Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi_ca.py", line 116, in <module> CertificateAuthority().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 375, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi_ca.py", line 94, in start Execute((run_ca_script, params.jdk64_home, ca_server_script, params.nifi_config_dir + '/nifi-certificate-authority.json', params.nifi_ca_log_file_stdout, params.nifi_ca_log_file_stderr, status_params.nifi_ca_pid_file), user=params.nifi_user) File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of '/var/lib/ambari-agent/tmp/run_ca.sh /usr/jdk64/jdk1.8.0_112 /var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.0.0-564/bin/tls-toolkit.sh /hadoop/nifi/conf/nifi-certificate-authority.json /hadoop/nifi/log/nifi-ca.stdout /hadoop/nifi/log/nifi-ca.stderr /var/run/nifi/nifi-ca.pid' returned 1. Timed out while waiting for CA server to start ... View more

Hello! How to monitoring (debugging) hive and tez queries with Ambari 2.7 (HDP-3.0)? If not exists the Hive/Tez views what we should using for this tasks? I want view a query status, resources used, query plan ... ... View more

Hello! Can anyone help me? I upload the CSV files into HDFS and then throw them in the Hive as:   create temporary external table test ( dt TIMESTAMP, uid INT )   row format delimited fields terminated by '|'   stored as textfile   location '/my_files/test';   insert into table my_table partition(name, md) select dt, uid, name, md from test; if the   files are small then everything is fine, but if the   size is large the process falls   on the insert with an error:   Error during job, getting debugging information ... Examining task ID: task_1522750657070_0091_m_000001 (and more) from job job_1522750657070_0091 Examining task ID: task_1522750657070_0091_r_000001 (and more) from job job_1522750657070_0091 Task with the most failures (4): ----- Task ID:   task_1522750657070_0091_m_000002 URL:   http://hadoop-namenode.mech.com:8088/taskdetails.jsp?jobid=job_1522750657070_0091&tipid=task_1522750... ----- Diagnostic Messages for this Task: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask in the YARN logs I see the following:   WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child: java.net.ConnectException: Call From hadoop-datanode02.prod.analytics.wz-ams.lo.mobbtech.com/127.0.1.1 to hadoop- datanode02.prod.analytics.wz-ams.lo.mobbtech.com:22232 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:   http://wiki.apache.org/hadoop/ConnectionRefused   all other messages INFO only    is this the cause of the insert error? what else can I look to solve the problem?   I met the mention that the error code 2 is a memory problem ... Is it so? ... View more

Hello! Can anyone help me? I upload the CSV files into HDFS and then throw them in the Hive as:   create temporary external table test ( dt TIMESTAMP, uid INT ) row format delimited fields terminated by '|' stored as textfile location '/my_files/test';   insert into table my_table partition(name, md) select dt, uid, name, md from test; if the files are small then everything is fine, but if the size is large the process falls on the insert with an error:   Error during job, getting debugging information ... Examining task ID: task_1522750657070_0091_m_000001 (and more) from job job_1522750657070_0091 Examining task ID: task_1522750657070_0091_r_000001 (and more) from job job_1522750657070_0091 Task with the most failures (4): ----- Task ID:   task_1522750657070_0091_m_000002 URL:   http://hadoop-namenode.mech.com:8088/taskdetails.jsp?jobid=job_1522750657070_0091&tipid=task_1522750657070_0091_m_000002 ----- Diagnostic Messages for this Task: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask in the YARN logs I see the following:   WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child: java.net.ConnectException: Call From hadoop-datanode02.prod.analytics.wz-ams.lo.mobbtech.com/127.0.1.1 to hadoop- datanode02.prod.analytics.wz-ams.lo.mobbtech.com:22232 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused   all other messages INFO only    is this the cause of the insert error? what else can I look to solve the problem?   I met the mention that the error code 2 is a memory problem ... Is it so? ... View more