@Jay Kumar SenSharma So we moved ahead and used another IP: XX.XX.XX.51 (which has internet access) to install HDP and then reverted HDP back to its original IP: XX.XX.XX.160 (which does't have internet access) and did the same to install HDF using IP: XX.XX.XX.51 (which has internet access) and then reverted HDF back to its original IP: XX.XX.XX.161 (which doesn't has internet access). When HDP and HDF were on IP: XX.XX.XX.51, all the services were up and running and restart-able but the moment those were reverted back i.e. HDP and HDF to those original IPs, everything got hung. Now services are in start mode as were when were on IP: XX.XX.XX.51 and we can't stop & restart and other things like hive is not accessible, NiFi UI does't appear etc. Please advice. ... View more

@Jay Kumar SenSharma Yup permissions are good. The problem is, there is no index.html file present in/repo/repodata directory. ... View more

@Jay Kumar SenSharma Hi, Apologies for delay. There you go with WebServer Log. -------------------------------------------------------------------------------------------------------------------------------------------- [root@ufm httpd]# tail error_log [Sun Sep 23 16:18:17.112853 2018] [core:error] [pid 29345] (13)Permission denied: [client10.200.40.51:53472] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path [Sun Sep 23 16:18:19.399155 2018] [core:error] [pid 29345] (13)Permission denied: [client10.200.40.51:53472] AH00035: access to /repo/ambari/centos7/index.html denied (filesystem path '/var/www/html/repo/ambari/centos7/index.html') because search permissions are missing on a component of the path, referer:http://10.200.40.160/repo/ [Sun Sep 23 16:18:20.584500 2018] [core:error] [pid 29345] (13)Permission denied: [client10.200.40.51:53472] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path, referer:http://10.200.40.160/repo/ambari/ [Sun Sep 23 16:18:22.762022 2018] [core:error] [pid 29345] (13)Permission denied: [client10.200.40.51:53472] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path, referer:http://10.200.40.160/repo/ambari/ [Sun Sep 23 16:18:23.613018 2018] [core:error] [pid 29345] (13)Permission denied: [client10.200.40.51:53472] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path, referer:http://10.200.40.160/repo/ambari/ [Sun Sep 23 16:18:41.563016 2018] [core:error] [pid 28765] (13)Permission denied: [client10.200.40.51:53473] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path, referer:http://10.200.40.160/repo/ambari/ [Sun Sep 23 18:11:29.286927 2018] [core:error] [pid 28765] (13)Permission denied: [client10.200.40.51:53653] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path [Sun Sep 23 18:13:02.298494 2018] [core:error] [pid 29345] (13)Permission denied: [client10.200.40.51:53660] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path, referer:http://ufm.hdp01.com/repo/HDP/ [Sun Sep 23 18:14:16.053361 2018] [core:error] [pid 1780] (13)Permission denied: [client10.200.40.51:53663] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path, referer:http://ufm.hdp01.com/repo/HDP-UTILS/ [Sun Sep 23 18:15:56.767202 2018] [core:error] [pid 1781] (13)Permission denied: [client10.200.40.51:53666] AH00035: access to /repo/repodata/index.html denied (filesystem path '/var/www/html/repo/repodata/index.html') because search permissions are missing on a component of the path, referer:http://ufm.hdp01.com/repo/HDP-GPL/ -------------------------------------------------------------------------------------------------------------------------------------------- Note: As this is offline line installation, we don't have index.html any where. Please advice. ... View more

@Jay Kumar SenSharma @Geoffrey Shelton Okot It seems, repodata/repomd.xml is the culprit. Please advice its uses and content with no-internet access as its content is; <?xml version="1.0" encoding="UTF-8"?> <repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm"> I feel above mentioned URLs should not go on internet. ... View more

@Jay Kumar SenSharma We moved ahead by disabling rest the repos and enabling ambari one but got stuck at cluster building. Note: When we paste this URL: http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml it does work. Following failed at installing this service: Timeline Service V1.5 Install failure: repodata/repomd.xml from HDP-3.0: [Errno 256] No more mirrors to try. http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable http://ufm.hadoop.com/repo/HDP/centos7/3.0.0.0-1634/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable 2018-09-18 16:05:51,775 - call[('ambari-python-wrap', u'/usr/bin/hdp-select', 'versions')] {} 2018-09-18 16:05:51,805 - call returned (0, '') 2018-09-18 16:05:51,805 - The 'hadoop-yarn-timelineserver' component did not advertise a version. This may indicate a problem with the component packaging. Command failed after 1 tries I AM SURE, WE DO NOT NEED INTERNET as a mandatory pre-requisite? ... View more

@Shu pmresult-67109368-60-201804260000-201804260100-ori.zip (Please unzip, its a csv file) sample-xml.xml Same result :-(. Also attaching XML and Source data file for testing. Empty Queue Avro Schema: { "type":"record", "name":"jazz", "fields":[ {"name":"hw_3g_result_time", "type":["null","string"]}, {"name":"hw_3g_granularity_period", "type":["null","string"]}, {"name":"hw_3g_bsc_name", "type":["null","string"]}, {"name":"hw_3g_bsc_type", "type":["null","string"]}, {"name":"hw_3g_cell_name", "type":["null","string"]}, {"name":"hw_3g_cell_id", "type":["null","string"]}, {"name":"hw_3g_cell_index", "type":["null","string"]}, {"name":"hw_3g_reliability", "type":["null","string"]}, {"name":"vsrabattestabamr", "type":["null","string"]}, {"name":"vsrabsuccestabcsamr", "type":["null","string"]}] } ... View more

@Shu It got successful but out out was empty. Source Header: Aim is to convert mentioned last 2 columns in lower case. hw_3g_result_time,hw_3g_granularity_period,hw_3g_bsc_name,hw_3g_bsc_type,hw_3g_cell_name,hw_3g_cell_id,hw_3g_cell_index,hw_3g_reliability,VSRABAttEstabAMR,VSRABSuccEstabCSAMR CSVWriter Properties Avro Schema UpdateRecord Property Source Header: Aim is to convert mentioned last 2 columns in lower case. hw_3g_result_time,hw_3g_granularity_period,hw_3g_bsc_name,hw_3g_bsc_type,hw_3g_cell_name,hw_3g_cell_id,hw_3g_cell_index,hw_3g_reliability,VSRABAttEstabAMR,VSRABSuccEstabCSAMR ... View more

@Shu UpdateRecord Error: Can not write to schema, schema is unknow. UpdateRecord Properties CSV Reader CSV Writer ... View more

@Shu Data coming in ConvertRecord from CSV to AVRO schema. Avro_Schema Outgoing Data As you can see, incoming data have column name in different format. I changed the Avro Schema to reflect the same column name format but then it loaded NULL values in all the columns. ... View more