We are planing to upgrade HDP from 2.6.1 to 3.1 along with new servers . Does any one have documentation or scripts which will allow us to create the same structure of folders at the destination version for hive,hdfs,hbase. ... View more

When i try to upgrade ambari from 2.6.1 to 2.7 its failing when i do server upgrade . please find the error below 2018-09-02 18:19:51,423 ERROR [main] SchemaUpgradeHelper:473 - Exception occurred during upgrade, failed org.apache.ambari.server.AmbariException: Trying to create a ServiceComponent not recognized in stack info, clusterName=HDF_LAB, serviceName=AMBARI_INFRA, componentName=INFRA_SOLR, stackInfo=HDF-3.1 at org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executePreDMLUpdates(SchemaUpgradeHelper.java:223) at org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:454) Caused by: java.lang.RuntimeException: Trying to create a ServiceComponent not recognized in stack info, clusterName=HDF_LAB, serviceName=AMBARI_INFRA, componentName=INFRA_SOLR, stackInfo=HDF-3.1 at org.apache.ambari.server.state.ServiceComponentImpl.updateComponentInfo(ServiceComponentImpl.java:154) at org.apache.ambari.server.state.ServiceComponentImpl.<init>(ServiceComponentImpl.java:178) at org.apache.ambari.server.state.ServiceComponentImpl$$EnhancerByGuice$$1d58f0d8.<init>(<generated>) at org.apache.ambari.server.state.ServiceComponentImpl$$EnhancerByGuice$$1d58f0d8$$FastClassByGuice$$280022dc.newInstance(<generated>) at com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:265) at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015) at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:776) at com.sun.proxy.$Proxy21.createExisting(Unknown Source) at org.apache.ambari.server.state.ServiceImpl.<init>(ServiceImpl.java:181) at org.apache.ambari.server.state.ServiceImpl$$EnhancerByGuice$$861a4bc.<init>(<generated>) at org.apache.ambari.server.state.ServiceImpl$$EnhancerByGuice$$861a4bc$$FastClassByGuice$$6de922a8.newInstance(<generated>) at com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:265) at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015) at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:776) at com.sun.proxy.$Proxy15.createExisting(Unknown Source) at org.apache.ambari.server.state.cluster.ClusterImpl.loadServices(ClusterImpl.java:451) at org.apache.ambari.server.state.cluster.ClusterImpl.<init>(ClusterImpl.java:344) at org.apache.ambari.server.state.cluster.ClusterImpl$$EnhancerByGuice$$13ee505a.<init>(<generated>) at org.apache.ambari.server.state.cluster.ClusterImpl$$EnhancerByGuice$$13ee505a$$FastClassByGuice$$73468d7.newInstance(<generated>) at com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:265) at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085) at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015) at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:776) at com.sun.proxy.$Proxy11.create(Unknown Source) at org.apache.ambari.server.state.cluster.ClustersImpl.loadClustersAndHosts(ClustersImpl.java:300) at org.apache.ambari.server.state.cluster.ClustersImpl.safelyLoadClustersAndHosts(ClustersImpl.java:266) at org.apache.ambari.server.state.cluster.ClustersImpl.getClustersByName(ClustersImpl.java:182) at org.apache.ambari.server.state.cluster.ClustersImpl.getClusters(ClustersImpl.java:677) at org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.getCheckedClusterMap(AbstractUpgradeCatalog.java:494) at org.apache.ambari.server.upgrade.UpgradeCatalog262.fixDesiredStack(UpgradeCatalog262.java:81) at org.apache.ambari.server.upgrade.UpgradeCatalog262.executePreDMLUpdates(UpgradeCatalog262.java:69) at org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.preUpgradeData(AbstractUpgradeCatalog.java:975) at org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executePreDMLUpdates(SchemaUpgradeHelper.java:220) ... 1 more ... View more

Team, I am running into below error when i try to start ambari agent post upgrade to HJDF3.1 from HDF 2.1. # ambari-server start Using python /usr/bin/python Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... DB configs consistency check failed. Run "ambari-server start --skip-database-check" to skip. If you use this "--skip-database-check" option, do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See "/var/log/ambari-server/ambari-server-check-database.log" for more details on the consistency issues. Ambari database consistency check finished In log i see the below error 2018-02-28 14:38:31,085 INFO - ******************************* Check database started ******************************* 2018-02-28 14:38:35,189 INFO - Checking for configs not mapped to any cluster 2018-02-28 14:38:35,261 INFO - Checking for configs selected more than once 2018-02-28 14:38:35,272 INFO - Checking for hosts without state 2018-02-28 14:38:35,273 INFO - Checking host component states count equals host component desired states count 2018-02-28 14:38:35,291 INFO - Checking services and their configs 2018-02-28 14:38:36,789 INFO - Processing HDF-2.1 / RANGER 2018-02-28 14:38:36,790 INFO - Processing HDF-2.1 / AMBARI_INFRA 2018-02-28 14:38:36,790 INFO - Processing HDF-2.1 / AMBARI_METRICS 2018-02-28 14:38:36,790 INFO - Processing HDF-2.1 / KAFKA 2018-02-28 14:38:36,790 INFO - Processing HDF-2.1 / NIFI 2018-02-28 14:38:36,790 INFO - Processing HDF-2.1 / ZOOKEEPER 2018-02-28 14:38:36,791 ERROR - Required config(s): nifi-logsearch-conf is(are) not available for service NIFI with service config version 20 in cluster HDF_LAB ... View more

I am getting below error when trying to connect to kerberos hbase. i have given all the details that are required . Any help would be greatful. ... View more

I am not able to run the below query using nifi . insert overwrite local directory '/hadoop_shared/xxx/xxx/xxx/' row format delimited fields terminated by '|' null defined as '' select xxx; Error :- Error message:PutHiveQL[id=8e81378c-822c-1a67-8f1d-461f14368aea] Failed to update Hive for StandardFlowFileRecord[uuid=6874a32d-5c58-4454-8963-eaa20e67094b,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1516598821063-18354, container=default, section=946], offset=119486, length=493],offset=0,name=23963691567831492.avro.avro.avro.avro.avro.avro,size=493] due to org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [bdcontour2] does not have [WRITE] privilege on [/hadoop_shared/*]; it is possible that retrying the operation will succeed, so routing to retry: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [bdcontour2] does not have [WRITE] privilege on [/hadoop_shared/*] ... View more

Team i have an requirement where NIFI should able to trigger an oozie coordinator which would schedule the workflow for every 15 mins upon receiving http call . i could see the API to trigger the workflow but not the coordinator . ... View more

i am getting syslogs from our filewall team to NIFI server, but NIFI processor is not able to read them .I can see the logs are being forwarded to the server on the port 7000. [root@catl0xlbxxxx nifi]# tcpdump -lnn -i eth0 'port 7000' tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 11:46:19.294809 IP 10.64.213.56.33684 > 10.64.56.111.7000: rx type 111 (1472) 11:46:19.294904 IP 10.64.213.56.33684 > 10.64.56.111.7000: rx type 111 (1472) 11:46:19.294915 IP 10.64.213.56.33684 > 10.64.56.111.7000: rx type 111 (1472) 11:46:19.294922 IP 10.64.213.56.33684 > 10.64.56.111.7000: rx type 111 (1472) 11:46:19.295264 IP 10.64.213.56.33684 > 10.64.56.111.7000: rx type 111 (1472) 11:46:19.295544 IP 10.64.213.56.33684 > 10.64.56.111.7000: rx type 111 (1472) ... View more

Team, We are trying to post messages to nifi server on SSL enabled port . we are getting certificate error . please find the error below . [bbhimava@dvtXXX ccm_json]$ curl -X POST -T /tmp/nifi/input/ccm_json/7c208164-f4e2-42c4-9988-69dfaec1f3fe https://dvtcxxxx.corp.cox.com:7000/xxxxxx curl: (60) Peer certificate cannot be authenticated with known CA certificates More details here: http://curl.haxx.se/docs/sslcerts.html curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option. If this HTTPS server uses a certificate signed by a CA represented in the bundle, the certificate verification probably failed due to a problem with the certificate (it might be expired, or the name might not match the domain name in the URL). If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option. ... View more

am getting below error when trying to run inbuilt kafka connector script . command :- bin/connect-standalone.sh config/connect-standalone.properties config/connect-file-source.properties config/connect-file-sink.properties error :- [2017-06-12 13:46:10,282] DEBUG EXCEPTION (org.eclipse.jetty.servlet.ServletHandler:875) java.lang.NoSuchMethodError: javax.ws.rs.core.Application.getProperties()Ljava/util/Map; at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:331) at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:392) at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:177) at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:369) at javax.servlet.GenericServlet.init(GenericServlet.java:241) ... View more