question Re: problem with making external cluster on HDP sandbox 2.4 in Support Questions

problem with making external cluster on HDP sandbox 2.4

hanna_lee — Tue, 17 May 2016 16:12:23 GMT

As following HDP falcon tutorial, I succeeded to test mirroring function from sandbox cluster to sandbox cluster.

Then using falcon mirror function, I want to make a replication from sandbox cluster to external cluster what I have. (the name is "edumst01.sbp.com")

But I failed to make a cluster entity. here are the information what I write on falcon web ui and the falcon.applicaiton.log message.

Please check it and tell me the right way to make a external cluster.

For your information, my cluster don't have falcon, jmx, only is installed hdfs, yarn, oozie.

Readonly hftp://edumst01.sbp.com:50070

Write hdfs://edumst01.sbp.com:8020

Execute edumst01.sbp.com:8050

Workflow http://edumst01.sbp.com:11000/oozie/

Messaging tcp://sandbox.hortonworks.com:61616?daemon=true

2016-05-17 08:53:21,308 INFO - [679586926@qtp-1820624068-3 - 8835df07-48b3-4f5b-9780-53efb2a9429e:ambari-qa:POST//entities/submit/cluster] ~ Retrying connect to server: edumst01.sbp.com/112.175.246.34:8050. Already tried 41 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS) (Client:869)

2016-05-17 08:53:23,328 INFO - [679586926@qtp-1820624068-3 - 8835df07-48b3-4f5b-9780-53efb2a9429e:ambari-qa:POST//entities/submit/cluster] ~ Retrying connect to server: edumst01.sbp.com/112.175.246.34:8050. Already tried 42 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS) (Client:869)

Re: problem with making external cluster on HDP sandbox 2.4

bvellanki — Tue, 17 May 2016 23:14:04 GMT

@Hanna Lee

Please make sure the execute endpoint is using port 8050. You can find this in Yarn configs under yarn.resourcemanager.address. This port can be 8032 on some nodes, but I think sandbox uses 8050. Once you have the right port, please restart falcon and try submitting cluster again. The restart is required because there could be a stale FileSystem object in Falcon server.

Re: problem with making external cluster on HDP sandbox 2.4

hanna_lee — Wed, 18 May 2016 08:34:25 GMT

Thank you! After I changed the execute port to 8032, I passed the step to validate a yarn resource manager.

But I still have a problem. I got a error message while I make the Locations. Here are the information what I write on falcon web UI and the falcon.application.log message.

On my external cluster, I made these two staging/working directory with ambari-qa user.

staging: /user/falcon/sbpRndCluster/staging

temp: /tmp

working: /user/falcon/sbpRndCluster/working

2016-05-18 01:28:54,989 INFO - [1431190494@qtp-1526367312-10 - cf529336-f2e6-4078-805c-e60346e79ed7:ambari-qa:POST//entities/submit/cluster] ~ Creating FS impersonating user ambari-qa (Hado opClientFactory:196)

2016-05-18 01:28:55,003 ERROR - [1431190494@qtp-1526367312-10 - cf529336-f2e6-4078-805c-e60346e79ed7:ambari-qa:POST//entities/submit/cluster] ~ Unable to persist entity object (AbstractEntit yManager:198) org.apache.falcon.entity.parser.ValidationException: Unable to validate the location with path: /apps/falcon/primaryCluster/staging for cluster:sbpRndCluster due to transient failures

[root@edumst01 ~]# hadoop fs -ls /user/falcon/sbpRndCluster Found 2 items

drwxrwxrwx - ambari-qa supergroup 0 2016-05-17 17:24 /user/falcon/sbpRndCluster/staging

drwxrwxrwx - ambari-qa supergroup 0 2016-05-17 17:24 /user/falcon/sbpRndCluster/working

Re: problem with making external cluster on HDP sandbox 2.4

hanna_lee — Thu, 19 May 2016 09:45:07 GMT

When I set up new external cluster on HDP sandbox, readonly/write/execute path is new cluster info but workflow, message path is sandbox info. Among external cluster and sandbox cluster, where can I make staging/temp/working directory?

Re: problem with making external cluster on HDP sandbox 2.4

hanna_lee — Thu, 19 May 2016 09:54:23 GMT

And Falcon is only installed on sandbox, now my cluster.

Re: problem with making external cluster on HDP sandbox 2.4

bvellanki — Fri, 20 May 2016 03:44:45 GMT

The working and staging dir should have the following permissions.

hadoop fs -chown -R falcon /user/falcon/sbpRndCluster/

hadoop fs -chmod -R 777 /user/falcon/sbpRndCluster/staging

hadoop fs -chmod -R 755 /user/falcon/sbpRndCluster/working

Once you set these permissions, submitting cluster should be straightforward. When you see errors like "Unable to validate the location with path: /apps/falcon/primaryCluster/staging for cluster:sbpRndCluster due to transient failures", can you please share the logs from falcon.application.log, this will give us more information about what is happening.

Re: problem with making external cluster on HDP sandbox 2.4

hanna_lee — Fri, 20 May 2016 07:49:09 GMT

I solved the problem!! I added the property "hadoop.proxyuser.falcon.groups", "hadoop.proxyuser.falcon.hosts" on my external cluster core-stie.xml.