Falcon UI not working

[#1]

I restarted Falcon Server and getting this error:

HTTP ERROR: 503 Problem accessing /index.html. Reason: SERVICE_UNAVAILABLE

[#2]

Please provde Falcon, HDP and Ambari versions, also provide logs. @Sushil Saxena

[#3]

Falcon: 0.6.1.2.3, HDP: 2.3.2.0-2950, Ambari: 2.1.2.1 Which log I need to provide?

Extract from application log:

2016-01-27 01:00:50,873 ERROR - [main:] ~ Failed to initialize service org.apache.falcon.entity.store.ConfigurationStore (ServiceInitializer:49)
org.apache.falcon.FalconException: Unable to restore configurations for entity type PROCESS
        at org.apache.falcon.entity.store.ConfigurationStore.loadEntity(ConfigurationStore.java:189)
        at org.apache.falcon.entity.store.ConfigurationStore.init(ConfigurationStore.java:152)
        at org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
        at org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
        at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
        at org.apache.falcon.Main.main(Main.java:83)
2016-01-27 01:00:50,875 ERROR - [main:] ~ Failed startup of context org.mortbay.jetty.webapp.WebAppContext@4759d881{/,/usr/hdp/current/falcon-server/server/webapp/falcon} (log:87)
java.lang.RuntimeException: org.apache.falcon.FalconException: org.apache.falcon.FalconException: Unable to restore configurations for entity type PROCESS
        at org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:59)
        at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
        at org.apache.falcon.Main.main(Main.java:83)
Caused by: org.apache.falcon.FalconException: org.apache.falcon.FalconException: Unable to restore configurations for entity type PROCESS
        at org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:50)
        at org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
        ... 11 more
Caused by: org.apache.falcon.FalconException: Unable to restore configurations for entity type PROCESS
        at org.apache.falcon.entity.store.ConfigurationStore.loadEntity(ConfigurationStore.java:189)
        at org.apache.falcon.entity.store.ConfigurationStore.init(ConfigurationStore.java:152)
        at org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
        ... 12 more
2016-01-27 01:00:50,879 INFO  - [main:] ~ Started SocketConnector@0.0.0.0:15000 (log:67)

[#4]

Please see the log in my previous response. The output of ps -ef | grep falcon

falcon 13996 1 3 01:00 ? 00:00:31 /usr/jdk64/jdk1.7.0_67/bin/java -Xmx1024m -noverify -Dfalcon.embeddedmq=True -Dfalcon.emeddedmq.port=61616 -Dfalcon.log.dir=/var/log/falcon -Dfalcon.embeddedmq.data=/hadoop/falcon/embeddedmq/data -Dfalcon.home=/usr/hdp/current/falcon-server -Dconfig.location=/usr/hdp/current/falcon-server/conf -Dfalcon.app.type=falcon -Dfalcon.catalog.service.enabled= -cp /usr/hdp/current/falcon-server/conf:/usr/hdp/2.3.2.0-2950/hadoop/conf:/usr/hdp/2.3.2.0-2950/hadoop/lib/*:/usr/hdp/2.3.2.0-2950/hadoop/.//*:/usr/hdp/2.3.2.0-2950/hadoop-hdfs/./:/usr/hdp/2.3.2.0-2950/hadoop-hdfs/lib/*:/usr/hdp/2.3.2.0-2950/hadoop-hdfs/.//*:/usr/hdp/2.3.2.0-2950/hadoop-yarn/lib/*:/usr/hdp/2.3.2.0-2950/hadoop-yarn/.//*:/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/lib/*:/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/.//*:::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/current/falcon-server/server/webapp/falcon/WEB-INF/classes:/usr/hdp/current/falcon-server/server/webapp/falcon/WEB-INF/lib/*:/usr/hdp/current/falcon-server/libext/* org.apache.falcon.Main -app /usr/hdp/current/falcon-server/server/webapp/falcon -port 15000

On Ambari, there is not error on Falcon Server. But when I tried to run Falcon UI, I am getting: HTTP ERROR: 503 Problem accessing /index.html. Reason: SERVICE_UNAVAILABLE

Please Note: I need to Update pig-action.xml to have hive in the share lib config and repackage

falcon-oozie-adaptor-0.6.1.2.3.2.0-2950.jar and replace jar at "/usr/hdp/current/falcon-server/webapp/falcon/WEB-INF/lib” and restart Falcon.

[#5 · Accepted solution]

The issue has been resolved. The entity was corrupt and we restored the falcon Store and WebApp.

[#6]

Hi, having same issue, followed steps in jira https://issues.apache.org/jira/browse/AMBARI-18046?jql=project%20%3D%20AMBARI and still getting This host-level alert is triggered if the Falcon Server Web UI is unreachable - 503. Any insights on what you did to get around this. I am running ambari 2.4 on HDP 2.5 tech preview. Steps from Jira listed here.

STR: 1) Upgrade Ambari from 2.2.1 to 2.4.0 2) Delete Falcon 3) Add Falcon

Result: Falcon UI is unavailable. From Falcon logs: java.lang.RuntimeException: org.apache.falcon.FalconException: Unable to get instance for org.apache.falcon.atlas.service.AtlasService

[#7]

Hi all,

I am getting a warning from Falcon like yours. I am using HDP-2.5.3.0, Ambari Version2.4.2.0 and Falcon 0.10.0. There is no valuable log into falcon.application.log. Falcon warns me like this:

HTTP 503 response from http://hostname:15000 in 0.000s (HTTP Error 503: Service Unavailable)

Do you have any ideas?

[#8]

As HortonWorks partner we have tested this an confirm the error.

HDP-2.5.3.0, Ambari Version2.4.2.0 and Falcon 0.10.0 using distro Ubuntu 14.0.4

HDP-2.5.3.0, Ambari Version2.4.1.0 and Falcon 0.10.0 using distro CentOS 7

If i am not mistaken this problem did NOT occur on HDP 2.5.0.0-1245

During install i get warning on one of the slave nodes

resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdp/current/falcon-client/bin/falcon admin -version' returned 255. ERROR: Unable to initialize Falcon Client object. Cause : Could not authenticate, Authentication failed, URL: http://master2.localhost:15000/api/options?user.name=ambari-qa&user.name=ambari-qa, status: 503, message: Service Unavailable

[#9]

I am also encountering the same issues (HTTP Error 503 when trying to access the Falcon web UI and Ambari Service Check for Falcon is consistently failing).

HDP 2.5.3.0-37, Ambari 2.4.2.0-136, Falcon 0.10.0.2.5.3.0-37 installed on SLES 11 SP3

This problem didn't occur when the cluster was installed with HDP 2.5.0.2 and Ambari 2.4.1.0.

I've opened the following Apache JIRA for the Ambari project for this issue:

https://issues.apache.org/jira/browse/AMBARI-19370

Excerpts from the falcon.application.log:

2017-01-04 14:08:56,436 ERROR - [main:] ~ Failed to initialize service org.apache.falcon.metadata.MetadataMappingService (ServiceInitializer:49)
java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [com.thinkaurelius.titan.core.TitanFactory].

2017-01-04 14:08:56,438 ERROR - [main:] ~ Failed startup of context org.mortbay.jetty.webapp.WebAppContext@96a75da{/,/usr/hdp/current/falcon-server/server/webapp/falcon} (log:87) 
java.lang.RuntimeException: org.apache.falcon.FalconException: java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [com.thinkaurelius.titan.core.TitanFactory].

[#10]

As of HDP 2.5.3, you need to install the Berkeley DB prior to upgrading or installing Falcon. That might be the problem for some of you. You might try the following: [Updated content below]

1. Download the required Berkeley DB implementation file.

   wget –O je-5.0.73.jar http://search.maven.org/remotecontent?filepath=com/sleepycat/je/5.0.73/je-5.0.73.jar

2. Log in to the Ambari server with administrator privileges.

   su – root

3. Copy the file to the Ambari server share folder.

   cp je-5.0.73.jar /usr/share/

4. Set permissions on the file to owner=read/write, group=read, other=read.

   chmod 644 /usr/share/je-5.0.73.jar

5. Configure the Ambari server to use the Berkeley DB driver.

   ambari-server setup --jdbc-db=bdb --jdbc-driver=/usr/share/je-5.0.73.jar

6. Restart the Ambari server.

   ambari-server restart

7. Restart the Falcon service from the Ambari UI.

   You need to have administrator privileges in Ambari to restart a service.

   1. In the Ambari web UI, click the Services tab and select the Falcon service in the left Services pane.
   2. From the Falcon Summary page, click Service Actions > Restart All.
   3. Click Confirm Restart All.

When the service is available, the Falcon status displays as Started on the Summary page.

Further information and manual install instructions are available in an article at https://community.hortonworks.com/articles/78274/prerequisite-to-installing-or-upgrading-falcon.html.