Falcon UI not working

[#1]

I restarted Falcon Server and getting this error:

HTTP ERROR: 503 Problem accessing /index.html. Reason: SERVICE_UNAVAILABLE

[#2]

Cut'n'pasteable / idempotent:

su - falcon
l_hdp_ver=$(ls -d /usr/hdp/[0-9]* | head -n 1 | sed -e 's#^.*/\([0-9].*\)#\1#')
wget http://download.oracle.com/maven/com/sleepycat/je/5.0.73/je-5.0.73.jar
yes | cp je-5.0.73.jar /usr/hdp/$l_hdp_ver/falcon/webapp/falcon/WEB-INF/lib
chmod 644 /usr/hdp/$l_hdp_ver/falcon/webapp/falcon/WEB-INF/lib/je-5.0.73.jar

[#3]

I also just hit this issue after upgrading to 2.5.3.0-37

[#4]

You need to register download and register the bdb jar with ambari server before starting Falcon server. This is required from HDP 2.5 or later now The following steps would help

1. wget -O je-5.0.73.jar http://search.maven.org/remotecontent?filepath=com/sleepycat/je/5.0.73/je-5.0.73.jar
2. cp je-5.0.73.jar /usr/share/
3. chmod 644 /usr/share/je-5.0.73.jar
4. ambari-server setup --jdbc-db=bdb --jdbc-driver=/usr/share/je-5.0.73.jar
5. ambari-server restart
6. Restart Falcon service

[#5]

It worked Perfectly Venkat R...

[#6]

Hi Venkat, I did the above, but the issue still persists on my cluster. Do you have any idea what else might be wrong or what's worth checking? Basically I get the same error whether I try to run a service check or see the falcon version

ERROR: Unable to initialize Falcon Client object. Cause : Could not authenticate, Authentication failed, URL: <a href="http://<host>:15000/api/options?user.name=falcon&user.name=falcon,">http://:15000/api/options?user.name=falcon&user.name=falcon,>:15000/api/options?user.name=falcon&user.name=falcon,</a>><host>:15000/api/options?user.name=falcon&user.name=falcon, status: 503, message: Service Unavailable

In the falcon.application.log there are the following errors. From what I understood, adding the je-5.0.73.jar file should've solved this problem, but it persists and I run out of ideas.

2017-07-12 09:07:42,568 ERROR - [main:] ~ Failed to initialize service org.apache.falcon.metadata.MetadataMappingService (ServiceInitializer:49)
java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [com.thinkaurelius.titan.core.TitanFactory].
        at com.tinkerpop.blueprints.GraphFactory.open(GraphFactory.java:50)
        at org.apache.falcon.metadata.MetadataMappingService.initializeGraphDB(MetadataMappingService.java:146)
        at org.apache.falcon.metadata.MetadataMappingService.init(MetadataMappingService.java:113)
        at org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
        at org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
        at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:549)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:58)
        at org.apache.falcon.FalconServer.main(FalconServer.java:118)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.tinkerpop.blueprints.GraphFactory.open(GraphFactory.java:45)
        ... 15 more
Caused by: java.lang.NoClassDefFoundError: com/sleepycat/je/LockMode
        at com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEStoreManager.<clinit>(BerkeleyJEStoreManager.java:47)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:42)
        at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:421)
        at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:361)
        at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1275)
        at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
        at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
        ... 20 more
Caused by: java.lang.ClassNotFoundException: com.sleepycat.je.LockMode
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 29 more
2017-07-12 09:07:42,569 ERROR - [main:] ~ Failed startup of context org.mortbay.jetty.webapp.WebAppContext@351f2244{/,/usr/hdp/current/falcon-server/server/webapp/falcon} (log:87)
java.lang.RuntimeException: org.apache.falcon.FalconException: java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [com.thinkaurelius.titan.core.TitanFactory].
        at org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:59)
...

[#7]

Great answer @Venkat Ranganathan your answer fixed our problem

[#8]

Currently my ambari server is setup using mysql DB and corresponding jdbc driver. If I do the setup again to specify a different DB for Falcon will that cause inconsistency? Or I can just setup mysql for ambari and bdb for falcon?

[#9]

@Swaapnika Guntaka

As of HDP 2.5.3, you need to install the Berkeley DB prior to upgrading or installing Falcon. That might be the problem you are encountering. You might try the following: [Updated content below]

1.Download the required Berkeley DB implementation file.

wget –O je-5.0.73.jar http://search.maven.org/remotecontent?filepath=com/sleepycat/je/5.0.73/je-5.0.73.jar

2.Log in to the Ambari server with administrator privileges.

su – root

3.Copy the file to the Ambari server share folder.

cp je-5.0.73.jar /usr/share/

4.Set permissions on the file to owner=read/write, group=read, other=read.

chmod 644 /usr/share/je-5.0.73.jar

5.Configure the Ambari server to use the Berkeley DB driver.

ambari-server setup --jdbc-db=bdb --jdbc-driver=/usr/share/je-5.0.73.jar

6.Restart the Ambari server.

ambari-server restart

7.Restart the Falcon service from the Ambari UI.

You need to have administrator privileges in Ambari to restart a service.

1.In the Ambari web UI, click the Services tab and select the Falcon service in the left Services pane.

2.From the Falcon Summary page, click Service Actions > Restart All.

3.Click Confirm Restart All.

When the service is available, the Falcon status displays as Started on the Summary page.

Further information and manual install instructions are available in an article at https://community.hortonworks.com/articles/78274/prerequisite-to-installing-or-upgrading-falcon.html.

[#10]

@Geoffrey Shelton Okot

I have downloaded the jar already. The question I have is since I’ve already setup the ambari server using mysql DB and mysql jdbc driver, would doing a setup again with bdb and it’s jar cause a conflict? Or can I set up ambari using any number of databases like for eg mysql for ambari, derby for oozie, bdb for falcon etc

[#11]

@Swaapnika Guntaka

That will run independently it won't interfere with the Mysql or any other database.Falcon doesn't use the classic db's like run. Before you proceed make sure Falcon is not running ,kill any rogue falcon process