Support Questions
Find answers, ask questions, and share your expertise

FALCON HDFS Mirroring not working

Re: FALCON HDFS Mirroring not working

Explorer

I can see this error in the oozie logs :

/apps/data-mirroring/workflows/lib] does not exist

There is a thread on this :

https://community.hortonworks.com/questions/7942/problem-while-mirroring-dataset-between-hadoop-clu....

The above talks about permissions in /apps/data-mirroring/workflows - which in my environment is 777.

drwxrwxrwx - falcon users 0 2017-02-27 16:56 /apps/data-mirroring

drwxrwxrwx - falcon users 0 2017-03-08 16:36 /apps/data-mirroring/workflows

But there is no 'lib' under /apps/data-mirroring/workflows - is this some library which is missing?

Re: FALCON HDFS Mirroring not working

Explorer

@Frank Lu @Sowmya Ramesh @Ram Baskaran @Neeraj Sabharwal @Akshay Shingote @Artem Ervits @Predrag Minovic - appreciate it if you could help me find a solution to my issue. Thanks.

Re: FALCON HDFS Mirroring not working

Explorer

BTW these are the versions I am using :

hdp 2.3.6.0-3796 / falcon 0.6.1.2.3 / namenode 2.7.1.2.3 / resource manager 2.7.1.2.3 / oozie 4.2.0.2.3

Re: FALCON HDFS Mirroring not working

Explorer

When creating falcon cluster there is this entry for messaging queue :

tcp://<host>:61616?daemon=true

How do we find the version for this?

BTW for the first two entries with ports 50070 and 8020 - I give the version of the namenode. The other two are ResourceManager and OOzie. Hope that is right.

Re: FALCON HDFS Mirroring not working

Rising Star

It looks like your staging directory (/apps/falcon/primaryCluster/staging) permissions are 755. Try setting to 777. That's the recommended setting for /staging. Don't know if it'll help, but something to try in the interim. From HWX doc: "Falcon stores artifacts of processes and feeds, such as the feed and process definitions and job logs, in /staging. When you schedule an entity, the workflow and library you specify in the entity are copied to the staging path."

Re: FALCON HDFS Mirroring not working

Expert Contributor

@n c Can you please share the falcon application log (/var/log/falcon/falcon.application.log) and oozie logs from /var/log/oozie to diagnose the issue, when you submit the falcon mirroring job.

Re: FALCON HDFS Mirroring not working

Explorer

Sorry peeyush, right now my falcon ui is not loading.

Re: FALCON HDFS Mirroring not working

Explorer

Hi Piyush, came into work today morning and the falcon ui is back up again - not sure what was the issue yesterday. So regarding 'submitting' the falcon job, we only submit it the first time right? After that it is running all the time (till the end of the specified timeframe). So do you want me to suspend and resume the job?

Re: FALCON HDFS Mirroring not working

Explorer

@peeyush I have attached the falcon and oozie logs for a timeframe. Pls see below. Thanks.

Re: FALCON HDFS Mirroring not working

Explorer

Thanks Beverly I set the permissions to 777 for the working dir. Staging was already 777. But unfortunately suddenly my Falcon Web UI is not loading - it is giving "ERROR 0". Not sure whats happening. BTW I think it was already showing this error before I make the change - so I don't think it has anything to do with changing the permissions.