Member since
07-31-2013
1924
Posts
462
Kudos Received
311
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1969 | 07-09-2019 12:53 AM | |
| 11879 | 06-23-2019 08:37 PM | |
| 9146 | 06-18-2019 11:28 PM | |
| 10133 | 05-23-2019 08:46 PM | |
| 4579 | 05-20-2019 01:14 AM |
08-04-2015
02:48 AM
1 Kudo
Please never disable that check. Checkpoints are very essential for the HDFS operation, and you do not want to be in a position with checkpoints failing for a technical reason and you never getting notified on that. Instead, look at your Standby or Secondary NN to figure out what the error is, and/or seek help with that identified information.
... View more
07-31-2015
12:34 AM
The trace is relieving in the fact that it fails on "readTransactionIdFile" method, which tries to read the file called "seen_txid" inside the NN's current/ local directory. Please try moving this file out (to /tmp or elsewhere) and then restart NN. Are you perchance running any form of disk encryption software that may not be active yet? The corrupted data is weird - the file is supposed to have a simple number in it.
... View more
07-30-2015
10:59 PM
Could you share the full exception trace? It appears your fsimage, or one of your edit logs, has somehow gotten some corrupt data in it. If its the edit log, you can perhaps attempt to skip a few entries in the 'hdfs namenode -recover' startup mode; but if its the fsimage that is corrupt, then we'll need to rollback to an older copy and replay edit logs on top. The full exception trace can help tell which file is corrupt.
... View more
07-28-2015
07:06 AM
Yes, do you not see it working? You'll need to pass the XML property via the workflow.xml under the action's configuration section.
... View more
07-28-2015
04:12 AM
Thanks, that'd explain your transition. What application is this? Is it an MR2 application, Spark app, or something custom?
... View more
07-28-2015
01:02 AM
2 Kudos
Recovery features deal with restarts of the service (RM or NM). An AM attempt is a separate feature that, like container retries in MR, is a regular runtime feature. Do you see your application ID attempt multiple AMs in the RM UI page for it? Do the RM logs indicate any form of kill or fail for the first 'appattempt' of the AM ID?
... View more
07-27-2015
07:54 PM
1 Kudo
There will not be a performance difference in the IO paths, as you will still be using the same HFiles, and they will behave as the same table.
... View more
07-27-2015
03:25 AM
Glad to hear; Please consider marking the topic as resolved so others with similar issues can find it easily.
... View more
07-27-2015
03:24 AM
Unlike your spark shell command, Oozie does not invoke/use scripts that setup local classpaths for its actions (as it needs to use distributed-caches for this). Take a look at how the ShareLib works, and how you can override them for your action to include a system one http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a17_HDFS_Share_Libraries_for_Workflow_Applications_since_Oozie_2.3. In your case, if you use the java action, you can make it include the "hive" share-lib, and that will include all Hive jars into the distributed cache classpath.
... View more