Hi, we are trying to use WebHDFS over Knox to access HDFS on our secured cluster from java. We are able to list files/folders there, but we are still struggling with the file creation. The problem is probably in Oracle's Java library, where the streaming does not seem to be supported when authentication is required:. In sun.net.www.protocol.http.HttpURLConnection.getInputStream0() there is something like if (j == 401) { /* 1635 */ if (streaming()) { /* 1636 */ disconnectInternal(); /* 1637 */thrownew HttpRetryException("cannot retry due to server authentication, in streaming mode", 401); /* */ } The streaming is not needed for some operation, like list/delete (and therefore it works), but it is required for file creation. Any suggestions how to handle this? Thanks a lot, Pavel ... View more

Hi, I am wondering if there is a general way how to determine what YARN applications were started by some application and vice-versa if some YARN application was started by some other. My use case is Oozie and sqoop where Oozie runs some launchers that in turn start some MR jobs to do the actual ingest. It is possible to browse through the logs to get ID of spawned application, but keep thinking that there should be some better way how to do it. This kind of relation must be stored somewhere, since when the Oozie workflow is killed all child processes are killed as well almost immediately. Thanks for any hints, Regards, Pavel ... View more

Hi, we are using org.apache.falcon.client.FalconClient API to update Falcon process from java: falconClient.update( EntityType.PROCESS.name(), <some-id>, <file-name>,true,doAs); where the local <file-name> is created like this: ... Marshaller marshaller = entityType.getMarshaller(); final File createTempFile = File.createTempFile(entityType.name().toLowerCase() + "_" + id, ".xml"); LOGGER.debug("Generated entity: {}", entity.toString()); marshaller.marshal(entity, createTempFile); return createTempFile.getPath(); and sometimes the update fails with this error: javax.xml.bind.UnmarshalException: [org.xml.sax.SAXParseException; Premature end of file.] at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException(AbstractUnmarshallerImpl.java:335) at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.createUnmarshalException(UnmarshallerImpl.java:523) at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:220) at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:189) at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157) at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204) at org.apache.falcon.entity.parser.EntityParser.parse(EntityParser.java:94) ... 61 more Caused by: org.xml.sax.SAXParseException; Premature end of file. at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source) at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source) at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:216) ... 65 more most of the updates pass, this error happens only sometimes, so I believe the file is created correctly on the client side and the error is caused possibly by some performance issue or race condition. Have you seen this behavior? Thanks, Pavel ... View more

Hi, I am trying to execute Hive query from java action that is part of Oozie workflow. My preferred way is to use the Beeline or JBDC rather than old Hive CLI. However I am struggling with this a little bit, since the connection is failing due to authentication errors. When the java action code is executed on some data node in a cluster, Oozie does not do kinit for the user and thus the connection fails. Both Beeline and JDBC connection string seem to support delegation tokens, but those token can only be obtained when user is logged in by Kerberos (kinited). We are currently using hive-0.14.0 and oozie-4.1. I have found out that new hive2 action introduced in oozie-4.2 seems to first create jdbc connection under Oozie Kerberos login, obtain delegation token from this connection and finally pass this token to the Beeline. Maybe the same approach could be used here as well. It would require a new custom oozie action (e.g. java-jdbc). . Seems possible but it is quite complicated; is there some easier way? Thanks for any comments, Pavel ... View more

Hi, we are trying to setup ingest of data from databases into Hive using Sqoop. Problem is that databases are used in production and we cannot use them too heavily during some working hours. There are many tables and some of them are quite huge ( > 2 GRows) so it is probable that we cannot ingest them all during the time window available. It is difficult to create some general delta queries that would run some given amount of time and no longer. I am thinking about possibility to implement such feature directly into Sqoop. I am not very familiar with Sqoop implementation, but I guess there is some loop where a row gets loaded from JDBC resultset, converted and stored into Hive table. All that would be required is to place a check in this loop and wait/sleep for some time if this is happening during the database working hours. This way the ingest will run at the full speed outside of the working hours and will be significantly reduced (but still running) when database is not supposed to be overloaded. What do you think? Does this sound like a feature that could be useful to someone else as well? Thanks, Pavel ... View more

Hi, our Falcon installation abruptly ceased to work and no feed could be created. It complained about file permission Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=kefi, access=WRITE, inode="/apps/falcon-MiddleGate/staging/falcon/workflows/feed":middlegate_test1:falcon:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6515) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6497) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6449) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4251) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4221) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4194) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:813) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:600) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.ipc.Server$Handler.run(Server.ja where 'kefi' is the user trying to create the feed and 'middlegate_test1' is another user that created some feed before. the folders on hdfs looked like this bash-4.1$ hadoop fs -ls /apps/falcon-MiddleGate/staging/falcon/workflows/ Found 2 items drwxr-xr-x - middlegate_test1 falcon 0 2015-12-02 09:13 /apps/falcon-MiddleGate/staging/falcon/workflows/feed drwxrwxrwx - middlegate_test1 falcon 0 2015-12-02 09:13 /apps/falcon-MiddleGate/staging/falcon/workflows/process I can think of two questions related to this: why the permission for the 'feed' folder is now 'drwxr-xr-x' whereas the 'process' folder has permissions 'drwxrwxrwx'? The feed creation has been working before, so I guess someone or something had to change it. It is not very probable that some user did it manually, is it possible that it was falcon itself who did it? it does not seem correct that such internal falcon system folders are owned by some normal user; quite probably the first one who ever tried to create some entity in falcon. Is it as expected, or rather some our misconfiguration of falcon? Thanks for any input, Regards, Pavel ... View more

Hello, I have got one question regarding how Falcon implements retention policy for feed instances. I have observed that the retention policy action(i.e. DELETE in my case) is executed only within the dataset's validity interval. It means that several instances (how many it depends of the dataset's frequency) close to end of dataset's validity are kept forever even when some retention is defined. Is it as expected or I am doing something wrong? Thanks for any input, Regards, Pavel ... View more