Member since
06-03-2017
8
Posts
0
Kudos Received
0
Solutions
09-23-2019
09:14 PM
I was trying to stream file from s3 bucket which is in another aws account. The way that i can access that account is through session token , access key and secret key. I am authenticating using fs.s3a.session.token, fs.s3a.access.key and fs.s3a.secret.key. The problem that i am facing is that , once i deploy this service it works for sometimes and then I will start getting this error: org.apache.hadoop.fs.s3a.AWSBadRequestException: getFileStatus on s3a://gehc-us-west-2-dl-80967290-1796-4131-a8a2-1aab359a3b92/c66bbb14-c91e-4175-9f1c-5ec658fc4cd0/00264.csv: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 6C3695CEF0A1E8A9; S3 Extended Request ID: xgWBNn+IpySOrnG/tw0gPJFdkXRbjHGQhqRRYpN3CZDipeRm+vU9q3B9g7sUxBDNyUbE/zMIXI8=), S3 Extended Request ID: xgWBNn+IpySOrnG/tw0gPJFdkXRbjHGQhqRRYpN3CZDipeRm+vU9q3B9g7sUxBDNyUbE/zMIXI8=:400 Bad Request: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 6C3695CEF0A1E8A9; S3 Extended Request ID: xgWBNn+IpySOrnG/tw0gPJFdkXRbjHGQhqRRYpN3CZDipeRm+vU9q3B9g7sUxBDNyUbE/zMIXI8=) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:212).
... View more
Labels:
02-16-2018
06:14 AM
@ Jay Kumar SenSharma I have set JAVA_HOME properly (Maven is working properly ) and yeah I am using run-nifi.bat
... View more
02-16-2018
05:59 AM
2018-02-16 11:17:37,109 INFO [main] o.a.nifi.web.server.HostHeaderHandler Created HostHeaderHandler for localhost:8080
2018-02-16 11:17:37,109 INFO [main] org.apache.nifi.web.server.JettyServer Created HostHeaderHandler [HostHeaderHandler for localhost:8080]
2018-02-16 11:17:37,249 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:581)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:672)
at java.base/java.util.ServiceLoader.access$1000(ServiceLoader.java:390)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1227)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1259)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1294)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1379)
at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:142)
at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:117)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:792)
at org.apache.nifi.NiFi.<init>(NiFi.java:160)
at org.apache.nifi.NiFi.main(NiFi.java:268)
Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3110)
at java.base/java.lang.Class.getConstructor0(Class.java:3315)
at java.base/java.lang.Class.getConstructor(Class.java:2108)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:659)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:656)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:667)
... 10 common frames omitted
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:466)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:563)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
... 18 common frames omitted
2018-02-16 11:17:37,328 INFO [Thread-0] org.apache.nifi.NiFi Initiating shutdown of Jetty web server... 2018-02-16 11:17:37,328 INFO [Thread-0] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
... View more
Labels:
10-19-2017
01:00 AM
example: Let the content of the flowfile be ABC before entering to a processor A. Processor A now changes the content to DEF, provenance repository contains Flow file attribute and pointer to the content repository , now how exactly do i get back the original content of the Flowfile i.e ABC. @Bryan Bende
... View more
Labels:
06-06-2017
07:25 AM
@Matt Clarke Thank you . Let me be more specific. How do we get the data Content from a Flowfile from inside a custom processor using the java code? Assuming that the content of the flowfile is like this: Name: Value Example Colour : Black I need to retrieve the colour(black) from the flowfile, with java code of my custom processor.
... View more