Member since
07-17-2016
23
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2579 | 05-08-2018 06:11 AM |
05-20-2018
12:08 AM
I am using HDP 2.6.4, with apache hive 2.1.0, seems the integration support from 2.2.0 version.
... View more
05-19-2018
05:48 AM
While I was running a query to create druid table from hive query, an error is returned SQL Query CREATE TABLE druid_table_1
(`data_timestamp_plus_seq` TIMESTAMP,
`stockcode` INT,
`seq` INT,
`price` FLOAT,
`flag` STRING,
`quantity` FLOAT,
`session` STRING
)
STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler';
Error Message org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hadoop.hive.druid.DruidStorageHandler Error StackTrace java.lang.Exception: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hadoop.hive.druid.DruidStorageHandler'
java.lang.Exception: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hadoop.hive.druid.DruidStorageHandler'
at org.apache.ambari.view.hive20.resources.jobs.JobService.getOne(JobService.java:147)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:287)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:132)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.apache.ambari.server.view.AmbariViewsMDCLoggingFilter.doFilter(AmbariViewsMDCLoggingFilter.java:54)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.apache.ambari.server.view.ViewThrottleFilter.doFilter(ViewThrottleFilter.java:161)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:212)
at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:201)
at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:150)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hadoop.hive.druid.DruidStorageHandler'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:264)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:250)
at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:309)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:250)
at org.apache.ambari.view.hive20.HiveJdbcConnectionDelegate.execute(HiveJdbcConnectionDelegate.java:49)
at org.apache.ambari.view.hive20.actor.StatementExecutor.runStatement(StatementExecutor.java:91)
at org.apache.ambari.view.hive20.actor.StatementExecutor.handleMessage(StatementExecutor.java:72)
at org.apache.ambari.view.hive20.actor.HiveActor.onReceive(HiveActor.java:38)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)
at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hadoop.hive.druid.DruidStorageHandler'
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:324)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:148)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:228)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:264)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:479)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:466)
at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy24.executeStatementAsync(Unknown Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:509)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1377)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1362)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.parse.SemanticException:Cannot find class 'org.apache.hadoop.hive.druid.DruidStorageHandler'
at org.apache.hadoop.hive.ql.parse.ParseUtils.ensureClassExists(ParseUtils.java:232)
at org.apache.hadoop.hive.ql.parse.StorageFormat.fillStorageFormat(StorageFormat.java:64)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:11210)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10391)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10477)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:219)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:465)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:321)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1224)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1218)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:146)
... 26 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.druid.DruidStorageHandler
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.hive.ql.parse.ParseUtils.ensureClassExists(ParseUtils.java:230)
... 37 more
(less...)
... View more
Labels:
- Labels:
-
Apache Hive
05-15-2018
06:20 AM
While I was trying to index a csv info druid for just 2 lines, it is extremely slow. The csv consists of 2 lines: 1,0,90.95,"P",385,"MA",2018-05-08
1,0,91,,6000,"MN",2018-05-08
The indexing json config file: { "type" : "index_hadoop", "spec" : { "dataSchema" : { "dataSource" : "hkexsales", "parser" : { "type" : "string", "parseSpec" : { "format" : "csv", "timestampSpec" : { "column" : "data_date", "format" : "iso" }, "columns" : ["stockcode","seq","price","flag","quantity","session","data_date"], "dimensionsSpec" : { "dimensions": ["stockcode","seq","price","flag","quantity","session"], "dimensionExclusions" : [], "spatialDimensions" : [] } } }, "metricsSpec" : [ { "type" : "count", "name" : "count" } ], "granularitySpec" : { "type" : "uniform", "segmentGranularity" : "DAY", "queryGranularity" : "NONE", "intervals" : [ "2013-08-31/2020-09-01" ] } }, "ioConfig" : { "type" : "hadoop", "inputSpec" : { "type" : "static", "paths" : "/data/HkexDayQuot/output/Sales/d180508e_1525922483177_skipheader2.htm" } }, "tuningConfig" : { "type" : "hadoop" } } }
Duration Spent is 8906764, it is unacceptable. Please advise
... View more
Labels:
- Labels:
-
Apache Hadoop
05-12-2018
10:08 AM
Thanks, it is a good answer, may I know how to run a command Execute Stream Command processor with pipeline "|" ?
... View more
05-11-2018
04:40 PM
Could I connect Apache Hive Table (Hive2) from Apache Superset?
... View more
Labels:
- Labels:
-
Apache Hive
05-11-2018
03:55 PM
After I changed the SECRET_KEY of Superset Database, then restarted server, it works.
... View more
05-11-2018
01:18 PM
Sorry, something went wrong 500 - Internal Server Error
Stacktrace
Traceback (most recent call last):
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1615, in full_dispatch_request
return self.finalize_request(rv)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1632, in finalize_request
response = self.process_response(response)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1858, in process_response
self.save_session(ctx.session, response)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 924, in save_session
return self.session_interface.save_session(self, session, response)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/sessions.py", line 363, in save_session
val = self.get_signing_serializer(app).dumps(dict(session))
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/itsdangerous.py", line 566, in dumps
rv = self.make_signer(salt).sign(payload)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/itsdangerous.py", line 412, in sign
return value + sep + self.get_signature(value)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/itsdangerous.py", line 347, in get_signature
key = self.derive_key()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/itsdangerous.py", line 336, in derive_key
mac = hmac.new(self.secret_key, digestmod=self.digest_method)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/hmac.py", line 144, in new
return HMAC(key, msg, digestmod)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/hmac.py", line 42, in __init__
raise TypeError("key: expected bytes or bytearray, but got %r" % type(key).__name__)
TypeError: key: expected bytes or bytearray, but got 'int'
Please advise how to fix.
... View more
- Tags:
- superset
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)
05-11-2018
07:52 AM
How to remove folder and everything inside that folder in Apache Nifi?
... View more
Labels:
- Labels:
-
Apache NiFi
05-10-2018
10:38 PM
Could you give me a concrete example for that? If you are not having fully qualified filename then we need to list all the files in the directory by using ExecuteStreamCommand processor by passing the dynamic generated directory name as an argument to list all the files in the directory then using Fetch File processor you can pull the required files into data flow.
... View more
05-10-2018
08:44 AM
I tried but it does not work, because it does not allow to run "unzip" command on the dataflow. Command Argument: Command Path: /usr/bin/unzip Ignore STDIN: false Working Directory: Argument Delimiter: ; Output Destination Attribute Max Attribute Length: 256 Please advise with working examples.
... View more
05-10-2018
08:40 AM
1 Kudo
Can I get the files in the middle of the data flow? I know I can get files by getfile processors but it is limited to the beginning of the data flow, please advise how can I get retrieve the files in the middle of the data flow? The reason is that I would like to pass the dynamic generated directory to be retrieve in the getfile / similar processor, it needs to be middle of the flow.
... View more
Labels:
- Labels:
-
Apache NiFi
05-08-2018
06:13 AM
May I know which processors support unzipping a file with password
... View more
Labels:
- Labels:
-
Apache NiFi
05-08-2018
06:11 AM
This is fixed by doing below:
Set default encoding using the JAVA_TOOL_OPTIONS environment variable: (nifi-env.sh) <-- currently this one not implemented export JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
Add default encoding parameter to NiFi’s bootstrap.conf file: java.arg.8=-Dfile.encoding=UTF8
Of course, adjust the argument’s number according to your configuration. And in nifi-env.sh export LANG="en_US.UTF-8" export LC_ALL="en_US.UTF-8"
... View more
05-08-2018
03:23 AM
Just tested, I are not able to write to local path "/tmp" neither
... View more
05-07-2018
02:49 AM
I am currently running apache nifi in centos 7, and mounted some network drives by samba ( windows share ), then in the nifi data flow, I use putfile to write some files with Chinese Characters filename into the network drives by samba. Then I got this error PutFile[id=e0ae7c13-0162-1000-f401-ecb9b39fa9de] Penalizing StandardFlowFileRecord[uuid=04563401-54ff-4a1f-9117-02e4472ff0c4,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1525659272432-6, container=default, section=6], offset=0, length=6981],offset=0,name=用户划款授权通知书.pdf,size=6981] and transferring to failure due to Malformed input or input contains unmappable characters: .用户划款授权通知书.pdf: java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: .用户划款授权通知书.pdf Please advise how to do.
... View more
Labels:
- Labels:
-
Apache NiFi
05-05-2018
11:33 PM
Attributes lost after running SelectHiveQL, do you know why? Is it a bug? And how to fix it? I have some attributes updated before SelectHiveQL, but all the attributes set disappear after running SelectHiveQL, please help.
... View more
Labels:
- Labels:
-
Apache NiFi
09-28-2017
07:39 AM
I am talking about the Guava in Phoenix Client Jar, it is rather old, version 12. The latest one is 23. My other jar depends on version 21. Both versions are not compatible because some functions are only available 12 or 21. Please advise.
... View more
09-28-2017
02:02 AM
I know I can get a phoenix client jar in the sandbox/cluster but one of the dependency in the jar is too old. Could someone advise me the newer/latest phoenix client jar?
... View more
Labels:
- Labels:
-
Apache Phoenix
09-16-2017
02:50 AM
Please advise how to install Cassandra on HDP 2.6
... View more
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)