Member since
10-24-2017
101
Posts
14
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2133 | 07-26-2017 09:57 PM | |
3983 | 12-13-2016 12:08 PM | |
1158 | 07-28-2016 08:41 PM | |
4212 | 06-15-2016 07:57 AM |
01-15-2019
02:30 PM
Listfile has the option to define a File Filter and a Path Filter. From your explanation I would expect that you need to define the Path Filter. In the Path Filter you can place a regular expression (in Java syntax). For regex there are a lot of resources available, I think something like this should do the trick: (?!(B|C)).*
... View more
05-25-2018
10:38 AM
It's still not working unfortunately
... View more
07-26-2017
09:57 PM
this is the code i came up with, is there a better approach? val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap {
case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF
... View more
05-05-2017
05:18 PM
@Ahmad Debbas Your exception is java.lang.ClassNotFoundException: com.microsoft.sqlserver.jdbc.SQLServerDriver Are you passing the jdbc driver to your program. Also you are specifying com.sqlserver.jdbc.Driver. It is looking for com.microsoft.sqlserver.jdbc.SQLServerDriver? I am not sure where is this coming from but your issue is not having a driver to connect to SQL Server.
... View more
04-06-2017
01:36 PM
i am trying the listhdfs processor, for some reason it is only retrieving around 5000 files
... View more
03-07-2017
06:53 AM
i have tried sending documents using Solr's rest api and i got the exact same error. The problem isn't with zip files. If a zip file contains pdf or word documents for example the zip is indexed well. However if the zip file contains an mdb file solr fails to index it. Is it possible to have solr ignore only the unsupported extensions rather than ignoring the entire document or file?
... View more
12-21-2016
03:20 PM
i did, now im getting this 2016-12-21 15:16:44.124 ERROR (qtp1450821318-15) [ ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: Error handling 'status' action
at org.apache.solr.handler.admin.CoreAdminOperation$4.call(CoreAdminOperation.java:192)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:153)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:439)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: file: BlockDirectory(HdfsDirectory@hdfs://db-hdp-nn1.com:8020/user/solr/CFRepo/core_node2/data/index lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@1451c9ff)
... View more
12-20-2016
04:47 PM
1 Kudo
@Ahmad Debbas FlolwFiles generated from the GetHDFS processor should have a "path" attribute set on them:
The path is set to the relative path of the file's directory on HDFS. For example, if the Directory property is set to /tmp, then files picked up from /tmp will have the path attribute set to "./". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "abc/1/2/3". Since it is only the relative path and not an absolute path, you would need to use an UpdateAttribute processor to prepend the configured directory path the that relative path if you need the absolute path for use later in your flow. Thanks, Matt
... View more
12-20-2016
04:20 PM
Can i update the _text_ field and have it stored and indexed? Also is it possible to update the values and rename the fields using the PutContentStream processor? I am want to be able to store the location of the file im pulling from HDFS in a field in Solr.
... View more
12-13-2016
12:08 PM
1 Kudo
I have set "blockUnknown":true" to false and basic authentiaction mode was disablde
... View more