Hello,        We have 4 node solr cluster and one of the collection size is around 12TB size and collection are split into 4 shards each holding 3TB of size and collection has around 40 fields , field types are XML, string, int everything where indexed due to customer requirement.         Now the problem is whatever query we execute it takes around 10 minutes to produce the result and sometimes the shards are going down.        Could you please tell me how to scale the collection and improve the query performance.     Thanks & Regards, J. Ganesh Kumar. ... View more

Hi all,    I like to know what is the difference between queries--->  fieldname: value  vs "fieldname": "value"   1) In the query menu of solr UI. if I give below query it fetches me all documents even it doesnot match the value in fieldname fieldname: value      2) "fieldname": "value"          It fetches correct result.   I was wondered what is the difference between 1 vs 2 query?   Please help me to understand the difference of both queries.     Thanks & Regards, J.Ganesh Kumar ... View more

Hi ,         We like to know if there any other way to do kerberos authentication for SOLR from java code with out using either System.setproperty or -D parameter during server startup,         We are using below code         System.setProperty("java.security.auth.login.config", "/tmp/jaas_client.conf");       HttpClientUtil.setConfigurer(new Krb5HttpClientConfigurer());   But in our infrastructure using Sytem.setProperty is not allowed. We have tried using UserGroupInfromation from hadoop security but does not work for SOLR.     Thanks & Regards, J.Ganesh Kumar. ... View more

Hi,       We have a kerberoized cluster and written a oozie job where from main workflow(workflow.xml) we are submitting the subworkflow(subworkflow.xml).        In Subworkflow we have shell script with below command.        curl -s --negotiate -u : "http://xxxx:8983/solr/xx_collection_shard1_replica3/select?q=xxx_TIME%3A%5B2018-07-30T00%3A00%3A00.000Z+TO+2018-07-30T23%3A59%3A59.999Z%5D&wt=json&indent=true&rows=0"          When it executing we are getting HTTP Status 401 - Authentication required error.        Same shell script are available in main workflow shell action and its working fine and returns us response.        Only we have problem when executing from subworkflow..         "Our obseration" is before each submittion of oozie job we do kinit. so main workflow taking the credentails from kinit. But in the subworkflow the main workflow is the user who submits the job may be due to this the credentials are lost.       Please help us on this to solve the issue.     Thanks & Regards, J.Ganesh Kumar.         ... View more

Hi,            I have followed the below document for accessing kerberosied solr from java application using solrj.            https://www.cloudera.com/documentation/enterprise/5-9-x/topics/search_using_kerberos.html   But doesnot work with the weblogic server when i set below property in my code   System.setProperty("java.security.auth.login.config", "/tmp/jaas-client.conf");      Getting authentication execption ---> HTTP Status 401 - Authentication required    Please tell us if anything other than this required for weblogic based server.     ... View more

Hi,   I like to understand what is the advantage of storing data in hbase and index in solr instead storing both data and index in solr itself for NRT based application.   Currently we have 4 node solr cluster where we have NRT usecase which index into the solr using the Solrj library.whose collection size is around 4 gb with 2 shard and 2 replica.   Is there any problem when the collection size grows while using solr for both data and index.   Thanks & Regards, J.Ganesh Kumar.     ... View more

Recently i have upgraded the oracle jdk from 1.7 to 1.8u131.   After the upgrade. and starting the cluster getting below error. and value we have is  dfs.datanode.max.locked.memory:3.36gb   java.lang.RuntimeException: Cannot start datanode because the configured max locked memory size (dfs.datanode.max.locked.memory) is greater than zero and native code is not available. at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1201) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:460) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2509) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2396) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2443) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2625) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2649) 2018-02-07 15:41:56,472 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1 2018-02-07 15:41:56,474 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: ... View more

No of solr nodes : 4   We have storm which continuously indexing the dcouments into the solr and getting solr Liveliness problem in the cloudera manager and affects the others collections query response time.   When we checked the cpu utilization of each node.. it only 12 -15% usage.   We like to any restriction on the resource utilization in the solr property and like to know the purpose of below properties.   Cgroup CPU Shares : 800 (value currently we set) Cgroup I/O Weight : 400 (value currently we set)   Planning to increase the Cgroup CPU Shares  and Cgroup I/O Weight   will it be helpful by increase value of the above property ? ... View more

Hi All,         I have some pdf files and each pdf file information like pdf creation date,pdf_id etc. and need to store pdf and its information in the hbase table.          And using java code i need to write new documents and read inserted documents.         I am not found any documentation related on this in the internet.         Please help me.   Thanks & Regards, J.Ganesh Kumar     ... View more

Hi all,         With the below steps we are able to copy the hbase data from cluster to another cluster.   snapshot "SOURCE_TABLE_NAME","SNAPSHOT_NAME" bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot SNAPSHOT_NAME -copy-to hdfs://DESTINATION_CLUSTER_ACTIVE_NAMENODE_ADDRESS:8020/hbase -mappers 16 disable "DEST_TABLENAME" restore_snapshot "SNAPSHOT_NAME" But the problem is Source cluster does not have access to the destination cluster namenode. To solve this we are planning to have one shared location which mounted in both cluster nodes and using those we are planing to do import and export the snapshot to avoid multiple time copy of the files since the hbase table data size is hug which is around 10TB of data.   Can you please tell me the command to  export the snapshot to the local mounted shared File location (instead of hdfs location) restore snapshot from the local   mounted shared file location. ( instead of hdfs location) Thank you, J.Ganesh Kumar ... View more

Is that possible to use solr for semistructured data like json.   I have nosql database and planning to index the data available in nosql sdatabases into the solr.     Thanks & Regards, J.Ganesh Kumar.       ... View more

We have 6 node cluster and solr installed in 4 nodes.   In near real time Storm indexing data into solr.   While indexing We always getting solr Server API Liveness issue in cloudera manager console.   When we look at the cloudera-scm-agent log we found that problem beause of the Timeout when calling the below url http://xxxx:8983/solr//admin/cores?wt=json&action=STATUS   What may be the issue for url to take long time to return status response?   Thanks & Regards, J.Ganesh Kumar. ... View more

Hi,     Please help us it is very critical issue we are facing one of the collection replica are down.      When we looked into the log Trace we found below error.     WARN org.apache.solr.update.UpdateLog: Log replay finished. recoveryInfo=RecoveryInfo{adds=0 deletes=0 deleteByQuery=0 errors=1 positionOfStart=3411522} DEBUG org.apache.solr.cloud.RecoveryStrategy: eeeeee:8983_solr replayed 4910970 INFO org.apache.solr.cloud.RecoveryStrategy: Replication Recovery was successful. core=xxxxxx_collection_shard1_replica1 INFO org.apache.solr.cloud.ZkController: publishing core=xxxxx_collection_shard1_replica1 state=active collection=xxxx_collection DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x15fda16a22c5a7d, packet:: clientPath:null serverPath:null finished:false header:: 18701,4 replyHeader:: 18701,343598899621,0 request:: '/solr/collections/xxxxx_collection/leader_initiated_recovery/shard1/core_node1,F response:: #7ba2020227374617465223a22646f776e222ca2020226372656174656442794e6f64654e616d65223a22626463656430362e73672e6762732e70726f3a383938335f736f6c72227d,s{343598889249,343598899525,1513852552137,1513855791909,45,0,0,0,73,0,343598889249} ERROR org.apache.solr.cloud.RecoveryStrategy: Could not publish as ACTIVE after succesful recovery org.apache.solr.common.SolrException: Cannot publish state of core 'xxxx_collection_shard1_replica1' as active without recovering first! at org.apache.solr.cloud.ZkController.publish(ZkController.java:1093) at org.apache.solr.cloud.ZkController.publish(ZkController.java:1056) at org.apache.solr.cloud.ZkController.publish(ZkController.java:1052) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:496) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:237) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) ERROR org.apache.solr.cloud.RecoveryStrategy: Recovery failed ... View more

Hi,       We have a collection and splitted into 2 shards and 2 replica.Each shard have around 200TB data.One of the shard is down and 2 replicas are failed. This leads to unable to use the collection since both leader and replicas of this shards are down.   I have tried the rolling restart of particular solr server as well But it doesn't recovered.   Is there anyway to recover it without complete restart of Solr?   Thanks & Regards, J.Ganesh Kumar. ... View more

Hi,        I have oozie job which executes the shell and shell command deletes the particular partition in the table. while deleting the partition I am getting below error message.   FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Got exception: java.io.IOException Failed to move to trash: xxxxxxx_hadoop_file       ... View more

Currently we have a collection with 2 shards, We want to split the 2 shards into 4 shards collection. Since each shards size has increased to 5TB of data.     Is there any option available to split the existing shards? ... View more

Hi All,          Is there any way to stream data from oracle table to hdfs in near real time. ... View more

Hi All,               We are having 6 nodes cluster and hbase region server are installed in all 6 nodes, But frequently the hbase region servers are stopped suddenly in 4 nodes where we also have yarn node server.   And we are using webhdfs (isilon) for storage    We are getting below error message before aborting the region server   java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:658) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) at org.apache.hadoop.util.DirectBufferPool.getBuffer(DirectBufferPool.java:72) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.reallocPacketBuf(PacketReceiver.java:272) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:165) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:102) at org.apache.hadoop.hdfs.RemoteBlockReader2.readNextPacket(RemoteBlockReader2.java:207) at org.apache.hadoop.hdfs.RemoteBlockReader2.read(RemoteBlockReader2.java:156) at org.apache.hadoop.hdfs.DFSInputStream$ByteArrayStrategy.doRead(DFSInputStream.java:772) at org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:828) at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:888) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:932) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:199) at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1444) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1713) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1542) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:445) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:913) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:301) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:199) at org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:350) at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:199) at org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2106) at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2096) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5545) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2570) at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2556) at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2537) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2411) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)         ... View more