About ali_mohammadi

ali_mohammadi · ‎03-07-2018

I had also been facing that issue in hive, but I solved by setting "use.hive.interactive.mode" to true and restart the Hive View to run in llap mode. thank you

ali_mohammadi · ‎03-03-2018

Dear @Girish Chaudhari, Unfortunately, I have faced this issue as well, but I could not understand the BEST ANSWER submitted by you. If it is possible, please help me. thanks a million, Ali

ali_mohammadi · ‎11-21-2017

@Jay Kumar SenSharma thanks

ali_mohammadi · ‎11-20-2017

Hi everybody, We have a cluster with 3 VM nodes on HP servers(64 GB ram and 10 TB capacity and 8 cores). Vsphere 5.5 has been used in this project to create virtual machines. Now we need some extra capacity to add to HDFS, but there is no space on the HP server and new VMs can not be constructed. The question is this: could I add a PC or a laptop with the same operating system as a new node to my cluster? Maybe, it is good to ask in this way: what kind of systems can be used together for constructing hdp clusters? thanks a million, Shanghoosh

ali_mohammadi · ‎10-14-2017

@pdarvas thanks, I am going to check it.

ali_mohammadi · ‎10-10-2017

@pdarvasi Thanks for your valuable and useful suggestion. Actually, my major problem is step one. I need to have openstack on current hp servers and also need to extend it whenever it is necessary. shall I install ESXi and then create a VM and after that install openstack? or I can install openstack on raw servers and then can create a extendable private cloud without ESXi. In fact, I am confusing with creating clusters. For example, I thought, I have to manage a cluster of servers on Vsphere, a cluster of nodes on openstack, and then a hadoop cluster on Ambari. It's still confusing for me. It would be appreciated if you could give me more information or introduce some tutorials. thanks a million, ALi

ali_mohammadi · ‎10-04-2017

Hi, I am new in making private clouds and hadoop ecosystem. We have three HP servers without ESX or openstack, and want to create a private cloud and then have an HDP cluster with Ambari. It would be appreciated, If somebody could explain the steps of creating a cloud and hadoop cluster by using (ESXi, openstack, cloudbreak, Ambari and ....). there are some tutorials to do this, but they are confusing to me. some of our requirements are as follows: 1- we must have at least 100TB HDFS capacity. 2- we should be able to add new nodes for increasing the capacity of HDFS and processing power. 3- we should be able to share the HDP cluster with multiple users. each user should be able to run own map/reduce job. Sincerely,

ali_mohammadi · ‎08-28-2017

Hi I read data from oracle to kafka with nifi and I want to analyze my records(for example sum of attributes) with spark streaming. I sum all of numbers in an rdd and want to add it by previous rdds value of summation. here, there is a problem! At the end of each stream, the type of variable that hold the sum of arrived data per 10 seconds is Dstream. how can I import this (just a number in dstream)to hive table? in the belove code, sum contains a number but in dstream format that I want to to insert it to hive. thanks a million Ali val words = lines.flatMap{ case(x, y) => y.split("\n")} val words1=words.map{aline=> aline.split(" ")(3).replaceAll("\"","").replaceAll("}","")} val sum= words1.map{ss => ss.toDouble}.reduce(_+_)

ali_mohammadi · ‎07-29-2017

Dear @Marco Gaido, I deleted my Json files in hdfs and then I ingested some other tweets in json format. Now, hive is working well. thanks a lot

ali_mohammadi · ‎07-26-2017

about tutorial: https://hortonworks.com/tutorial/analyzing-social-media-and-customer-sentiment-with-apache-nifi-and-hdp-search/ I have been following above tutorial and everything worked fine but when calculating whether a tweet was positive, neutral, or negative using this next Hive command : create table IF NOT EXISTS tweets_sentiment stored as orc as select tweet_id, case when sum( polarity ) > 0 then 'positive' when sum( polarity ) < 0 then 'negative' else 'neutral' end as sentiment from l3 group by tweet_id; I got the following error message: tanks a million java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1501058816878_0021_1_01, diagnostics=[Task failed, taskId=task_1501058816878_0021_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1501058816878_0021_1_01 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1501058816878_0021_1_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:2, Vertex vertex_1501058816878_0021_1_02 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 (less...)

ali_mohammadi · ‎07-26-2017

@Luis Martinez hi if you have solved the problem, please let us know

ali_mohammadi · ‎07-24-2017

may be you lose some data when you remove the docker

ali_mohammadi · ‎07-24-2017

add port 50010 like this https://community.hortonworks.com/articles/65914/how-to-add-ports-to-the-hdp-25-virtualbox-sandbox.html also you should add 50010 to VMbox

ali_mohammadi · ‎04-09-2017

I am trying to implement a movie recommender system for users. I entirely implemented the ALS algorithm and now I want to get new user rates and after that to recommend some other movies. I programmed my code as follows in zeppelin to get information: %pyspark myRatedMovies = [ (0, 1,1.0), (0, 2,1.5), (0, 3,1.1), (0, 4,1), (0,5,1.5), (0,6,1.0), (0, 5,5), (0,52,5), (0,96,5.0) ] myRatingsRDD = sc.parallelize(myRatedMovies) I created an RDD myself and after that I append it in my train dataset and so on, but I want to have a user interface to get information. Actually I want to have a dynamic UI. I saw this page "https://zeppelin.apache.org/docs/0.6.1/manual/dynamicform.html", but it did not help me because I need to get a lot of tuples and I did not find some helpful information about command buttons on that web page. Also I know, I can use angular in zeppelin, but I am new in zeppelin and angulary, and it is very hard to me. For example I can not use the binded variables in pyspark. I would like to add user rates to an RDD when the user clicks on the button "Add My Rate" . Thank you very much for your time and assistance in this matter. Sincerely, Shanghoosh

ali_mohammadi · ‎04-05-2017

Hi @Jay SenSharma Thank you for excellent notes. I solved the problem before reading your notes as follows: from hdfs import InsecureClient client = InsecureClient('http://localhost:50070') # for reading a file with client.read('/tmp/tweets_staging/tweets-082940117.json') as reader: features = reader.read() # for writing a file with client.write('/tmp/tweets_staging/1.json', overwrite=True) as writer: writer.write(features) but I am going to try your solution too. Regards, Shanghoosh

ali_mohammadi · ‎04-04-2017

I want to read and write files to and from a remote HDFS. I program by Pycharm in local machine and I want to connect to a remote hdfs (HDP 2.5). Is there any solution? How can I configure hdfs and how can I refer to a file in hdfs? Thanks a million, Shanghoosh

ali_mohammadi · ‎02-28-2017

thank you Actually I had clicked the checkbox. I think, I should wait for a new version of Ambari; my Ambari is 2.4 .

ali_mohammadi · ‎02-28-2017

Thank you @Artem Ervits Yes, I should wait for Ambari 2.5.

ali_mohammadi · ‎02-26-2017

Hello guys First of all, I am using HDP 2.5. In a tutorial with this address " https://community.hortonworks.com/articles/82964/getting-started-with-apache-ambari-workflow-design.html " thanks to @Artem Ervits that he showed we can create workflows, coordinatores and bundles. In the tutorial I see a view the same as following: but I can not see them in my ambari. I just see workflows as the following photo: How can I solve the problem? thanks a million shanghoosh

ali_mohammadi · ‎02-26-2017

thanks for great tutorial. unfortunately I couldn't see the output. where can I see? And also I do not see create coordinators button 😞

ali_mohammadi · ‎02-18-2017

Hi @Raf Mohammed I have a same problem. Can you help me? I do not know about stale configs for solr. thank you

ali_mohammadi · ‎02-18-2017

Hi @kkawamura Finally I solved it. I installed Solr by using ambari (befor, Solr was in VM but it was not in sandbox) and after that I set "localhost:2181/solr " to PutSolrContentStream processor. Now I am working with Banana,Solr,Nifi and Hive so comfortable. Thanks Koji a million.

ali_mohammadi · ‎02-15-2017

Thank you @Ali Bajwa for good tutoral. I am trying this example with a difference, My nifi is local and I try to put tweets in a remote Solr. Solr is in a VM that contains Hortonworks sandbox. Unfortunately I am getting this error on PutSolrContentStream processor: PutSolrContentStream[id=f6327477-fb7d-4af0-ec32-afcdb184e545] Failed to send StandardFlowFileRecord[uuid=9bc39142-c02c-4fa2-a911-9a9572e885d0,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1487148463852-14, container=default, section=14], offset=696096, length=2589],offset=0,name=103056151325300.json,size=2589] to Solr due to org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; routing to connection_failure: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; Could you help me? thanks, Shanghoosh

ali_mohammadi · ‎02-15-2017

Thank you @kkawamura for nice summarization. I tried any case in hosts files locally and also in VM, but it didn't work. Are there any other processors in NiFi to help me? I mean, Can I work with Solr with another processor?

ali_mohammadi · ‎02-14-2017

Another thing, When I try bin/solr status It gives me Found 1 Solr nodes: Solr process 11364 running on port 8983 { "solr_home":"/opt/lucidworks-hdpsearch/solr/server/solr/", "version":"5.2.1 1684708 - shalin - 2015-06-10 23:20:13", "startTime":"2017-02-14T05:24:44.191Z", "uptime":"0 days, 3 hours, 36 minutes, 37 seconds", "memory":"101.6 MB (%20.7) of 490.7 MB", "cloud":{ "ZooKeeper":"localhost:2181", "liveNodes":"1", "collections":"2"}} and when I try bin/solr healthcheck -c tweets -z localhost:2181 It gives { "collection":"tweets", "status":"healthy", "numDocs":1, "numShards":1, "shards":[{ "shard":"shard1", "status":"healthy", "replicas":[{ "name":"core_node1", "url":"http://172.17.0.2:8983/solr/tweets_shard1_replica1/", "numDocs":1, "status":"active", "uptime":"0 days, 3 hours, 37 minutes, 5 seconds", "memory":"27.4 MB (%5.6) of 490.7 MB", "leader":true}]}]} but when I try bin/solr healthcheck -c tweets It gives ERROR: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper localhost:9983 within 10000 ms what is the default port of zookeeper ? Is it possible solr nodes can't to communicate with zookeeper?

ali_mohammadi · ‎02-14-2017

Hi Koji thank you 🙂 Yes, I want to try this example " https://github.com/abajwa-hw/ambari-nifi-service ", with a difference: my Nifi is in my local machine and I want to connect to a remote lucidworks locally. The lucidworks(solr) is in a VM(with sandbox). screenshot-from-2017-02-14-11-36-39.png(128.3 kB) I am trying to add tweets in solr.

ali_mohammadi · ‎02-13-2017

Hello @kkawamura First of all, I am new in Solr. When I tried the curl command "curl -i http://172.17.0.2:8983/solr " in VM (sandbox), I got a message as below: HTTP/1.1 302 Found Location: http://172.17.0.2:8983/solr/ Content-Length: 0 but when I try in the local machine (ubuntu 16), I don't get any messages. Also, when I try "curl -i localhost:8983/solr" locally, I get a message as follows: HTTP/1.1 302 Found Location: http://localhost:8983/solr/ Content-Length: 0 Actually, I use PutSolrContentStream processor for sending tweets to a remote Solr (lucidworks) in a VM (sandbox). I set Solr type= cloud and Solr Location= localhost:2181. NiFi is in my local machin. Unfortunately, I get an error as follows: PutSolrContentStream[id=f6327477-fb7d-4af0-e990-512eebdb1c1c] Failed to send StandardFlowFileRecord[uuid=c478c4dc-f16e-4ae7-814e-a7de0985985e,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1486893589035-10, container=default, section=10], offset=612767, length=7103],offset=0,name=77207261802126.json,size=7103] to Solr due to org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; routing to connection_failure: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1 Excuse me for my poor English. Thanks a Million, Shanghoosh

ali_mohammadi · ‎02-13-2017

I am trying to connect to a remote solr but I faced with this error on server can not find: /configs/tweets/admin-extra.html and also faced to this error org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr when I tried to connect to Solr from NiFi thank you guys

ali_mohammadi · ‎02-12-2017

Notice: some information will be lost when you do what we did. for example I had a lucidworks but now I should install it again.

ali_mohammadi · ‎02-12-2017

Hi Guys I am trying to add tweets in a remote solr but I faced with this error. PutSolrContentStream[id=f6327477-fb7d-4af0-e990-512eebdb1c1c] Failed to send StandardFlowFileRecord[uuid=34e67c1a-3a18-4280-a738-9943d6716962,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1486876718587-1, container=default, section=1], offset=53886, length=2508],offset=0,name=60294765850236.json,size=2508] to Solr due to org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; routing to connection_failure: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1 I checked: NiFi is trying to connect http://172.17.0.2:8983/solr/tweets_shard1_replica1 but the tweets_shard1_replical is in below address and i can see it in my browser http://localhost:8983/solr/#/tweets_shard1_replica1 8983 and 2181 ports are added in my virtualbox. My solr server is on. All properties of PutSolrContentStream are configures the as this example "https://github.com/abajwa-hw/ambari-nifi-service" I appreciate your help with this error. Unfortunately I have some problems in playing with ports, host names and connections and I would like to read more about them, is there a good reference? thanks,

Online	Offline
Last Visited	‎03-07-2018 05:32 AM

Date Registered	‎02-08-2017 05:17 AM
Last Visited	‎03-07-2018 05:32 AM
Total Messages Posted	33
Total Kudos Received	1

Re: Hive query failed over JDBC.

Re: Hive query failed over JDBC.

Re: Is it possible to add a new node with differe...

Is it possible to add a new node with different a...

Re: what are steps of easiest way to create an HDP...

Re: what are steps of easiest way to create an HDP...

what are steps of easiest way to create an HDP clu...

how to save a single number from dstram into hive ...

Re: tweets sentinment hive 2.0 problem when I try ...

tweets sentinment hive 2.0 problem when I try to c...

Re: Hive query error when calculating tweets_senti...

Re: NiFi --PutHDFS error

Re: NiFi --PutHDFS error

How can I get some information of user by using a ...

Re: How to connect to a remote hdfs (for example h...

How to connect to a remote hdfs (for example horto...

Re: Apache Ambari Workflow Manager View for Apache...

Re: My workflow manager is not complete. I can not...

My workflow manager is not complete. I can not cre...

Re: Apache Ambari Workflow Manager View for Apache...

Re: Solr 5.5 on Sandbox 2.5 Has Stopped Running - ...

Re: How to get rid of "Can not find: /configs/twee...

Re: Sample HDF/NiFi flow to Push Tweets into Solr/...

Re: How to get rid of "Can not find: /configs/twee...

Re: How to get rid of "Can not find: /configs/twee...

Re: How to get rid of "Can not find: /configs/twee...

Re: How to get rid of "Can not find: /configs/twee...

How to get rid of "Can not find: /configs/tweets/a...

Re: "File /user/root/tmp/test.txt" could only be r...

How to connect from NiFi to Solr: org.apache.solr....