ali_mohammadi
Explorer

Re: Hive query failed over JDBC.

by Explorer in Support Questions
‎03-07-2018 05:34 AM
‎03-07-2018 05:34 AM
I had also been facing that issue in hive, but I solved by setting "use.hive.interactive.mode" to true and restart the Hive View to run in llap mode. thank you ... View more

Re: Hive query failed over JDBC.

by Explorer in Support Questions
‎03-03-2018 07:41 AM
‎03-03-2018 07:41 AM
Dear @Girish Chaudhari, Unfortunately, I have faced this issue as well, but I could not understand the BEST ANSWER submitted by you. If it is possible, please help me. thanks a million, Ali ... View more

Re: Is it possible to add a new node with differe...

by Explorer in Support Questions
‎11-21-2017 08:18 AM
‎11-21-2017 08:18 AM
@Jay Kumar SenSharma thanks ... View more

Is it possible to add a new node with different a...

by Explorer in Support Questions
‎11-20-2017 10:33 AM
‎11-20-2017 10:33 AM
Hi everybody, We have a cluster with 3 VM nodes on HP servers(64 GB ram and 10 TB capacity and 8 cores). Vsphere 5.5 has been used in this project to create virtual machines. Now we need some extra capacity to add to HDFS, but there is no space on the HP server and new VMs can not be constructed. The question is this: could I add a PC or a laptop with the same operating system as a new node to my cluster? Maybe, it is good to ask in this way: what kind of systems can be used together for constructing hdp clusters? thanks a million, Shanghoosh ... View more

Re: what are steps of easiest way to create an HDP...

by Explorer in Support Questions
‎10-14-2017 06:05 AM
‎10-14-2017 06:05 AM
@pdarvas thanks, I am going to check it. ... View more

Re: what are steps of easiest way to create an HDP...

by Explorer in Support Questions
‎10-10-2017 06:19 AM
‎10-10-2017 06:19 AM
@pdarvasi Thanks for your valuable and useful suggestion. Actually, my major problem is step one. I need to have openstack on current hp servers and also need to extend it whenever it is necessary. shall I install ESXi and then create a VM and after that install openstack? or I can install openstack on raw servers and then can create a extendable private cloud without ESXi. In fact, I am confusing with creating clusters. For example, I thought, I have to manage a cluster of servers on Vsphere, a cluster of nodes on openstack, and then a hadoop cluster on Ambari. It's still confusing for me. It would be appreciated if you could give me more information or introduce some tutorials. thanks a million, ALi ... View more

what are steps of easiest way to create an HDP clu...

by Explorer in Support Questions
‎10-04-2017 12:33 PM
‎10-04-2017 12:33 PM
Hi, I am new in making private clouds and hadoop ecosystem. We have three HP servers without ESX or openstack, and want to create a private cloud and then have an HDP cluster with Ambari. It would be appreciated, If somebody could explain the steps of creating a cloud and hadoop cluster by using (ESXi, openstack, cloudbreak, Ambari and ....). there are some tutorials to do this, but they are confusing to me. some of our requirements are as follows: 1- we must have at least 100TB HDFS capacity. 2- we should be able to add new nodes for increasing the capacity of HDFS and processing power. 3- we should be able to share the HDP cluster with multiple users. each user should be able to run own map/reduce job. Sincerely, ... View more

how to save a single number from dstram into hive ...

by Explorer in Support Questions
‎08-28-2017 10:28 AM
‎08-28-2017 10:28 AM
Hi I read data from oracle to kafka with nifi and I want to analyze my records(for example sum of attributes) with spark streaming. I sum all of numbers in an rdd and want to add it by previous rdds value of summation. here, there is a problem! At the end of each stream, the type of variable that hold the sum of arrived data per 10 seconds is Dstream. how can I import this (just a number in dstream)to hive table? in the belove code, sum contains a number but in dstream format that I want to to insert it to hive. thanks a million Ali val words = lines.flatMap{ case(x, y) => y.split("\n")} val words1=words.map{aline=> aline.split(" ")(3).replaceAll("\"","").replaceAll("}","")} val sum= words1.map{ss => ss.toDouble}.reduce(_+_) ... View more

Re: tweets sentinment hive 2.0 problem when I try ...

by Explorer in Support Questions
‎07-29-2017 05:33 AM
‎07-29-2017 05:33 AM
Dear @Marco Gaido, I deleted my Json files in hdfs and then I ingested some other tweets in json format. Now, hive is working well. thanks a lot ... View more

tweets sentinment hive 2.0 problem when I try to c...

by Explorer in Support Questions
‎07-26-2017 11:16 AM
‎07-26-2017 11:16 AM
about tutorial: https://hortonworks.com/tutorial/analyzing-social-media-and-customer-sentiment-with-apache-nifi-and-hdp-search/ I have been following above tutorial and everything worked fine but when calculating whether a tweet was positive, neutral, or negative using this next Hive command : create table IF NOT EXISTS tweets_sentiment stored as orc as select tweet_id, case when sum( polarity ) > 0 then 'positive' when sum( polarity ) < 0 then 'negative' else 'neutral' end as sentiment from l3 group by tweet_id; I got the following error message: tanks a million java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1501058816878_0021_1_01, diagnostics=[Task failed, taskId=task_1501058816878_0021_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"tweet_id":890076549655003136,"created_unixtime":1501045760917,"created_time":"Wed Jul 26 05:09:20 +0000 2017","lang":"en","displayname":"2ne1legend21","time_zone":"","msg":"RT hyung_rose_bts BTS_twt The cup couldnt have said it better at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: Unterminated string at 237 [character 238 line 1] at org.openx.data.jsonserde.JsonSerDe.onMalformedJson(JsonSerDe.java:424) at org.openx.data.jsonserde.JsonSerDe.deserialize(JsonSerDe.java:183) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:113) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:554) ... 18 more ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1501058816878_0021_1_01 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1501058816878_0021_1_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:2, Vertex vertex_1501058816878_0021_1_02 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 (less...) ... View more

Re: Hive query error when calculating tweets_senti...

by Explorer in Support Questions
‎07-26-2017 11:09 AM
‎07-26-2017 11:09 AM
@Luis Martinez hi if you have solved the problem, please let us know ... View more

Re: NiFi --PutHDFS error

by Explorer in Support Questions
‎07-24-2017 09:31 AM
‎07-24-2017 09:31 AM
may be you lose some data when you remove the docker ... View more

Re: NiFi --PutHDFS error

by Explorer in Support Questions
‎07-24-2017 09:28 AM
‎07-24-2017 09:28 AM
add port 50010 like this https://community.hortonworks.com/articles/65914/how-to-add-ports-to-the-hdp-25-virtualbox-sandbox.html also you should add 50010 to VMbox ... View more

How can I get some information of user by using a ...

by Explorer in Support Questions
‎04-09-2017 11:42 AM
‎04-09-2017 11:42 AM
I am trying to implement a movie recommender system for users. I entirely implemented the ALS algorithm and now I want to get new user rates and after that to recommend some other movies. I programmed my code as follows in zeppelin to get information: %pyspark myRatedMovies = [ (0, 1,1.0), (0, 2,1.5), (0, 3,1.1), (0, 4,1), (0,5,1.5), (0,6,1.0), (0, 5,5), (0,52,5), (0,96,5.0) ] myRatingsRDD = sc.parallelize(myRatedMovies) I created an RDD myself and after that I append it in my train dataset and so on, but I want to have a user interface to get information. Actually I want to have a dynamic UI. I saw this page "https://zeppelin.apache.org/docs/0.6.1/manual/dynamicform.html", but it did not help me because I need to get a lot of tuples and I did not find some helpful information about command buttons on that web page. Also I know, I can use angular in zeppelin, but I am new in zeppelin and angulary, and it is very hard to me. For example I can not use the binded variables in pyspark. I would like to add user rates to an RDD when the user clicks on the button "Add My Rate" . Thank you very much for your time and assistance in this matter. Sincerely, Shanghoosh ... View more
Labels:

Re: How to connect to a remote hdfs (for example h...

by Explorer in Support Questions
‎04-05-2017 05:26 AM
‎04-05-2017 05:26 AM
Hi @Jay SenSharma Thank you for excellent notes. I solved the problem before reading your notes as follows: from hdfs import InsecureClient client = InsecureClient('http://localhost:50070') # for reading a file with client.read('/tmp/tweets_staging/tweets-082940117.json') as reader: features = reader.read() # for writing a file with client.write('/tmp/tweets_staging/1.json', overwrite=True) as writer: writer.write(features) but I am going to try your solution too. Regards, Shanghoosh ... View more

How to connect to a remote hdfs (for example horto...

by Explorer in Support Questions
‎04-04-2017 11:25 AM
‎04-04-2017 11:25 AM
I want to read and write files to and from a remote HDFS. I program by Pycharm in local machine and I want to connect to a remote hdfs (HDP 2.5). Is there any solution? How can I configure hdfs and how can I refer to a file in hdfs? Thanks a million, Shanghoosh ... View more
Labels:

Re: Apache Ambari Workflow Manager View for Apache...

by Explorer in Community Articles
‎02-28-2017 04:41 AM
‎02-28-2017 04:41 AM
thank you Actually I had clicked the checkbox. I think, I should wait for a new version of Ambari; my Ambari is 2.4 . ... View more

Re: My workflow manager is not complete. I can not...

by Explorer in Support Questions
‎02-28-2017 04:31 AM
‎02-28-2017 04:31 AM
Thank you @Artem Ervits Yes, I should wait for Ambari 2.5. ... View more

My workflow manager is not complete. I can not cre...

by Explorer in Support Questions
‎02-26-2017 08:13 AM
‎02-26-2017 08:13 AM
Hello guys First of all, I am using HDP 2.5. In a tutorial with this address " https://community.hortonworks.com/articles/82964/getting-started-with-apache-ambari-workflow-design.html " thanks to @Artem Ervits that he showed we can create workflows, coordinatores and bundles. In the tutorial I see a view the same as following: but I can not see them in my ambari. I just see workflows as the following photo: How can I solve the problem? thanks a million shanghoosh ... View more

Re: Apache Ambari Workflow Manager View for Apache...

by Explorer in Community Articles
‎02-26-2017 07:18 AM
‎02-26-2017 07:18 AM
thanks for great tutorial. unfortunately I couldn't see the output. where can I see? And also I do not see create coordinators button 😞 ... View more

Re: Solr 5.5 on Sandbox 2.5 Has Stopped Running - ...

by Explorer in Support Questions
‎02-18-2017 12:18 PM
‎02-18-2017 12:18 PM
Hi @Raf Mohammed I have a same problem. Can you help me? I do not know about stale configs for solr. thank you ... View more

Re: How to get rid of "Can not find: /configs/twee...

by Explorer in Support Questions
‎02-18-2017 04:42 AM
‎02-18-2017 04:42 AM
Hi @kkawamura Finally I solved it. I installed Solr by using ambari (befor, Solr was in VM but it was not in sandbox) and after that I set "localhost:2181/solr " to PutSolrContentStream processor. Now I am working with Banana,Solr,Nifi and Hive so comfortable. Thanks Koji a million. ... View more

Re: Sample HDF/NiFi flow to Push Tweets into Solr/...

by Explorer in Community Articles
‎02-15-2017 11:11 AM
‎02-15-2017 11:11 AM
Thank you @Ali Bajwa for good tutoral. I am trying this example with a difference, My nifi is local and I try to put tweets in a remote Solr. Solr is in a VM that contains Hortonworks sandbox. Unfortunately I am getting this error on PutSolrContentStream processor: PutSolrContentStream[id=f6327477-fb7d-4af0-ec32-afcdb184e545] Failed to send StandardFlowFileRecord[uuid=9bc39142-c02c-4fa2-a911-9a9572e885d0,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1487148463852-14, container=default, section=14], offset=696096, length=2589],offset=0,name=103056151325300.json,size=2589] to Solr due to org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; routing to connection_failure: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; Could you help me? thanks, Shanghoosh ... View more

Re: How to get rid of "Can not find: /configs/twee...

by Explorer in Support Questions
‎02-15-2017 09:33 AM
‎02-15-2017 09:33 AM
Thank you @kkawamura for nice summarization. I tried any case in hosts files locally and also in VM, but it didn't work. Are there any other processors in NiFi to help me? I mean, Can I work with Solr with another processor? ... View more

Re: How to get rid of "Can not find: /configs/twee...

by Explorer in Support Questions
‎02-14-2017 09:17 AM
‎02-14-2017 09:17 AM
Another thing, When I try bin/solr status It gives me Found 1 Solr nodes: Solr process 11364 running on port 8983 { "solr_home":"/opt/lucidworks-hdpsearch/solr/server/solr/", "version":"5.2.1 1684708 - shalin - 2015-06-10 23:20:13", "startTime":"2017-02-14T05:24:44.191Z", "uptime":"0 days, 3 hours, 36 minutes, 37 seconds", "memory":"101.6 MB (%20.7) of 490.7 MB", "cloud":{ "ZooKeeper":"localhost:2181", "liveNodes":"1", "collections":"2"}} and when I try bin/solr healthcheck -c tweets -z localhost:2181 It gives { "collection":"tweets", "status":"healthy", "numDocs":1, "numShards":1, "shards":[{ "shard":"shard1", "status":"healthy", "replicas":[{ "name":"core_node1", "url":"http://172.17.0.2:8983/solr/tweets_shard1_replica1/", "numDocs":1, "status":"active", "uptime":"0 days, 3 hours, 37 minutes, 5 seconds", "memory":"27.4 MB (%5.6) of 490.7 MB", "leader":true}]}]} but when I try bin/solr healthcheck -c tweets It gives ERROR: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper localhost:9983 within 10000 ms what is the default port of zookeeper ? Is it possible solr nodes can't to communicate with zookeeper? ... View more

Re: How to get rid of "Can not find: /configs/twee...

by Explorer in Support Questions
‎02-14-2017 08:11 AM
‎02-14-2017 08:11 AM
Hi Koji thank you 🙂 Yes, I want to try this example " https://github.com/abajwa-hw/ambari-nifi-service ", with a difference: my Nifi is in my local machine and I want to connect to a remote lucidworks locally. The lucidworks(solr) is in a VM(with sandbox). screenshot-from-2017-02-14-11-36-39.png(128.3 kB) I am trying to add tweets in solr. ... View more

Re: How to get rid of "Can not find: /configs/twee...

by Explorer in Support Questions
‎02-13-2017 01:17 PM
‎02-13-2017 01:17 PM
Hello @kkawamura First of all, I am new in Solr. When I tried the curl command "curl -i http://172.17.0.2:8983/solr " in VM (sandbox), I got a message as below: HTTP/1.1 302 Found Location: http://172.17.0.2:8983/solr/ Content-Length: 0 but when I try in the local machine (ubuntu 16), I don't get any messages. Also, when I try "curl -i localhost:8983/solr" locally, I get a message as follows: HTTP/1.1 302 Found Location: http://localhost:8983/solr/ Content-Length: 0 Actually, I use PutSolrContentStream processor for sending tweets to a remote Solr (lucidworks) in a VM (sandbox). I set Solr type= cloud and Solr Location= localhost:2181. NiFi is in my local machin. Unfortunately, I get an error as follows: PutSolrContentStream[id=f6327477-fb7d-4af0-e990-512eebdb1c1c] Failed to send StandardFlowFileRecord[uuid=c478c4dc-f16e-4ae7-814e-a7de0985985e,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1486893589035-10, container=default, section=10], offset=612767, length=7103],offset=0,name=77207261802126.json,size=7103] to Solr due to org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; routing to connection_failure: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1 Excuse me for my poor English. Thanks a Million, Shanghoosh ... View more

How to get rid of "Can not find: /configs/tweets/a...

by Explorer in Support Questions
‎02-13-2017 08:07 AM
‎02-13-2017 08:07 AM
I am trying to connect to a remote solr but I faced with this error on server can not find: /configs/tweets/admin-extra.html and also faced to this error org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr when I tried to connect to Solr from NiFi thank you guys ... View more

Re: "File /user/root/tmp/test.txt" could only be r...

by Explorer in Support Questions
‎02-12-2017 08:55 AM
‎02-12-2017 08:55 AM
Notice: some information will be lost when you do what we did. for example I had a lucidworks but now I should install it again. ... View more

How to connect from NiFi to Solr: org.apache.solr....

by Explorer in Support Questions
‎02-12-2017 06:18 AM
1 Kudo
‎02-12-2017 06:18 AM
1 Kudo
Hi Guys I am trying to add tweets in a remote solr but I faced with this error. PutSolrContentStream[id=f6327477-fb7d-4af0-e990-512eebdb1c1c] Failed to send StandardFlowFileRecord[uuid=34e67c1a-3a18-4280-a738-9943d6716962,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1486876718587-1, container=default, section=1], offset=53886, length=2508],offset=0,name=60294765850236.json,size=2508] to Solr due to org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; routing to connection_failure: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1 I checked: NiFi is trying to connect http://172.17.0.2:8983/solr/tweets_shard1_replica1 but the tweets_shard1_replical is in below address and i can see it in my browser http://localhost:8983/solr/#/tweets_shard1_replica1 8983 and 2181 ports are added in my virtualbox. My solr server is on. All properties of PutSolrContentStream are configures the as this example "https://github.com/abajwa-hw/ambari-nifi-service" I appreciate your help with this error. Unfortunately I have some problems in playing with ports, host names and connections and I would like to read more about them, is there a good reference? thanks, ... View more
Contact Me
Online
Offline
Last Visited
‎03-07-2018 05:32 AM
Public Statistics
Date Registered ‎02-08-2017 05:17 AM
Last Visited ‎03-07-2018 05:32 AM
Total Messages Posted 33
Total Kudos Received 1