Member since
02-17-2016
49
Posts
43
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
773 | 05-27-2016 08:18 AM | |
1240 | 02-22-2016 10:50 AM |
05-27-2016
08:18 AM
Fixed it. I removed the atlas.pid file and restarted the atlas-server. rm /var/run/atlas/atlas.pid
python /usr/hdp/current/atlas-server/bin/atlas_start.py
... View more
05-26-2016
08:59 AM
Hi, I'm having trouble with Hive and Atlas. From what i understood, the hive error coms from atlas but i can't find anything to fix my atlas issue... I don't know which logs you need so please just tell me and i'll get them 🙂 Thank you in advance.
... View more
Labels:
- Labels:
-
Apache Atlas
-
Apache Hive
05-19-2016
08:51 AM
Works great. Thanks @Ravi Mutyala
... View more
05-18-2016
03:16 PM
Hi, I would like to get rid of some duplicates in my Hive table. The duplicates differs by the variable DateExtraction. DateExtraction is a date formated as "yyyy/MM/dd". In case of duplicate i would like to keep the most recent observation. Any clues are most welcome 🙂 Here is the query i used to create my table: DROP TABLE IF EXISTS netodb.oneTable;
CREATE EXTERNAL TABLE IF NOT EXISTS netodb.oneTable(
Entreprise string,Industrie string,Source string,Type string,Auteur string,
Id string,Texte string,DateCreation string,Likes bigint,Partages bigint,
ObjetParent string,Users_mentionned string,Hashtags_uses string,View_count string,
Dislike_count string,Comment_count string,DateExtraction date)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY "|"
LOCATION "/tmp/NetoDB/OneTable"
TBLPROPERTIES("transactional"="true");
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
03-23-2016
08:44 AM
Yes i use it to request the API i'm interested in once a week (except for the twitter streaming api of course).
... View more
03-22-2016
12:25 PM
1 Kudo
Well, most of the tutorials i found on Oozie were written 2 or 3 years ago so i was wondering if it was still the recommended tech for this kind of things.
... View more
03-22-2016
09:47 AM
3 Kudos
Hi, I would like to create/update Hive tables let's say once per hour. Which tool should i use to schedule the execution of my hql script? Thank you in advance
... View more
Labels:
- Labels:
-
Apache Hive
03-16-2016
11:08 AM
Hi @Chris Nauroth. I changed the dfs.permissions.superusergroup in /etc/hadoop/2.3.2.0-2950/0/hdfs-site.xml . Then i restarted all components for HDFS from the Ambari view. It appears that the change i had made in hdfs-site.xml has vanished. Am i doing it wrong or something?
... View more
03-14-2016
10:22 AM
1 Kudo
@Neeraj Sabharwal Thank you for your answer. Unfortunately i can't access ambari dashboard. For some reason, the username/password i've been using for one month doesn't work anymore. I thought it was related to this safemode thing but apparently it wasn't..
... View more
03-14-2016
10:16 AM
@Sagar Shimpi Thank you very much.
... View more
03-14-2016
09:06 AM
3 Kudos
Hi, It looks like i reached the memory limit for my sandbox. However i can't delete files since the Safemode is On. So i tried to turn it off manually with : hdfs dfsadmin -safemode leave but it makes nothing. From what i understood, it is because the memory is still full when i turn it off so the system immediatly goes back to safemode. How can i delete some files?
... View more
Labels:
- Labels:
-
Apache Hadoop
03-11-2016
04:45 PM
1 Kudo
@mburgess Thank you very much. I tried to change the remote Owner but apparently i was not executing nifi as super user. So i went to the shell, stopped nifi and typed : sudo bash /opt/nifi-1.1.1.0-12/bin/nifi.sh start However i'm still not considered as super user. Should i change something in the configuration of nifi?
... View more
03-11-2016
03:45 PM
1 Kudo
Hi, I'm having some permission issue with Hive. I wrote a file in HDFS using the PutHDFS nifi processor. When i try to create a table from this location in Hive Ambari view i get this error message : H110 Unable to submit statement. Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [admin] does not have [READ] privilege on [hdfs://sandbox.hortonworks.com:8020/tmp/fb_post_name2] [ERROR_STATUS] I can fix that by typing : sudo -u hdfs hadoop fs chown -R admin /tmp/fb_post_name2
sudo -u hdfs hadoop fs chmod 777 /tmp/fb_post_name2 But do i have a way to make it automatic (with the executeProcess processor for instance)? Thank you in advance.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi
03-09-2016
02:45 PM
1 Kudo
@mclark Thank you very much. Just to complete a bit your answer for those as new to nifi as i am : - the invokeHttp processor must send the "original" to replaceText - assuming the "Put Response Body in Attribute" property has been set to "Plouf"; the "Replacement Value" property in replaceText must be set to "${Plouf}"
... View more
03-09-2016
12:45 PM
a bit more information : to make my http request, i used routeOnAttribute --> InvokeHttp which works fine
... View more
03-09-2016
11:39 AM
1 Kudo
Hi, I'm having trouble manipulating flow file content in nifi : i have a flow file which contains a json. I use one of the field of this json to make a http request, up to this point everything works fine. I would like to merge the answer of this request with my original json. The merge content processor doesn't seem appropriate so i'm kinda stuck. Thank you in advance.
... View more
Labels:
- Labels:
-
Apache NiFi
03-04-2016
08:11 AM
1 Kudo
Thank you very much. It works splendidly. I'm not too sure of what was wrong in my json but it doesn't matter anymore.
... View more
03-03-2016
01:31 PM
1 Kudo
@Abdelkrim Hadjidj Thank you for your answer, the always output response property will certainly be usefull. I'm not trying to post anything on facebook. I want to make multiple requests to the graph API. According to the facebook documentation, i need to post a json with a batch of my requests (or something close). If you have any infos about that or an example, please post it under this question : https://community.hortonworks.com/questions/20104/trying-to-make-multiple-request-to-facebook-graph.html
... View more
03-03-2016
01:19 PM
1 Kudo
Adding the update processor with the mime.type attribute set to "application/json" solved the problem. *insert thank you gif*
... View more
03-02-2016
10:36 PM
1 Kudo
Thank you very much. I'll have a look at it tomorrow.
... View more
03-02-2016
08:23 AM
2 Kudos
Hi, I made a Nifi template with 3 processor : GetFile -->InvokeHttp-->PutHdfs. InvokeHttp makes a POST request to the facebook graph api and receives a JSON which is wrote in hdfs by PutHdfs. The issue i have is that the request i make doesn't seem to depend on the file read by GetFile. I read the Nifi documentation on InvokeHttp but i didn't find/understand the answer, hence my question. Thank you in advance.
... View more
Labels:
- Labels:
-
Apache NiFi
03-01-2016
11:10 AM
1 Kudo
@mburgess Thank you for your answer. InvokeHttp seems to work since i got an answer from the graph API that i can write in HDFS. However, i didn't manage to make this answer depend on my original JSON : whatever is writen in the file taken by the GetFile processor, even if it is not a JSON formated file, the answer is the same. I looked at the nifi documentation on InvokeHttp and tried to put something in the property "Attributes to send" but it did nothing so far. Do you know how InvokeHttp interacts with the FlowFiles coming from GetFile?
... View more
02-29-2016
10:56 AM
1 Kudo
Hi, I need to make a lot of request to the Facebook Graph API using Nifi. I looked at the documentation https://developers.facebook.com/docs/graph-api/making-multiple-requests which tells me that i need to POST a JSON which contains the request i want to make in a batch format to a specific endpoint. So i created a Nifi template with 3 processors : GetFile -->PostHTTP-->PutHDFS. GetFile reads a very simple JSON file i made to test. When i run the template, i have no error message. However, the HDFS file wrote is just a copy of my initial JSON file. Any explanation / solution? Thank you in advance.
... View more
Labels:
- Labels:
-
Apache NiFi
02-26-2016
09:00 AM
1 Kudo
@Andy LoPresto Thank you for your answer, one question to be sure : if i don't pay too much attention to security, can i use the graph api without adding the Facebook CA into my trustore?
... View more
02-25-2016
05:53 PM
1 Kudo
Definetly not an answer but i'll leave it here : i "solved" the issue by putting the same file and password for the keystore than i had for the truststore (the ones mentionned in the questions/9509). Probably not something to do since i have absolutly no understanding of SSL but it allowed me to move forward.. I'm still interested in a real answer.
... View more
02-25-2016
09:35 AM
5 Kudos
Hi, I've been trying to pull data from the facebook graph api using nifi. As i need to make a lot of request, i want to be able to post a json to http://graph.facebook.com as explained here https://developers.facebook.com/docs/graph-api/making-multiple-requests. So i created a PostHTTP processor on NiFi but it requires a SSL context. Now i found some answers thanks to @Aldrin Piri and his answer to https://community.hortonworks.com/questions/9509/connecting-to-datasift-https-api-using-nifi.html but i still don't know how to fill the fields "Keystore Filename" and "Keystore Password". Thank you in advance.
... View more
Labels:
- Labels:
-
Apache NiFi
02-23-2016
04:44 PM
never mind. Found it, thank you very much. 🙂
... View more
02-23-2016
04:39 PM
1 Kudo
it tells me that queue isn't empty for one of the connection. How can i clear the queue? or get rid of it anyway ?
... View more
02-23-2016
04:26 PM
1 Kudo
Everything is in the question. I suppressed the template from the template management table which is now empty. Yet the canvas isn't and i didn't find how to clear it. Please help it's driving me nuts :'(
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache NiFi