Member since
04-29-2016
33
Posts
8
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1517 | 05-02-2016 06:54 PM | |
2723 | 05-02-2016 04:36 PM | |
549 | 05-02-2016 04:31 PM | |
2748 | 05-02-2016 03:05 PM |
07-27-2016
06:27 AM
generally working now. except that every time it runs, it returns one and exactly one result .json file. i find it hard to believe that there is exactly one tweet meeting my criteria ever 30 seconds.
... View more
07-27-2016
05:44 AM
I've just finished reading up on twitter's API and perhaps rate-limiting is the issue. I'm using a template pulled from HWX and it looks like it had the scheduling set to zero seconds. I have increased it to 30 seconds and still it does not work. Will reset my keys and see if that fixes it. perhaps twitter cuts off access for those identifiers. What is a "reasonable" schedule time that won't run into the rate limit? Here are some log messages: 2016-07-27 05:38:49,352 INFO [hosebird-client-io-thread-0] com.twitter.hbc.httpclient.ClientBase GetTwitter[id=b2e83358-2e38-42c5-88a7-9f8a3f8307ba] Establishing a connection 2016-07-27 05:38:50,837 WARN [hosebird-client-io-thread-0] com.twitter.hbc.httpclient.ClientBase GetTwitter[id=b2e83358-2e38-42c5-88a7-9f8a3f8307ba] Error connecting w/ status code - 420, reason - Enhance Your Calm 2016-07-27 05:39:07,949 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.twitter.GetTwitter GetTwitter[id=b2e83358-2e38-42c5-88a7-9f8a3f8307ba] Received error HTTP_ERROR: HTTP/1.1 420 Enhance Your Calm. Will attempt to reconnect
... View more
07-27-2016
04:04 AM
Get Twitter is working some but not all times. If I connect to the sample endpoint, it retrieves data. If I connect to the filter endpoint with a valid filter that contains popular topics, it retrieves nothing. If I go back to the sample endpoint, it eventually stops getting any data. If I regenerate my twitter keys, it seems to work for a while and then stops getting data. I saw the issue regarding server clock synchronization and I have made sure that nntp is running properly and the clock is right. not sure what else to adjust.
... View more
Labels:
- Labels:
-
Apache NiFi
07-27-2016
03:15 AM
thanks! had this same problem and getting ntpd running properly fixed it.
... View more
05-03-2016
05:49 PM
I got now(72,0) to work. don't know the ramifications....
... View more
05-03-2016
05:39 PM
thanks! that's what I was trying to do but couldn't figure out the right syntax.
... View more
05-03-2016
05:22 PM
The tutorial says to use "now(0,0)" as the instance times, this is rejected with exception "start instance is before start of feed". What does "now(0,0)" mean, and what would be the proper value use? I suspect this is some kind of time relative to the current time. I searched for help on what "now(0,0)" means and didn't find anything.
... View more
Labels:
05-02-2016
06:54 PM
1 Kudo
solved it. the problem is introduced by copy/paste text from the tutorial page. it must introduce some garbage characters. someone else noted this as a problem with starting one of the other interpreters.
... View more
05-02-2016
04:36 PM
1 Kudo
What browser are you using? I've had this same problem and it seemed to be caused by Ambari + IE 11 compatibility issues. I switched over to chrome and it worked fine.
... View more
05-02-2016
04:31 PM
@Viren Papaiyawala what browser are you using? I had same problem with IE 11. Switched over to chrome and it worked.
... View more
05-02-2016
04:16 PM
@nmaillard thanks. checked as you say and all is as expected. tried again and got the same error. same stack trace. If this helps, I can run spark from within zeppelin. Problem seems to be only with hive interpreter.
... View more
05-02-2016
03:40 PM
1 Kudo
I see a green "connected" icon. I check the zeppelin interpreter page, and hive listed. I restart it from that page. I can run the hive cli. I check the yarn application list and there is nothing running. when I try to run simple query from the tutorial %hive SELECT * FROM finalresult I get "hive interpreter not found" stack trace from the zeppelin log file: ERROR [2016-05-02 15:27:45,626] ({qtp61918702-31} NotebookServer.java[afterStatusChange]:800) - Error
org.apache.zeppelin.interpreter.InterpreterException: hive
interpreter not found
at org.apache.zeppelin.notebook.NoteInterpreterLoader.get(NoteInterpreterLoader.java:148)
at org.apache.zeppelin.notebook.Note.run(Note.java:368)
at org.apache.zeppelin.socket.NotebookServer.runParagraph(NotebookServer.java:711)
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:143)
at org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
at org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
... View more
Labels:
05-02-2016
02:46 PM
@Neeraj Sabharwal thanks. I killed all of the yarn applications and tried again. Both the hive CLI and the ambari hive query editor still hang up. the initial issue (reported separately) was with the ambari hive query: CREATE TABLE events
STORED AS ORC
AS
SELECT driverid, city, state FROM geolocation; as per the tutorial. so perhaps the issue is with creating ORC files? and that is hung and cascading back to any hive activity? All I am doing is trying to follow the tutorial. very frustrating. here is yarn application list: ^C[hive@sandbox hive]$ yarn application -list
16/05/02 14:38:35 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
16/05/02 14:38:35 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.15:8050
Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):1
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
application_1461958087140_0034 HIVE-c01d9d1e-b461-4770-822b-b46a7ceeecf2 TEZ hive default RUNNING UNDEFINED 0% http://sandbox.hortonworks.com:44396/ui/
[hive@sandbox hive]$ yarn application -list
16/05/02 14:41:24 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
16/05/02 14:41:24 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.15:8050
Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):2
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
application_1461958087140_0035 HIVE-caf7b780-54c9-436b-9c90-28bd6db769dd TEZ hive default RUNNING UNDEFINED 0% http://sandbox.hortonworks.com:51481/ui/
application_1461958087140_0034 HIVE-c01d9d1e-b461-4770-822b-b46a7ceeecf2 TEZ hive default RUNNING UNDEFINED 0% http://sandbox.hortonworks.com:44396/ui/
[hive@sandbox hive]$
... View more
05-02-2016
02:10 PM
hivelog.txtattached is the hive log. just tried this again and same thing happened. just a nuisance, but the upload refuses to accept files with extension of .log. sigh....
... View more
05-01-2016
02:53 AM
I start the hive shell. it doesn't give me an kind of prompt. I can type away, it echoes what I type but no response: [hive@sandbox root]$ hive
WARNING: Use "yarn jar" to launch YARN applications. Logging initialized using configuration in file:/etc/hive/2.4.0.0-169/0/hive-log4j.properties
select * from truck_mileage;
go
^C[hive@sandbox root]$
[hive@sandbox root]$ hive
WARNING: Use "yarn jar" to launch YARN applications. Logging initialized using configuration in file:/etc/hive/2.4.0.0-169/0/hive-log4j.properties
hello there
anyone home?
;
;
;
;;
... View more
Labels:
05-01-2016
12:01 AM
1 Kudo
@hduraiswamy this seems to be an IE 11 compatibility issue. It works fine with Chrome. Chrome shows all of the pig scripts that were created. IE shows some of them. it seems to show only those that were created during a prior session. I created two new ones just now. one with chrome, one with IE. Chrome shows all of the scripts (created previously, new from chrome, new from IE), IE doesn't show the ones that were just created (either with chrome or IE).
... View more
04-30-2016
11:48 PM
2 Kudos
@hduraiswamy I know the problem. It is an IE 11 feature that I encountered and dealt with for one of my own apps. I can do the upload with Chrome. IE 11 passes the full local file pathname at times (depends upon user browser options) . Ambari needs to strip off the path prefix when forming the server filename.
... View more
04-30-2016
11:27 PM
yes it is the sandbox, out of the box, no config changes made. just downloaded it the other day, so it is the most recent version. browser IE 11 I've noticed a couple of other weird things happening, and I have a suspicion of browser compatibility issues with IE 11. will try this with chrome. trying to save into the default directory, just following the tutorial scripts. so whatever directories, permissions etc. that are set in the sandbox VM are what are being used. will check on permissions.
... View more
04-30-2016
11:19 PM
thanks @hduraiswamy used filezilla to do the uploads. would be nice it this ambari feature worked because it is really convenient.
... View more
04-30-2016
04:50 AM
create a pig script. save it confirmation message comes back to confirm that it has been saved go to something else, like "tez view" go back to pig it says that there are no scripts. my work has vanished. in one case, I was able to run the script, go over to tez to see what had happened, and when returned to pig it was gone (even after saving) tried this a couple of times and the same thing happened. scripts disappeared each time.
... View more
Labels:
04-29-2016
08:34 PM
1 Kudo
attempts to upload a file from ambari fails by generating an invalid target file name. it creates a target filename (hdfs) by removing all of the path delimiters from the source file name, resulting in a ridiculously long file name that additionally contains "c:". I don't see any way of controlling this. running it all on windows. all I do is browse for the local file and then upload. exception: 500 Pathname /tmp/maria_dev/data/C:UsersXXXXXDownloadsHortonworksGeolocationgeolocation.csv from /tmp/maria_dev/data/C:UsersXXXXXDownloadsHortonworksGeolocationgeolocation.csv is not a valid DFS filename. Collapse Stack Trace java.lang.IllegalArgumentException: Pathname /tmp/maria_dev/data/C:UsersklyborDownloadsHortonworksGeolocationgeolocation.csv from /tmp/maria_dev/data/C:UsersklyborDownloadsHortonworksGeolocationgeolocation.csv is not a valid DFS filename
... View more
Labels: