1973
Posts
1225
Kudos Received
124
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1948 | 04-03-2024 06:39 AM | |
| 3063 | 01-12-2024 08:19 AM | |
| 1671 | 12-07-2023 01:49 PM | |
| 2449 | 08-02-2023 07:30 AM | |
| 3413 | 03-29-2023 01:22 PM |
11-11-2016
04:20 PM
2 Kudos
I have a table with a few thousand rows in a small dev cluster and I am running a count on a ORC table and Tez is enabled. The count takes a really really really long time or times out. Any suggestions?
... View more
Labels:
- Labels:
-
Apache Hive
11-10-2016
05:05 PM
is this UTF-8?
... View more
11-08-2016
08:00 PM
It got a different error, when I removed that attribute. I also turned off use internal attributes into convert to JSON. Maybe be internal fields are an issue.
... View more
11-08-2016
07:56 PM
2016-11-08 19:35:09,121 WARN [Timer-Driven Process Thread-9] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConvertJSONToAvro[id=10091199-14c4-1ad1-fdc6-9c4abcfa4349] due to uncaught Exception: java.lang.IllegalArgumentException: Schemas for JSON files should be record
2016-11-08 19:35:09,123 WARN [Timer-Driven Process Thread-9] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalArgumentException: Schemas for JSON files should be record
at org.kitesdk.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) ~[na:na]
at org.kitesdk.data.spi.filesystem.JSONFileReader.initialize(JSONFileReader.java:84) ~[na:na]
at org.apache.nifi.processors.kite.ConvertJSONToAvro$1.process(ConvertJSONToAvro.java:144) ~[na:na]
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2370) ~[nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
at org.apache.nifi.processors.kite.ConvertJSONToAvro.onTrigger(ConvertJSONToAvro.java:139) ~[na:na]
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) ~[nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_101]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_101]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_101]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_101]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Error on inferAvroSchema from JSON file.
... View more
Labels:
- Labels:
-
Apache NiFi
11-08-2016
07:56 PM
Facebook only let's you see information that you are allowed to see. You have set your Facebook security settings and your app settings. You also will only have access to information that users want you to see. Work it out in the Facebook graph explorer first and then use that API call. That may be unicode. http://stackoverflow.com/questions/3806215/facebook-graph-api-non-english-album-names Are you using an American English ID or another language that may be stored in unicode due to other characters not in UTF-8? Characters like these will be unicode encoded in JSON: பதவியேற்ற
... View more
11-08-2016
05:36 PM
1. Make sure you have the SSLcontext created. 2. it is either encrypted or uuencoded. try to decode with UUDecoder in nifi
... View more
11-08-2016
03:13 PM
The one thing I didn't do in the article Milind posted was the paging aspect. That can be done it just takes a few extra steps and processors to step through to get potentially more. If you are running this constantly you will not need it.
... View more
11-08-2016
03:11 AM
https://issues.apache.org/jira/browse/NIFI-353 https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-content-viewer/src/main/webapp/WEB-INF/jsp https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-content-viewer/src/main/webapp/META-INF/nifi-content-viewer --- seems you could add some more there. You can UpdateAttribute and set mime.type to a viewable type
... View more
11-07-2016
04:48 PM
hdfs dfs -cat /music/meta/amclassical_beethoven_fur_elise.mp3
{"xmpDM:trackNumber":"","invokehttp.tx.id":"b4690a5a-ec60-4f68-8d1a-de9344eced8d","xmpDM:releaseDate":"2001","Server":"Apache","gethttp.remote.source":"192.168.1.2","xmpDM:artist":"A-M Classical","fragment.identifier":"d8b6da55-c512-42c5-b9a0-a93b0f32fd71","link":"http://www.amclassical.com/mp3/amclassical_beethoven_fur_elise.mp3","dc:creator":"A-M Classical","Last-Modified":"Sun, 05 Oct 2008 20:59:37 GMT","title":"Für Elise","xmpDM:audioChannelType":"Stereo","uuid":"d1e6b6cb-63cc-46a8-ac63-a588da01ecff","invokehttp.request.url":"http://www.amclassical.com/mp3/amclassical_beethoven_fur_elise.mp3","path":"./","xmpDM:logComment":"eng -","xmpDM:audioSampleRate":"44100","dc:title":"Für Elise","OkHttp-Sent-Millis":"1478537014168","segment.original.filename":"mp3.json","Content-Length":"3393536","Content-Type":"audio/mpeg","samplerate":"44100","Keep-Alive":"timeout=2, max=95","xmpDM:genre":"Classical","xmpDM:composer":"Ludwig van Beethoven","X-Parsed-By":"org.apache.tika.parser.DefaultParser, org.apache.tika.parser.mp3.Mp3Parser","creator":"A-M Classical","xmpDM:album":"","meta:author":"A-M Classical","invokehttp.status.code":"200","Connection":"Keep-Alive","fragment.index":"0","xmpDM:audioCompressor":"MP3","mime.type":"audio/mpeg","version":"MPEG 3 Layer III Version 1","Date":"Mon, 07 Nov 2016 16:38:13 GMT","Accept-Ranges":"bytes","descr":"Beethoven: Für Elise","filename":"amclassical_beethoven_fur_elise.mp3","OkHttp-Received-Millis":"1478537014194","channels":"2","ETag":"\"33c800-45887d8256040\"","Author":"A-M Classical","fragment.count":"28","invokehttp.status.message":"OK","xmpDM:duration":"211996.546875"} I added an ExtractMediaMetaData processor and examined the mp3s
... View more
11-07-2016
04:31 PM
Next steps, use the audio processor. Next step generate music with DL4J and TensorFlow. Next steps connect NIFI to a MIDI synthesizer or Raspberry PI SonicPi.
... View more