Member since
10-14-2016
32
Posts
1
Kudos Received
0
Solutions
11-23-2018
02:16 PM
I had limited resource on the yarn queue that I used for my spark job.
... View more
11-22-2018
01:59 PM
Hello All, I try to run a spark 2 job with oozie on HDP 2.6.5. Oozie starts successful, but my spark 2 job keeps in running state. on Resource manager UI, I got
Application Type:
SPARK
Application Tags:
oozie-7bf4f1cfb7c848de2db37230975b4450
Application Priority:
0 (Higher Integer value indicates higher priority)
YarnApplicationState:
ACCEPTED: waiting for AM container to be allocated, launched and register
with RM.
Kindly suggest.
... View more
Labels:
03-29-2018
06:40 AM
A there ways to solve this problem. Is it possible to compile the PutHDFS code with a new version of Hadoop code ?
... View more
10-12-2017
12:50 PM
I have done this, but this gives performance issues. I found how to do this with the JoltTransform processor.
... View more
10-12-2017
10:59 AM
Hello All, I want to flatten json records using nifi. The records are store in kafka as AVRO. I use the ConsumeKafkaRecord processor to convert avro to json. For flattening I would like to use the JoltTransformJSON processor, but this doesn't work since it expect as input a single json and not an array of json records. Any ideas, how I can do this ? Ideally is that I don't have to convert the records to json, but it seems that currently there is no processor to flatten avro. Regards, Chris
... View more
Labels:
- Labels:
-
Apache NiFi
10-12-2017
07:53 AM
1 Kudo
Yes, you can use one schema for read and a different one for write
... View more
08-03-2017
11:20 AM
Hello All, Does Stream Analytics manager support spark streaming ?
... View more
Labels:
- Labels:
-
Apache Spark
-
Cloudera DataFlow (CDF)
05-10-2017
05:07 AM
can you change outputStream.wrtie(text) with outputStream.write(text.encode('latin-1'))
... View more
05-09-2017
09:59 AM
Hello,
In the "process" method you can read for instance data from the inputstream, change some things and output it see https://community.hortonworks.com/content/kbentry/75032/executescript-cookbook-part-1.html for more information on executescript
text = IOUtils.toString(inputStream, IOUtils.toString(inputStream,StandardCharsets.ISO_8859_1)
...
outputStream.write(text)
... View more
02-03-2017
10:06 AM
Hi, I try to use the executeScript Processor with python for converting strings with special characters like é . For encoding I use latin-1. The script is : text = IOUtils.toString(inputStream, StandardCharsets.ISO_8859_1)
....
outputStream.write(bytearray(out.encode('latin-1')))
using this I get the follow error: org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: TypeError: write(): 1st arg can't be coerced to int, byte[] in <script> at line number 33 if I loop over the bytearray : text = IOUtils.toString(inputStream, StandardCharsets.ISO_8859_1)
....
out = bytearray(out.encode('latin-1'))
for o in out :
outputStream.write(o) I don't get this error. Thanks for your help Chris
... View more
Labels:
- Labels:
-
Apache NiFi