@Mukesh Kumar Not sure why the article is protected, but it essentially says to switch the superset database to mysql/postgreSQL instead of sqlite for a temporary work around. You can adjust this in Ambari by going to the druid service and modifying the config for the "Superset Database name". ... View more

@mayki wogno It should already be available in your script, if you use this: flowFile = session.get() flowFile.getAttribute('filename') This should return your filename in your executeScript. ... View more

@Rohit Ravishankar That should be fine, you could also use splitText processor before the extractText processor. So basically each flowfile would contain a single record. ... View more

@Praveen PentaReddy Can you please mark an answer as accepted, and then create a new question with your new problem? This way it is indexed and other people with a similar new problem will be able to benefit? Please post your Flume conf file as well, i believe you might have a bind property off in it which is why the sink is not working. Thanks. ... View more

@qfdk Would you mind sending a copy of the data that comes out of the "replaceText" processor? I would like to try on my local system to post to ES. Do you have the mapping for that index as well in ES? Is the data as simple as: { "gid" : 799092, "coordinates" : [ 6.933880894889725, 48.0070784663541 ]}, { "gid" : 799092, "coordinates" : [ 6.933880894889725, 48.0070784663541 ]},{ "gid" : 799092, "coordinates" : [ 6.933880894889725, 48.0070784663541 ]} It looks to me like you should be using just the PutElasticSearch processor which uses the bulk api. I see in your "PostHTTP" processor in your video that you are pushing out to the "_bulk" endpoint. Looking at your error log, it seems it is not formatted correctly when being indexed in ES: "Caused by: java.lang.IllegalArgumentException: Malformed content, found extra data after parsing: START_OBJECT" ... View more

@qfdk In your flow, note that the "PutElasticSearch" processor actually uses the ES bulk api: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-bulk.html. So if you are specifying a reasonable amount in the batch size you should essentially be "bulk inserting". What exception in index did you receive from ElasticSearch? Were you pushing documents/data to the wrong index with an incorrect mapping? Posting the error from ES will help. The bulk index api in ES can sometimes be a bit clunky, so instead I use the ES HTTP processor and adjust the batchsize accordingly. This makes sending the raw JSON to the ES a little easier and straightforward. For the data being stuck, can you run a nifi.sh dump and then post the contents of the bootstrap.log? ... View more