@ankurkapoor_wor Hi, Even I am facing the same issue as @mqureshi. I am trying to fetch data from SQL server in Avro fomat through NiFi and load it to Redshift through copy command. But the generated Avro file is converting the date and timestamp datatypes to string because of which copy command is loading all NULL values in the target table. So I tried to follow your approach, In my case I'm using ExecuteSQLRecord processor to fetch the data from SQL server and writing it to json format and then trying to convert it to Avro format using ConvertJsonToAvro processor but then I am unable to parse Record Schema. Could you please help me also to resolve this issue.  Thanks in advance! Anusha ... View more

Hi Nishant, I have an update, so utiliizing your suggestion of adding the "columns": [ ... ] solved my problem and now I am able to successfully ingest my data into Druid. For users who want to see the ingestion spec, I have included it below: { "type" : "index_hadoop", "spec" : { "dataSchema" : { "dataSource" : "usgs", "parser" : { "type" : "hadoopyString", "parseSpec" : { "format" : "tsv", "timestampSpec" : { "column" : "dates", "format" : "auto" }, "dimensionsSpec" : { "dimensions": ["staid","val","dates"], "dimensionExclusions" : [], "spatialDimensions" : [] }, "columns" : ["staid","val","dates"] } }, "metricsSpec" : [ { "type" : "count", "name" : "count" }, { "type" : "doubleSum", "name" : "avgFlowCuFtsec", "fieldName" : "val" } ], "granularitySpec" : { "type" : "uniform", "segmentGranularity" : "MONTH", "queryGranularity" : "NONE", "intervals" : [ "1963-01-01/2013-12-31" ] } }, "ioConfig" : { "type" : "hadoop", "inputSpec" : { "type" : "static", "paths" : "/tmp/druid/napa-flow.tsv.gz" } }, "tuningConfig" : { "type": "hadoop", "targetPartitionSize" : 10000, "maxRowsInMemory" : 75000 } } } Note: I will add the "index" ingestion spec soon too, so users can ingest data into Druid from their local file system or they can go with hadoop file system. ... View more

@jmedel Hello, I have the same issue. Have you solved it please? Thanks. Regards, ... View more

Thank you Pierre! ... View more

hi all, i'm still getting "PutKafka Error : failed while waiting for acks from kafka" error though i see some values under "In" & "Read". but i see attached consumer console error. Also,i see something strange here. When i remove out Kafka from "maintenance mode" on Ambari and start the kafka broker, it gets stopped by itself after a while. please help me on this & find supporting attachments. @jmedel @Predrag Minovic @Artem Ervits Note: I'm using HDP 2.6 Regards, Akshay putkafka-processor-nifi-properties.png ... View more

Hello Michael for now an alternative way to access Zeppelin and Storm is through their web addresses: Storm UI http://127.0.0.1:8744/ Zeppelin Notebook http://127.0.0.1:9995/#/ Note: the latest sandbox refresh will be available soon. ... View more

Hey guys. The tutorial mentioned above has been updated and is also compatible with the latest Sandbox HDP 2.4. It addresses the issue of permissions. Here is the link: http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/ When you a chance, can you go through the tutorial on our new Sandbox? ... View more