Member since
03-07-2016
37
Posts
12
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2488 | 07-21-2016 08:00 PM | |
2070 | 05-17-2016 05:45 PM |
06-29-2018
08:46 PM
Hey @rajat puchnanda First I would recommend extracting the attributes from JSON using EvaluateJSONPath processor, then utilize AttributesToCSV processor to represent the FlowFile attributes as CSV format. For seeing how to use evaluateJSONPath processor, check out this part of the tutorial series: https://hortonworks.com/tutorial/analyze-transit-patterns-with-apache-nifi/section/5/ AttributesToCSV processor is similar to AttributesToJSON processor, the following tutorial shows how to use AttributesToJSON, apply the similar idea to using AttributesToCSV processor: https://hortonworks.com/tutorial/analyze-transit-patterns-with-apache-nifi/section/6/ For info on AttributesToCSV processor: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.7.0/org.apache.nifi.processors.standard.AttributesToCSV/index.html
... View more
04-01-2018
07:33 PM
Hi Nishant, I have an update, so utiliizing your suggestion of adding the "columns": [ ... ] solved my problem and now I am able to successfully ingest my data into Druid. For users who want to see the ingestion spec, I have included it below: {
"type" : "index_hadoop",
"spec" : {
"dataSchema" : {
"dataSource" : "usgs",
"parser" : {
"type" : "hadoopyString",
"parseSpec" : {
"format" : "tsv",
"timestampSpec" : {
"column" : "dates",
"format" : "auto"
},
"dimensionsSpec" : {
"dimensions": ["staid","val","dates"],
"dimensionExclusions" : [],
"spatialDimensions" : []
},
"columns" : ["staid","val","dates"]
}
},
"metricsSpec" : [
{
"type" : "count",
"name" : "count"
},
{
"type" : "doubleSum",
"name" : "avgFlowCuFtsec",
"fieldName" : "val"
}
],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "MONTH",
"queryGranularity" : "NONE",
"intervals" : [ "1963-01-01/2013-12-31" ]
}
},
"ioConfig" : {
"type" : "hadoop",
"inputSpec" : {
"type" : "static",
"paths" : "/tmp/druid/napa-flow.tsv.gz"
}
},
"tuningConfig" : {
"type": "hadoop",
"targetPartitionSize" : 10000,
"maxRowsInMemory" : 75000
}
}
}
Note: I will add the "index" ingestion spec soon too, so users can ingest data into Druid from their local file system or they can go with hadoop file system.
... View more
03-30-2018
06:08 PM
overlord-log.txtHi Nishant, The input file doesn't have a header row.napa-flowtsv.txt But, I utilized your suggestion about adding the "hasHeaderRow = true" and that caused the task to complete successfully. Here is my latest Stack trace with the ERROR I noticed: 2018-03-29 15:54:37,879 pool-1-thread-1 ERROR Unable to register shutdown hook because JVM is shutting down. java.lang.IllegalStateException: Not started
at io.druid.common.config.Log4jShutdown.addShutdownCallback(Log4jShutdown.java:46)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.addShutdownCallback(Log4jContextFactory.java:273)
at org.apache.logging.log4j.core.LoggerContext.setUpShutdownHook(LoggerContext.java:256)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:216)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:253)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:273)
at org.apache.hadoop.hdfs.LeaseRenewer.<clinit>(LeaseRenewer.java:72)
at org.apache.hadoop.hdfs.DFSClient.getLeaseRenewer(DFSClient.java:830)
at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:968)
at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:1214)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2886)
at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2903)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
... View more
03-29-2018
03:16 PM
I am developing an HDP sandbox tutorial that teaches users how they can use Druid's Batch Ingestion Method to load local files into Druid. After I store the USGS data into a local file and submit an ingestion spec referring to the local file to Druid Overlord, I get that the task fails meaning the data didn't load into Druid. Here is my batch ingestion spec: {
"type" : "index",
"spec" : {
"dataSchema" : {
"dataSource" : "usgs",
"parser" : {
"type" : "string",
"parseSpec" : {
"format" : "tsv",
"timestampSpec" : {
"column" : "dates",
"format" : "auto"
},
"dimensionsSpec" : {
"dimensions": ["staid","val","dates"],
"dimensionExclusions" : [],
"spatialDimensions" : []
}
}
},
"metricsSpec" : [
{
"type" : "count",
"name" : "count"
},
{
"type" : "doubleSum",
"name" : "avgFlowCuFtsec",
"fieldName" : "val"
}
],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "MONTH",
"queryGranularity" : "NONE",
"intervals" : [ "1963-01-01/2013-12-31" ]
}
},
"ioConfig" : {
"type" : "index",
"firehose" : {
"type" : "local",
"baseDir" : "/tmp/usgs/",
"filter" : "napa-flow.tsv"
}
},
"tuningConfig" : {
"type": "index",
"targetPartitionSize" : 10000,
"maxRowsInMemory" : 75000
}
}
}
Here is the stack trace from Druid's Overlord: 2018-03-29T14:09:28,013 INFO [main] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2018-03-29T14:09:28,015 INFO [main] io.druid.guice.PropertiesModule - Loading properties from runtime.properties
2018-03-29T14:09:28,042 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final
2018-03-29T14:09:28,411 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.segment.NullValueHandlingConfig] from props[druid.null.handling.] as [io.druid.segment.NullValueHandlingConfig@6995bf68]
2018-03-29T14:09:28,460 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory='/usr/hdp/current/druid-middlemanager/extensions', hadoopDependenciesDir='/usr/hdp/current/druid-middlemanager/hadoop-dependencies', hadoopContainerDruidClasspath='null', addExtensionsToHadoopContainer=false, loadList=[druid-datasketches, druid-hdfs-storage, druid-kafka-indexing-service]}]
2018-03-29T14:09:28,510 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-datasketches] for class [interface io.druid.cli.CliCommandCreator]
2018-03-29T14:09:28,518 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-datasketches/druid-datasketches-0.10.1.2.6.4.0-91.jar]
2018-03-29T14:09:28,518 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-datasketches/sketches-core-0.8.4.jar]
2018-03-29T14:09:28,518 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-datasketches/memory-0.8.4.jar]
2018-03-29T14:09:28,520 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-hdfs-storage] for class [interface io.druid.cli.CliCommandCreator]
2018-03-29T14:09:28,522 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-common-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,522 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/okio-1.4.0.jar]
2018-03-29T14:09:28,522 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/curator-client-2.11.0.jar]
2018-03-29T14:09:28,522 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-beanutils-1.7.0.jar]
2018-03-29T14:09:28,522 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jsp-api-2.1.jar]
2018-03-29T14:09:28,522 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/snappy-java-1.0.4.1.jar]
2018-03-29T14:09:28,523 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/stax-api-1.0-2.jar]
2018-03-29T14:09:28,523 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/servlet-api-2.5.jar]
2018-03-29T14:09:28,523 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-client-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,523 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-mapreduce-client-jobclient-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,523 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/okhttp-2.4.0.jar]
2018-03-29T14:09:28,523 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-logging-1.1.1.jar]
2018-03-29T14:09:28,523 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/api-util-1.0.0-M20.jar]
2018-03-29T14:09:28,524 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/apacheds-i18n-2.0.0-M15.jar]
2018-03-29T14:09:28,524 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/curator-recipes-2.11.0.jar]
2018-03-29T14:09:28,524 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/netty-all-4.1.11.Final.jar]
2018-03-29T14:09:28,524 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/aws-java-sdk-kms-1.10.77.jar]
2018-03-29T14:09:28,524 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/druid-hdfs-storage-0.10.1.2.6.4.0-91.jar]
2018-03-29T14:09:28,525 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/nimbus-jose-jwt-3.9.jar]
2018-03-29T14:09:28,525 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/xmlenc-0.52.jar]
2018-03-29T14:09:28,525 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/htrace-core-3.1.0-incubating.jar]
2018-03-29T14:09:28,525 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/xz-1.0.jar]
2018-03-29T14:09:28,525 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-beanutils-core-1.8.0.jar]
2018-03-29T14:09:28,525 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/gson-2.2.4.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jackson-core-2.4.6.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/azure-storage-5.4.0.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/curator-framework-2.11.0.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-mapreduce-client-core-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jackson-xc-1.9.13.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-configuration-1.6.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jackson-dataformat-cbor-2.5.3.jar]
2018-03-29T14:09:28,526 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-yarn-api-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,527 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-yarn-client-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,527 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/aws-java-sdk-core-1.10.77.jar]
2018-03-29T14:09:28,527 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-hdfs-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,527 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-yarn-registry-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,527 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/azure-keyvault-core-0.8.0.jar]
2018-03-29T14:09:28,527 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-yarn-common-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,527 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/api-asn1-api-1.0.0-M20.jar]
2018-03-29T14:09:28,528 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-yarn-server-common-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,528 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-lang3-3.4.jar]
2018-03-29T14:09:28,528 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-auth-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,528 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-net-3.1.jar]
2018-03-29T14:09:28,528 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/paranamer-2.3.jar]
2018-03-29T14:09:28,528 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jersey-client-1.9.jar]
2018-03-29T14:09:28,528 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-mapreduce-client-common-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,529 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/avro-1.7.4.jar]
2018-03-29T14:09:28,529 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jackson-databind-2.4.6.jar]
2018-03-29T14:09:28,529 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jaxb-api-2.2.2.jar]
2018-03-29T14:09:28,529 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jackson-annotations-2.4.6.jar]
2018-03-29T14:09:28,529 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/httpclient-4.5.1.jar]
2018-03-29T14:09:28,529 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jackson-jaxrs-1.9.13.jar]
2018-03-29T14:09:28,529 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/xml-apis-1.3.04.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-compress-1.4.1.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/xercesImpl-2.9.1.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/joda-time-2.9.9.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jetty-sslengine-6.1.26.hwx.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-digester-1.8.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-aws-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/json-smart-1.1.1.jar]
2018-03-29T14:09:28,530 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/apacheds-kerberos-codec-2.0.0-M15.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/netty-3.6.2.Final.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/aws-java-sdk-s3-1.10.77.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/httpcore-4.4.3.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/guava-16.0.1.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-mapreduce-client-shuffle-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-collections-3.2.2.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/hadoop-mapreduce-client-app-2.7.3.2.6.4.0-91.jar]
2018-03-29T14:09:28,531 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/jcip-annotations-1.0.jar]
2018-03-29T14:09:28,532 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/commons-math3-3.6.1.jar]
2018-03-29T14:09:28,532 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-hdfs-storage/leveldbjni-all-1.8.jar]
2018-03-29T14:09:28,547 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service] for class [interface io.druid.cli.CliCommandCreator]
2018-03-29T14:09:28,547 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-kafka-indexing-service/snappy-java-1.1.2.6.jar]
2018-03-29T14:09:28,547 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-kafka-indexing-service/slf4j-api-1.6.4.jar]
2018-03-29T14:09:28,547 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-kafka-indexing-service/druid-kafka-indexing-service-0.10.1.2.6.4.0-91.jar]
2018-03-29T14:09:28,548 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-kafka-indexing-service/lz4-1.3.0.jar]
2018-03-29T14:09:28,548 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/hdp/current/druid-middlemanager/extensions/druid-kafka-indexing-service/kafka-clients-0.10.2.0.jar]
2018-03-29T14:09:28,616 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ModulesConfig] from props[druid.modules.] as [ModulesConfig{excludeList=[]}]
2018-03-29T14:09:28,772 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-datasketches] for class [interface io.druid.initialization.DruidModule]
2018-03-29T14:09:28,775 INFO [main] io.druid.initialization.Initialization - Adding implementation [io.druid.query.aggregation.datasketches.theta.SketchModule] for class [interface io.druid.initialization.DruidModule] from local file system extension
2018-03-29T14:09:28,775 INFO [main] io.druid.initialization.Initialization - Adding implementation [io.druid.query.aggregation.datasketches.theta.oldapi.OldApiSketchModule] for class [interface io.druid.initialization.DruidModule] from local file system extension
2018-03-29T14:09:28,776 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-hdfs-storage] for class [interface io.druid.initialization.DruidModule]
2018-03-29T14:09:28,778 INFO [main] io.druid.initialization.Initialization - Adding implementation [io.druid.storage.hdfs.HdfsStorageDruidModule] for class [interface io.druid.initialization.DruidModule] from local file system extension
2018-03-29T14:09:28,778 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service] for class [interface io.druid.initialization.DruidModule]
2018-03-29T14:09:28,779 INFO [main] io.druid.initialization.Initialization - Adding implementation [io.druid.indexing.kafka.KafkaIndexTaskModule] for class [interface io.druid.initialization.DruidModule] from local file system extension
2018-03-29T14:09:29,499 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-03-29T14:09:30,292 WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2018-03-29T14:09:30,672 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@24eeac69]
2018-03-29T14:09:30,679 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[class com.metamx.metrics.JvmMonitor]}]
2018-03-29T14:09:30,698 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName='druid/middlemanager', host='sandbox-hdp.hortonworks.com', port=8100}]
2018-03-29T14:09:30,774 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.lookup.LookupConfig] from props[druid.lookup.] as [LookupConfig{snapshotWorkingDir=''}]
2018-03-29T14:09:30,844 INFO [main] io.druid.server.metrics.MetricsModule - Adding monitor[com.metamx.metrics.JvmMonitor@485caa8f]
2018-03-29T14:09:30,847 INFO [main] io.druid.server.metrics.MetricsModule - Adding monitor[io.druid.query.ExecutorServiceMonitor@4d157493]
2018-03-29T14:09:30,848 INFO [main] io.druid.server.metrics.MetricsModule - Adding monitor[io.druid.server.initialization.jetty.JettyServerModule$JettyMonitor@76e3b45b]
2018-03-29T14:09:30,854 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.log.StartupLoggingConfig] from props[druid.startup.logging.] as [io.druid.server.log.StartupLoggingConfig@3e28fee1]
2018-03-29T14:09:30,854 INFO [main] io.druid.cli.CliPeon - Starting up with processors[4], memory[308,281,344], maxMemory[1,908,932,608].
2018-03-29T14:09:30,857 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.storage.hdfs.HdfsKerberosConfig] from props[druid.hadoop.security.kerberos.] as [io.druid.storage.hdfs.HdfsKerberosConfig@0]
2018-03-29T14:09:30,864 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskConfig] from props[druid.indexer.task.] as [io.druid.indexing.common.config.TaskConfig@2eda072]
2018-03-29T14:09:30,867 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.http.DruidHttpClientConfig] from props[druid.global.http.] as [io.druid.guice.http.DruidHttpClientConfig@4091b9c3]
2018-03-29T14:09:30,928 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.indexing.IndexingServiceSelectorConfig] from props[druid.selectors.indexing.] as [io.druid.client.indexing.IndexingServiceSelectorConfig@7e7e962d]
2018-03-29T14:09:30,931 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.curator.CuratorConfig] from props[druid.zk.service.] as [io.druid.curator.CuratorConfig@7499eac7]
2018-03-29T14:09:30,934 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.curator.ExhibitorConfig] from props[druid.exhibitor.service.] as [io.druid.curator.ExhibitorConfig@d88f893]
2018-03-29T14:09:30,938 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29
2018-03-29T14:09:30,959 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.CuratorDiscoveryConfig] from props[druid.discovery.curator.] as [io.druid.server.initialization.CuratorDiscoveryConfig@5b000fe6]
2018-03-29T14:09:30,980 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.RetryPolicyConfig] from props[druid.peon.taskActionClient.retry.] as [io.druid.indexing.common.RetryPolicyConfig@7186b202]
2018-03-29T14:09:30,982 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.storage.hdfs.HdfsDataSegmentPusherConfig] from props[druid.storage.] as [io.druid.storage.hdfs.HdfsDataSegmentPusherConfig@1a538ed8]
2018-03-29T14:09:30,983 INFO [main] io.druid.storage.hdfs.HdfsDataSegmentPusher - Configured HDFS as deep storage
2018-03-29T14:09:30,986 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.segment.loading.LocalDataSegmentPusherConfig] from props[druid.storage.] as [io.druid.segment.loading.LocalDataSegmentPusherConfig@270a620]
2018-03-29T14:09:30,991 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.DruidServerConfig] from props[druid.server.] as [io.druid.client.DruidServerConfig@4dfe8b37]
2018-03-29T14:09:30,994 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.BatchDataSegmentAnnouncerConfig] from props[druid.announcer.] as [io.druid.server.initialization.BatchDataSegmentAnnouncerConfig@3c232051]
2018-03-29T14:09:30,996 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ZkPathsConfig] from props[druid.zk.paths.] as [io.druid.server.initialization.ZkPathsConfig@22e2266d]
2018-03-29T14:09:31,002 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.coordination.DataSegmentAnnouncerProvider] from props[druid.announcer.] as [io.druid.server.coordination.BatchDataSegmentAnnouncerProvider@59fc6d05]
2018-03-29T14:09:31,004 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.coordinator.CoordinatorSelectorConfig] from props[druid.selectors.coordinator.] as [io.druid.client.coordinator.CoordinatorSelectorConfig@7e87ef9e]
2018-03-29T14:09:31,007 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.segment.realtime.plumber.CoordinatorBasedSegmentHandoffNotifierConfig] from props[druid.segment.handoff.] as [io.druid.segment.realtime.plumber.CoordinatorBasedSegmentHandoffNotifierConfig@7c241a9f]
2018-03-29T14:09:31,009 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [50000000] for [druid.processing.buffer.sizeBytes] on [io.druid.query.DruidProcessingConfig#intermediateComputeSizeBytes()]
2018-03-29T14:09:31,011 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.computation.buffer.poolCacheMaxCount, ${base_path}.buffer.poolCacheMaxCount] on [io.druid.query.DruidProcessingConfig#poolCacheMaxCount()]
2018-03-29T14:09:31,012 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.fifo] on [io.druid.query.DruidProcessingConfig#isFifo()]
2018-03-29T14:09:31,012 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [2] for [druid.processing.numThreads] on [io.druid.query.DruidProcessingConfig#getNumThreadsConfigured()]
2018-03-29T14:09:31,012 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.numMergeBuffers] on [io.druid.query.DruidProcessingConfig#getNumMergeBuffersConfigured()]
2018-03-29T14:09:31,013 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.columnCache.sizeBytes] on [io.druid.query.DruidProcessingConfig#columnCacheSizeBytes()]
2018-03-29T14:09:31,013 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.tmpDir] on [io.druid.query.DruidProcessingConfig#getTmpDir()]
2018-03-29T14:09:31,013 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [processing-%s] for [${base_path}.formatString] on [io.druid.java.util.common.concurrent.ExecutorServiceConfig#getFormatString()]
2018-03-29T14:09:31,083 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.client.cache.CacheProvider] from props[druid.cache.] as [io.druid.client.cache.LocalCacheProvider@182e7eda]
2018-03-29T14:09:31,091 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.cache.CacheConfig] from props[druid.realtime.cache.] as [io.druid.client.cache.CacheConfig@27dc627a]
2018-03-29T14:09:31,099 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ServerConfig] from props[druid.server.http.] as [ServerConfig{numThreads=50, maxIdleTime=PT5M, defaultQueryTimeout=300000, maxScatterGatherBytes=9223372036854775807}]
2018-03-29T14:09:31,107 INFO [main] org.eclipse.jetty.util.log - Logging initialized @3834ms
2018-03-29T14:09:31,221 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.search.search.SearchQueryConfig] from props[druid.query.search.] as [io.druid.query.search.search.SearchQueryConfig@622d7e4]
2018-03-29T14:09:31,226 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.metadata.SegmentMetadataQueryConfig] from props[druid.query.segmentMetadata.] as [io.druid.query.metadata.SegmentMetadataQueryConfig@26582ca]
2018-03-29T14:09:31,229 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.groupby.GroupByQueryConfig] from props[druid.query.groupBy.] as [GroupByQueryConfig{defaultStrategy='v2', singleThreaded=false, maxIntermediateRows=50000, maxResults=500000, bufferGrouperMaxSize=2147483647, bufferGrouperMaxLoadFactor=0.0, bufferGrouperInitialBuckets=0, maxMergingDictionarySize=100000000, maxOnDiskStorage=0, forcePushDownLimit=false, forceHashAggregation=false}]
2018-03-29T14:09:31,244 INFO [main] io.druid.offheap.OffheapBufferGenerator - Allocating new intermediate processing buffer[0] of size[50,000,000]
2018-03-29T14:09:31,254 INFO [main] io.druid.offheap.OffheapBufferGenerator - Allocating new intermediate processing buffer[1] of size[50,000,000]
2018-03-29T14:09:31,263 INFO [main] io.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[0] of size[50,000,000]
2018-03-29T14:09:31,314 INFO [main] io.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[1] of size[50,000,000]
2018-03-29T14:09:31,370 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.scan.ScanQueryConfig] from props[druid.query.scan.] as [ScanQueryConfig{legacy=false}]
2018-03-29T14:09:31,373 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.select.SelectQueryConfig] from props[druid.query.select.] as [io.druid.query.select.SelectQueryConfig@707865bd]
2018-03-29T14:09:31,375 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.topn.TopNQueryConfig] from props[druid.query.topN.] as [io.druid.query.topn.TopNQueryConfig@3ebc955b]
2018-03-29T14:09:31,379 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.log.RequestLoggerProvider] from props[druid.request.logging.] as [io.druid.server.log.NoopRequestLoggerProvider@6ffd4c0d]
2018-03-29T14:09:31,380 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.security.AuthConfig] from props[druid.auth.] as [AuthConfig{enabled=false}]
2018-03-29T14:09:31,388 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.lookup.LookupListeningAnnouncerConfig] from props[druid.lookup.] as [ListeningAnnouncerConfig{listenersPath='/druid/listeners'}]
2018-03-29T14:09:31,389 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.start()] on object[io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@3dbd7107].
2018-03-29T14:09:31,391 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.emitter.service.ServiceEmitter.start()] on object[com.metamx.emitter.service.ServiceEmitter@5a090f62].
2018-03-29T14:09:31,392 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.query.lookup.LookupReferencesManager.start()] on object[io.druid.query.lookup.LookupReferencesManager@89caf47].
2018-03-29T14:09:31,393 INFO [main] io.druid.query.lookup.LookupReferencesManager - LookupReferencesManager is starting.
2018-03-29T14:09:31,393 INFO [main] io.druid.query.lookup.LookupReferencesManager - LookupReferencesManager is started.
2018-03-29T14:09:31,394 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.metrics.MonitorScheduler.start()] on object[com.metamx.metrics.MonitorScheduler@be164d8].
2018-03-29T14:09:31,403 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.storage.hdfs.HdfsStorageAuthentication.authenticate()] on object[io.druid.storage.hdfs.HdfsStorageAuthentication@2e56b4d].
2018-03-29T14:09:31,405 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.http.client.NettyHttpClient.start()] on object[com.metamx.http.client.NettyHttpClient@36ad229b].
2018-03-29T14:09:31,405 INFO [main] io.druid.curator.CuratorModule - Starting Curator
2018-03-29T14:09:31,405 INFO [main] org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.6-91--1, built on 01/04/2018 09:27 GMT
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:host.name=sandbox-hdp.hortonworks.com
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_161
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-3.b14.el6_9.x86_64/jre
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/usr/hdp/current/druid-middlemanager/conf/_common:/usr/hdp/current/druid-middlemanager/conf/middleManager:/usr/hdp/current/druid-middlemanager/lib/antlr4-runtime-4.5.1.jar:/usr/hdp/current/druid-middlemanager/lib/druid-sql-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/plexus-interpolation-1.19.jar:/usr/hdp/current/druid-middlemanager/lib/guice-servlet-4.1.0.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-jaxrs-smile-provider-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/curator-client-2.11.0.jar:/usr/hdp/current/druid-middlemanager/lib/tesla-aether-0.0.5.jar:/usr/hdp/current/druid-middlemanager/lib/wagon-provider-api-2.4.jar:/usr/hdp/current/druid-middlemanager/lib/extendedset-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-dataformat-smile-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/maven-model-3.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/json-path-2.1.0.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-util-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/commons-compiler-2.7.6.jar:/usr/hdp/current/druid-middlemanager/lib/log4j-api-2.5.jar:/usr/hdp/current/druid-middlemanager/lib/jcl-over-slf4j-1.7.12.jar:/usr/hdp/current/druid-middlemanager/lib/druid-server-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-proxy-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/commons-pool-1.6.jar:/usr/hdp/current/druid-middlemanager/lib/jsr311-api-1.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/jsr305-2.0.1.jar:/usr/hdp/current/druid-middlemanager/lib/druid-indexing-service-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/commons-logging-1.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/okhttp-1.0.2.jar:/usr/hdp/current/druid-middlemanager/lib/RoaringBitmap-0.5.18.jar:/usr/hdp/current/druid-middlemanager/lib/http-client-1.1.0.jar:/usr/hdp/current/druid-middlemanager/lib/curator-recipes-2.11.0.jar:/usr/hdp/current/druid-middlemanager/lib/aopalliance-1.0.jar:/usr/hdp/current/druid-middlemanager/lib/netty-all-4.1.11.Final.jar:/usr/hdp/current/druid-middlemanager/lib/derbyclient-10.11.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/bytebuffer-collections-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/protobuf-java-3.1.0.jar:/usr/hdp/current/druid-middlemanager/lib/aether-spi-0.9.0.M2.jar:/usr/hdp/current/druid-middlemanager/lib/derbynet-10.11.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/compress-lzf-1.0.3.jar:/usr/hdp/current/druid-middlemanager/lib/disruptor-3.3.6.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-http-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/maven-settings-3.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/bcprov-jdk15on-1.52.jar:/usr/hdp/current/druid-middlemanager/lib/classmate-1.0.0.jar:/usr/hdp/current/druid-middlemanager/lib/opencsv-2.3.jar:/usr/hdp/current/druid-middlemanager/lib/druid-aws-common-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/aws-java-sdk-ec2-1.10.77.jar:/usr/hdp/current/druid-middlemanager/lib/druid-services-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/javax.inject-1.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-datatype-joda-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/commons-dbcp-1.4.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-datatype-guava-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-core-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/jline-0.9.94.jar:/usr/hdp/current/druid-middlemanager/lib/base64-2.3.8.jar:/usr/hdp/current/druid-middlemanager/lib/calcite-core-1.12.0.jar:/usr/hdp/current/druid-middlemanager/lib/mapdb-1.0.8.jar:/usr/hdp/current/druid-middlemanager/lib/curator-framework-2.11.0.jar:/usr/hdp/current/druid-middlemanager/lib/log4j-core-2.5.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-servlets-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/druid-console-0.0.2.jar:/usr/hdp/current/druid-middlemanager/lib/maven-model-builder-3.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/config-magic-0.9.jar:/usr/hdp/current/druid-middlemanager/lib/emitter-0.4.5.jar:/usr/hdp/current/druid-middlemanager/lib/slf4j-api-1.6.4.jar:/usr/hdp/current/druid-middlemanager/lib/jboss-logging-3.1.3.GA.jar:/usr/hdp/current/druid-middlemanager/lib/commons-cli-1.2.jar:/usr/hdp/current/druid-middlemanager/lib/guice-4.1.0.jar:/usr/hdp/current/druid-middlemanager/lib/druid-common-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-dataformat-cbor-2.5.3.jar:/usr/hdp/current/druid-middlemanager/lib/derby-10.11.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/aether-api-0.9.0.M2.jar:/usr/hdp/current/druid-middlemanager/lib/commons-io-2.5.jar:/usr/hdp/current/druid-middlemanager/lib/netty-3.10.6.Final.jar:/usr/hdp/current/druid-middlemanager/lib/javax.el-3.0.0.jar:/usr/hdp/current/druid-middlemanager/lib/janino-2.7.6.jar:/usr/hdp/current/druid-middlemanager/lib/plexus-utils-3.0.15.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-jaxrs-base-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/aws-java-sdk-core-1.10.77.jar:/usr/hdp/current/druid-middlemanager/lib/javax.servlet-api-3.1.0.jar:/usr/hdp/current/druid-middlemanager/lib/log4j-1.2-api-2.5.jar:/usr/hdp/current/druid-middlemanager/lib/avatica-core-1.9.0.jar:/usr/hdp/current/druid-middlemanager/lib/aether-impl-0.9.0.M2.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-security-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/spymemcached-2.12.3.jar:/usr/hdp/current/druid-middlemanager/lib/maven-aether-provider-3.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/aggdesigner-algorithm-6.0.jar:/usr/hdp/current/druid-middlemanager/lib/aether-connector-okhttp-0.0.9.jar:/usr/hdp/current/druid-middlemanager/lib/commons-pool2-2.2.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-databind-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/asm-tree-5.2.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-continuation-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/java-util-0.28.2.jar:/usr/hdp/current/druid-middlemanager/lib/maven-repository-metadata-3.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-annotations-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/calcite-linq4j-1.12.0.jar:/usr/hdp/current/druid-middlemanager/lib/jersey-servlet-1.19.3.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/druid-middlemanager/lib/asm-5.2.jar:/usr/hdp/current/druid-middlemanager/lib/asm-commons-5.2.jar:/usr/hdp/current/druid-middlemanager/lib/httpclient-4.5.1.jar:/usr/hdp/current/druid-middlemanager/lib/lz4-1.3.0.jar:/usr/hdp/current/druid-middlemanager/lib/avatica-metrics-1.9.0.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-servlet-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/druid-middlemanager/lib/log4j-slf4j-impl-2.5.jar:/usr/hdp/current/druid-middlemanager/lib/javax.el-api-3.0.0.jar:/usr/hdp/current/druid-middlemanager/lib/irc-api-1.0-0014.jar:/usr/hdp/current/druid-middlemanager/lib/java-util-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/jersey-guice-1.19.3.jar:/usr/hdp/current/druid-middlemanager/lib/activation-1.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/druid-indexing-hadoop-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/commons-lang-2.6.jar:/usr/hdp/current/druid-middlemanager/lib/jersey-server-1.19.3.jar:/usr/hdp/current/druid-middlemanager/lib/aether-connector-file-0.9.0.M2.jar:/usr/hdp/current/druid-middlemanager/lib/geoip2-0.4.0.jar:/usr/hdp/current/druid-middlemanager/lib/joda-time-2.9.9.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-server-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/druid-hll-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/guice-multibindings-4.1.0.jar:/usr/hdp/current/druid-middlemanager/lib/curator-x-discovery-2.11.0.jar:/usr/hdp/current/druid-middlemanager/lib/druid-api-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/commons-lang3-3.2.jar:/usr/hdp/current/druid-middlemanager/lib/maxminddb-0.2.0.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-io-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/maven-settings-builder-3.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/jdbi-2.63.1.jar:/usr/hdp/current/druid-middlemanager/lib/aether-util-0.9.0.M2.jar:/usr/hdp/current/druid-middlemanager/lib/icu4j-54.1.1.jar:/usr/hdp/current/druid-middlemanager/lib/validation-api-1.1.0.Final.jar:/usr/hdp/current/druid-middlemanager/lib/jets3t-0.9.4.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-module-jaxb-annotations-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/server-metrics-0.2.8.jar:/usr/hdp/current/druid-middlemanager/lib/jetty-client-9.3.19.v20170502.jar:/usr/hdp/current/druid-middlemanager/lib/avatica-server-1.9.0.jar:/usr/hdp/current/druid-middlemanager/lib/httpcore-4.4.3.jar:/usr/hdp/current/druid-middlemanager/lib/airline-0.7.jar:/usr/hdp/current/druid-middlemanager/lib/jackson-jaxrs-json-provider-2.4.6.jar:/usr/hdp/current/druid-middlemanager/lib/druid-processing-0.10.1.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/zookeeper-3.4.6.2.6.4.0-91.jar:/usr/hdp/current/druid-middlemanager/lib/guava-16.0.1.jar:/usr/hdp/current/druid-middlemanager/lib/commons-codec-1.7.jar:/usr/hdp/current/druid-middlemanager/lib/jersey-core-1.19.3.jar:/usr/hdp/current/druid-middlemanager/lib/commons-dbcp2-2.0.1.jar:/usr/hdp/current/druid-middlemanager/lib/hibernate-validator-5.1.3.Final.jar:/usr/hdp/current/druid-middlemanager/lib/java-xmlbuilder-1.1.jar:/usr/hdp/current/druid-middlemanager/lib/commons-math3-3.6.1.jar:/usr/hdp/current/druid-middlemanager/lib/rhino-1.7R5.jar:/usr/hdp/current/druid-middlemanager/lib/log4j-jul-2.5.jar:/usr/hdp/current/druid-middlemanager/lib/fastutil-7.2.0.jar:/usr/hdp/2.6.4.0-91/hadoop/conf
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.version=4.4.114-1.el7.elrepo.x86_64
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.name=druid
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/druid
2018-03-29T14:09:31,413 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/druid
2018-03-29T14:09:31,414 INFO [main] org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=sandbox-hdp.hortonworks.com:2181 sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@29013ef2
2018-03-29T14:09:31,434 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.discovery.ServerDiscoverySelector.start() throws java.lang.Exception] on object[io.druid.curator.discovery.ServerDiscoverySelector@13cc0b90].
2018-03-29T14:09:31,474 INFO [main-SendThread(sandbox-hdp.hortonworks.com:2181)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server sandbox-hdp.hortonworks.com/172.17.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
2018-03-29T14:09:31,481 INFO [main-SendThread(sandbox-hdp.hortonworks.com:2181)] org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /172.17.0.2:42090, server: sandbox-hdp.hortonworks.com/172.17.0.2:2181
2018-03-29T14:09:31,491 INFO [main-SendThread(sandbox-hdp.hortonworks.com:2181)] org.apache.zookeeper.ClientCnxn - Session establishment complete on server sandbox-hdp.hortonworks.com/172.17.0.2:2181, sessionid = 0x16271e552860012, negotiated timeout = 30000
2018-03-29T14:09:31,494 INFO [main-EventThread] org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
2018-03-29T14:09:31,590 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.announcement.Announcer.start()] on object[io.druid.curator.announcement.Announcer@1f129467].
2018-03-29T14:09:31,591 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.discovery.ServerDiscoverySelector.start() throws java.lang.Exception] on object[io.druid.curator.discovery.ServerDiscoverySelector@646cd766].
2018-03-29T14:09:31,612 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.indexing.worker.executor.ExecutorLifecycle.start() throws java.lang.InterruptedException] on object[io.druid.indexing.worker.executor.ExecutorLifecycle@65eb3597].
2018-03-29T14:09:31,662 INFO [main] io.druid.indexing.worker.executor.ExecutorLifecycle - Running with task: {
"type" : "index",
"id" : "index_usgs_2018-03-29T14:09:27.159Z",
"resource" : {
"availabilityGroup" : "index_usgs_2018-03-29T14:09:27.159Z",
"requiredCapacity" : 1
},
"spec" : {
"dataSchema" : {
"dataSource" : "usgs",
"parser" : {
"type" : "string",
"parseSpec" : {
"format" : "tsv",
"timestampSpec" : {
"column" : "dates",
"format" : "auto"
},
"dimensionsSpec" : {
"dimensions" : [ "staid", "val", "dates" ],
"dimensionExclusions" : [ ],
"spatialDimensions" : [ ]
}
}
},
"metricsSpec" : [ {
"type" : "count",
"name" : "count"
}, {
"type" : "doubleSum",
"name" : "avgFlowCuFtsec",
"fieldName" : "val",
"expression" : null
} ],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "MONTH",
"queryGranularity" : {
"type" : "none"
},
"rollup" : true,
"intervals" : [ "1963-01-01T00:00:00.000Z/2013-12-31T00:00:00.000Z" ]
}
},
"ioConfig" : {
"type" : "index",
"firehose" : {
"type" : "local",
"baseDir" : "/tmp/usgs",
"filter" : "napa-flow.tsv",
"parser" : null
},
"appendToExisting" : false
},
"tuningConfig" : {
"type" : "index",
"targetPartitionSize" : 10000,
"maxRowsInMemory" : 75000,
"numShards" : null,
"indexSpec" : {
"bitmap" : {
"type" : "concise"
},
"dimensionCompression" : "lz4",
"metricCompression" : "lz4",
"longEncoding" : "longs"
},
"maxPendingPersists" : 0,
"buildV9Directly" : true,
"forceExtendableShardSpecs" : false,
"reportParseExceptions" : false,
"publishTimeout" : 0
}
},
"context" : null,
"groupId" : "index_usgs_2018-03-29T14:09:27.159Z",
"dataSource" : "usgs"
}
2018-03-29T14:09:31,664 INFO [main] io.druid.indexing.worker.executor.ExecutorLifecycle - Attempting to lock file[/apps/druid/tasks/index_usgs_2018-03-29T14:09:27.159Z/lock].
2018-03-29T14:09:31,666 INFO [main] io.druid.indexing.worker.executor.ExecutorLifecycle - Acquired lock file[/apps/druid/tasks/index_usgs_2018-03-29T14:09:27.159Z/lock] in 1ms.
2018-03-29T14:09:31,671 INFO [main] io.druid.indexing.common.actions.RemoteTaskActionClient - Performing action for task[index_usgs_2018-03-29T14:09:27.159Z]: LockTryAcquireAction{interval=1963-01-01T00:00:00.000Z/2014-01-01T00:00:00.000Z}
2018-03-29T14:09:31,679 INFO [main] io.druid.indexing.common.actions.RemoteTaskActionClient - Submitting action for task[index_usgs_2018-03-29T14:09:27.159Z] to overlord[http://sandbox-hdp.hortonworks.com:8090/druid/indexer/v1/action]: LockTryAcquireAction{interval=1963-01-01T00:00:00.000Z/2014-01-01T00:00:00.000Z}
2018-03-29T14:09:31,688 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,728 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,729 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,730 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,730 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,731 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,731 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,732 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,732 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,732 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,732 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,733 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,733 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,733 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,734 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,734 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,734 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,735 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,735 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,735 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://sandbox-hdp.hortonworks.com:8090
2018-03-29T14:09:31,821 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Running task: index_usgs_2018-03-29T14:09:27.159Z
2018-03-29T14:09:31,824 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_usgs_2018-03-29T14:09:27.159Z] location changed to [TaskLocation{host='sandbox-hdp.hortonworks.com', port=8100}].
2018-03-29T14:09:31,824 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_usgs_2018-03-29T14:09:27.159Z] status changed to [RUNNING].
2018-03-29T14:09:31,829 INFO [task-runner-0-priority-0] io.druid.indexing.common.task.IndexTask - Determining intervals and shardSpecs
2018-03-29T14:09:31,831 INFO [main] org.eclipse.jetty.server.Server - jetty-9.3.19.v20170502
2018-03-29T14:09:31,880 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[IndexTask{id=index_usgs_2018-03-29T14:09:27.159Z, type=index, dataSource=usgs}]
java.lang.IllegalArgumentException: Instantiation of [simple type, class io.druid.data.input.impl.DelimitedParseSpec] value failed: If columns field is not set, the first row of your data must have your header and hasHeaderRow must be set to true.
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2774) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700) ~[jackson-databind-2.4.6.jar:2.4.6]
at io.druid.segment.indexing.DataSchema.getParser(DataSchema.java:111) ~[druid-server-0.10.1.2.6.4.0-91.jar:0.10.1.2.6.4.0-91]
at io.druid.indexing.common.task.IndexTask.determineShardSpecs(IndexTask.java:269) ~[druid-indexing-service-0.10.1.2.6.4.0-91.jar:0.10.1.2.6.4.0-91]
at io.druid.indexing.common.task.IndexTask.run(IndexTask.java:188) ~[druid-indexing-service-0.10.1.2.6.4.0-91.jar:0.10.1.2.6.4.0-91]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.1.2.6.4.0-91.jar:0.10.1.2.6.4.0-91]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.1.2.6.4.0-91.jar:0.10.1.2.6.4.0-91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Instantiation of [simple type, class io.druid.data.input.impl.DelimitedParseSpec] value failed: If columns field is not set, the first row of your data must have your header and hasHeaderRow must be set to true.
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapException(StdValueInstantiator.java:405) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:234) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:167) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:398) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:536) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2769) ~[jackson-databind-2.4.6.jar:2.4.6]
... 10 more
Caused by: java.lang.IllegalArgumentException: If columns field is not set, the first row of your data must have your header and hasHeaderRow must be set to true.
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:125) ~[guava-16.0.1.jar:?]
at io.druid.data.input.impl.DelimitedParseSpec.<init>(DelimitedParseSpec.java:65) ~[druid-api-0.10.1.2.6.4.0-91.jar:0.10.1.2.6.4.0-91]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_161]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_161]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_161]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_161]
at com.fasterxml.jackson.databind.introspect.AnnotatedConstructor.call(AnnotatedConstructor.java:125) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:230) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:167) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:398) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:536) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41) ~[jackson-databind-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2769) ~[jackson-databind-2.4.6.jar:2.4.6]
... 10 more
2018-03-29T14:09:31,887 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_usgs_2018-03-29T14:09:27.159Z] status changed to [FAILED].
2018-03-29T14:09:31,898 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
"id" : "index_usgs_2018-03-29T14:09:27.159Z",
"status" : "FAILED",
"duration" : 66
}
2018-03-29T14:09:32,015 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider as a provider class
2018-03-29T14:09:32,015 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider as a provider class
2018-03-29T14:09:32,016 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering io.druid.server.initialization.jetty.CustomExceptionMapper as a provider class
2018-03-29T14:09:32,016 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering io.druid.server.StatusResource as a root resource class
2018-03-29T14:09:32,019 INFO [main] com.sun.jersey.server.impl.application.WebApplicationImpl - Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM'
2018-03-29T14:09:32,086 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.server.initialization.jetty.CustomExceptionMapper to GuiceManagedComponentProvider with the scope "Singleton"
2018-03-29T14:09:32,089 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider to GuiceManagedComponentProvider with the scope "Singleton"
2018-03-29T14:09:32,096 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider to GuiceManagedComponentProvider with the scope "Singleton"
2018-03-29T14:09:32,368 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.server.http.security.StateResourceFilter to GuiceInstantiatedComponentProvider
2018-03-29T14:09:32,387 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.server.http.SegmentListerResource to GuiceInstantiatedComponentProvider
2018-03-29T14:09:32,398 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.server.QueryResource to GuiceInstantiatedComponentProvider
2018-03-29T14:09:32,400 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.segment.realtime.firehose.ChatHandlerResource to GuiceInstantiatedComponentProvider
2018-03-29T14:09:32,403 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.query.lookup.LookupListeningResource to GuiceInstantiatedComponentProvider
2018-03-29T14:09:32,405 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.query.lookup.LookupIntrospectionResource to GuiceInstantiatedComponentProvider
2018-03-29T14:09:32,406 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding io.druid.server.StatusResource to GuiceManagedComponentProvider with the scope "Undefined"
2018-03-29T14:09:32,420 WARN [main] com.sun.jersey.spi.inject.Errors - The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void io.druid.server.http.SegmentListerResource.getSegments(long,long,long,javax.servlet.http.HttpServletRequest) throws java.io.IOException, MUST return a non-void type.
2018-03-29T14:09:32,428 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@63de4fa{/,null,AVAILABLE}
2018-03-29T14:09:32,435 INFO [main] org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@2fba0dac{HTTP/1.1,[http/1.1]}{0.0.0.0:8100}
2018-03-29T14:09:32,435 INFO [main] org.eclipse.jetty.server.Server - Started @5163ms
2018-03-29T14:09:32,436 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.server.listener.announcer.ListenerResourceAnnouncer.start()] on object[io.druid.query.lookup.LookupResourceListenerAnnouncer@7be38eba].
2018-03-29T14:09:32,450 INFO [main] io.druid.server.listener.announcer.ListenerResourceAnnouncer - Announcing start time on [/druid/listeners/lookups/__default/sandbox-hdp.hortonworks.com:8100]
2018-03-29T14:09:32,455 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.server.listener.announcer.ListenerResourceAnnouncer.stop()] on object[io.druid.query.lookup.LookupResourceListenerAnnouncer@7be38eba].
2018-03-29T14:09:32,455 INFO [main] io.druid.curator.announcement.Announcer - unannouncing [/druid/listeners/lookups/__default/sandbox-hdp.hortonworks.com:8100]
2018-03-29T14:09:32,484 INFO [main] io.druid.server.listener.announcer.ListenerResourceAnnouncer - Unannouncing start time on [/druid/listeners/lookups/__default/sandbox-hdp.hortonworks.com:8100]
2018-03-29T14:09:32,488 INFO [main] org.eclipse.jetty.server.AbstractConnector - Stopped ServerConnector@2fba0dac{HTTP/1.1,[http/1.1]}{0.0.0.0:8100}
2018-03-29T14:09:32,489 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@63de4fa{/,null,UNAVAILABLE}
2018-03-29T14:09:32,496 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.indexing.worker.executor.ExecutorLifecycle.stop() throws java.lang.Exception] on object[io.druid.indexing.worker.executor.ExecutorLifecycle@65eb3597].
2018-03-29T14:09:32,497 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.indexing.overlord.ThreadPoolTaskRunner.stop()] on object[io.druid.indexing.overlord.ThreadPoolTaskRunner@37b1149b].
2018-03-29T14:09:32,497 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.discovery.ServerDiscoverySelector.stop() throws java.io.IOException] on object[io.druid.curator.discovery.ServerDiscoverySelector@646cd766].
2018-03-29T14:09:32,501 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.announcement.Announcer.stop()] on object[io.druid.curator.announcement.Announcer@1f129467].
2018-03-29T14:09:32,501 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.discovery.ServerDiscoverySelector.stop() throws java.io.IOException] on object[io.druid.curator.discovery.ServerDiscoverySelector@13cc0b90].
2018-03-29T14:09:32,501 INFO [main] io.druid.curator.CuratorModule - Stopping Curator
2018-03-29T14:09:32,502 INFO [Curator-Framework-0] org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2018-03-29T14:09:32,506 INFO [main] org.apache.zookeeper.ZooKeeper - Session: 0x16271e552860012 closed
2018-03-29T14:09:32,506 INFO [main-EventThread] org.apache.zookeeper.ClientCnxn - EventThread shut down
2018-03-29T14:09:32,506 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.http.client.NettyHttpClient.stop()] on object[com.metamx.http.client.NettyHttpClient@36ad229b].
2018-03-29T14:09:32,527 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.storage.hdfs.HdfsStorageAuthentication.stop()] on object[io.druid.storage.hdfs.HdfsStorageAuthentication@2e56b4d].
2018-03-29T14:09:32,527 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.metrics.MonitorScheduler.stop()] on object[com.metamx.metrics.MonitorScheduler@be164d8].
2018-03-29T14:09:32,528 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.query.lookup.LookupReferencesManager.stop()] on object[io.druid.query.lookup.LookupReferencesManager@89caf47].
2018-03-29T14:09:32,528 INFO [main] io.druid.query.lookup.LookupReferencesManager - LookupReferencesManager is stopping.
2018-03-29T14:09:32,528 INFO [LookupReferencesManager-MainThread] io.druid.query.lookup.LookupReferencesManager - Lookup Management loop exited, Lookup notices are not handled anymore.
2018-03-29T14:09:32,529 INFO [main] io.druid.query.lookup.LookupReferencesManager - LookupReferencesManager is stopped.
2018-03-29T14:09:32,529 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.emitter.service.ServiceEmitter.close() throws java.io.IOException] on object[com.metamx.emitter.service.ServiceEmitter@5a090f62].
2018-03-29T14:09:32,529 INFO [main] io.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.stop()] on object[io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@3dbd7107].
2018-03-29 14:09:32,567 pool-1-thread-1 ERROR Unable to register shutdown hook because JVM is shutting down. java.lang.IllegalStateException: Not started
at io.druid.common.config.Log4jShutdown.addShutdownCallback(Log4jShutdown.java:46)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.addShutdownCallback(Log4jContextFactory.java:273)
at org.apache.logging.log4j.core.LoggerContext.setUpShutdownHook(LoggerContext.java:256)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:216)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:253)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:273)
at org.apache.hadoop.hdfs.LeaseRenewer.<clinit>(LeaseRenewer.java:72)
at org.apache.hadoop.hdfs.DFSClient.getLeaseRenewer(DFSClient.java:830)
at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:968)
at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:1214)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2886)
at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2903)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Any help would be very much appreciated.
... View more
03-11-2017
11:20 PM
Hello @Josh Elser, Thank you for your prompt response. I executed instruction: echo ruok | nc sandbox-hdp.hortonworks.com 2181 I received message: "imok" I believe it is not a firewall issue because I tried telnet <hostname-hdp> 2181 against HDP sandbox from HDF sandbox. I also tried telnet <hostname-hdf> 2181 against HDF sandbox from HDP sandbox. The output message I get from console illustrates HDF is connected to zookeeper port running on HDP: [root@sandbox ~]# telnet 172.17.0.3 2181
Trying 172.17.0.3...
Connected to 172.17.0.3.
Escape character is '^]'. Note: 172.17.0.3 is the ip address of HDP sandbox. What approach would you take to debug a port forwarding issue? Thanks, James
... View more
03-11-2017
05:32 PM
1 Kudo
Description of my setup:
I have HDF sandbox node-A running NiFi while HDP sandbox node-B runs HBase. Both sandboxes have zookeper activated. In NiFi, I setup HBaseClientService for the PutHbaseJson processor. I added hbase-site.xml to my HBaseClientService under the Hadoop Configuration Files property.hbase-clientservice-error.pnghbase-config-hdp-setup.png Originally I acquired hbase-site.xml from HDP sandbox in path "/var/lib/ambari-server/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-site.xml". I modified hbase-site.xml: hbase.rootdir, hbase.zookeeper.quorum and added hbase.regionserver.port, so NiFi knows where to find HBase. I have narrowed the problem down to a configuration problem either on NiFi or HBase side and not a networking issue because I have successfully tested that NiFi can dump data into HDFS using the PutHDFS processor.
The Problem:
A snippet of the stack trace from the nifi-app.log points out that hbase.MasterNotRunningException and can't get connection to Zookeeper. So, I conclude the problem is coming from Zookeeper. I don't understand why this problem is happening because I included both NiFi's and HBase's IP address for the zookeeper.quorum property value in the hbase-site.xml. 2017-03-11 16:18:22,066 ERROR [StandardProcessScheduler Thread-1] o.a.n.c.s.StandardControllerServiceNode HBase_1_1_2_ClientService[id=4d4142fa-0159-1000-a996-9604038fb2af] Failed to invoke @OnEnabled method due to org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions:
Sat Mar 11 16:18:22 UTC 2017, RpcRetryingCaller{globalStartTime=1489249087052, pause=100, retries=1}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to ZooKeeper: KeeperErrorCode = OperationTimeout Update March 11, 2017: I found other interesting insights from HDF and HDP sandbox logs HDF sandbox logs: NiFi logs: The NiFi logs indicate there is a connection problem with Zookeeper. So, I go to check zookeeper logs. o.a.h.h.zookeeper.RecoverableZooKeeper Unable to create ZooKeeper Connectionjava.net.UnknownHostException: sandbox.hortonworks.com: Name or service not knowno.a.h.hbase.zookeeper.ZooKeeperWatcher hconnection-0x30c93240x0, quorum=sandbox-hdp.hortonworks.com:2181, sandbox.hortonworks.com:2181, baseZNode=/hbase-unsecure Received unexpected KeeperException, re-throwing exceptionCaused by: org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to ZooKeeper: KeeperErrorCode = OperationTimeoutCaused by: org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to ZooKeeper: KeeperErrorCode = OperationTimeoutCaused by: org.apache.zookeeper.KeeperException$OperationTimeoutException: KeeperErrorCode = OperationTimeout Zookeeper logs: Zookeeper logs show there is no node for Primary Node and Cluster Coordinate. Also I see the client closed the socket, so it can’t read any more data from the client. 2017-03-11 19:55:49,257 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15abb279ca20026 type:delete cxid:0x8 zxid:0x22f5 txntype:-1 reqpath:n/a Error Path:/nifi/leaders/Primary Node/_c_f44978a2-7e01-487d-878b-d99d79fee7a9-lock-0000000005 Error:KeeperErrorCode = NoNode for /nifi/leaders/Primary Node/_c_f44978a2-7e01-487d-878b-d99d79fee7a9-lock-00000000052017-03-11 19:55:49,257 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15abb279ca20026 type:delete cxid:0x9 zxid:0x22f6 txntype:-1 reqpath:n/a Error Path:/nifi/leaders/Cluster Coordinator/_c_17d97f01-625d-459b-bcb9-850d98da42a6-lock-0000000005 Error:KeeperErrorCode = NoNode for /nifi/leaders/Cluster Coordinator/_c_17d97f01-625d-459b-bcb9-850d98da42a6-lock-0000000005EndOfStreamException: Unable to read additional data from client sessionid 0x15abb279ca20035, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:230) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) at java.lang.Thread.run(Thread.java:745) Ambari Infra Solr logs: Solr logs indicate a config file of Zookeeper does not exist. Solr also cannot talk to zookeeper.znode.parent. Caused by: org.apache.solr.common.SolrException: Unable to create core [hadoop_logs_shard1_replica1]Caused by: org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper: hadoop_logs2017-03-11 21:35:06,404 [OverseerCollectionConfigSetProcessor-97596120747868221-172.17.0.2:8886_solr-n_0000000000] WARN [ ] org.apache.solr.cloud.OverseerTaskProcessor (OverseerTaskProcessor.java:249) - Overseer cannot talk to ZK2017-03-11 21:35:06,409 [OverseerHdfsCoreFailoverThread-97596120747868221-172.17.0.2:8886_solr-n_0000000000] ERROR [ ] org.apache.solr.common.SolrException (SolrException.java:159) - OverseerAutoReplicaFailoverThread had an error in its thread work loop.:org.apache.solr.common.SolrException: Error reading cluster properties HDP sandbox logs: HBase logs: HBase logs show there is a server refused connection on solr’s side. WARN [timeline] timeline.HadoopTimelineMetricsSink: Unable to send metrics to collector by address:http://sandbox.hortonworks.com:6188/ws/v1/timeline/metricsERROR [org.apache.ranger.audit.queue.AuditBatchQueue0] impl.CloudSolrClient: Request to collection ranger_audits failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 0Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://172.17.0.2:8886/solr/ranger_audits_shard1_replica1]Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://172.17.0.2:8886/solr/ranger_audits_shard1_replica1Caused by: java.net.ConnectException: Connection refused (Connection refused) Zookeeper logs: Zookeeper logs illustrate there is an invalid configuration and that only one server can be specified, it looks like this issue is related to Zookeeper Quorum. My question is then, when you have multiple zookeeper services running in your cluster, don’t you have to specify their <IP-address> or hostname in the hbase-site.xml on both sandboxes? 2017-03-11 20:07:57,177 - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /usr/hdp/current/zookeeper-server/conf/zoo.cfg2017-03-11 20:07:57,178 - ERROR [main:QuorumPeerConfig@287] - Invalid configuration, only one server specified (ignoring)2017-03-11 21:01:16,208 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exceptionEndOfStreamException: Unable to read additional data from client sessionid 0x15abefc1ba0000d, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) at java.lang.Thread.run(Thread.java:745)2017-03-11 21:07:27,068 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15abefc1ba00011 type:create cxid:0x1 zxid:0xdbe txntype:-1 reqpath:n/a Error Path:/consumers Error:KeeperErrorCode = NodeExists for /consumers2017-03-11 21:07:27,068 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15abefc1ba00011 type:create cxid:0x2 zxid:0xdbf txntype:-1 reqpath:n/a Error Path:/consumers/ranger_entities_consumer Error:KeeperErrorCode = NodeExists for /consumers/ranger_entities_consumer2017-03-11 21:07:27,068 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15abefc1ba00011 type:create cxid:0x3 zxid:0xdc0 txntype:-1 reqpath:n/a Error Path:/consumers/ranger_entities_consumer/ids Error:KeeperErrorCode = NodeExists for /consumers/ranger_entities_consumer/ids Ambari Infra Solr logs: Solr logs indicate that it could not read the data. I believe this log message refers to solr not being able to read the data coming in from HDF sandbox. [OverseerExitThread] ERROR [ ] org.apache.solr.cloud.Overseer$ClusterStateUpdater (Overseer.java:311) - could not read the dataorg.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader2017-03-11 21:35:06,630 [OverseerHdfsCoreFailoverThread-97600331946393615-172.17.0.3:8886_solr-n_0000000001] ERROR [ ] org.apache.solr.common.SolrException (SolrException.java:159) - OverseerAutoReplicaFailoverThread had an error in its thread work loop.:org.apache.solr.common.SolrException: Error reading cluster propertiesCaused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterprops.json Any help for resolving this issue would be greatly appreciated. Thank you. Regards, James hbase-site.xml nifi-app.txt zookeeper-zookeeper-server-sandboxhortonworkscom.txt
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache NiFi
07-23-2016
11:29 PM
Hi @Srikanth Gundam, I tested your NiFi template on my local machine and the dataflow works fine. No problems. "connectionTimeout error" occurs usually due to network connectivity issues or firewall could be dropping the API HTTP request. I don't think you have a network connectivity issue because you were able to use SOAP UI. Can you send me the logs after you ran your NiFi Dataflow and the error happened? The logs folder is in HDF-1.2.0.1-1/nifi/logs.
... View more
07-21-2016
11:31 PM
Hi @Srikanth Gundam Can you share your NiFi template, so I can look through your dataflow and help you find the solution?
... View more
07-21-2016
08:00 PM
Hi @Harish Padmaraju, I updated the install_maven.sh script, so now it downloads the most up to date version of maven from the maven site. Here is a link to lab 0. Let me know if you come across any other issues. http://hortonworks.com/hadoop-tutorial/realtime-event-processing-nifi-kafka-storm/#section_3
... View more
07-08-2016
05:51 PM
Tutorial has been updated to solve issue 163. Here is the updated tutorial: https://hortonworks.com/hadoop-tutorial/learning-ropes-apache-nifi/#section_4 @slachterman @dpelaezpri
... View more