Member since
03-21-2016
3
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3263 | 03-29-2016 12:19 PM |
03-29-2016
12:19 PM
1 Kudo
Hi, i solved this issue changing the java memory configuration. I see this same issue in another topic. Thanks anyway. Regards!
... View more
03-28-2016
10:44 AM
Hi guys, im having some problems with morphline solr sink. This is my flume conf file tier1.sources = fuente_lt
tier1.channels = memoryChannel_lt
tier1.sinks = solrSink_lt
tier1.sources.fuente_lt.type = netcat
tier1.sources.fuente_lt.max-line-lenght = 512
tier1.sources.fuente_lt.bind = 0.0.0.0
tier1.sources.fuente_lt.port = 6667
tier1.channels.memoryChannel_lt.type = memory
tier1.channels.memoryChannel_lt.capacity = 100
tier1.channels.memoryChannel_lt.transactionCapacity = 100
tier1.sinks.solrSink_lt.type = org.apache.flume.sink.solr.morphline.MorphlineSolrSink
tier1.sinks.solrSink_lt.morphlineFile = /home/xxxx/morphline.conf
tier1.sinks.solrSink_lt.morphlineId = morphline1
tier1.sources.fuente_lt.channels = memoryChannel_lt
tier1.sinks.solrSink_lt.channel = memoryChannel_lt I've created the collection in Solr, modified my schema.xml file and that i think its working fine. This is my morphline.conf file SOLR_LOCATOR : {
collection : logCollection
zkHost : "x.x.x.x:2181/solr"
}
morphlines : [
{
id : morphline1
importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
commands : [
{
readCSV {
separator: " "
columns: [ip,tipo,dispositivo,descripcion]
charset : UTF-8
}
}
{
generateUUID {
field: id
}
}
{
loadSolr {
solrLocator : ${SOLR_LOCATOR}
}
}
]
}
] When i restart the flume service im getting this ....
mar. 28, 13:24:31.652 INFO org.apache.flume.instrumentation.MonitoredCounterGroup
Component type: SINK, name: solrSink_lt started
mar. 28, 13:24:31.660 INFO org.apache.flume.source.NetcatSource
Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/0.0.0.0:6667]
mar. 28, 13:24:31.693 INFO org.mortbay.log
Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
mar. 28, 13:24:31.746 INFO org.mortbay.log
jetty-6.1.26.cloudera.4
mar. 28, 13:24:31.794 INFO org.mortbay.log
Started SelectChannelConnector@0.0.0.0:41414
mar. 28, 13:24:32.156 INFO org.kitesdk.morphline.api.MorphlineContext
Importing commands and from that point after a couple of seconds it starts again from this point mar. 28, 13:25:12.339 INFO org.apache.flume.node.PollingPropertiesFileConfigurationProvider
Configuration provider starting
mar. 28, 13:25:12.355 INFO org.apache.flume.node.PollingPropertiesFileConfigurationProvider
Reloading configuration file:/var/run/cloudera-scm-agent/process/290-flume-AGENT/flume.conf
mar. 28, 13:25:12.359 INFO org.apache.flume.conf.FlumeConfiguration
Processing:solrSink_lt
mar. 28, 13:25:12.359 INFO org.apache.flume.conf.FlumeConfiguration
Processing:solrSink_lt
mar. 28, 13:25:12.360 INFO org.apache.flume.conf.FlumeConfiguration
Added sinks: solrSink_lt Agent: tier1
....
.... anybody experienced this kind of issue? regards.-
... View more
Labels:
- Labels:
-
Apache Flume
-
Apache Solr