The data is streamed so the file size should be irrelevant. Perhaps there are too many reducer slots configured, i.e. the machine doesn?t have enough RAM to handle that many reducers concurrently (there?s also a --reducers CLI option which always rounds up to a multiple of the number of solr shards) To see if the JVM -Xmx settings are applied correctly, you could, for example, add the registerJVMMetrics and startReportingMetricsToSLF4J commands to your morphline http://kitesdk.org/docs/current/kite-morphlines/morphlinesReferenceGuide.html#/registerJVMMetrics http://kitesdk.org/docs/current/kite-morphlines/morphlinesReferenceGuide.html#startReportingMetricsToSLF4J Wolfgang. ... View more

See the answer here: https://groups.google.com/a/cloudera.org/forum/#!topic/cdk-dev/4PkFFmG59vk Wolfgang. ... View more

To make Solr & XML Parser happy consider removing non-valid characters from input strings. Perhaps plug some sanity fixup logic into a custom morphline command, along similar lines as these: https://github.com/kite-sdk/kite/blob/master/kite-morphlines/kite-morphlines-solr-cell/src/main/java/org/kitesdk/morphline/solrcell/StripNonCharSolrContentHandlerFactory.java#L56-71 Wolfgang. ... View more

If indeed the data in HBase contains the XML tags, then it sounds like your tokenizer/analyzer chain in Solr schema.xml is stripping info away, i.e. schema.xml isn?t configured to do what you want it to do. You could confirm that the morphline is doing what it?s supposed to do by adding some debug log message like this to your morphline: logInfo { format : "my record: {}", args : ["@{}"] } Also see http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters and https://cwiki.apache.org/confluence/display/solr/Field+Types+Included+with+Solr Wolfgang. ... View more

Interceptors are executed prior to Sinks. If the UUIDInterceptor does nothing it's probably misconfigured or attached to the wrong channel in flume.conf, or similar. Alternatively, consider replacing the UUIDInterceptor with a MorphlineInterceptor that uses the generateUUID command, or move the generateUUID command into the morphline config of the MorphlineSolrSink. Also see http://kitesdk.org/docs/current/kite-morphlines/morphlinesReferenceGuide.html#/generateUUID Wolfgang. ... View more