Reply
Highlighted
Explorer
Posts: 21
Registered: ‎04-15-2014

Schedule batch indexing solr

Hi,

I have a string of batch indexing solr.
I need to run to index new files every X time.
I'm currently using a crontab but I would use oozie.

The right path is a shell-action or a java-action?
Is there any example to follow as action java?

I'm trying to make a shell action, but I get errors in a morphline where I put java code. 

 

Stdoutput Caused by: org.kitesdk.morphline.api.MorphlineCompilationException: Cannot compile script near: {
Stdoutput # GrpHdr.conf: 29
Stdoutput "code" : "String msg = record.get(message).toString();if (!msg.contains(<GrpHdr)) {return false;}return child.process(record);",
Stdoutput # GrpHdr.conf: 28
Stdoutput "imports" : "import java.util.*;import java.lang.String;"
Stdoutput }
Stdoutput at org.kitesdk.morphline.stdlib.JavaBuilder.build(JavaBuilder.java:54)
Stdoutput at org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
Stdoutput at org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
Stdoutput at org.kitesdk.morphline.stdlib.TryRulesBuilder$TryRules.<init>(TryRulesBuilder.java:82)
Stdoutput at org.kitesdk.morphline.stdlib.TryRulesBuilder.build(TryRulesBuilder.java:59)
Stdoutput at org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
Stdoutput at org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
Stdoutput at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:46)
Stdoutput at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)
Stdoutput at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:126)
Stdoutput at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:55)
Stdoutput at org.apache.solr.hadoop.morphline.MorphlineMapRunner.<init>(MorphlineMapRunner.java:157)
Stdoutput at org.apache.solr.hadoop.morphline.MorphlineMapper.setup(MorphlineMapper.java:75)
Stdoutput at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
Stdoutput at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
Stdoutput at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
Stdoutput at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
Stdoutput at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
Stdoutput at java.util.concurrent.FutureTask.run(FutureTask.java:262)
Stdoutput at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
Stdoutput at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
Stdoutput at java.lang.Thread.run(Thread.java:744)
Stdoutput Caused by: javax.script.ScriptException: Cannot compile script: String msg = record.get(message).toString();if (!msg.contains(<GrpHdr)) {return false;}return child.process(record); caused by compilation failed: > expected
Stdoutput ')' expected

 

fails to take the java code, I tried to put everything on the same line to change quotes and double quotes, but I can not find the solution.

 

I'm using CDH 5.0.2

Thanks

Albert