While running bigbench Benchmark on HDP 2.3.0.0 using ambari, the following error occurred in data generation stage
-------------------------------------------------------------------------------------------------------------------------------------------------------
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51-2.4.5.5.el7.x86_64/jre-abrt/bin/java
/tmp/pdgfLog/1/pdgf.log:
DEBUG main pdgf.generator.BigBenchReviewGenerator - 'Clothing & Accessories_Tops & Tees'
DEBUG main pdgf.generator.BigBenchReviewGenerator - 'Toys & Games_Electronics for Kids'
DEBUG main pdgf.generator.BigBenchReviewGenerator - 'Toys & Games_Vehicles & Remote-Control'
DEBUG main pdgf.core.dataGenerator.scheduler.DefaultPartitioner - Using default Pre-Partitioner from class pdgf.core.dataGenerator.scheduler.TemplatePartitioner
11: <generation>
84: <schema name="default">
85: <tables>
554: <table name="product_reviews">
555: <scheduler name="DefaultScheduler">
556: <partitioner name="pdgf.core.dataGenerator.scheduler.TemplatePartitioner">
DEBUG main pdgf.output.FileOutputSkeleton - def path != null: '"/user/root/benchmarks/bigbench/data_refresh/"+table.getName()+"/"' && !Constants.OUTPUT_FILE_KEEP_OUTPUTDIR:false => ignoring specified <ouputDir> nodes
DEBUG main pdgf.core.dataGenerator.scheduler.DefaultPartitioner - Using default Pre-Partitioner from class pdgf.core.dataGenerator.scheduler.TemplatePartitioner
11: <generation>
59: <scheduler name="DefaultScheduler">
60: <partitioner name="pdgf.core.dataGenerator.scheduler.TemplatePartitioner" staticTableOnAllNodes="false">
DEBUG main pdgf.output.FileOutputSkeleton - def path != null: '"/user/root/benchmarks/bigbench/data_refresh/"+table.getName()+"/"' && !Constants.OUTPUT_FILE_KEEP_OUTPUTDIR:false => ignoring specified <ouputDir> nodes
DEBUG main pdgf.core.dataGenerator.DataGenerator - MemoryAllocatorInterface: add Element: <schema name="bigbench"><table name="store"><field name="s_rec_start_date"><gen name="DateTimeGenerator">
WARN main pdgf.core.dataGenerator.DataGenerator - A 'pdgf.core.exceptions.ConfigurationException Exception occurred during initialization.
Message: The template contains errors: java.lang.RuntimeException: java.io.IOException: invalid constant type: 18
Copy this class in an IDE of your choice to ease debugging:
private class TemplateTester extends pdgf.generator.template.NextValueTemplate {
public void getValue(pdgf.plugin.AbstractPDGFRandom rng,pdgf.core.dataGenerator.beans.FieldValueDTO fvdto, pdgf.core.dataGenerator.beans.GenerationContext gc) throws Exception{
fvdto.setBothValues(generator(0, rng, gc, fvdto) + " " + generator(1, rng, gc, fvdto));
}
}
Location:
Location:
14: <schema name="bigbench">
2076: <table name="store">
2170: <field name="s_manager" primary="false" size="40" type="VARCHAR">
2171: <gen_NullGenerator name="NullGenerator" probability="${NULL_CHANCE}">
2172: <gen_TemplateGenerator name="TemplateGenerator">
DebugInformation:
:pdgf.core.exceptions.ConfigurationException: The template contains errors: java.lang.RuntimeException: java.io.IOException: invalid constant type: 18
Copy this class in an IDE of your choice to ease debugging:
private class TemplateTester extends pdgf.generator.template.NextValueTemplate {
public void getValue(pdgf.plugin.AbstractPDGFRandom rng,pdgf.core.dataGenerator.beans.FieldValueDTO fvdto, pdgf.core.dataGenerator.beans.GenerationContext gc) throws Exception{
fvdto.setBothValues(generator(0, rng, gc, fvdto) + " " + generator(1, rng, gc, fvdto));
}
}
at pdgf.generator.template.NextValueTemplate.instance(NextValueTemplate.java:97)
at pdgf.generator.TemplateGenerator.initialize(TemplateGenerator.java:102)
at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:514)
at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528)
at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528)
at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528)
at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528)
at pdgf.core.dbSchema.Project.initStage8_initialize_(Project.java:722)
at pdgf.core.dataGenerator.DataGenerator.initRootProject(DataGenerator.java:171)
at pdgf.core.dataGenerator.DataGenerator.initialize(DataGenerator.java:139)
at pdgf.core.dataGenerator.DataGenerator.start(DataGenerator.java:214)
at pdgf.actions.StartAction.execute(StartAction.java:112)
at pdgf.actions.ActionPrioritySortObject.execute(ActionPrioritySortObject.java:50)
at pdgf.Controller.parseCmdLineArgs(Controller.java:1248)
at pdgf.Controller.start(Controller.java:1385)
at pdgf.Controller.main(Controller.java:1226)
