Support Questions
Find answers, ask questions, and share your expertise

Kylin sync issue with hive partitioned tables and views

Kylin sync issue with hive partitioned tables and views

New Contributor

Hortonworks version: 2.6.1.0

Kylin version: 2.2.0

I am trying to sync up a table from a list of hive table into kylin in order to build a OLAP cube in Kylin. The issue here is with tables which have partition columns(date) and also with tables which are created as views. I am able to sync up the metadata but not the underlying data for these tables. There is no issue for tables without partitions, I am able to sync both metadata and data. When I try seeing the associated logs, here is what I find:

NoSuchObjectException(message:DEFAULT.KYLIN_INTERMEDIATE_XYZ_TABLE_NAME table not found) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:115) at org.apache.kylin.source.hive.cardinality.HiveColumnCardinalityJob.run(HiveColumnCardinalityJob.java:94) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:129) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: NoSuchObjectException(message:DEFAULT.KYLIN_INTERMEDIATE_BVP_STG_VIEWS_UVERSE_PROGRAM_METADATA_VOD table not found) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:112) ... 9 more Caused by: NoSuchObjectException(message:DEFAULT.KYLIN_INTERMEDIATE_BVP_STG_VIEWS_UVERSE_PROGRAM_METADATA_VOD table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:44359) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:44327) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:44258) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)

Please let me know what would be the issue? I could successfully sync up few other tables from hive into kylin along with the data(not only metadata) to be used for kylin OLAP cubes and models. Not sure why this happens with few tables and not with others.