Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kylin sync issue with hive partitioned tables and views

Highlighted

Kylin sync issue with hive partitioned tables and views

New Contributor

Hortonworks version: 2.6.1.0

Kylin version: 2.2.0

I am trying to sync up a table from a list of hive table into kylin in order to build a OLAP cube in Kylin. The issue here is with tables which have partition columns(date) and also with tables which are created as views. I am able to sync up the metadata but not the underlying data for these tables. There is no issue for tables without partitions, I am able to sync both metadata and data. When I try seeing the associated logs, here is what I find:

NoSuchObjectException(message:DEFAULT.KYLIN_INTERMEDIATE_XYZ_TABLE_NAME table not found) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:115) at org.apache.kylin.source.hive.cardinality.HiveColumnCardinalityJob.run(HiveColumnCardinalityJob.java:94) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:129) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: NoSuchObjectException(message:DEFAULT.KYLIN_INTERMEDIATE_BVP_STG_VIEWS_UVERSE_PROGRAM_METADATA_VOD table not found) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:112) ... 9 more Caused by: NoSuchObjectException(message:DEFAULT.KYLIN_INTERMEDIATE_BVP_STG_VIEWS_UVERSE_PROGRAM_METADATA_VOD table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:44359) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:44327) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:44258) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)

Please let me know what would be the issue? I could successfully sync up few other tables from hive into kylin along with the data(not only metadata) to be used for kylin OLAP cubes and models. Not sure why this happens with few tables and not with others.

Don't have an account?
Coming from Hortonworks? Activate your account here