Member since
10-29-2017
4
Posts
1
Kudos Received
0
Solutions
11-02-2017
11:17 AM
My Bad, the split calculation works as expected there was a class loading conflict at runtime as I had also named my InputFormat as org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat which already existed in the hive-exec jar.
... View more
10-31-2017
01:13 PM
I have an external Hive table which is partitioned by date where new files would be created, however, when I launch a query on this table I want to filter out certain files i.e do not include as part of the InputSplit while launching the job. I tried my own InputFormat and excluded the files matching a certain pattern in the split calculation, but didn't have any effect. Kindly let me know is there a way to achieve this?
... View more
Labels:
10-30-2017
07:02 AM
1 Kudo
Thanks a lot for your time.
... View more
10-29-2017
03:45 PM
I have loaded just around 214MB worth of data with the year, month, day and hour wise partitioned Hive ACID table through a merge query, however when I launch a simple count(*) query its taking 3645 mappers, which the same data on a non-transactional table on Hive just takes 12 mappers. Is this an expected behavior? I followed the steps from here.
... View more
Labels: