Member since
09-13-2015
59
Posts
18
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2678 | 12-19-2017 08:14 PM | |
2095 | 10-11-2017 02:21 PM | |
2886 | 06-12-2017 09:26 PM | |
4489 | 06-08-2017 01:36 PM | |
1700 | 11-04-2016 08:35 PM |
06-08-2017
01:36 PM
It appears that this is expected behavior. I discovered a jira pertaining to it here: https://issues.apache.org/jira/browse/HIVE-13865
... View more
06-07-2017
07:18 PM
I have noticed an issue in Hive 1.2 where char data is rtrimmed when converted to string. This doesn't occur when converting varchar to string. Is this a bug? or a feature? select length(cast('abc ' as char(10))),
length(cast('abc ' as varchar(10))),
length(cast('abc ' as string));
_c0,_c1,_c2
3,7,7
In the above example I would expect the length of the char field to be 10.
... View more
Labels:
- Labels:
-
Apache Hive
06-06-2017
07:41 PM
The individual file sizes is not stored in the metastore, so there is not a way to query them directly From within the hive shell you can execute HDFS commands such as dfs -ls /path/to/table to see the individual files and their sizes. If you're interested in the total data size of the table, you can execute: DESCRIBE FORMATTED table_name; and look for the table parameter named totalSize.
... View more
05-24-2017
03:55 PM
The hive table column names are stored in the metastore. Assuming you are using an external hive table on the txt file(s) you will need to either update the DDL for the table, or drop and re-create the table with the correct columns. Dropping an external table does not delete the underlying data.
... View more
04-20-2017
08:36 PM
It includes Spark 1.6 and 2.1. Hbase 1.1.2 https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2017/03/Asparagus-2.6.png
... View more
02-06-2017
12:25 AM
I am having an issue where occasionally a node will drop out of the cluster and will not rejoin after restarting. I am running NiFi 1.1.0 on VMs. 3 node HDF cluster, with 2 of the nodes running NiFi and the third running ambari/ranger/LogSearch/. On the running node it will display 1/2 nodes connected. On the failed node the UI stays at the "voting on a flow" screen. The only way I can get the node to reconnect to the cluster is by removing the flowfile_repository directory. Unfortunately I can't post logs without significant cleansing, but if there is something in particular I can post.
... View more
Labels:
- Labels:
-
Apache NiFi
11-04-2016
08:38 PM
1 Kudo
I had the same issue. The root cause of my issue was the hive user account (that the compactor was running under) did not have write permissions to the directory that the hive database was in. After adjusting the permissions I was able to run the compactor succesfully.
... View more
11-04-2016
08:35 PM
1 Kudo
Discovered the problem was the hive user account did not have write permissions to the directory containing the hive database. After fixing the permissions the compactor ran succesfully.
... View more
11-03-2016
02:02 AM
If the date is an attribute named 'dt' you can use an attribute variable in the directory path using the syntax ${dt} .
... View more
11-03-2016
01:44 AM
1 Kudo
In my lab environment I have a transactional hive table setup that is receiving data from NiFi. Everything works except the number of delta directories is blowing up because the compactor is not successfully running. It appears to be looking for a _tmp file that does not exist. This is the error that I am getting: No of maps and reduces are 0 job_1478131263487_0009
Job commit failed: java.io.FileNotFoundException: File hdfs://node2.localdomain:8020/apps/hive/warehouse/stock.db/hdp/_tmp_80e50691-c9fe-485a-8e1e-ba20331c7d97 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:966)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:962)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:962)
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorOutputCommitter.commitJob(CompactorMR.java:776)
at org.apache.hadoop.mapred.OutputCommitter.commitJob(OutputCommitter.java:291)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:285)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi
- « Previous
- Next »