Member since
05-16-2016
785
Posts
114
Kudos Received
39
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1810 | 06-12-2019 09:27 AM | |
3004 | 05-27-2019 08:29 AM | |
5024 | 05-27-2018 08:49 AM | |
4403 | 05-05-2018 10:47 PM | |
2747 | 05-05-2018 07:32 AM |
10-13-2016
07:43 PM
kerjo I was thinking a work around of type casting in the hive side . I understand that your map-column-hive is being ignored . Correct me if I am wrong.
... View more
10-13-2016
06:37 AM
Would consider trying type casting BIGINT TO Timestamp . Also please refer this document , I read it long back. I am quoting it from the cloudera manul document If you use Sqoop to convert RDBMS data to Parquet, be careful with interpreting any resulting values from DATE, DATETIME, or TIMESTAMP columns. The underlying values are represented as the Parquet INT64 type, which is represented as BIGINT in the Impala table. The Parquet values represent the time in milliseconds, while Impala interprets BIGINT as the time in seconds. Therefore, if you have a BIGINT column in a Parquet table that was imported this way from Sqoop, divide the values by 1000 when interpreting as the TIMESTAMP type. I guess there is underlying problem with Timestamp when you use Parquet file. http://www.cloudera.com/documentation/archive/impala/2-x/2-1-x/topics/impala_parquet.html#parquet_data_types_unique_1
... View more
10-12-2016
09:07 PM
1 Kudo
impalad daemon is the one that is not able to access the jar for query processing since you have set the hdfs permission as 700. Your assumption is right and thats what I was refering in my previous post by stating Impala does not support HDFS-level user impersonation.
... View more
10-12-2016
07:22 AM
When you run impala-shell it would not run as "impala", it would run as the current user. Impala does not support HDFS-level user impersonation .if you need grandular level authorization / user permission you might want to use Sentry . please refer this link. https://www.cloudera.com/documentation/enterprise/5-2-x/topics/cm_sg_sentry_service.html
... View more
10-08-2016
05:02 AM
Could you let us know the timestamp format - is it something like the below yyyy-mm-dd hh:mm:ss?
... View more
10-06-2016
05:10 AM
3 Kudos
Please add some more memory by editing the mapred-site.xml <property>
<name>mapred.child.java.opts</name>
<value>-Xmx4096m</value>
</property> The above tag i have used 5gb. Let me know if that helped you alternatively you can also edit the hadoop-env.sh file add export HADOOP_OPTS="-Xmx5096m"
... View more
09-20-2016
09:26 AM
1 Kudo
1. I assume you are using my sql Check if MySQL connector jar file accessible to Hive 2. Check Meta store status sudo service hive-metastore status 3. ps -ef | grep RunJar 4. Check Hive Server2 status sudo service hive-server2 status 5. Finally check your hive-site.xml to see if you have missed the driver name
... View more
09-01-2016
12:05 AM
1 Kudo
You can refer Hdfs sink timestamp escape sequence , there is alot of them you can use accordingly . example U can use hdfs bucketing , for every one hour. agen1.sinks.hdfsSinks.hdfs.path = /data/flume/%{aa}/%y/%m/%d/%H/%M
agent1.sinks.hdfsSinks.hdfs.round = true
agen1.sinks.hdfsSinks.roundUnit = hour
agen1.sinks.hdfsSinks.roundValue = 1
... View more
- « Previous
- Next »