2206
Posts
230
Kudos Received
82
Solutions
About
My expertise is not in hadoop but rather online communities, support and social media. Interests include: photography, travel, movies and watching sports.
My Accepted Solutions
Title | Views | Posted |
---|---|---|
440 | 05-07-2025 11:41 AM | |
910 | 02-27-2025 12:49 PM | |
2786 | 06-29-2023 05:42 AM | |
2366 | 05-22-2023 07:03 AM | |
1730 | 05-22-2023 05:42 AM |
06-19-2018
02:02 PM
I received a similar error, but for the LzopCodec (not LzoCodec) not being found. In this case, I actually wanted to use the LzoCodec as my default compression codec (for legacy reasons...). In order to get Hive working with Lzo on CDH 5.14.x, I did the following: 1)
# Add GPLEXTRAS parcel to CM and distribute to all nodes:
https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_ig_install_gpl_extras.html
2)
# Configure HDFS to use it:
http://www.roggr.com/2014/06/enabling-lzo-compression-for-hive-to.html
ClouderaManager -> Hive -> Configuration -> Service-wide -> Advanced -> Hive Auxiliary JARs Directory : /opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib
# Verify it works by invoking 'show tables' in hive with debug-logging enabled. There should not be any errors. The error condition if not working complains about LzopCode (not LzoCodec) not found.
hive --hiveconf hive.root.logger=DEBUG,console
3)
# Configure HDFS
ClouderaManager -> HDFS -> Configuration -> Service-wide -> Compression Codes (io.compression.codecs):
+ com.hadoop.compression.lzo.LzoCodec
+ com.hadoop.compression.lzo.LzopCodec
4)
# Configure YARN/MR2
# http://blog.cloudera.com/blog/2013/07/one-engineers-experience-with-parcel/
ClouderaManager -> YARN/MR2 -> Configuration -> SearchFor: compress ->
mapreduce.output.fileoutputformat.compress: checked
mapreduce.output.fileoutputformat.compress.codec: com.hadoop.compression.lzo.LzoCodec
mapreduce.map.output.compress: checked
mapreduce.map.output.compress.codec: com.hadoop.compression.lzo.LzoCodec
... View more
06-19-2018
07:23 AM
Following up on this. The 404 error has been corrected. 🙂
I'm sorry it took so long.
... View more
06-05-2018
05:37 AM
Of course I wondered what @KamaJinny was saying so I figured others may as well. Here is the translation via google translate:
... View more
05-08-2018
01:13 AM
Do not specify driver in sqoop arguments. Using the –driver parameter will always force Sqoop to use the Generic JDBC Connector regardless of if a more specialized connector is available. For example, if the MySQL specialized connector would be used because the URL starts with jdbc:mysql://, specifying the –driver option will force Sqoop to use the generic connector instead. As a result, in most cases, you should not need to use the –driver option at all. Thanks, Ankit Gaurav Sinha
... View more
04-21-2018
05:52 AM
Yup, I will wait till Tuesday. Thanks for the reply.
... View more
04-20-2018
01:27 PM
Thinks for bringing this up @AcharkiMed. I've seen this type of thing happen on other communities as well. While the view count is nice I also consider things like number of kudos and replies when guaging the popularity of a post. I'm assuming your concern is people thinking their post is popular when it could be them, or others, checking on it to see if there are updates. Am I correct?
If the reason for refreshing the page is to see if there is a new reply, I would suggest checking your subscription settings. I have mine set to notify me immediately of new activity plus I look often at the notification count on most pages.
... View more
04-11-2018
05:57 PM
1 Kudo
There are dirty data in the roles table, I deleted it
... View more
03-28-2018
11:22 AM
All I'm seeing are messages like the following when restarting haproxy: 2018-03-28T11:32:41-04:00 172.28.2.234 haproxy[26642]: Proxy main started.
2018-03-28T11:32:41-04:00 172.28.2.234 haproxy[26642]: Proxy static started.
2018-03-28T11:32:41-04:00 172.28.2.234 haproxy[26642]: Proxy impala started.
2018-03-28T11:32:41-04:00 172.28.2.234 haproxy[26642]: Proxy impalajdbc started.
2018-03-28T11:32:41-04:00 172.28.2.234 haproxy[26642]: Proxy main started.
2018-03-28T11:32:41-04:00 172.28.2.234 haproxy[26642]: Proxy impalajdbc started.
2018-03-28T11:32:41-04:00 172.28.2.234 haproxy[26642]: Proxy app started. And sometimes other messages like the following in the haproxy log - but they don't come in immediately after running an Impala query over JDBC such as from Hue or beeline: 2018-03-27T15:51:47-04:00 172.28.2.234 haproxy[4286]: 172.28.6.234:37768 [27/Mar/2018:15:51:47.621] impalajdbc impalajdbc/impalajdbc1 0/0/+0 +0 -- 2/2/2/1/0 0/0
2018-03-27T18:08:08-04:00 172.28.2.234 haproxy[4286]: 172.28.2.20:39978 [27/Mar/2018:18:08:08.888] impala impala/impalad1 0/0/+0 +0 -- 3/1/1/1/0 0/0 However, no logs come in when accessing from a BI tool such as Tableau using its native Cloudera Impala connector or the Cloudera Impala ODBC driver. Is there a way to increase the logging for haproxy so that we can know which Impala Daemon a query is executing on for the purpose of debugging potential issues of someone accessing from a BI application? I already have it said to debug under the listen section for impalajdbc. listen impalajdbc :21051
mode tcp
option tcplog
balance roundrobin
log 172.28.xx.xx local2 debug
server impalajdbc1 hdp104v.cmssvc.local:21050
server impalajdbc2 hdp105v.cmssvc.local:21050 Thanks, Braz
... View more