Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

RegexSerDe in hue/hive-editor does not work

Solved Go to solution

RegexSerDe in hue/hive-editor does not work

New Contributor

While working with CDH5, hive and RegexSerDe there are some issues in hive-editor in hue environment. Initially a table is created:

CREATE TABLE access_log (
        `ip`                STRING,
        `time_local`        STRING,
        `method`            STRING,
        `uri`               STRING,
        `protocol`          STRING,
        `status`            STRING,
        `bytes_sent`        STRING,
        `referer`           STRING,
        `useragent`         STRING
        )
    ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
    WITH SERDEPROPERTIES (
    'input.regex'='^(\\S+) \\S+ \\S+ \\[([^\\[]+)\\] "(\\w+) (\\S+) (\\S+)" (\\d+) (\\d+) "([^"]+)" "([^"]+)".*'
)
STORED AS TEXTFILE
LOCATION '/user/test/apachelogs/access_log-20140622';

 
Then data is moved into the table.

LOAD DATA INPATH /user/test/apachelogs/access_log-20140622' into table access_log;

 
 Up to here all is fine. But when executing the follwoing statement, a mapreduce job is started and freezes at 5%:

select count(*) from access_log ;

 
The problem is caused by a classpath issue, so the RegexSerDe is not found within the hive editor. I started the hive shell on commandline and added a jar:

add jar /path/to/hive-contrib.jar;
select count(*) from access_log;

 
 Here the query runs and returns the correct results.

Question: how to run the query in hue/hive-editor? I tried settings and add jar resource, but the query does not run ...

1 ACCEPTED SOLUTION

Accepted Solutions

Re: RegexSerDe in hue/hive-editor does not work

New Contributor

Now it is working!

 

Main problems seems to be the web frontend, which does not give feedback. There was an error in the jar path and it could not be found. The shell on command line is more intuitive :-)

 

thx!

4 REPLIES 4

Re: RegexSerDe in hue/hive-editor does not work

Could you add the jar in the "Settings" tab?

[image: Inline image 1]

Romain

Re: RegexSerDe in hue/hive-editor does not work

New Contributor

Now it is working!

 

Main problems seems to be the web frontend, which does not give feedback. There was an error in the jar path and it could not be found. The shell on command line is more intuitive :-)

 

thx!

Re: RegexSerDe in hue/hive-editor does not work

Glad to hear!

For info I just tried with a wrong path and I fot the error (Hue 3.5+)
*:*


*Your query has the following error(s):*

Error while processing statement:
hdfs://localhost:8020/user/romain/other_vhosts_access.jar does not exist.

Romain

Highlighted

Re: RegexSerDe in hue/hive-editor does not work

New Contributor

What if I am not able to add the jar via setting tab [error - Insufficient privileges to execute ADD] ?
Also if I want this jar to be available to all the users, how can i do that ?
Please suggest.