Member since
12-10-2015
58
Posts
24
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1759 | 02-17-2016 04:12 AM | |
2946 | 02-03-2016 05:15 AM | |
1633 | 01-27-2016 09:13 AM | |
4133 | 01-27-2016 07:00 AM | |
2099 | 01-02-2016 03:29 PM |
01-27-2016
07:00 AM
3 Kudos
I guess it is not considering param file try this, pig -param_file=hdfs://ip-XXX-XX-XX-XXX.ec2.internal:8020/home/hadoop/adh_time /home/hadoop/test.pig writing param_file at the end encountering same issue for me too
... View more
01-05-2016
10:08 AM
1 Kudo
I have some number of files having format like 1)filename+date.filefomat 2)filename.fileformat now i need to copy only files which have some number before .(dot).
... View more
Labels:
- Labels:
-
Apache Hadoop
01-05-2016
04:37 AM
@Kuldeep Kulkarni Is there any way like ,Using "set" in directly grunt shell??? eg: set exectype=tez;
... View more
01-05-2016
04:21 AM
I would like to know what are the different ways to enable hcatalog and tez in writing pig scripts.
... View more
Labels:
- Labels:
-
Apache Pig
01-04-2016
02:15 PM
@Benjamin Leonhardi Thank you.Yeah,The script setting the environmental variable and then executing pig script in $PIG_HOME like exec /usr/hdp/2.2.8.0-3150/pig/bin/pig.distro "$@"
... View more
01-04-2016
01:53 PM
@Benjamin Leonhardi I used dump after illustrate.So i got error.So the problem is with " illustrate " command. Actually i have a habit to use illustrate for every pig command i used in grunt shell to check the output.
... View more
01-04-2016
09:28 AM
I would like to print or read pig_home from terminal.Is there any way???
... View more
Labels:
- Labels:
-
Apache Pig
01-04-2016
08:56 AM
Apache Pig version 0.12.1.2.1.7.0-784 I have data where one of the field doesn't have data like 2015,,08
2015,,09
2015,,11
2015,,04
2015,,05 Now i run the pig command like grunt> given_input = load '/pigtest/flightdelays/' using PigStorage(',') as (year,month,day);
grunt> ori = foreach given_input generate month;
grunt> illustrate ori;
generating error like : Caused by: java.lang.RuntimeException: No (valid) input data found! when i replace the loader with CSVExcelStorage like grunt> given_input = load '/pigtest/flightdelays/' using org.apache.pig.piggybank.storage.CSVExcelStorage(',') as (year,month,day);
grunt> ori = foreach given_input generate month;
grunt> illustrate ori; getting output like -------------------------------------------------------------------------------
| given_input | year:bytearray | month:bytearray | day:bytearray |
-------------------------------------------------------------------------------
| | 2015 | | 05 |
-------------------------------------------------------------------------------
--------------------------------
| ori | month:bytearray |
--------------------------------
| | |
-------------------------------- So,I would like to know 1)What is the problem with Pigstorage. 2)Is it loader problem or pig version problem. 3)If i want to use PigStoarage in this,How is should??? Not only illustrate even dump behaves the same.
... View more
Labels:
- Labels:
-
Apache Pig
01-02-2016
03:29 PM
1 Kudo
@Vidya SK DISTINCT in pig is a relational operator.So it will apply or perform on relations rather than fields or some other.consider the following. given_input = load '/given/path' using PigStorage(',') as (col1 ,col2,col3); consider the following situations. 1)Suppose i want to maintain unique values in col1 then, unique_col1 = foreach given_input generate col1;
unique_values= DISTINCT unique_col1; (DISTINCT only perform on relations i.e unique_col1). suppose col1 contains data like hortonworks
hortonworks
cloudera then u get cloudera
hortonworks 2)Suppose i want to maintain unique values in col1 and col2 then unique_two_fields = forech given_input generate col1 ,col2;
unique_values = DISTINCT unique_two_fields; (DISTINCT only performs on relations) suppose col1 and col2 contains data like hortonworks,clouera
hortonworks,clouera
hortonwors,hortonworks u get like hortonworks,clouera
hortonwors,hortonworks
Like this we should get the data that u want to make unique in one relation and then apply the distinct operator.Suppose if u want to perform any aggregations then go for group and apply aggregations.
... View more
12-31-2015
06:05 AM
@Guilherme Braccialli yupp...its working.
... View more