Member since
12-11-2015
213
Posts
87
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 7061 | 12-20-2016 03:27 PM | |
| 15804 | 07-26-2016 06:38 PM |
02-24-2016
11:10 PM
I tried running the command: pig -x tez -useHCatalog Pigscript but there result is the same. It's saying : 2016-02-24 23:04:54,675 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist
Details at logfile: /hadoop/yarn/local/usercache/ppunj/appcache/application_1456334730705_0001/container_e05_1456334730705_0001_01_000002/pig_1456355093669.log
2016-02-24 23:04:54,722 [main] INFO org.apache.pig.Main - Pig script completed in 1 second and 467 milliseconds (1467 ms)
... View more
02-24-2016
03:57 AM
@Neeraj Sabharwal. Looks like Ranger is failing to install because of the error below. I am using MySql db. what is the root user. Looks at the screen shot.. 2016-02-24 03:52:25,732 [I] ---------- Creating Ranger Admin db user ----------
2016-02-24 03:52:25,732 [JISQL] /usr/jdk64/jdk1.8.0_60/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://hdp-m.asotc/mysql -u rangeradmin -p '********' -noheader -trim -c \; -query "SELECT version();"
SQLException : SQL state: 28000 java.sql.SQLException: Access denied for user 'rangeradmin'@'hdp-m.asotc' (using password: YES) ErrorCode: 1045
... View more
02-24-2016
03:31 AM
@Neeraj Sabharwal Looks like I had mysql, so I am using instruction for MySql and installing Ranger. Hopefully this will go smooth...
... View more
02-24-2016
02:05 AM
1 Kudo
Few basic administration question. For a user of Ambari view, how can I do the following: Restrict their access only to a folder in HDFS I donnot want a regular user to see any other folder when they go to HDFS browser, Currently a user can view all the folders I want to restrict them to upload no more than 20gb Is it a good practice to create a file/hive/pig view instance for every user ? Can I create just one general HDFS / Hive browser and have users upload data in their folder only. Currently I create a separate folder for every user on the host machine using HDFS and then create a user in Ambari to work within that folder. Thanks
... View more
Labels:
- Labels:
-
Apache Ambari
02-23-2016
12:43 PM
1 Kudo
Thanks @Neeraj Sabharwal and @Saurabh Kumar. I think it would be helpful to identify users and their jobs running. May be in future release of Ambari..
... View more
02-23-2016
03:33 AM
@Artem Ervits Below is the code that I m using in pig script.
a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();
b = filter a by event != 'normal';
c = foreach b generate driverid, event, (int) '1' as occurance;
d = group c by driverid;
e = foreach d generate group as driverid, SUM(c.occurance) as t_occ;
g = LOAD 'drivermileage' using org.apache.hive.hcatalog.pig.HCatLoader();
h = join e by driverid, g by driverid;
final_data = foreach h generate $0 as driverid, $1 as events, $3 as totmiles, (float) $3/$1 as riskfactor;
store final_data into 'riskfactor' using org.apache.hive.hcatalog.pig.HCatStorer();
ERROR that I AM GETTING IS:
ARNING: Use "yarn jar" to launch YARN applications.
16/02/23 03:31:35 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/02/23 03:31:35 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/02/23 03:31:35 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2016-02-23 03:31:35,609 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0.2.3.4.0-3485 (rexported) compiled Dec 16 2015, 04:30:33
2016-02-23 03:31:35,609 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0013/container_e03_1455824286177_0013_01_000002/pig_1456198295607.log
2016-02-23 03:31:38,312 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist
Details at logfile: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0013/container_e03_1455824286177_0013_01_000002/pig_1456198295607.log
2016-02-23 03:31:38,339 [main] INFO org.apache.pig.Main - Pig script completed in 3 seconds and 412 milliseconds (3412 ms)
... View more
02-22-2016
06:35 PM
2 Kudos
Is there a way to find out number of active users on hadoop using Ambari dashboard
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
02-20-2016
06:59 PM
1 Kudo
Need to know how can I enforce restriction on Ambari view and stuff. Below is what is happening:
a.) I create an user account in Ambari
b.) Created File and Hive views in Ambari
c.) Assign permission only to this user account to the above created views
d.) created an account via HDFS ( command below)
What I noticed is account created this way can view everyone else's directory via Ambari file browser view. what I want is all he should be seeing is his user folder and access to upload data and create view.
How can I enforce this restriction.
[hdfs@hdp-m ~]$ hadoop fs -mkdir /user/guest1
[hdfs@hdp-m ~]$ hadoop fs -chown admin:hadoop /user/guest1
... View more
Labels: