There are some factors which are affecting performance of sqoop import. 1. Network Bandwidth between source server (Oracle, MySQL, PostgreSQL) and destination server (Datanodes). 2. Source server's connection policy for clients. 3. CPU, RAM, DISK Performance listing of servers. ... View more

What is the application_1504517816511_0001~4? I'm not sure, if a run application_1504517816511 before enabled ResourceManager HA, then kill this application first. ... View more

As I know, sqoop import "--append" options is not compatible with "--hcatalog-*". Just remove "--append". ... View more

Check your mounted disk status whether or not to write on disk. ... View more

I'm just interesting. Why does it setup these options in hive config from ambari web? Well, actually, that means is if I want to use hive ORC file format with advanced TBLPROPERTIES such as "orc.compress, orc.compress.size, orc.stripe.size, orc,create.index....etc", I have to specify these tblproperties options every times when I'm trying to create hive table ORC file format. ... View more

As I know, you don't need to specify oracle jdbc driver parameter in cmd. ... View more

The jar path you typed is wrong path pattern. ADD JAR hdfs:////user/test/lib/my-custom-format-0.0.1-SNAPSHOT.jar It's not working properly. Try it again below. Add a jar in local path, then file:///home/username/some_lib/...jar or just use /home/username/some_lib/...jar. Add a jar in hdfs path, then hdfs://namenode_fqdn:port(8020)/user/username/lib/...jar or if you setup the namenode HA, just use the nameserviceid such as hdfs:///nameserviceid/user/username/lib/...jar. ... View more

1st. If you were executed spark cmd with master(local), then check the connection host and port in that local server. 2nd. Check your firewall & iptables status whether it is of or off. ... View more

I think, the hivemetastore is not loaded hadoop&hive env properly. Check this conf file and hive library from installed server with hive packages. 1. /etc/hive/conf/hive-env.sh ... # Set HADOOP_HOME to point to a specific hadoop install directory HADOOP_HOME=${HADOOP_HOME:-/usr/hdp/current/hadoop-client} export HIVE_HOME=${HIVE_HOME:-/usr/hdp/current/hive-client} # Hive Configuration Directory can be controlled by: export HIVE_CONF_DIR=${HIVE_CONF_DIR:-/usr/hdp/current/hive-client/conf} ... 2. /usr/hdp/2.4.2.0-258/hive-metastore/lib & /usr/hdp/2.4.2.0-258/hive/lib Check library files. /usr/hdp/2.4.2.0-258/hive-metastore/lib ..... hive-jdbc-1.2.1000.2.5.3.0-37-standalone.jar hive-jdbc-1.2.1000.2.5.3.0-37.jar hive-jdbc.jar -> hive-jdbc-1.2.1000.2.5.3.0-37-standalone.jar hive-metastore-1.2.1000.2.5.3.0-37.jar hive-metastore.jar -> hive-metastore-1.2.1000.2.5.3.0-37.jar hive-serde-1.2.1000.2.5.3.0-37.jar hive-serde.jar -> hive-serde-1.2.1000.2.5.3.0-37.jar hive-service-1.2.1000.2.5.3.0-37.jar hive-service.jar -> hive-service-1.2.1000.2.5.3.0-37.jar .... /usr/hdp/2.4.2.0-258/hive/lib libthrift-0.9.3.jar ... View more

It's a normal, cause' you don't have access ownership & permission of that path. The path, /tmp/ambari-qa, ownership & permission is ambari:hdfs:700. That means is only access to the path by ambari-qa user include a sub-path (staging). So you have to access another user such as nifi, you should acquire ambari-qa ownership or hdfs ownership not group. I'm not recommend to change ownership or permission /tmp/../path, but if you using with only nifi user necessarily, then change the permission "755" of the path "/tmp/ambari-qa" and a sub-path by hdfs user or ambari-qa user. ... View more

Is that the same distribution hadoop version between CentOS and Ubuntu? ... View more

Try to change hdp-utils.repo's baseurl to "http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6" ... View more

The best way to connect to Hadoop Cluster as client server, register client server to ambari-server. If you don't have same OS version between ambari-server and client server, then you should setup same version's hadoop library and config files in client server. To handle easily for HA services like NameNode, ReourceManager, Hive,...etc, I'll recommend to use ZooKeeper Curatorframework. ... View more

jdbc:hive2//fqdn/10000 .... You have returned 'invalid URL'. Check the metastore url setting value in Ambari > Hive. ... View more

Did you install hive service with metastore clearly? Could you let me show the Hive Summary Menu in ambari. ... View more

Could you attach your hivemetastore.log after started hive service from ambari? ... View more

1. Connect to mysql as root. 2. Execute these queries. mysql> use mysql; mysql> select * User,Host,Password from mysql; Normally, that returned two or three results for user 'hive'. hive | localhost | *encodedPassword hive | adrien.cluster | *encodedPassword hive | % | *encodedPassword if you can't get above results, then execute queries. mysql> create user 'hive'@'adrien.cluster' identified by 'PASSWORD' mysql> create user 'hive'@'%' identified by 'PASSWORD' mysql> grant all privileges on *.* to 'hive'@'adrien.cluster' mysql> grant all privileges on *.* to 'hive'@'%' mysql> flush privileges; 3. Try connection test in Ambari Web's Hive menu. ... View more

Did you check the option "dfs.namenode.acls.enabled=true"? ... View more