Hi Guys, I am trying to check the service status with Ambari rest API, However I am not able to find any documention which explain thing in great details. For example, If I hit following REST URL http://localhost:8080/api/v1/clusters/Sandbox/services/HDFS I get following output "maintenance_state": "OFF", "repository_state": "CURRENT", "service_name": "HDFS", "state": "STARTED" }, "alerts_summary": { "CRITICAL": 0, "MAINTENANCE": 8, "OK": 293, "UNKNOWN": 4, "WARNING": 0 }, However, I am not sure how to interpret this things. Should I care about MAINTENANCE, UNKNOWN and WARNING and just checking that nothing CRITICAL is good enough. This is mainly for developers to understand and track the time how much any service is down. ... View more

I am trying to understand the hive query plan for a simple distinct query and I have small confusion regrading output of one of the stage. I have simple table with just two column, id and value. and just 4 rows as mentioned below. Data: <code>hive> select * from temp.test_distinct; OK 1 100 2 100 3 100 4 150 Plan <code>hive> explain select distinct value from temp.test_distinct; OK Plan not optimized by CBO. Vertex dependency in root stage Reducer 2 <- Map 1 (SIMPLE_EDGE) Stage-0 Fetch Operator limit:-1 Stage-1 Reducer 2 File Output Operator [FS_6] compressed:false Statistics:Num rows: 2 Data size: 10 Basic stats: COMPLETE Column stats: NONE table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"} Group By Operator [GBY_4] | keys:KEY._col0 (type: string) | outputColumnNames:["_col0"] | Statistics:Num rows: 2 Data size: 10 Basic stats: COMPLETE Column stats: NONE |<-Map 1 [SIMPLE_EDGE] Reduce Output Operator [RS_3] key expressions:_col0 (type: string) Map-reduce partition columns:_col0 (type: string) sort order:+ Statistics:Num rows: 4 Data size: 20 Basic stats: COMPLETE Column stats: NONE Group By Operator [GBY_2] keys:value (type: string) outputColumnNames:["_col0"] Statistics:Num rows: 4 Data size: 20 Basic stats: COMPLETE Column stats: NONE Select Operator [SEL_1] outputColumnNames:["value"] Statistics:Num rows: 4 Data size: 20 Basic stats: COMPLETE Column stats: NONE TableScan [TS_0] alias:test_distinct Statistics:Num rows: 4 Data size: 20 Basic stats: COMPLETE Column stats: NONE Time taken: 0.181 seconds, Fetched: 35 row(s) Confusion: TableScan, Select Operator and Group By Operator shows that they processed 4 rows which make sense to me. But shouldn't the next stage after Group By Operator get only 2 rows the process. As group by will remove other rows. In my DAG I can see the output of mapper is just two rows and not four, however that doesn't see to match with Plan. May i looking it wrong? ... View more

I am trying to connect to hive metastore to my HDP sandbox. However it's throwing following error. Hive Catalogonnector.name=hive-hadoop2 hive.metastore.uri=thrift://sandbox-hdp.hortonworks.com:9083 hive.metastore.authentication.type=NONE Error: Query 20180810_005352_00000_umgac failed: Failed connecting to Hive metastore: [sandbox-hdp.hortonworks.com:9083] I tried using following values for hive.metastore.uri however I am gettting the same error. thrift://localhost:9083 thrift://127.0.0.1:9083 thrift://<IP of docker container>:9083 thrift://<IP of local machine>:9083 ... View more

hi Guys. I am trying to load data files into my hive table and facing issue. if files are located on local it doesn't work. if I move the file to hdfs then it works without any issue. Following command is not working in beeline, however it works perfectly in hive load data local inpath '/home/gaurang.shah/test.json' into table temp.test; Data is located on the node where one of the instance of hiverserver2 is running. I have given it all the permission as well. [gaurang.shah@aa ~] pwd /home/gaurang.shah [gaurang.shah@aa ~]$ ll test.json -rwxrwxrwx 1 gaurang.shah domain users 56 Jul 11 13:54 test.json ... View more