Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.
GokuNaveen
Explorer

SSL certification validation ignoring "-k" via pyt...

by Explorer in Support Questions
‎08-29-2018 07:39 AM
‎08-29-2018 07:39 AM
Dear Team,   I am trying to connect to cm api client with below python script.   Code: import sys from datetime import datetime, timedelta from cm_api.api_client import ApiResource cm_host = "172.21.1.40" cm_port = "7183" cdh5 = None cm_host = "<IP Address>" api = ApiResource(cm_host, 7183, username=" ********", password="*********", use_tls=True) for c in api.get_all_clusters():     print c.name   I get below error ... Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.7/site-packages/cm_api/api_client.py", line 131, in get_all_clusters return clusters.get_all_clusters(self, view) File "/usr/lib/python2.7/site-packages/cm_api/endpoints/clusters.py", line 66, in get_all_clusters params=view and dict(view=view) or None) File "/usr/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 139, in call ret = method(path, params=params) File "/usr/lib/python2.7/site-packages/cm_api/resource.py", line 110, in get return self.invoke("GET", relpath, params) File "/usr/lib/python2.7/site-packages/cm_api/resource.py", line 73, in invoke headers=headers) File "/usr/lib/python2.7/site-packages/cm_api/http_client.py", line 181, in execute return self._opener.open(request) File "/usr/lib64/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 449, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 1258, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open raise URLError(err) urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)>   I can overrided above error with below command manually[-k] curl -X post  -k -u userid:**** https://******************:7183/api/v19/clusters/ADH-DEVELOPMENT001/commands/stop   Now the problem statment is . I am not sure how to add -k option in python code basically don't know how to "Ignore SSL certification validation"     Thanks and Regards, Naveen Srikanth D ... View more
Labels:

For Pyspark Sql Query delay in picking up tasks fo...

by Explorer in Support Questions
‎08-15-2018 03:19 AM
‎08-15-2018 03:19 AM
Dear All,   I am using pyspark2.3.0 . With below syntax         pyspark2 --driver-memory 4G --executor-cores 3 --driver-cores 3 --num-executors 16 --executor-memory 4G   On execution of above statement making I am reserving the above cores mentioned   Now the problem is I am firing below query, Below mentioned table is partitioned and it is of 450GB         spark.sql("select * from tabl_name limit 20").show()           On firing above query we have ,There are 10000 tasks created and execution starts at end of 4th min.    Is there any help so that I could reduce the time to wait till 4 min. Please let me know if any parameter I should be setting     Thanks and Regards, Naveen Srikanth D   ... View more
Labels:

Re: Not able to retrieve value from "show table st...

by Explorer in Support Questions
‎07-26-2018 03:54 AM
‎07-26-2018 03:54 AM
Thanks for the response saranvisa .   Reiterating my requirement , and is below . I fire below statement show table stats tblname; My output will look like below: Partition_Name, Rows, Files,Ingesteddate...etc val1 val2                       Total Rows   Now my problem statement is how do I query this table alone, For example out of all the partition values I should get total rows out . How can I do it ? ... View more

Not able to retrieve value from "show table stats ...

by Explorer in Support Questions
‎07-26-2018 03:15 AM
‎07-26-2018 03:15 AM
Dear Team,   No command for retrieving value from show table stats t2; From Below command ,I am able to see the values show table stats t2; But, How to get each column values.For example. How do I get the Total value from column #Rows   Please do the needful.   Thanks and Regards, Naveen Srikanth D ... View more
Labels:

Re: Hive and impala results are differing

by Explorer in Support Questions
‎07-11-2018 11:09 PM
‎07-11-2018 11:09 PM
In below mentioned post when I fire query like below, works fine    select * from working.ddr2_raw_actual_test where dest_icao='NULL'   But with actual file it is not working. I am getting resultset in hive which is correct but same query not able to get in impala.   Thanks and Regards, Naveen Srikanth D ... View more

Re: Hive and impala results are differing

by Explorer in Support Questions
‎07-11-2018 10:32 PM
‎07-11-2018 10:32 PM
Dear Naveen,   Thanks for picking up the issue.   Below is the data  $NToI_$NToJ,LTB,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL $NWVa,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL BENTU,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL $NXTe_B,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL $MoaR,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL LER,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL   Query: select * from tbl_name where c is null; Problem is: Above query was running on 2 million records and returns attached file 6 records in hive and same query doesn't result in impala.   Now what I have done is have taken this resultset alone and created new external table . Now query resultset not able to fetch in hive and impala both.  Actual files are delimited with space     Below is the ddl of the query:   CREATE TABLE `working.tbl_test`( `a` string, `b` string, `c` string, `d` string, `e` string, `f` string, `g` smallint, `h` smallint, `i` tinyint, `j` string, `k` string, `l` string, `m` float, `n` float, `o` float, `p` float, `q` int, `r` smallint, `s` float, `t` tinyint) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION '/tmp/tbl_test';   I will be reachable over +919791081769 and +919008482114. We could also have webex call if it is feasible for you   THanks and Regards, Naveen Srikanth D ... View more

Hive and impala results are differening

by Explorer in Support Questions
‎07-09-2018 04:14 AM
‎07-09-2018 04:14 AM
Dear All,   Below is my problem statement: I am trying to create the external table through hive. I am getting problems when I query 1)When I query count(*) from Hive and impala results are differing . 2)When I query an column in Hive with isnull condition I am getting resultset 6 rows.     In these 6 rows first 3 coulmns has data with values and from 4th column values are with NULL.     But when I fire the same query in impala I am not getting any rowsl   What could be the isssue please help me. Below are the last lines in ddl query   ROW FORMAT DELIMITED FIELDS TERMINATED BY ' |' LOCATION '/x/f/w/'   Thanks and Regards, Naveen Srikanth D   ... View more

Re: Impala memory configuration

by Explorer in Support Questions
‎03-21-2018 03:32 AM
‎03-21-2018 03:32 AM
Yes Admission control is enabled now ... View more

Impala memory configuration

by Explorer in Support Questions
‎03-11-2018 05:50 AM
‎03-11-2018 05:50 AM
Dear All,   I am using Cloudera 14 version. Cluster size is of 30 tb 8 node cluster   Issue is "After couple of query executions in impala hue/console getting conole memory exceeded by 131 mb error" or message like "Query expired or Rerun the Query if required"   Configuration of Impala is done by AWS load balancer .   Please suggest the memory recommened approach or any bechmark that can be configured in Impala .      Thanks and Regards, Naveen ... View more
Labels:

Resolved:Re: Unable to configure the Cloudera Mana...

by Explorer in Support Questions
‎11-15-2017 04:17 AM
‎11-15-2017 04:17 AM
Thanks alot Team. Issue resolved. Solution:             Need to attach roles and policies for the cloudera director url . Got the help from aws cloudera and aws documentations.After doing I got the issue resolved   We need to implement content present in below link properly  https://www.cloudera.com/documentation/director/latest/topics/director_iam_methods.html ... View more
Contact Me
Online
Offline
Last Visited
‎09-17-2019 06:34 AM
Public Statistics
Date Registered ‎03-03-2017 10:36 PM
Last Visited ‎09-17-2019 06:34 AM
Total Messages Posted 15