Member since
07-07-2020
63
Posts
4
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
343 | 06-13-2022 10:24 PM | |
1241 | 06-09-2022 09:56 PM | |
535 | 10-12-2021 07:13 AM | |
907 | 09-22-2021 10:54 PM |
03-21-2023
12:01 AM
Hello , - Please share the complete impala-shell command that you are using? - Is there any load balancer configured for impala? Thanks
... View more
07-19-2022
09:31 PM
For the workflows that were created on Hue, you can follow the next steps to export/import the workflows:
>To export:
Go to Documents> Select the workflow(s)> Select the button with three dots and select the Export option
>To import:
Go to Documents> Select the button with three dots and select the Import option.
... View more
06-26-2022
06:16 PM
As upsert is not SQL-92 syntax, the Impala JDBC driver does not currently support it. However, we are currently testing a newer version of the JDBC driver and hope to support upsert in the near future. For the time being, please break these statements into a combination of insert and update statements.
... View more
06-23-2022
03:56 AM
For a permanent fix, you need to install the python version 2.7.5.
... View more
06-22-2022
07:51 AM
Yes, you need to switch to venv everytime or you need to reinstall python with the 2.7.5 version.
... View more
06-22-2022
07:34 AM
You can try the below steps in edge node B virtualenv venv -p python2 cd venv source bin/activate (venv) impala-shell
... View more
06-22-2022
07:31 AM
Hello Syed, Yes, the impala is compatible with Python version 2.7.5.
... View more
06-13-2022
10:24 PM
Oozie Email Action supports attachments of HDFS files. Because standard output files are written as local files, you need to copy them to HDFS first, then specify the path in the <attachment> tag of the email action. <workflow-app name="[WF-DEF-NAME]" xmlns="uri:oozie:workflow:0.1"> ... <action name="[NODE-NAME]"> <email xmlns="uri:oozie:email-action:0.2"> <to>[COMMA-SEPARATED-TO-ADDRESSES]</to> <cc>[COMMA-SEPARATED-CC-ADDRESSES]</cc> <!-- cc is optional --> <subject>[SUBJECT]</subject> <body>[BODY]</body> <content_type>[CONTENT-TYPE]</content_type> <!-- content_type is optional --> <attachment>[COMMA-SEPARATED-HDFS-FILE-PATHS]</attachment> <!-- attachment is optional --> </email> <ok to="[NODE-NAME]"/> <error to="[NODE-NAME]"/> </action> ... </workflow-app>
... View more
06-09-2022
09:56 PM
Hello Team, We have tested the java code internally and it worked fine for us. ClouderaJDBC version:- 2.6.27.1032 Java code:- ======== import java.sql.*; import java.math.*; public class test{ public static void main(String args[]){ try{ Class.forName("com.cloudera.impala.jdbc41.Driver"); Connection con=DriverManager.getConnection("jdbc:impala://<<hostname>> :21050;UseNativeQuery=1"); String sql = "upsert into user_info(id, name, address, email, insert_time) values (?,?,?,?,?)"; PreparedStatement statement = con.prepareStatement(sql); statement.setInt(1, 102); statement.setString(2, "Peter"); statement.setString(3, "New York"); statement.setString(4, "John@xyz.com"); statement.setTimestamp(5, java.sql.Timestamp.valueOf(java.time.LocalDateTime.now())); statement.addBatch(); statement.executeBatch(); statement.close(); con.close(); } catch(Exception e){ System.out.println(e); } } } Please let us know if it helps.
... View more
06-09-2022
12:29 AM
Please elaborate a little more on the issue:- Also please share the steps you are performing and share the table DDL for the same.
... View more
11-25-2021
01:23 AM
From where the query is submitted[Jdbc/ODBC, Impala-shell, Hue]. If it is from JDBC/ODBC then the query is generated to figure out column names. You can disable the feature that generates it with the PreparedMetadataLimitZero flag - see [page 90] https://www.cloudera.com/documentation/other/connectors/impala-jdbc/latest/Cloudera-JDBC-Driver-for-...
... View more
10-14-2021
03:41 AM
Hello, Please try the below command and let us know:- $ sqoop import --connect jdbc:mysql://localhost/employees --username hive --password hive --table departments --hcatalog-database default --hcatalog-table my_table_orc --create-hcatalog-table --hcatalog-storage-stanza --hcatalog-storage-stanza "stored as orc tblproperties (\"transactional\"=\"false\")" If it doesn’t work then the workaround is the two step process. 1. Create the ORC table in hive with the keyword external and set transactional to false 2. Then use the sqoop command to load the data into the orc table.
... View more
10-14-2021
02:51 AM
Hello, Please try to run the sqoop command as below and let us know how it goes:- $ sqoop import --connect jdbc:mysql://localhost/employees --username hive --password hive --table departments --hcatalog-database default --hcatalog-table my_table_orc --create-hcatalog-table --hcatalog-storage-stanza "stored as orcfile"
... View more
10-14-2021
12:22 AM
1 Kudo
Hello, The partition clause in the drop partition expect CONSTANT VALUE on the right-hand side and the functions inside the drop partition clause are not supported The correct syntax would be:- ALTER TABLE audit_logs DROP PARTITION (evt_date<‘some constant value ' )
... View more
10-13-2021
02:25 AM
Hello, Please try with the below connection URL and let us know how it goes jdbc:impala://nightly57-3.gce.cloudera.com:21050/default;AuthMech=1;SSL=1;KrbRealm=GCE.CLOUDERA.COM;KrbHostFQDN=nightly57-4.gce.cloudera.com;KrbServiceName=impala;SSLTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks
... View more
10-12-2021
07:13 AM
1 Kudo
Hello, There is no such way to kill the query in one go. You need to either do it by one of the following methods:- You can kill the query from the Impala Daemon web UI of the Impala Daemon coordinating the query. or you can try killing from the browser. https://<query_coordinator_server_name>:25000/close_session?session_id=<session_id>
... View more
10-11-2021
10:29 PM
Hello, You can kill the query from the Impala Daemon web UI of the Impala Daemon coordinating the query. or you can try killing from the browser. https://<query_coordinator_server_name>:25000/close_session?session_id=<session_id> Please let us know if it helps
... View more
10-04-2021
01:58 AM
Hello, Are you trying to connect impala from spark via JDBC? if yes, we don't support this feature yet. please refer to the below document. https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_621_unsupported_features.html#spark
... View more
10-04-2021
01:49 AM
Hello, Please use the either of the methods and let us know if it helps:- Step 1:- Pass the --hs2-url correctly sqoop import --connect <<Connection string>> --username <<username>> --password <<password>> --table “ta”ble_name --target-dir “<<“directory path>> --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab" If you are launching the sqoop job from CLI, you don’t need to pass the hs2-user and hs2 key tab. You can remove the parameters and run the sqoop command after doing the kinit from CLI or you can pass in the sqoop command itself like above. Step 2:- If you don’t want to add the hs2_url Add the following 2 properties in the sqoop configuration and restart sqoop 1. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml : Name: sqoop.beeline.env.preserve Value: HADOOP_CLIENT_OPTS 2. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh: export HADOOP_CLIENT_OPTS="-Djline.terminal=jline.UnsupportedTerminal"
... View more
10-04-2021
01:28 AM
Hello, Please find the sample impala oozie action and check if it helps. 1 Create a file impala_invalidate.sh export PYTHON_EGG_CACHE=/tmp/impala_eggs impalacmd="impala-shell -k -s impala --quiet --impalad=host-10-17-100-110.coe.cloudera.com:21000" ${impalacmd} --query="show databases;" > /tmp/impala_databases 2 Create the workflow for the shell action <workflow-app name="Impala_redirect" xmlns="uri workflow:0.5"> <start to="shell-11a3"/> <kill name="Kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <action name="shell-11a3"> <shell xmlns="uri shell-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <exec>impala_invalidate.sh</exec> <file>/user/systest/lib/impala_invalidate.sh#impala_invalidate.sh</file> <file>/user/systest/lib/impala.keytab#impala.keytab</file> <capture-output/> </shell> <ok to="End"/> <error to="Kill"/> </action> <end name="End"/> </workflow-app>
... View more
09-23-2021
01:39 AM
Hello Can you try giving space between -- & import as it is a sub tool name. sqoop job --create job1 -- import --connect jdbc:oracle:thin:xxxxx --username xxxxx --password xxxxxxxx --as-parquetfile --target-dir /user/cdh/testgb/raw/adminunits --fields-terminated-by ',' --lines-terminated-by '\n' --table develop.adminunits --m 1 --hive-import --hive-database metdev_testgb --hive-table ADMINUNITS
... View more
09-22-2021
10:54 PM
2 Kudos
1. For total memory configured you can check (impala daemon memory * a total number of demons ) , these values should be displayed on top of Impala admission control as well that this much if memory is allocated to the impala. 2. You can check other memory metrics from the cluster utilization report, please note that how much memory is consumed per pool feature is not currently captured in impala metrics. a) Max Allocated Peak Allocation Time – The time when Impala reserved the maximum amount of memory for queries. Click the drop-down list next to the date and time and select View Impala Queries Running at the Time to see details about the queries. Max Allocated – The maximum memory that was reserved by Impala for executing queries. If the percentage is high, consider increasing the number of hosts in the cluster. Utilized at the Time – The amount of memory used by Impala for running queries at the time when maximum memory was reserved. Click View Time Series Chart to view a chart of peak memory allocations. Histogram of Allocated Memory at Peak Allocation Time – Distribution of memory reserved per Impala daemon for executing queries at the time Impala reserved the maximum memory. If some Impala daemons have reserved memory close to the configured limit, consider adding more physical memory to the hosts. b) Max Utilized Peak Usage Time – The time when Impala used the maximum amount of memory for queries. Click the drop-down list next to the date and time and select View Impala Queries Running at the Time to see details about the queries. Max Utilized – The maximum memory that was used by Impala for executing queries. If the percentage is high, consider increasing the number of hosts in the cluster. Reserved at the Time – The amount of memory reserved by Impala at the time when it was using the maximum memory for executing queries. Click View Time Series Chart to view a chart of peak memory utilization. Histogram of Utilized Memory at Peak Usage Time – Distribution of memory used per Impala daemon for executing queries at the time Impala used the maximum memory. If some Impala daemons are using memory close to the configured limit, consider adding more physical memory to the hosts. [1] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_cluster_util_custom.html#concept_jp4_4bh_hx [2] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_metrics_impala_daemon.html [3] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_metrics_impala_daemon_resource_pool.html
... View more