Member since
07-19-2018
613
Posts
101
Kudos Received
117
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 5095 | 01-11-2021 05:54 AM | |
| 3421 | 01-11-2021 05:52 AM | |
| 8789 | 01-08-2021 05:23 AM | |
| 8385 | 01-04-2021 04:08 AM | |
| 36687 | 12-18-2020 05:42 AM |
04-29-2020
01:17 PM
@mzidy Have you tried to append the data you want to pass in the Remote Url? Some Apis will accept the values in the $_GET request. For example: DELETE: https://api.cloudera.com/api/service/?value1=value1&value2=value2 Some apis could be: DELETE: https://api.cloudera.com/api/service/1/2/ In both these examples the $_GET variables are used not a $_POST. I believe this may be typical for most DELETE calls and may explain why DELETE doesn't send a $_POST (body).
... View more
04-28-2020
05:15 AM
@Damian_S You cannot delete and recreate a hadoop cluster in this manner. The data in the metastore is very specific to the previous installation. A proper migration will back up the required artifacts from the original cluster, and migrate them to a new existing ambari cluster that has already completed the initial cluster installation.
... View more
04-28-2020
05:09 AM
@san_re The configuration of Remote Url in InvokeHTTP should be as follows: /j_security_check?j_username=${j_username}&j_password=${j_password} Notice the credential values are passed to the url with the ?. This is the only change in the processor config. Here is a demo flow: GenerateFlowFile - starts flow for testing UpdateAttribute - sets user, pass,and mime type attributes needed in InvokeHttp InvokeHttp - sends POST You can find this template on my GitHub to inspect the entire flow: https://github.com/steven-dfheinz/NiFi-Templates/blob/master/InvokeHttp_Demo.xml
... View more
04-24-2020
06:16 AM
@Former Member I think you misunderstood the Variable Scoping. All sensitive values in templates and registry controlled process groups do not migrate to the new instance. These sensitive values should be set at the Root NiFi Canvas level per environment. Then when you import a template, or in your case, a version controlled process group, the values are already set per environment. For example: Let's say we have 3 environments: dev, staging, and prod. Each host has its own "database" host with different database tables, users and passwords. I want all 3 environments to run a single flow with NiFi Registry Version Control. I create the flows database controller services using ${host}, ${user}, ${pass}, ${database}, etc. Now in each environment, at the root level, I define each variable. Since the sensitive value is defined at a higher Variable Scoping level (root canvas level or other higher level) it will be available to any template or version controlled process group at any lower Variable Scoping level.
... View more
04-22-2020
11:31 AM
@ruthika Here is a working hive 3.0 (hdp3 - no ssl, no kerberos) config for hue 4.6.0: [beeswax]
# Host where HiveServer2 is running.
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=hdp.cloudera.com
# Binary thrift port for HiveServer2.
#hive_server_port=10000
# Http thrift port for HiveServer2.
#hive_server_http_port=10001
# Host where LLAP is running
## llap_server_host = localhost
# LLAP binary thrift port
## llap_server_port = 10500
# LLAP HTTP Thrift port
## llap_server_thrift_port = 10501
# Alternatively, use Service Discovery for LLAP (Hive Server Interactive) and/or Hiveserver2, this will override server and thrift port
# Whether to use Service Discovery for LLAP
## hive_discovery_llap = true
# is llap (hive server interactive) running in an HA configuration (more than 1)
# important as the zookeeper structure is different
## hive_discovery_llap_ha = false
# Shortcuts to finding LLAP znode Key
# Non-HA - hiveserver-interactive-site - hive.server2.zookeeper.namespace ex hive2 = /hive2
# HA-NonKerberized - (llap_app_name)_llap ex app name llap0 = /llap0_llap
# HA-Kerberized - (llap_app_name)_llap-sasl ex app name llap0 = /llap0_llap-sasl
## hive_discovery_llap_znode = /hiveserver2-hive2
# Whether to use Service Discovery for HiveServer2
hive_discovery_hs2 = true
# Hiveserver2 is hive-site hive.server2.zookeeper.namespace ex hiveserver2 = /hiverserver2
hive_discovery_hiveserver2_znode = /hiveserver2
# Applicable only for LLAP HA
# To keep the load on zookeeper to a minimum
# ---- we cache the LLAP activeEndpoint for the cache_timeout period
# ---- we cache the hiveserver2 endpoint for the length of session
# configurations to set the time between zookeeper checks
## cache_timeout = 60
# Host where Hive Metastore Server (HMS) is running.
# If Kerberos security is enabled, the fully-qualified domain name (FQDN) is required.
#hive_metastore_host=hdp.cloudera.com
# Configure the port the Hive Metastore Server runs on.
#hive_metastore_port=9083
# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/etc/hive/conf
# Timeout in seconds for thrift calls to Hive service
## server_conn_timeout=120
# Choose whether to use the old GetLog() thrift call from before Hive 0.14 to retrieve the logs.
# If false, use the FetchResults() thrift call from Hive 1.0 or more instead.
## use_get_log_api=false
# Limit the number of partitions that can be listed.
## list_partitions_limit=10000
# The maximum number of partitions that will be included in the SELECT * LIMIT sample query for partitioned tables.
## query_partitions_limit=10
# A limit to the number of rows that can be downloaded from a query before it is truncated.
# A value of -1 means there will be no limit.
## download_row_limit=100000
# A limit to the number of bytes that can be downloaded from a query before it is truncated.
# A value of -1 means there will be no limit.
## download_bytes_limit=-1
# Hue will try to close the Hive query when the user leaves the editor page.
# This will free all the query resources in HiveServer2, but also make its results inaccessible.
## close_queries=false
# Hue will use at most this many HiveServer2 sessions per user at a time.
# For Tez, increase the number to more if you need more than one query at the time, e.g. 2 or 3 (Tez has a maximum of 1 query by session).
## max_number_of_sessions=1
# Thrift version to use when communicating with HiveServer2.
# Version 11 comes with Hive 3.0. If issues, try 7.
thrift_version=11
# A comma-separated list of white-listed Hive configuration properties that users are authorized to set.
## config_whitelist=hive.map.aggr,hive.exec.compress.output,hive.exec.parallel,hive.execution.engine,mapreduce.job.queuename
# Override the default desktop username and password of the hue user used for authentications with other services.
# e.g. Used for LDAP/PAM pass-through authentication.
## auth_username=hive
## auth_password=hive
# Use SASL framework to establish connection to host.
use_sasl=true Pay close attention the values I have uncommented. Especially the zk discovery, thrift version, and SASL Regarding your second issue: please monitor /var/log/hue/error.log for any errors while operating dashboard. Share those with us so we can be helpful. Additionally, you may need to use browser Dev Tool and report any client side errors associated to dashboard tools & widgets. You can also post and find help on the Hue Discourse.
... View more
04-22-2020
05:14 AM
@stevenmatison Thank you very much!
... View more
04-21-2020
06:25 AM
Hi @cjervis i resolved it by adding the below parameter hive.server2.parallel.ops.in.session=true under Hive > Config > Advanced > "Custom hive-site" .
... View more
04-19-2020
10:14 AM
It will create a new table but the requirement is to show the results of SQL in tab delimited. If we create a now table as \t delimited and then again select * from that new format table , wouldn't it will show with default "|" delimited.
... View more
04-17-2020
10:03 PM
Thank for you this reply! This has been quite difficult for me to troubleshoot, but I finally figured it out. These machines I've been using had chrony on them all along, but the previous machines I set up did not have chrony installed. Chrony and ntpd were both enabled, and ntpd was getting exited on reboot. Because the host monitor issues "ntpq -np", and ntpd was loaded but inactive, it would report a failure to query the server, even though chrony was running. I had no idea that chrony was installed, and thus, the whole problem could've been solved by just disabling/uninstalling ntpd. I spent WAY too many hours to come to such a simple solution. It may be very helpful to someone who doesn't understand network time protocols very well if there was a suggestion to explain potential conflicts between ntpd and chronyd in the documentation, or even to take a second to check which (if any) you already have installed. Maybe it won't be an issue for most people, but for me, assuming that I didn't have chrony already running cost me a bunch of time getting my cluster healthy. I would check, find ntpd dead, see no problems reported on Host Monitor, wonder why the hell ntpd died, kill ntpd, run ntpdate, restart ntpdate, restart scm-agent, and that would "fix" it, but on reboot it would go back to using chrony and exit ntpd, and host monitor would report failure to query ntp service, even though the machine was using chrony and synced just fine all along. I appreciate your help!
... View more