Member since
03-23-2015
1288
Posts
114
Kudos Received
98
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3297 | 06-11-2020 02:45 PM | |
5016 | 05-01-2020 12:23 AM | |
2815 | 04-21-2020 03:38 PM | |
2620 | 04-14-2020 12:26 AM | |
2322 | 02-27-2020 05:51 PM |
01-02-2020
03:53 PM
1 Kudo
Yes looks like PIVOT would definitely work but that ticket appears to be 4 years old so I won't hold my breath on it 😉 In the mean time, I'll continue to use HIVE which provides this functionality with a simple dictionary like syntax (not quite PIVOT, but works and is intuitive for python programmers) `select name, props["age"], props["fav_animal"] from table;`
... View more
12-29-2019
08:41 PM
@Cl0ck Each host's name is stored in CM's backend database with an UUID attached, please refer to table HOSTS. Example as below: HOST_ID: 12
OPTIMISTIC_LOCK_VERSION: 148
HOST_IDENTIFIER: bfaf4b71-01e2-4157-b46f-d1c13566b69a
NAME: host-xxx-xxx.xxx
IP_ADDRESS: xx.xx.xx.xx
RACK_ID: /default
STATUS: NA
CONFIG_CONTAINER_ID: 1
MAINTENANCE_COUNT: 0
DECOMMISSION_COUNT: 0
CLUSTER_ID: 1
NUM_CORES: 1
TOTAL_PHYS_MEM_BYTES: 1929342976
PUBLIC_NAME: NULL
PUBLIC_IP_ADDRESS: NULL
CLOUD_PROVIDER: NULL Where HOST_IDENTIFIER is the UUID, and is stored under /var/lib/cloudera-scm-agent/uuid on each host. Maybe you can try to update the table here for NAME field and see if that can help? Cheers Eric
... View more
12-13-2019
09:02 AM
Hi @kvinod ,
Here is the snippet from the article:
For CM 5.13.x and below, use the following process to remove the license manually from the database.
1. Log in to Cloudera Manager and stop the Cloudera Management Services. (Host Monitor, Event server, Alert Publisher, etc)
2. From the Cloudera Manager server shell, Stop Cloudera Manager.
service cloudera-scm-server stop
3. Confirm the Cloudera Managerdatabase instance name and credentials. grep com.cloudera.cmf.db /etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.type=$DBTYPE com.cloudera.cmf.db.host=$DBHOST:$DBPORT com.cloudera.cmf.db.name=$DBNAME com.cloudera.cmf.db.user=$DBUSER com.cloudera.cmf.db.password=$DBPASS
For the remainder of this process the $DB variables will be used, please replace with your configured values. This document will cover $DBTYPE of Postgres or MySQL. $DBPORT may not exist in the properties file if using the default port.
4.Backup the database instance. IMPORTANT: Ensure to create the database backup for this step.
For MySQL: # mysqldump -h $DBHOST -P $DBPORT -u $DBUSER -p $DBNAME > /root/cmbak.`date +%Y-%m-%d:%H:%M`.sql
For Postgres: # pg_dump -h $DBHOST -p $DBPORT -U $DBUSER -W $DBNAME > /root/cmbak.`date +%Y-%m-%d:%H:%M`.sql
5. Log in to the database For MySQL: # mysql -h $DBHOST -P $DBPORT -u $DBUSER -p $DBNAME
For Postgres: # psql -h $DBHOST -p $DBPORT -U $DBUSER -W $DBNAME
Type the $DBPASS when requested.
6. Update the record to remove the license key. update CONFIGS set VALUE = '' where ATTR = 'license_key'; 7. Confirm the license key is removed.
select CONFIG_ID, ATTR, VALUE from CONFIGS where ATTR = 'license_key'; +-----------+-------------+-------+ | CONFIG_ID | ATTR | VALUE | +-----------+-------------+-------+ | 79 | license_key | | +-----------+-------------+-------+ 1 row in set (0.00 sec)
8. Quit database:
For MySQL: mysql> quit For Postgres: scm> \q
9. Start Cloudera Manager from the server shell: service cloudera-scm-server start
10. Login to Cloudera Manager and start the Cloudera Management Services. (Host Monitor, Event server, Alert Publisher, etc)
11. Restart the cluster services to confirm everything is okay.
Thanks and hope this helps,
Li
... View more
12-12-2019
01:28 PM
Hi @sahilk I worked on this last year, and it turned out at the time, it was not supported, as HS2 and HMS will generate the credentials themselves, please see my blog post using below link: Access S3 in Hive Through hadoop.security.credential.provider.path However, it was based on CDH, looks like you are on HDP, so stories might be different, but might need other Hive guys' input here. Cheers Eric
... View more
12-09-2019
05:11 PM
Thank for the answer.
... View more
12-09-2019
03:04 PM
@arunkumarc Is there any particular reason you need to keep schema between Hive and ORC out of sync? Can you ALTER the Hive table schema to match with ORC data? Spark might be more restrictive on this checking. Cheers Eric
... View more
11-28-2019
09:37 PM
Hi Tim, Thanks for you reply, we ran couple of other queries and don't encounter this issue, we will resolve this as a one of case, appreciate your explanation in this regard.
... View more
11-28-2019
07:30 PM
I ended up creating a new column in new data frame via withColumn and used regex to populate the new column with the trimmed vals thanks
... View more
11-28-2019
05:19 PM
1 Kudo
Hi @parthk, No problems. Impala + Ranger is under construction for CDP release. From what I can see Phase one is done and there are a few more phases to go through. So it is still early stage and I do not have ETA. You probably just have to wait and ask the question again a few months down the track. Cheers Eric
... View more
11-25-2019
07:40 AM
Great @EricL !! Thanks a lot for your advice. I'm going to update my CDH and try again 😉
... View more