Re: CDH parcel repo URL

buntu · ‎05-02-2017

Thanks, this is very useful. How would one go about getting the application name? Is it the app name or the app ID or something else? Thanks!

buntu · ‎05-01-2017

Thanks for the update. > Are you looking for support on a particular environment? Yes, looking for Debian (Wheezy or Jessie) support.

buntu · ‎04-28-2017

Trying to understand the requirements for CDSW which seems to be only available on RHEL/CentOS: https://www.cloudera.com/documentation/data-science-workbench/latest/topics/cdsw_requirements_supported_versions.html#operating_system_req But docker containers are support across other linux distributions. Are there plans to make it available on other distributions?

buntu · ‎04-12-2017

I got 5 clusters managed by CM and everything is on the latest CDH 5.10. In order to evaluate the Java upgrade, I would like to test the upgrade on a test cluster and minimize the issues when upgrading the prod clusters.

buntu · ‎04-12-2017

Thanks for the response. Since CM host manages the clusters, I wanted to check if the hosts for one of the cluster (running the services) be running say Java 8 while CM host still be on Java 7?

buntu · ‎04-12-2017

Is it possible to have CM host running on a different version of Java compared to the Java version all the hosts of a cluster? Thanks!

buntu · ‎04-04-2017

Can anyone weigh in on the possibility of running the cluster hosts, except the cloudera manager host, on JDK 1.8? Thanks!

buntu · ‎03-17-2017

I run into this error when I try to start the NodeManager: ~~~~ Error starting NodeManager org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: Disallowed NodeManager from <host>, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:215) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:288) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:522) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:568) Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: Disallowed NodeManager from <host>, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:283) at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:209) ~~~~~~ Any idea how to overcome this error and start the NodeManagers? Thanks!

buntu · ‎03-17-2017

While attempting to apply a host template, I keep running into this error/notification: Host must have a single version of CDH installed I didn't notice multiple cm agents running on the host, is there anything else I need to check? Thanks!

buntu · ‎03-16-2017

CDH and Cloudera Manager requirements are mentioned here: https://www.cloudera.com/documentation/enterprise/release-notes/topics/rn_consolidated_pcm.html "... Cloudera does not support mixed environments, all nodes in your cluster must be running the same major JDK version." If I want to run the hosts of the cluster on JDK 8, can I still have CM running on JDK 7? Thanks!

buntu · ‎03-12-2017

I currently have my CDH 5.9 cluster currently on JDK 1.7 and wanted to know if its safe to upgrade the hosts to JDK 1.8. Also, is it possible to have the Cloudera manager to be on JDK 1.7 while the cluster to be upgraded to JDK 1.8. Thanks!

buntu · ‎02-02-2017

How does one go about deploying a Spark streaming job on a CDH cluster? Does the job need to be deployed as a CSD parcel? Thanks!

buntu · ‎02-01-2017

Ok, I do notice the CDH 5.10 parcel and requires Cloudera Manager to be updated before updating CDH parcel.

buntu · ‎02-01-2017

Is this still the right way to configure the repo URL and get the latest CDH when available: http://archive.cloudera.com/cdh5/parcels/{latest_supported}/ I don't seem to find CDH 5.10 via Cloudera Manager. If I look at the 'latest' directory I notice the CDH version is 5.3.10: http://archive.cloudera.com/cdh5/parcels/latest/ Please let me know if the repo URL needs to be updated. Thanks!

buntu · ‎01-16-2017

Is there a R package/parcel that is available to be installed across the CDH cluster? Thanks!

buntu · ‎01-12-2017

Given the size of the dataset, I believe the data fits in memory and its not providing any additional performance improvement. Thanks!

buntu · ‎01-11-2017

I got a table of size ~1gb and I tried to setup hdfs caching as described in this 'Using HDFS caching with Impala" doc, with replication factor of 2 I noticed that without hdfs caching the queries seem to be performing better. I'm using CDH 5.8.2. Is there anything I might be missing or can check why that is the case? Thanks!

buntu · ‎10-31-2016

Thanks for the tutorial. Are there any workflow examples of how to handle the update and delete transactions and how to replicate it into HBase/Phoenix?

buntu · ‎02-22-2016

Thanks srowen! I did notice that I'm able to import 'spark.graphx._' but in the known issues for CDH 5.5 its marked as not supported so was checking: http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_spark_ki.html "GraphX not supported Cloudera does not support GraphX." It probably means its not supported in these forums 🙂

buntu · ‎02-22-2016

Is Graphx avaiable in the latest CDH distro? How to check if a specific pkg or lib is available in CDH in general? Thanks!

buntu · ‎09-16-2015

I got the oozie job to run after adding all the args under Parameters. thanks!

buntu · ‎09-16-2015

Thanks Harsha! I'm using Oozie editor in Hue 3.6 and do not see any option to add 'args'. I attempted to use 'parameters' but upon submitting got an exception regarding invalid xml. Please let me know how to add the args.

buntu · ‎09-16-2015

I've configured an Oozie workflow to run this Sqoop job: <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>my-node:8032</job-tracker> <name-node>hdfs://my-node:8020</name-node> <command>import --connect "jdbc:mysql://<url>" --username "dummy" --password "pwd" --query "SELECT col1, col2, CAST(REPLACE(SUBSTR(datecol,1,7), '-', '') AS UNSIGNED) as dateyyymm FROM src_table WHERE \$CONDITIONS" --target-dir /user/hive/warehouse/hive_table --fetch-size 0 --hive-import --hive-drop-import-delims --hive-table hive_table -m1</command> <file>/tmp/hive-site.xml#hive-site.xml</file> </sqoop> The Sqoop command works fine if ran from command prompt, but I keep running into these errors using Oo: [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing arguments for import: [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: col1 [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: col2 [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: CAST(REPLACE(SUBSTR(datecol,1,7), [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: '-', [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: '') [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: AS [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: UNSIGNED) [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: as [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: dateyyymm .. ...

buntu · ‎06-19-2015

Thanks for the info. I did the 'Deploy client configuration' but still do not see the hive-site.xml updated with the configuration I provided in the safety valve field via CM. Please let me know if there is any other way to verify the deployment. Thansk!

buntu · ‎06-19-2015

I'm facing same error as well when I add this configuration to Hue's hue_safety_valve.ini via CM: ~~~ [hbase] hbase_clusters=(Cluster|some-hbase-thrift-server:9090) ~~~~ * I've verified that the ' Enable HBase Thrift Server Framed Transport' is unchecked * thrift_transport is set to 'buffered' in Hue Is there a way to download the hbase-site.xml via CM? Please let me know if there is any other config to update. Thanks!

buntu · ‎06-19-2015

I've updated the configuration for Hive using CM under " Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml" providing this configuration: <property> <name>hbase.zookeeper.quorum</name> <value>some.zookeeper.node</ value> </property> After restarting the Hive service, I don't see this config in the /etc/hive/conf.cloudera.hive/hive-site.xml config file. It seems to already have this property set to some other value. Does adding a safety value with an existing property overwrite or append the value to the hive-site.xml? Thanks!

buntu · ‎04-20-2015

I need to process nested JSON. How did you go about mapping the fields to the nested AVRO schema?

buntu · ‎03-12-2015

Sorry, was a copy/paste error. I did have a comma between pagename and year but got the error I pasted. Thanks!

buntu · ‎03-12-2015

I got a 'log' table which is currently partitioned by year, month and day. I'm looking to create a partitioned view on top of 'log' table but running into this error: ~~~~ hive> CREATE VIEW log_view PARTITIONED ON (pagename,year,month,day) AS SELECT pagename year,month,day,uid,properties FROM log; FAILED: SemanticException [Error 10093]: Rightmost columns in view output do not match PARTITIONED ON clause ~~ Whats the right way to create a partitioned views? I'm using Hive 0.13 in CDH 5.3.2. Thanks!

buntu · ‎01-15-2015

Thanks Joey, looks like "/usr/bin/flume-ng agent ... -Duser.home=/user/xyz -c /path/to/jar1:/path/to/jar2" seem to have done the trick by prepending to the java.class.path.

Online	Offline
Last Visited	‎10-18-2018 01:40 AM

Date Registered	‎07-21-2014 02:20 PM
Last Visited	‎10-18-2018 01:40 AM
Total Messages Posted	141
Total Kudos Received	8

Re: CDH parcel repo URL

Re: NPE if kafka has null record key

Re: Flume metrics

Re: Google Vision & Apache NiFi - Making Advanced ...

Re: CDSW requirements

CDSW requirements

Re: Mixed versions of Java

Re: Mixed versions of Java

Mixed versions of Java

Re: CDH with JDK 1.8

Yarn NodeManager fails to start

Apply host template - Host must have a single vers...

Re: CDH with JDK 1.8

CDH with JDK 1.8

Running a spark streaming job in production

Re: CDH parcel repo URL

CDH parcel repo URL

Installing R package on CDH cluster

Re: Impala performance with HDFS caching enabled

Impala performance with HDFS caching enabled

Re: Incrementally Streaming RDBMS Data to Your Had...

Re: Graphx in latest CDH

Graphx in latest CDH

Re: Sqoop fails with "Error parsing arguments for ...

Re: Sqoop fails with "Error parsing arguments for ...

Sqoop fails with "Error parsing arguments for impo...

Re: Custom configuration using safety valve for Hi...

Re: Hue hbase Api Error: TSocket read 0 bytes

Custom configuration using safety valve for Hive

Re: JSON to Avro, Sub-records in Avro

Re: How to create partitioned view

How to create partitioned view

Re: NPE if kafka has null record key