@sanjaysubramani CDP Trial is available at : https://docs.cloudera.com/cdp-private-cloud/latest/release-guide/topics/cdpdc-trial-download-information.html   But there is no older version which is available for trial due to Paywall Changes.  ... View more

Considering the amount of effort required to get Cloudera to work would it not just be easier to install and configure Hadoop on your own? ... View more

@jjjjjjhao,   The bits of errors provided don't tell enough of the story to indicate what may be wrong.   I would run: service cloudera-scm-agent restart  and then see what happens in the agent log.    Also, what is the actual problem?  What is wrong in Cloudera Manager, etc.  It is unclear what you are trying to do or see and what actually happens.  Once that is clarified, the community can help.   Ben ... View more

Check if your ntp is synhronised  ... View more

That query probably has multiple big joins and aggregations and needs more memory to complete. A very rough rule of thumb for minimum memory in releases CDH5.9-CDH5.12 is the following.   For each hash join, the minimum of 150MB or the amount of data on the right side of the node (e.g. if you have a few thousand rows on the right side, maybe a MB or two). For each merge aggregation, the minimum of 300MB or the size of grouped data in-memory (e.g. if you only have a few thousand groups, maybe a MB or two). For each sort, about 50-60MB For each analytic, about 20MB If you add all those up and add another 25% you'll get a ballpark number for how much memory the query will require to execute.   I'm working on reducing those numbers and making the system give a clearer yes/no answer on whether it can run the query before it starts executing. ... View more

Thats awesome thanks a ton Mike. Early this morning before your mails came in - I grew impatient 🙂  as is my nature - and did give it a shot to the Cloudera director as-is-wher-is scripts.   1. Used the Cloud Formation template here     https://s3.amazonaws.com/quickstart-reference/cloudera/latest/templates/Template2-Cloudera-AWS-ExistingVPC.template   2. Created a "ClusterLauncher Instance" on AWS   3. SSH to "ClusterLauncher Instance"     cloudera-director bootstrap cloudera/setup-default/aws.reference.conf  Process logs can be found at /home/ec2-user/.cloudera-director/logs/application.log Plugins will be loaded from /var/lib/cloudera-director-plugins OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0 Cloudera Director 2.1.0 initializing ... The configuration file aws.reference.conf is not present or cannot be read. [ec2-user@ip-10-219-178-74 ~]$ cloudera-director bootstrap cloudera/setup-default/aws.reference.conf Process logs can be found at /home/ec2-user/.cloudera-director/logs/application.log Plugins will be loaded from /var/lib/cloudera-director-plugins OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0 Cloudera Director 2.1.0 initializing ... Installing Cloudera Manager ... * Starting ..... done * Requesting an instance for Cloudera Manager .......................... done * Installing screen package (1/1) ....... done * Running custom bootstrap script on [10.219.177.189, ip-10-219-177-189.us-west-2.compute.internal, 52.43.22.181, ec2-52-43-22-181.us-west-2.compute.amazonaws.com] .......... done * Waiting for SSH access to [10.219.177.189, ip-10-219-177-189.us-west-2.compute.internal, 52.43.22.181, ec2-52-43-22-181.us-west-2.compute.amazonaws.com], default port 22 ..... done * Inspecting capabilities of 10.219.177.189 .......... done * Normalizing a3508870-3de7-4dc0-84a5-f69c77610c89 ..... done * Installing ntp package (1/4) ..... done * Installing curl package (2/4) ..... done * Installing nscd package (3/4) ..... done * Installing gdisk package (4/4) ..................... done * Resizing instance root partition ......... done * Mounting all instance disk drives ............ done * Waiting for new external database servers to start running ........ done * Installing repositories for Cloudera Manager ....... done * Installing oracle-j2sdk1.7 package (1/3) ..... done * Installing cloudera-manager-daemons package (2/3) ..... done * Installing cloudera-manager-server package (3/3) ...... done * Setting up embedded PostgreSQL database for Cloudera Manager ...... done * Installing cloudera-manager-server-db-2 package (1/1) ..... done * Starting embedded PostgreSQL database ...... done * Starting Cloudera Manager server ... done * Waiting for Cloudera Manager server to start ..... done * Setting Cloudera Manager License ... done * Enabling Enterprise Trial ... done * Configuring Cloudera Manager ..... done * Deploying Cloudera Manager agent ...... done * Waiting for Cloudera Manager to deploy agent on 10.219.177.189 ... done * Setting up Cloudera Management Services ............ done * Backing up Cloudera Manager Server configuration ...... done * Inspecting capabilities of 10.219.177.189 ...... done * Done ... Cloudera Manager ready. Creating cluster C5-Reference-AWS ... * Starting ..... done * Requesting 11 instance(s) in 3 group(s) ....................................... done * Preparing instances in parallel (20 at a time) .............................................................. done * Waiting for Cloudera Manager installation to complete ... done * Installing Cloudera Manager agents on all instances in parallel (20 at a time) ........ done * Waiting for new external database servers to start running ... done * Creating CDH5 cluster using the new instances ... done * Creating cluster: C5-Reference-AWS .... done * Downloading parcels: CDH-5.7.2-1.cdh5.7.2.p0.18,KAFKA-2.0.2-1.2.0.2.p0.5 ... done * Distributing parcels: KAFKA-2.0.2-1.2.0.2.p0.5,CDH-5.7.2-1.cdh5.7.2.p0.18 ... done * Activating parcels: KAFKA-2.0.2-1.2.0.2.p0.5,CDH-5.7.2-1.cdh5.7.2.p0.18 ...... done * Configuring Hive to use Sentry ... done * Creating Sentry Database ... done * Calling firstRun on cluster C5-Reference-AWS ... done * Waiting for firstRun on cluster C5-Reference-AWS .... done * Running cluster post creation scripts ...... done * Adjusting health thresholds to take into account optional instances. ... done * Done ...   ... View more