Member since
07-20-2017
5
Posts
0
Kudos Received
0
Solutions
07-27-2017
07:48 AM
Thank you Viniciu, I am in the holliday now but I will test your advice when I will come back to the office and I will let you now about the results. Thank you again, Aurel
... View more
07-20-2017
11:26 PM
I'am trying to install CDH Express with CM 5.12.0 on ununtu 16.04. I'am getting this error "Installation failed. Failed to receive heartbeat from agent" I take the following log /var/log/clodera-scm-agent/cloudera-scm-agent.log (please suggest another one if this one does not help you) . I know it is a common problem but I think you can realize what is happend from the log What do you think it is wrong? [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO SCM Agent Version: 5.12.0 [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Agent Protocol Version: 4 [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Using Host ID: 694d1118-4c53-4e69-9198-68ac8834a61c [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Using directory: /run/cloudera-scm-agent [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Using supervisor binary path: /usr/lib/cmf/agent/build/env/bin/supervisord [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Neither verify_cert_file nor verify_cert_dir are configured. Not performing validation of server certificates in HTTPS communication. These options can be configured in this agent's config.ini file to enable certificate validation. [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Agent Logging Level: INFO [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO No command line vars [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if you're not using this database type) [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if you're not using this database type) [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Found database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar [20/Jul/2017 16:42:39 +0000] 14191 MainThread agent INFO Agent starting as pid 14191 user root(0) group root(0). [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Created /run/cloudera-scm-agent/cgroups [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/cgroups to 0751 [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found existing subsystem memory at /sys/fs/cgroup/memory [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found existing subsystem blkio at /sys/fs/cgroup/blkio [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found existing subsystem cpu at /sys/fs/cgroup/cpu,cpuacct [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found existing subsystem cpuacct at /sys/fs/cgroup/cpu,cpuacct [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found cgroups subsystem: cpu [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO cgroup pseudofile /sys/fs/cgroup/cpu,cpuacct/cpu.rt_runtime_us does not exist, skipping [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found cgroups subsystem: cpuacct [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found cgroups subsystem: blkio [20/Jul/2017 16:42:41 +0000] 14191 MainThread cgroups INFO Found cgroups subsystem: memory [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Found cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes': 8796093022207, 'default_memory_soft_limit_in_bytes': 8796093022207, 'writable_cgroup_dot_procs': True, 'default_cpu_rt_runtime_us': -1, 'has_cpu': True, 'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct': True, 'has_blkio': True} [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Setting up supervisord event monitor. [20/Jul/2017 16:42:41 +0000] 14191 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] [20/Jul/2017 16:42:41 +0000] 14191 MainThread filesystem_map INFO Using timeout of 2.000000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread filesystem_map INFO Using join timeout of 0.100000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread filesystem_map INFO Using tolerance of 60.000000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] [20/Jul/2017 16:42:41 +0000] 14191 MainThread kt_renewer INFO Agent wide credential cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_0 [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Using metrics_url_timeout_seconds of 30.000000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json [20/Jul/2017 16:42:41 +0000] 14191 MainThread ntp_monitor INFO Using timeout of 2.000000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread dns_names INFO Using timeout of 30.000000 [20/Jul/2017 16:42:41 +0000] 14191 MainThread __init__ INFO Created DNS monitor. [20/Jul/2017 16:42:41 +0000] 14191 MainThread stacks_collection_manager INFO Using max_uncompressed_file_size_bytes: 5242880 [20/Jul/2017 16:42:41 +0000] 14191 MainThread __init__ INFO Importing metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala' , 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games' , 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE' , 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUSTEE_SERVER_HOME': '/usr/lib/keytrustee-server' , 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms' , 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper' , 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce' , 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs' , 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka' , 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Created /run/cloudera-scm-agent/process [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/process to 0751 [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Created /run/cloudera-scm-agent/supervisor [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/supervisor to 0751 [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Created /run/cloudera-scm-agent/flood [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Chowning /run/cloudera-scm-agent/flood to cloudera-scm (122) cloudera-scm (129) [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/flood to 0751 [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Created /run/cloudera-scm-agent/supervisor/include [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751 [20/Jul/2017 16:42:41 +0000] 14191 MainThread agent ERROR Failed to connect to previous supervisor. Traceback (most recent call last): File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor self.configure_supervisor_clients() File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")]) File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize Options.realize(self, *arg, **kw) File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize self.process_config() File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config self.process_config_file(do_usage) File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file self.usage(str(msg)) File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage self.exit(2) SystemExit: 2 [20/Jul/2017 16:42:41 +0000] 14191 Dummy-1 daemonize WARNING Stopping daemon. [20/Jul/2017 16:42:41 +0000] 14191 Dummy-1 agent INFO Stopping agent... [20/Jul/2017 16:42:41 +0000] 14191 Dummy-1 agent INFO No extant cgroups; unmounting any cgroup roots
... View more
Labels:
05-18-2016
12:46 PM
Hi @Pierre Villard , In the link you give me yesterday I choose Ambari automated install guide (is right , isn't it?) . But when I want to install Ambari with wget -nv http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.4.2/updates/2.2.2.0/ambari.list I get ERROR 404: Not Found . Can you help me? Thanks a lot, Aurel
... View more
05-17-2016
10:51 AM
Many thanks @Pierre Villard . I have read more carefully Ambari Automated Install Guide and I have found out I should use wget to get locally the software from Horton works repository. I hope tomorrow to make an pseudo-distributed installation of HDP 2.4. on a I5 machine with 8GB with Ubuntu 14.04 - this is my development machine but now has windows 10. Thank you again, Aurel
... View more
05-17-2016
07:30 AM
Hello, I would like to download HDP - the lastest version 2.7 , for building a datawarehouse on Hive ; use sqoop for data loading from my operational database and use Spark for querying datawarehouse (if you want make some comments on this "architecture") .But what I need most is a link for downloading the HDP. Thanks, Aurel
... View more
Labels: