rss.livelink.threads-in-node

Hbase Role Log is not generated.

PeterLuo — Tue, 26 Jun 2018 19:12:54 GMT

Hbase Role Log is not generated.

I got errors below when I clicked on Role Log -> Full Log File

There was an error communicating with the server:

[Errno 2] No such file or directory: '/var/log/hbase/hbase-cmf-HBASE-MASTER-ip-10-11-1-105.ec2.internal.log.out'

sudo ls -alF /var/log/hbase

total 4

drwxr-xr-x 3 hbase hbase 20 Jun 26 11:20 ./

drwxr-xr-x. 17 root root 4096 Jun 26 11:20 ../

drwxr-xr-x 2 hbase hbase 6 Jun 26 11:20 stacks/

Could not find yarn-site.xml, make sure to deploy yarn client in UI

PeterLuo — Tue, 26 Jun 2018 18:56:27 GMT

Deploy hbase failed. It looks for the folder /etc/hadoop/conf.cloudera.yarn. But, at the master server. the folder name is conf.cloudera.YARN and conf.cloudera.HDFS. "YARN" and "HDFS" are capital case. See below. How to solve it?

Error Messages:

++ echo 'Copying yarn-site.xml to hbase directory'

++ '[' -r /etc/hadoop/conf.cloudera.yarn/yarn-site.xml ']'

++ echo 'Could not find yarn-site.xml, make sure to deploy yarn client in UI'

Could not find yarn-site.xml, make sure to deploy yarn client in UI

ls -alF /etc/hadoop

total 12

drwxr-xr-x 4 root root 70 Jun 26 11:29 ./

drwxr-xr-x. 101 root root 8192 Jun 26 11:27 ../

lrwxrwxrwx 1 root root 29 Jun 26 11:29 conf -> /etc/alternatives/hadoop-conf/

drwxr-xr-x 2 root root 214 Jun 26 11:29 conf.cloudera.HDFS/

drwxr-xr-x 2 root root 258 Jun 26 11:29 conf.cloudera.YARN/

ls -alF /etc/hadoop/conf.cloudera.YARN

total 48

drwxr-xr-x 2 root root 258 Jun 26 11:29 ./

drwxr-xr-x 4 root root 70 Jun 26 11:29 ../

-rw-r--r-- 1 root root 20 Jun 26 11:29 __cloudera_generation__

-rw-r--r-- 1 root root 66 Jun 26 11:29 __cloudera_metadata__

-rw-r--r-- 1 root root 3870 Jun 26 11:29 core-site.xml

-rw-r--r-- 1 root root 617 Jun 26 11:29 hadoop-env.sh

-rw-r--r-- 1 root root 1772 Jun 26 11:29 hdfs-site.xml

-rw-r--r-- 1 root root 314 Jun 26 11:29 log4j.properties

-rw-r--r-- 1 root root 5107 Jun 26 11:29 mapred-site.xml

-rw-r--r-- 1 root root 315 Jun 26 11:29 ssl-client.xml

-rw-r--r-- 1 root root 630 Jun 26 11:29 topology.map

-rwxr-xr-x 1 root root 1594 Jun 26 11:29 topology.py*

-rw-r--r-- 1 root root 3661 Jun 26 11:29 yarn-site.xml

ls -alF /etc/hadoop/conf.cloudera.HDFS

total 36

drwxr-xr-x 2 root root 214 Jun 26 11:29 ./

drwxr-xr-x 4 root root 70 Jun 26 11:29 ../

-rw-r--r-- 1 root root 20 Jun 26 11:29 __cloudera_generation__

-rw-r--r-- 1 root root 66 Jun 26 11:29 __cloudera_metadata__

-rw-r--r-- 1 root root 3557 Jun 26 11:29 core-site.xml

-rw-r--r-- 1 root root 2696 Jun 26 11:29 hadoop-env.sh

-rw-r--r-- 1 root root 1772 Jun 26 11:29 hdfs-site.xml

-rw-r--r-- 1 root root 314 Jun 26 11:29 log4j.properties

-rw-r--r-- 1 root root 315 Jun 26 11:29 ssl-client.xml

-rw-r--r-- 1 root root 630 Jun 26 11:29 topology.map

-rwxr-xr-x 1 root root 1594 Jun 26 11:29 topology.py*

Limit Query Result Rows from a Hive Table on Hue

rengon — Tue, 26 Jun 2018 18:05:51 GMT

Hi,

how could I limit the rows returned from a query on a Hive table using Hue 4.1.0 without using the LIMIT syntax or the parameters %_partitions_limit.

Failed to install Cloudera Manager Agent on all Nodes

gfrair — Tue, 26 Jun 2018 14:28:06 GMT

Good Day,

Setting up my first 17 node cluster and have gotten to the point of deploying Cloudera Manager Agent (5.15.0) through Cloudera Manager across the nodes in the cluster (running CentOS 7.5). Every nodes reports the following error:

--------------------------------------------------------------

Error: Package: glibc-2.17-222.el7.i686 (CentOS)
Requires: libfreebl3.so
Error: Package: krb5-devel-1.15.1-18.el7.x86_64 (CentOS)
Requires: libkadm5(x86-64) = 1.15.1-18.el7
Installed: libkadm5-1.15.1-19.el7.x86_64 (@updates)
libkadm5(x86-64) = 1.15.1-19.el7
Available: libkadm5-1.15.1-18.el7.x86_64 (CentOS)
libkadm5(x86-64) = 1.15.1-18.el7
Error: Package: glibc-2.17-222.el7.i686 (CentOS)
Requires: libfreebl3.so(NSSRAWHASH_3.12.3)
Error: Package: krb5-devel-1.15.1-18.el7.x86_64 (CentOS)
Requires: krb5-libs(x86-64) = 1.15.1-18.el7
Installed: krb5-libs-1.15.1-19.el7.x86_64 (@updates)
krb5-libs(x86-64) = 1.15.1-19.el7
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest

--------------------------------------------------------------

I spoke to my server admin and was informed that we cannot deploy the version of the components it is requesting on CentOS 7.5, as these are core components that come at this version level with the OS template.

I did open a case and the Cloudera support person is attempting to setup an environment using the same versions, although I thought that maybe someone could offer some additional suggestions or advice as this is a high-priority issue.

Any assistance would be greatly appreciated.

Thanks

Greg

How to automatically sync a Hive external table with a MySQL table without using Sqoop?

sumit2687 — Tue, 26 Jun 2018 13:55:56 GMT

Hi All,

I'm already having a MySQL table in my local machine (Linux) itself, and I have a Hive external table with the same schema as the MySQL table.

I want to sync my hive external table whenever new record is inserted or updated.Batch update is ok with me say hourly.
What is the best possible approach to achieve the same.

Thanks,

Sumit

Kudu start up - ksck: table consistency check errors

razee — Tue, 26 Jun 2018 13:22:37 GMT

Hi everyone,

We use CDH to manage our data using Kudu and Impala. Whenever we restart Kudu, we see very high start up times to the tune of an hour or more. When we run ksck during this time we get the following (summary at the end):

==================
Errors:
==================
table consistency check error: Corruption: 51 out of 74 table(s) are bad

FAILED
Runtime error: ksck discovered errors

All the other tables are either under replicated or unavailable. Is this normal? Any help in making this faster will be appreciated.

Thanks,

Razee

Failed to become active master org.apache.hadoop.security.AccessControlException: Permission denied:

PeterLuo — Mon, 25 Jun 2018 22:20:48 GMT

I cannot start hbase service. I got Permission denied.

Failed to become active master

org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)

at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)

at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:240)

at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:162)

......

Unhandled exception. Starting shutdown.

org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)

at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)

......

Output of ls:

sudo -u hdfs hdfs dfs -ls /user

Found 2 items

drwxrwxrwx - mapred mapred 0 2018-06-25 14:48 /user/history

drwxrwxrwx - mapred mapred 0 2018-06-25 13:59 /user/spark

sudo -u hdfs hdfs dfs -ls /hbase

Found 1 items

drwxr-xr-x - hdfs hbase 0 2018-06-25 15:06 /hbase/data

sudo -u hdfs hdfs dfs -ls /hbase/data/hbase

The /hbase/data/hbase is empty.

Failed to start Cloudera Manager Agent

Siv — Mon, 25 Jun 2018 19:39:05 GMT

Hi, Please help me to address below issue.. after agent installation , while starting cloudera manager agent on second VM machine I am getting following error ..

----------------- logs is as shown below --------------

Starting cloudera-scm-agent: [60G[[0;31mFAILED[0;39m]
END (1)
agent logs:
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/'
>> [--orphan_process_directory_refresh_interval ORPHAN_PROCESS_DIRECTORY_REFRESH_INTERVAL]
>> [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
>> PACKAGE_DIR [--parcel_dir PARCEL_DIR]
>> [--supervisord_path SUPERVISORD_PATH]
>> [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
>> [--standalone STANDALONE] [--master MASTER]
>> [--environment ENVIRONMENT] [--host_id HOST_ID]
>> [--disable_supervisord_events] --hostname HOSTNAME
>> --ip_address IP_ADDRESS
>> [--reported_hostname REPORTED_HOSTNAME] [--use_tls]
>> [--client_key_file CLIENT_KEY_FILE]
>> [--client_cert_file CLIENT_CERT_FILE]
>> [--verify_cert_file VERIFY_CERT_FILE]
>> [--verify_cert_dir VERIFY_CERT_DIR]
>> [--client_keypw_file CLIENT_KEYPW_FILE]
>> [--client_keypw_cmd CLIENT_KEYPW_CMD]
>> [--max_cert_depth MAX_CERT_DEPTH] [--logfile LOGFILE]
>> [--logdir LOGDIR] [--optional_token] [--clear_agent_dir]
>> [--sudo_command SUDO_COMMAND] [--pidfile PIDFILE]
>> [--comm_name COMM_NAME]
>>cmf-agent: error: argument --hostname is required
>>[26/Jun/2018 00:17:43 +0000] 11328 MainThread agent INFO SCM Agent Version: 5.13.0
>>[26/Jun/2018 00:17:43 +0000] 11328 MainThread agent ERROR Could not determine hostname or ip address; proceeding.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.13.0-py2.6.egg/cmf/agent.py", line 2686, in parse_arguments
>> ip_address = socket.gethostbyname(fqdn)
>>gaierror: [Errno -3] Temporary failure in name resolution
>>usage: cmf-agent [-h] [--agent_dir AGENT_DIR] [--daemon] [--lib_dir LIB_DIR]
>> [--orphan_process_directory_staleness_threshold ORPHAN_PROCESS_DIRECTORY_STALENESS_THRESHOLD]
>> [--orphan_process_directory_refresh_interval ORPHAN_PROCESS_DIRECTORY_REFRESH_INTERVAL]
>> [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
>> PACKAGE_DIR [--parcel_dir PARCEL_DIR]
>> [--supervisord_path SUPERVISORD_PATH]
>> [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
>> [--standalone STANDALONE] [--master MASTER]
>> [--environment ENVIRONMENT] [--host_id HOST_ID]
>> [--disable_supervisord_events] --hostname HOSTNAME
>> --ip_address IP_ADDRESS
>> [--reported_hostname REPORTED_HOSTNAME] [--use_tls]
>> [--client_key_file CLIENT_KEY_FILE]
>> [--client_cert_file CLIENT_CERT_FILE]
>> [--verify_cert_file VERIFY_CERT_FILE]
>> [--verify_cert_dir VERIFY_CERT_DIR]
>> [--client_keypw_file CLIENT_KEYPW_FILE]
>> [--client_keypw_cmd CLIENT_KEYPW_CMD]
>> [--max_cert_depth MAX_CERT_DEPTH] [--logfile LOGFILE]
>> [--logdir LOGDIR] [--optional_token] [--clear_agent_dir]
>> [--sudo_command SUDO_COMMAND] [--pidfile PIDFILE]
>> [--comm_name COMM_NAME]
>>cmf-agent: error: argument --hostname is required
>> [--orphan_process_directory_refresh_interval ORPHAN_PROCESS_DIRECTORY_REFRESH_INTERVAL]
>> [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
>> PACKAGE_DIR [--parcel_dir PARCEL_DIR]
>> [--supervisord_path SUPERVISORD_PATH]
>> [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
>> [--standalone STANDALONE] [--master MASTER]
>> [--environment ENVIRONMENT] [--host_id HOST_ID]
>> [--disable_supervisord_events] --hostname HOSTNAME
>> --ip_address IP_ADDRESS
>> [--reported_hostname REPORTED_HOSTNAME] [--use_tls]
>> [--client_key_file CLIENT_KEY_FILE]
>> [--client_cert_file CLIENT_CERT_FILE]
>> [--verify_cert_file VERIFY_CERT_FILE]
>> [--verify_cert_dir VERIFY_CERT_DIR]
>> [--client_keypw_file CLIENT_KEYPW_FILE]
>> [--client_keypw_cmd CLIENT_KEYPW_CMD]
>> [--max_cert_depth MAX_CERT_DEPTH] [--logfile LOGFILE]
>> [--logdir LOGDIR] [--optional_token] [--clear_agent_dir]
>> [--sudo_command SUDO_COMMAND] [--pidfile PIDFILE]
>> [--comm_name COMM_NAME]
>>cmf-agent: error: argument --hostname is required
>>[26/Jun/2018 00:17:43 +0000] 11328 MainThread agent INFO SCM Agent Version: 5.13.0
>>[26/Jun/2018 00:17:43 +0000] 11328 MainThread agent ERROR Could not determine hostname or ip address; proceeding.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.13.0-py2.6.egg/cmf/agent.py", line 2686, in parse_arguments
>> ip_address = socket.gethostbyname(fqdn)
>>gaierror: [Errno -3] Temporary failure in name resolution
>>usage: cmf-agent [-h] [--agent_dir AGENT_DIR] [--daemon] [--lib_dir LIB_DIR]
>> [--orphan_process_directory_staleness_threshold ORPHAN_PROCESS_DIRECTORY_STALENESS_THRESHOLD]
>> [--orphan_process_directory_refresh_interval ORPHAN_PROCESS_DIRECTORY_REFRESH_INTERVAL]
>> [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
>> PACKAGE_DIR [--parcel_dir PARCEL_DIR]
>> [--supervisord_path SUPERVISORD_PATH]
>> [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
>> [--standalone STANDALONE] [--master MASTER]
>> [--environment ENVIRONMENT] [--host_id HOST_ID]
>> [--disable_supervisord_events] --hostname HOSTNAME
>> --ip_address IP_ADDRESS
>> [--reported_hostname REPORTED_HOSTNAME] [--use_tls]
>> [--client_key_file CLIENT_KEY_FILE]
>> [--client_cert_file CLIENT_CERT_FILE]
>> [--verify_cert_file VERIFY_CERT_FILE]
>> [--verify_cert_dir VERIFY_CERT_DIR]
>> [--client_keypw_file CLIENT_KEYPW_FILE]
>> [--client_keypw_cmd CLIENT_KEYPW_CMD]
>> [--max_cert_depth MAX_CERT_DEPTH] [--logfile LOGFILE]
>> [--logdir LOGDIR] [--optional_token] [--clear_agent_dir]
>> [--sudo_command SUDO_COMMAND] [--pidfile PIDFILE]
>> [--comm_name COMM_NAME]
>>cmf-agent: error: argument --hostname is required
END (0)
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/'
tail: cannot open `/var/log/cloudera-scm-agent//cloudera-scm-agent.log' for reading: No such file or directory
tail: cannot open `/var/log/cloudera-scm-agent//cloudera-scm-agent.log' for reading: No such file or directory
END (0)

Summary: Exchange operation taking too much time on a single node Impala

himanshuag — Mon, 25 Jun 2018 17:28:16 GMT

Hi,

Our system has single node impala cluster on top of 3 kudu nodes. As you can see the summary below, Exchange is taking too much time. I want to understand is this normal, or do I need to make any changes in Impala configurations.

Thanks!

Full authentication is required to access this resource

don1123 — Mon, 25 Jun 2018 15:42:15 GMT

I am currently using Unix to access Cloudera manager (SAML protected). I have access to links such as

https://servername.net:1111/static/release/js/cloudera/cmf/

but, I am unable to gain access to links with "/api"

example:

https://servername.net:1111/api/v13/clusters/

Here is my error message that I managed to get.

I am using get requests from python to gain access to Cloudera.

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 401 Full authentication is required to access this resource</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /api/v13/clusters/. Reason:
<pre>    Full authentication is required to access this resource</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>
<br/>
<br/>

Maven Error With Cloudera Repository

bthelm2 — Mon, 25 Jun 2018 14:14:26 GMT

When trying to compile a maven project, I'm receiving the following error:

"(https://repository.cloudera.com/artifactory/cloudera-repos/): Remote host closed connection during handshake: SSL peer shut down incorrectly"

Here is how I have the repository configured:

<repository>
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>

My maven version is 3.0.5

.sanpshot file in kafka log directory

RajeshBodolla — Mon, 25 Jun 2018 13:24:00 GMT

Hi,

What is the use of .sanpshot files in the data.log directory of a kafka topic and how do I make use of it? Is this present in apache kafka as well or available only on cloudera?

spark 1.6 csd for CDH5.5

RajeshBodolla — Mon, 25 Jun 2018 09:25:26 GMT

Hi,

Do we have csd for spark 1.6 supporting on CDH5.5.X? Any pointer are appreciated.

[kafka] Error sending message when leader is not master node

Qing — Mon, 25 Jun 2018 08:04:05 GMT

Hi,

My current setup has 1 management node and 3 worker node.
Server A(master) - kafka broker, ...
Server B - zookeeper, kafka broker, ...
Server C - zookeeper, kafka broker, ...
Server D - zookeeper, kafka broker, ...

KAFKA-3.0.0-1.3.0.0.p0.40
CDH-5.14.2-1.cdh5.14.2.p0.3

I am using a client machine in the same network to create topic, produce and consume message using kafka bat files.

I realise when i create a topic, and the leader of the topic happens to be the broker on server A, everything works.

But if the leader of the topic is any broker on server b/c/d, when i attempt to send any messages, it prompt the following message.
WARN - Got error produce response with correlation id _ on topic-partition ___, retrying. Error: UNKNOWN_TOPIC_OR_PARTITION
WARN - Receive unknown topic or partition error in produce request on partition ___. The topic/partition may not exist or the user may not have Describe access to it
ERROR - Error when sending message to topic ___ with key: null, value... UnknownTopicOrPartitionException: This server does not host this topic-partition.

I am suspecting something to do with the zookeeper, but have no idea where to look deeper into.
I am a beginner in this area, any help would be appreciated.

Thanks in advance!

Cloudera Navigator Error Message

dans_vander — Sun, 24 Jun 2018 04:54:07 GMT

Hello everyone,

I'm using an application to connect to Cloudera Navigator API (v10), scan some databases and collect metrics and I'm receiving the following return error:

Can not map JSON result for API call to relations?entityIds=3125134&types=PARENT_CHILD&roles=PARENT

It looks like it is not finding a JSON clause.

I cheked my configurations and everything is correct.

Could someone please explain to me what this error means? And what can I do to solve this?

P.s: In my application, I received a warning message before this error: "Exception has occurred while scanning database "databaseName". Error message: "null".

Can these two message be related?

Thanks,

Danilo

NULL columns importing csv data into table

uur — Sat, 23 Jun 2018 11:59:31 GMT

Hi everyone,

I'm trying to import a csv file to a table. But after I created the table and load the data into the table some columns (data types except STRING) is getting NULL. Here is the create table statement I used:

CREATE TABLE deneme6 (framenumber int,frametime TIMESTAMP, ipsrc STRING, ipdst STRING, protocol STRING, flag int, windowsize int, info STRING)

ROW FORMAT DELIMITED

FIELDS TERMINATED BY ','

STORED AS TEXTFILE ;

And then I'm loading my data file:

load data inpath 'hdfs:///user/hive/warehouse/deneme4/deneme4.csv' into table deneme6;

But for the columns framenumber, frametime, flag, windowsize data is returning NULL. These are the columns that their data type are not STRING. What can I do for the issue? And here is an example of the csv file:

frame.number,frame.time_relative,ip.src,ip.dst,_ws.col.Protocol,tcp.flags,tcp.window_size_value,_ws.col.Info

1,"0.000000000","147.32.84.165","91.212.135.158","TCP","0x00000002","64240","1040 â†’ 5678 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1"

2,"0.000009000","147.32.84.165","91.212.135.158","TCP","0x00000002","64240","[TCP Out-Of-Order] 1040 â†’ 5678 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1"

3,"0.062970000","91.212.135.158","147.32.84.165","TCP","0x00000012","65535","5678 â†’ 1040 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 SACK_PERM=1"

And this is the result for the table:

NULL NULL "147.32.84.165" "91.212.135.158" "TCP" NULL NULL "1040 → 5678 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1"
NULL NULL "147.32.84.165" "91.212.135.158" "TCP" NULL NULL "[TCP Out-Of-Order] 1040 → 5678 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1"

NULL NULL "91.212.135.158" "147.32.84.165" "TCP" NULL NULL "5678 → 1040 [SYN

Something similar to PL/HQL available on Cloudera?

Nachopa — Sat, 23 Jun 2018 07:23:57 GMT

Hi,

I need to process some data from Hive tables through loops. I have seen that PL/HQL would be perfect for my goal but it is not available on Cloudera. Anybody knows about something similar?

Thanks for your hepl!

Name node not starting after computer restarts

kokokoko — Sat, 23 Jun 2018 01:40:19 GMT

Hello,

My cluster has the following configuration 2 NN , 5 DN , 3 JN.

After restarting the cluster ,none of the name node started .Please see the attached screenshot & logs.

Is there any suggestion to start the name node ? This is DEV enviroment,so if i lost the data i can easily reload it.

Thanks

9:02:01.896 PM

WARN

FSEditLog

Unable to determine input streams from QJM to [192.168.83.30:8485, 192.168.83.28:8485, 192.168.83.5:8485]. Skipping.
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 1 successful responses:
192.168.83.5:8485: [[1292263,1292312], [1292313,1292376], [1292377,1292426], [1292427,1292488], [1292489,1292538], [1292539,1292588], [1292589,1292664], [1292665,1292714], [1292715,1292778], [1292779,1292846], [1292847,1292985], [1292986,1293049], [1293050,1293099], [1293100,1293164], [1293165,1293214], [1293215,1293264], [1293265,1293355], [1293356,1293405], [1293406,1293469], [1293470,1293520], [1293521,1293579], [1293580,1293643], [1293644,1293706], [1293707,1293775], [1293776,1293796]]
2 exceptions thrown:
192.168.83.28:8485: Journal Storage Directory /tmp/jrnldata/HAnameService not formatted
	at org.apache.hadoop.hdfs.qjournal.server.Journal.checkFormatted(Journal.java:472)
	at org.apache.hadoop.hdfs.qjournal.server.Journal.getEditLogManifest(Journal.java:655)

9:02:02.665 PM

FATAL

FSEditLog

Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [192.168.83.30:8485, 192.168.83.28:8485, 192.168.83.5:8485], stream=null))
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 1 successful responses:
192.168.83.5:8485: lastPromisedEpoch: 23
httpPort: 8480
fromURL: "http://ntiger:8480"

2 exceptions thrown:
192.168.83.30:8485: Journal Storage Directory /tmp/jrnldata/HAnameService not formatted
	at org.apache.hadoop.hdfs.qjournal.server.Journal.checkFormatted(Journal.java:472)
	at org.apache.hadoop.hdfs.qjournal.server.Journal.getLastPromisedEpoch(Journal.java:245)

java.lang.ClassNotFoundException: com.splicemachine.hbase.SpliceMasterObserver

PeterLuo — Sat, 23 Jun 2018 00:55:17 GMT

I am using CDH5.12.2. I Cannot start HBASE Master. I Got,

java.lang.ClassNotFoundException: com.splicemachine.hbase.SpliceMasterObserver

At Java Configuration Option for HBASE Master in Cloudera Manager, I configured,

-Dsplice.spark.driver.extraClassPath=/opt/cloudera/parcels/SPLICEMACHINE/lib/*:/opt/cloudera/parcels/CDH/lib/hbase/conf

-Dsplice.spark.executor.extraClassPath=/opt/cloudera/parcels/CDH/lib/hbase/conf:/opt/cloudera/parcels/SPLICEMACHINE/lib/*

-Dsplice.spark.yarn.jars=/opt/cloudera/parcels/SPLICEMACHINE/lib/*

The jars are at

/opt/cloudera/parcels/SPLICEMACHINE/lib/.

But, I got errors:

ERROR org.apache.hadoop.hbase.coprocessor.CoprocessorHost: The coprocessor com.splicemachine.hbase.SpliceMasterObserver threw java.lang.ClassNotFoundException: com.splicemachine.hbase.SpliceMasterObserver

java.lang.ClassNotFoundException: com.splicemachine.hbase.SpliceMasterObserver

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:158)

at org.apache.hadoop.hbase.master.MasterCoprocessorHost.<init>(MasterCoprocessorHost.java:92)

at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:741)

at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:194)

at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1834)

at java.lang.Thread.run(Thread.java:745)

2018-06-22 09:24:15,482 FATAL org.apache.hadoop.hbase.master.HMaster: The coprocessor com.splicemachine.hbase.SpliceMasterObserver threw java.lang.ClassNotFoundException: com.splicemachine.hbase.SpliceMasterObserver

java.lang.ClassNotFoundException: com.splicemachine.hbase.SpliceMasterObserver

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:158)

at org.apache.hadoop.hbase.master.MasterCoprocessorHost.<init>(MasterCoprocessorHost.java:92)

at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:741)

at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:194)

at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1834)

at java.lang.Thread.run(Thread.java:745)

[ANNOUNCE] Cloudera Data Science Workbench 1.4 Released

ameet — Fri, 22 Jun 2018 23:32:15 GMT

Cloudera is delighted to announce the release of Cloudera Data Science Workbench 1.4.0. With this release, Cloudera Data Science Workbench extends the machine learning platform experience from research to production. Data scientists can now build, train, and deploy models in a unified workflow with two new key capabilities: experiments and models.

Experiments: Experiments let data scientists train, compare, and reproduce versioned models. With this feature,, data scientists can run a batch job that will:

create a snapshot of model code, dependencies, and configuration parameters necessary to train the model
build and execute the training run in an isolated container
track model metrics, performance, and any model artifacts the user specifies

Models: Models let data scientists build, deploy, and manage models as REST APIs to serve predictions. With this feature, data scientists can simply select a Python or R function within a project file, and Cloudera Data Science Workbench will:

create a snapshot of model code, saved model parameters, and dependencies
build an immutable executable container with the trained model and serving code
add a REST endpoint that automatically accepts input parameters matching the function signature, and that returns a data structure matching the function’s return type
save the built model container, along with metadata like who built or deployed it
deploy and start a specified number of model API replicas, automatically load balanced
let the user document, test, and share the model

In addition, Cloudera Data Science Workbench 1.4 also includes security enhancements that help automate user administration.

Simplified user administration: Previous CDSW releases offered LDAP and SAML authentication but allowed every user to log in. The consequence was user sprawl and unintended license consumption. Site administrators had to be manually entitled in the Cloudera Data Science Workbench UI.

With version 1.4 you can now designate LDAP and SAML groups for both users and administrators. With automatic synchronisation, the ability to log in or administer CDSW now depends on a user’s group membership. These groups can be assigned in your existing centralised LDAP/SAML authentication system.

We’ve added two new properties to CDSW:

LDAP/SAML User Groups - Groups whose users can log in to CDSW
LDAP/SAML Admin Groups - Groups whose users are automatically made site administrators in CDSW

For a complete list of new features and bug fixes in this release, please see the Release Notes.

For more information on downloading, installing and using Cloudera Data Science Workbench, please see the links below:

As always, we welcome your feedback. Please send your comments and suggestions on our community forums.