Member since
07-31-2013
1924
Posts
462
Kudos Received
311
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1969 | 07-09-2019 12:53 AM | |
| 11881 | 06-23-2019 08:37 PM | |
| 9147 | 06-18-2019 11:28 PM | |
| 10136 | 05-23-2019 08:46 PM | |
| 4580 | 05-20-2019 01:14 AM |
11-07-2017
10:25 PM
Can you check if the host you're executing 'pyspark' on has a Spark (1.6) Gateway plus a YARN Gateway role deployed on it? These would translate to valid /etc/hadoop/conf/ and /etc/spark/conf/ directories.
... View more
11-07-2017
09:55 PM
If you're instead using tarball or an unmanaged installation, the command to run the failover controller is: $ hadoop-daemon.sh start zkfc Or for a more interactive style: $ hdfs zkfc
... View more
11-07-2017
09:48 PM
1 Kudo
If you're using Cloudera Manager, you can see the Failover Controller role instances and their states under the HDFS -> Instances tab. If you're managing CDH without Cloudera Manager, then you can check on the NameNode host(s) with the below command: $ sudo service hadoop-hdfs-zkfc status
... View more
11-07-2017
08:54 PM
Is the Failover Controller daemon running on the remaining NameNode? If not, start it up so it may elect its local NameNode into the ACTIVE state
... View more
11-02-2017
04:40 AM
1 Kudo
You can pass an input directory to the ImportTSV tool, where your directory can carry any number of files. It is not limited to a single file unless you pass a single file (instead of a directory) to it.
... View more
11-02-2017
04:39 AM
1 Kudo
You are right that its all just byte sequences to HBase, and that it sorts everything lexicographically. You do not require a separator character when composing your key for HBase to understand them as boundaries (cause it would not serve as one), unless you prefer the extra bytes for better readability or for recovering back the individual data elements from (variable length) keys if that's a use-case. HBase 'sharding' (splitting) can be manually specified at table create time if you are aware of your key pattern and ranges - this is strongly recommended to scale from the beginning. Otherwise, HBase computes key midpoints by analysing them in byte form and splits them based on that, whenever a split size threshold is reached for a region range.
... View more
10-23-2017
10:32 PM
1 Kudo
That's not enitirely right - The block count warning per DataNode stems from the fact that the DataNode's carrying too many replicas (of blocks). The FSCK total block count is a number that does not include the replica multiplier (x3, typically) and is global (across all DNs). Have you looked at the Live DataNodes page on your NameNode UI as indicated in the earlier post? It should have the reported replica count of every DN alive in the cluster.
... View more
10-19-2017
11:39 AM
1 Kudo
You should place that config XML snippet under HDFS - Configuration - 'NameNode Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml' field. For more on what the 'Advanced Configuration Snippet' / 'Safety Valve' fields are: https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_config_snippet.html#xd_583c10bfdbd326ba--43d5fd93-1410993f8c2--7ea7
... View more
10-19-2017
10:53 AM
Unfortunately the JDBC standard's strength ends at its APIs. The connection string, perhaps aptly named, is merely a string and there's no feature-based standards around it - every JDBC provider is uniquely different and using each requires consulting its own documentation.
... View more
10-19-2017
10:26 AM
Have you taken a look at http://www.oracle.com/technetwork/topics/wp-oracle-jdbc-thin-ssl-130128.pdf? The useSSL and requireSSL are keywords used by MySQL JDBC drivers, not by Oracle JDBC. The syntax required for Oracle is detailed in the white-paper above, and following that should help you proceed. In general on the shell, it is better to quote detailed strings with lots of non-alphabetic characters, such as is done in your (d) example. That's what would've most likely worked if you were using MySQL in this instance (given that ; and & are both special in shell).
... View more