Member since
04-22-2014
1218
Posts
341
Kudos Received
157
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
19705 | 03-03-2020 08:12 AM | |
10698 | 02-28-2020 10:43 AM | |
3218 | 12-16-2019 12:59 PM | |
2557 | 11-12-2019 03:28 PM | |
4352 | 11-01-2019 09:01 AM |
08-27-2018
10:45 PM
1 Kudo
@krb, What you provided appears to be an agent log message that indicates an attempt to kinit with the HTTP principal on the host where HTTPFS role runs was not successful. Check on the host where the httpfs role runs and make sure the krb5.conf file is correct. This shoud not impact HDFS as a whole since HTTPFS is a client of HDFS really. Cloudera Manager should merge the HTTP principal automatically, so please run the following to make sure the keytab has the right keys: # klist -kte /run/cloudera-scm-agent/process/1330-hdfs-HTTPFS/httpfs.keytab
... View more
08-27-2018
10:25 PM
1 Kudo
@aasf, I think it has been consollodated on this page: https://www.cloudera.com/content/dam/www/marketing/resources/datasheets/cloudera-enterprise-datasheet.pdf.landing.html#cmfeig_topic_5_1 See page 2 of the embedded doc. If you have any other questions, we are happy to assist.
... View more
08-27-2018
05:31 PM
@DanielWhite, I got an email showing your update, but for some reason I don't see it here. What I did notice was that the stack trace says: ERROR /path/filename org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1508298398-10.9.129.86-1406065203774:blk_2079737512_1100628731148 file=filename at org.apache.hadoop.hdfs.DFSInputStream.refetchLocations( The inablity by a client to retrieve blocks results in the BlockMissingException. This might be a bit misleading. Rather, I'd check to verify that all the DataNodes in the source cluster are accessible during replication and that all the nodes in your destination cluster can connect to DataNodes in the source clusters. Note that I think there may be more information in the BDR job logs in YARN. It could be that there is a firewall or something else preventing mappers from retrieving blocks. Does the same problem happen every time, somtimes, etc.?
... View more
08-27-2018
05:19 PM
@sim6, The DNSTest class does simple tests like this to get your "hostname": InetAddress addr = InetAddress.getLocalHost(); String hostname = addr.getHostName(); I believe that Java calls 'uname' under the hood to get the host name. I believe that 'uname' gets the hostname from the proc location: /proc/sys/kernel/hostname Anyway, try checking /etc/sysconfig/network /etc/hosts Make sure the FQDN is specified Check "uname -a" Check "cat /proc/sys/kernel/hostname"
... View more
08-27-2018
04:08 PM
@DanielWhite, I don't quite follow. Can you show us the errors or messages you are seeing and some log context around them?
... View more
08-27-2018
01:28 PM
@DanielWhite, files are split up into blocks and stored on DataNodes. By default, each block is stored on 3 datanoes (block replication factor of 3). In BDR, the mappers will request blocks (as instructed by the NameNode) from the DataNodes that have them. If no DataNodes contain the blocks, for that file, the file itself cannot be copied. Recommendation: Check to see what files have missing blocks in the source cluster and find address the issue. BDR/distcp copies files, not individual blocks at this time so if one block of a file is missing from the source, the remaining blocks are not copied.
... View more
08-27-2018
01:24 PM
@pollard, Sorry, don't have enough time to dig too deep on this. I did use tcpdump and Wireshark to see this is how the ids are sent: csrfmiddlewaretoken=NlsLy4K7LBHIiSHsalA8isMiQtIDk4FQ&is_embeddable=true&user_ids=17&user_ids=18&user_ids=12 Passwords cannot be downloaded from CM. The CM API only allows deletion of a CM user (stored in CM's database). Hue Users are stored in the Hue database. The curl example is for CM only. The other example (using Hue command line) is for Hue. Use TLS for your CM API connections so the plain text password submitted is encrypted over the wire. Same when accessing Hue UI... TLS. There is no way to encrypt the password before sending at this time.
... View more
08-27-2018
10:45 AM
@pollard, I think it is a JavaScript list passed in as parameter to the post. You may be able to use Wireshark to view how the form is submitted on the web page. Internally, we use: ids = request.POST.getlist('user_ids')
... View more
08-24-2018
11:27 PM
@hadoopuser2000, Are you still having trouble? The screen shots indicate you need to uninstall before installing again, so I'd try that. I wasn't clear how the screen shots were related to the problem connecting to CM. First, make sure CM is running; # netstat -nap |grep 7180 If you see that it is listening, make sure that you can connect to the host on which Cloudera Manager is installed from whatever client you are using to connect to Cloudera Manager. By public address, the doc means that you should use whatever hostname will resolve to an IP that lets you connect to that host and port.
... View more
08-24-2018
08:56 PM
@supahcraig, I am glad you were able to resolve the problem. Usually if the upgrade stalls during distribution you can view details about which hosts have successfully downloaded the parcel via the the parcels page. Clicking the progress bar will open a page that shows the progress for each host. Once you find a host or hosts that have errors or are not completing, you can mouse over or click on the bar to view more details including the hostname. Once you have the hostname, you can ssh to that host and review the agent log there (/var/log/cloudera-scm-agent/cloudera-scm-agent.log) The agent actually downloads the parcel file so you can see information about any problems in the agent log. Here is a page that shows more about the parcels: https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cm_ig_parcels.html Cheers!
... View more