Member since
09-08-2017
53
Posts
2
Kudos Received
0
Solutions
04-18-2019
03:36 PM
@Braz wrote: Is there not a Kudu command which will allow for obtaining table size information? If not, then how does Cloudera Manager perform this? We would like to be able to replicate this behavior so that we can configure e-mail alerts to be sent whenever a table reaches a particular size. Thanks, Braz CM is scrapping and aggregating the /metrics pages from the tablet server instances for each tablet/table. Have you reviewed CM triggers/alerts?[1] You might be able to configure email alerts with a similar trigger rule for table sizes. Alternatively, you could implement what CM currently does by scraping each tablet server's /metrics page and aggregating the data together per tablet/table. [1] https://www.cloudera.com/documentation/enterprise/latest/topics/cm_dg_triggers_usecases.html
... View more
04-18-2019
03:31 PM
Hi Sree, I'm not entirely sure of the syntax you're using for your curl request. The HttpFS role still uses the same API syntax as the WebHDFS API. If you haven't already, I would suggest reviewing the API method for creating and writing to a file via the WebHDFS API[1]. Your command, when executed correctly, should look similar to the following: curl -i -k -X PUT "https://your.host.com:14000/webhdfs/v1/tmp/file.txt?op=CREATE" The HDFS service should then spit back a URL to follow through to actually upload your file to. In the above example, that returned looked like this: HTTP/1.1 307 Temporary Redirect
Date: Thu, 18 Apr 2019 22:23:31 GMT
Cache-Control: no-cache
Expires: Thu, 18 Apr 2019 22:23:31 GMT
Date: Thu, 18 Apr 2019 22:23:31 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
WWW-Authenticate: Negotiate YGwGCSqGSIb3EgECAgIAb10wW6ADAgEFoQMCAQ+iTzBNoAMCARCiRgRE/UEMlOPP/tGaYJaWJMk3AyJMlMu9cHklguw/oEKRQwULVYvOiRhJAaTHo8FL9x3Lqhn/F4XpCscNa8IWg/LCflyYci8=
Set-Cookie: hadoop.auth="u=hdfs&p=hdfs@YOUR.DOMAIN.COM&t=kerberos-dt&e=1555662211059&s=TWQoO1FJ/fNNEHxLTNOyIivzsEieJ46M1oUv+DzB7OY="; Path=/; Secure; HttpOnly
Location: https://your.host.com:14000/webhdfs/v1/tmp/file.txt?op=CREATE&data=true <======*THIS IS THE URL WE WRITE TO*
Content-Type: application/json;charset=utf-8
Content-Length: 0 At that point, we actually submit the payload against the returned path from HDFS: curl -i -k -H "Content-Type:application/octet-stream" -X PUT -T file.txt "https://your.host.com:14000/webhdfs/v1/tmp/file.txt?op=CREATE&data=true" And the data shows up: hdfs dfs -cat /tmp/file.txt
>>this is a file [1] https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Create_and_Write_to_a_File
... View more