About hemant_dindi

hemant_dindi · ‎10-08-2018

Complete list of properties configured in core-site - fs.s3a.access.key= fs.s3a.secret.key= fs.s3a.endpoint= fs.s3a.path.style.access=false fs.s3a.attempts.maximum=20 fs.s3a.connection.establish.timeout=5000 fs.s3a.connection.timeout=200000 fs.s3a.paging.maximum=5000 fs.s3a.threads.max=10 fs.s3a.socket.send.buffer=8192 fs.s3a.socket.recv.buffer=8192 fs.s3a.threads.keepalivetime=60 fs.s3a.max.total.tasks=5 fs.s3a.multipart.size=100M fs.s3a.multipart.threshold=2147483647 fs.s3a.multiobjectdelete.enable=true fs.s3a.buffer.dir=/tmp/s3a fs.s3a.block.size=64M fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem fs.AbstractFileSystem.s3a.impl=org.apache.hadoop.fs.s3a.S3A fs.s3a.readahead.range=64K fs.s3a.etag.checksum=true

hemant_dindi · ‎10-05-2018

Hi All, I am facing the below issue while creating an dir on HDFS with Minio S3 as storage backend - com.amazonaws.AmazonClientException: Unable to verify integrity of data upload. Client calculated content hash (contentMD5: 1B2M2Y8AsgTpgAmY7PhCfg== in base 64) didn't match hash (etag: null in hex) calculated by Amazon S3. You may need to delete the data stored in Amazon S3. (metadata.contentMD5: null, md5DigestStream: com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream@2d6861a6, bucketName: hadoopsa, key: dindi2/): Unable to verify integrity of data upload. Client calculated content hash (contentMD5: 1B2M2Y8AsgTpgAmY7PhCfg== in base 64) didn't match hash (etag: null in hex) calculated by Amazon S3. You may need to delete the data stored in Amazon S3. Below error while adding file to HDFS root@hemant-insa:~# hdfs dfs -put abc.txt s3a://sample/ put: saving output on abc.txt._COPYING_: com.amazonaws.AmazonClientException: Unable to verify integrity of data upload. Client calculated content hash (contentMD5: 1B2M2Y8AsgTpgAmY7PhCfg== in base 64) didn't match hash (etag: null in hex) calculated by Amazon S3. You may need to delete the data stored in Amazon S3. (metadata.contentMD5: 1B2M2Y8AsgTpgAmY7PhCfg==, md5DigestStream: null, bucketName: sample, key: abc.txt._COPYING_): Unable to verify integrity of data upload. Client calculated content hash (contentMD5: 1B2M2Y8AsgTpgAmY7PhCfg== in base 64) didn't match hash (etag: null in hex) calculated by Amazon S3. You may need to delete the data stored in Amazon S3. (metadata.contentMD5: 1B2M2Y8AsgTpgAmY7PhCfg==, md5DigestStream: null, bucketName: sample, key: abc.txt._COPYING_) I have configured the properties as attached. Any hints/solutions - @Predrag Minovic @Jagatheesh Ramakrishnan @Pardeep @Neeraj Sabharwal @Anshul Sisodia

hemant_dindi · ‎07-26-2017

I had the same issue which was due to mixed case of hostname. I ensured that the hostname is uniform(lower case) and restarted the server. This fixed the issue.

hemant_dindi · ‎07-25-2017

Did you try HDFS Service check? Is it successful?

hemant_dindi · ‎02-16-2017

Can you please share the REST API Command which you are using to fetch the groups?

hemant_dindi · ‎07-04-2016

@Jobin George This is real cool. I have tried to build a similar functionality tool using Flink as my Stream Processor. I call it Applerts :).

hemant_dindi · ‎05-12-2016

Can we add an existing KNOX Service to Ambari?

hemant_dindi · ‎05-12-2016

How to add/manage multiple KNOX Gateways in Ambari?

hemant_dindi · ‎03-18-2016

Thanks @Predrag. It worked when I changed it through Ambari.

hemant_dindi · ‎03-18-2016

I have modified the statement at line 66 in the file oozie-env.sh export OOZIE_CLIENT_OPTS="${OOZIE_CLIENT_OPTS} -Doozie.connection.retry.count=2 to export OOZIE_CLIENT_OPTS="${OOZIE_CLIENT_OPTS} -Doozie.connection.retry.count=2" We are getting the same error. What could be the possible value for OOZIE_CLIENTS_OPTS in HDP 2.4? Thanks.

Online	Offline
Last Visited	‎10-08-2018 06:51 PM

Member Since	‎12-10-2015 02:48 AM
Last Visited	‎10-08-2018 06:51 PM
Posts	32
Kudos received	11

Cloudera Community

Cloudera Community

Re: Ambari:Problem accessing /webhdfs/v1/ats/done ...

Re: How to : HTTP Stream in flink

Re: YARN Capacity Scheduler Queue utilisation grap...

Re: hash issue while using S3 as Storage Backend f...

hash issue while using S3 as Storage Backend for H...

Re: Ambari:Problem accessing /webhdfs/v1/ats/done ...

Re: Ambari:Problem accessing /webhdfs/v1/ats/done ...

Re: Add group in Ranger using REST API

Re: YARN Application Monitoring with NiFi

Re: How to add/manage multiple KNOX Gateways in Am...

How to add/manage multiple KNOX Gateways in Ambari...

Re: Upgrade from HDP 2.3.2 to HDP 2.4 - Oozie serv...

Re: Upgrade from HDP 2.3.2 to HDP 2.4 - Oozie serv...