Member since
07-05-2017
19
Posts
0
Kudos Received
0
Solutions
07-31-2017
07:06 AM
Hi, dynamic allocation is enabled and the value of max dynamic allocation executors is set to 10. My yarn container min. size is 4GB, executor size is 10 GB, overhead memory is 384MB, so the containers are launching with 12GB. We are ok with 12 GB container size and also we are clear on the memory utilization. But, we failed to understand how spark is launching more containers than the no. of executors. I think for driver also it uses AM container rather than launching separate container for the driver process.
... View more
07-20-2017
06:54 PM
decrypt operation needs POST instead of GET. Check the samples form KMS doc below. Decrypt Encrypted Key REQUEST: POST http://HOST:PORT/kms/v1/keyversion/<version-name>/_eek?ee_op=decrypt
Content-Type: application/json
{
"name" : "<key-name>",
"iv" : "<iv>", //base64
"material" : "<material>", //base64
}
RESPONSE: 200 OK
Content-Type: application/json
{
"name" : "EK",
"material" : "<material>", //base64
}
... View more
07-20-2017
10:38 PM
Hello @mravipati, In order to decrypt an EEK, first you'll need to generate one. Here's what is working for me: 1. Generate EEK # curl --negotiate -u : -iv -X GET "http://<kms-host-fqdn>:9292/kms/v1/key/hr1-ez-key/_eek?eek_op=generate&num_keys=1" 2. Store the EEK data into a json: # cat eek.json
{
"name" : "hr1-ez-key",
"iv" : "38Eb8nTxxxxxUG7Rc6_wrA",
"material" : "K8Q0bv8Hxxxxxxn9ByVnT-xxx-Bmwxxxxxy1buCk"
} 3. Call decrypt EEK while passing above EEK data: # curl --negotiate -u : -iv -X POST -H "Content-Type: application/json" -d @./eek.json "http://<kms-host-fqdn>:9292/kms/v1/keyversion/hr1-ez-key@0/_eek?eek_op=decrypt" Please note the change in REST URL and keyname while making decrypt request. The error you are getting is because you are trying to do a GET request on a URL which is expecting a POST request. Hope this helps!
... View more
07-17-2017
01:54 PM
Those tables have referential integrity and should not be able to get out-of-sync. Are you using MySQL as your Ambari database? If so, is your default engine InnoDB or MyISAM? If it's MyISAM, that's very bad as it doesn't support transactions or foreign keys. You would need perform several steps to convert your database into InnoDB or this could happen again in the future.
... View more