I would say go with 256 gb ram. but 128 is def do-able and often seen in the field. Go with Dual 10Gbps NIC. I would go with at least 12-2TB drives broken down into 2 OS drives with raid-1 mirror. 6 drives in raid-10 (metadata) 4 spare drives ... View more

I found the answer to this. ListenSMTP can sit behind LB if the processor will run on all nodes if single DNS is to be exposed. you can also have processor only run on single node (primary). Another option, not tested or fully vetted. have SMTP response payload into nifi rest processor. from there push to process group which will have SMTPListen processor. This will automatically load balance @Artem Ervits I provided this answer after some research. no response from other. please accept if this answer is good. ... View more

Sunile, that post you linked references an outdates processor. Instead, use Consume/PublishJMS, it has been tested to work with WMQ. Though you are referencing MQ Series (this was rebranded to WebSphere MQ years ago), so it it's older than 7.x, no guarantees. ... View more

@Sunile Manjee Yes, you can get the entity GUID by using the entity name in the dsl search query(hive_table where name='<entity_name>'). For a hive table: regional_reporting, query looks like below. curl -u admin:admin 'http://atlas-testing-unsecure-1.openstacklocal:21000/api/atlas/discovery/search/dsl?limit=25&query=hive_table+where+name%3D%22regional_reporting%22' The response for this query will have GUID for that specific entity. { "count": 1, "dataType": { "attributeDefinitions": [ { "dataTypeName": "hive_db", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 1, "upper": 1 }, "name": "db", "reverseAttributeName": null }, { "dataTypeName": "date", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "createTime", "reverseAttributeName": null }, { "dataTypeName": "date", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "lastAccessTime", "reverseAttributeName": null }, { "dataTypeName": "string", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "comment", "reverseAttributeName": null }, { "dataTypeName": "int", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "retention", "reverseAttributeName": null }, { "dataTypeName": "hive_storagedesc", "isComposite": true, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "sd", "reverseAttributeName": null }, { "dataTypeName": "array<hive_column>", "isComposite": true, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "partitionKeys", "reverseAttributeName": null }, { "dataTypeName": "array<string>", "isComposite": true, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "aliases", "reverseAttributeName": null }, { "dataTypeName": "array<hive_column>", "isComposite": true, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "columns", "reverseAttributeName": null }, { "dataTypeName": "map<string,string>", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "parameters", "reverseAttributeName": null }, { "dataTypeName": "string", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "viewOriginalText", "reverseAttributeName": null }, { "dataTypeName": "string", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "viewExpandedText", "reverseAttributeName": null }, { "dataTypeName": "string", "isComposite": false, "isIndexable": false, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "tableType", "reverseAttributeName": null }, { "dataTypeName": "boolean", "isComposite": false, "isIndexable": true, "isUnique": false, "multiplicity": { "isUnique": false, "lower": 0, "upper": 1 }, "name": "temporary", "reverseAttributeName": null } ], "hierarchicalMetaTypeName": "org.apache.atlas.typesystem.types.ClassType", "superTypes": [ "DataSet" ], "typeDescription": null, "typeName": "hive_table" }, "query": "hive_table where name=\"regional_reporting\"", "queryType": "dsl", "requestId": "qtp1678008151-18 - 6a678cdc-fa25-48fd-862a-f560a10481ad", "results": [ { "$id$": { "$typeName$": "hive_table", "id": "9a2edbe6-c736-4189-af9a-4c15dead3f42", "state": "ACTIVE", "version": 0 }, "$typeName$": "hive_table", "aliases": null, "columns": [ { "$id$": { "$typeName$": "hive_column", "id": "2b123293-fad7-4220-9e79-b211d0438b32", "state": "ACTIVE", "version": 0 }, "$typeName$": "hive_column", "comment": null, "description": null, "name": "imp", "owner": "hive", "qualifiedName": "demo.regional_reporting.imp@cl1", "table": { "$typeName$": "hive_table", "id": "9a2edbe6-c736-4189-af9a-4c15dead3f42", "state": "ACTIVE", "version": 0 }, "type": "string" }, { "$id$": { "$typeName$": "hive_column", "id": "8e1e8795-72ab-41ea-8c15-2d9be8dd009a", "state": "ACTIVE", "version": 0 }, "$typeName$": "hive_column", "comment": null, "description": null, "name": "hits", "owner": "hive", "qualifiedName": "demo.regional_reporting.hits@cl1", "table": { "$typeName$": "hive_table", "id": "9a2edbe6-c736-4189-af9a-4c15dead3f42", "state": "ACTIVE", "version": 0 }, "type": "int" } ], "comment": null, "createTime": "2016-10-18T18:49:44.000Z", "db": { "$typeName$": "hive_db", "id": "0194de74-46ba-4ea3-8ebe-28afd1719428", "state": "ACTIVE", "version": 0 }, "description": null, "lastAccessTime": "2016-10-18T18:49:44.000Z", "name": "regional_reporting", "owner": "hive", "parameters": { "COLUMN_STATS_ACCURATE": "{\"BASIC_STATS\":\"true\"}", "numFiles": "0", "numRows": "0", "rawDataSize": "0", "totalSize": "0", "transient_lastDdlTime": "1476816584" }, "partitionKeys": null, "qualifiedName": "demo.regional_reporting@cl1", "retention": 0, "sd": { "$id$": { "$typeName$": "hive_storagedesc", "id": "3b700b63-a8a5-4519-8d0b-52d3de3302fa", "state": "ACTIVE", "version": 0 }, "$typeName$": "hive_storagedesc", "bucketCols": null, "compressed": false, "inputFormat": "org.apache.hadoop.mapred.TextInputFormat", "location": "hdfs://atlas-testing-unsecure-5.openstacklocal:8020/apps/hive/warehouse/demo.db/regional_reporting", "numBuckets": -1, "outputFormat": "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat", "parameters": null, "qualifiedName": "demo.regional_reporting@cl1_storage", "serdeInfo": { "$typeName$": "hive_serde", "name": null, "parameters": { "serialization.format": "1" }, "serializationLib": "org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe" }, "sortCols": null, "storedAsSubDirectories": false, "table": { "$typeName$": "hive_table", "id": "9a2edbe6-c736-4189-af9a-4c15dead3f42", "state": "ACTIVE", "version": 0 } }, "tableType": "MANAGED_TABLE", "temporary": false, "viewExpandedText": null, "viewOriginalText": null } ] } ... View more

@srinivas reddy gaddam Using ambari to kerberos your cluster is very straight forward. Please follow directions on how to (the guide) here: http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.1.0/bk_ambari-security/content/ch_configuring_amb_hdp_for_kerberos.html ... View more

@Ashnee Sharma Take a look this HCC post https://community.hortonworks.com/questions/7165/how-to-copy-hdfs-file-to-aws-s3-bucket-hadoop-dist.html It outlines options to move hdfs data to s3 ... View more