Created 01-23-2019 11:13 AM
Created 01-23-2019 07:11 PM
In a nutshell, NO Schema Analytics Manager is part of HDF and doesn't need HDP to operationalize. I have a 3 node HDF cluster running SAM its drag and drop future akin of NiFI UI helps to build a working streaming application without writing a single line of code.
It allows you to add/connect to any remote HDF/HDP cluster by adding the Ambari URL
See attached screenshot
Created 01-24-2019 07:21 AM
Thanks for the quick reply @Geoffrey Shelton Okot its extremely kind of you !!
Please have a look at this issue I am having:-
https://community.hortonworks.com/questions/233052/sam-cannot-store-and-run-any-test-case-gives-nosu...
Created 01-24-2019 02:27 PM
I have seen your other post too how can I reproduce your case? As reiterated I set up a 3 node cluster with SAM and if I could reproduce your use case, basically there are 3 ways to install HDF see attached screenshot
Please share your walk through
Created 01-24-2019 02:58 PM
Thanks @Geoffrey Shelton Okot for responding it was extremely kind of you
I have setup the HDF cluster on a single machine, with all the services installed on the same host. No HDP installed.
I created a single Kafka source and sink in SAM.
This is my schema
{
"namespace": "hdf.heap.com",
"type": "record",
"name": "PatientField",
"fields": [
{
"name": "Patient_name",
"type": "string"
}
]
}
and this is my test data
{
"Patient_name": "john"
}
I get an exception when I try to run my application (NoSuchMethodException)
When try to run my test case it gives me a storageException
SAM is not allowing me to even delete the test cases whilst giving the same exception !!
I am using a console producer to talk to the kafka source(consumer) inside SAM but that is also not showing any activity.
Any help is much appreciated !!!
Created 01-26-2019 03:04 PM
@Geoffrey Shelton OkotHow many kafka brokers are required for a kafka source and sink in the dataflow that i created ??
Created on 01-28-2019 07:19 PM - edited 08-17-2019 02:54 PM
All depends on the resilence your application is built to meet. The basic config is one broker. As a rule of thumb just like for the zookeeper ensemble to avoid the split brain decision you will need at least 3 brokers A three(3) node {zk or Kafka broker} cluster can survive the loss of 1 node see attached screenshot.
I have set up a 3 node HDF cluster (3 zookeeper & 3 brokers ) using virtual box each with 6 GB of RAM for test purposes.