Can somebody tell me what would be the real time cluster configuration? I have setup Hortonworks in my home system however it is standalone in real time project what would be the cluster configuration like how many nodes, Cluster memory & RAM, Node memory & RAM, backup of cluster and all? And while submitting Spark job to YARN how can we decide executors, memory and all those properties? ... View more

Hello, I have trained myself on hadoop, I know how to work with MR,Pig,HIVE,SPARK,SCALA,SQOOp and all however I worked on all these components in my personal system and in singlenode architecture. Now I need to know that how real time LIVE project works? How multi node structure works? If I am trying to process one CSV file then How do I access spark and hive and all which are installed on different nodes? And How do I access those? I need detailed documents if somebody have or any article that anyone is aware of which shows complete steps and process to access different components. I feel helpless as nobody in my group or in my connection works on real time Hadoop ecosystem ... View more

I have gone through below URL to understand how to load data into HIVE using spark in orc format. I understood how to create table in HIVE using spark howvere I have one question that how would spark identify that in which database this table should be created or if I have same table name in two different HIVE DB in which table spark is going to insert values I have gone through below URL: https://hortonworks.com/tutorial/using-hive-with-orc-from-apache-spark/ ... View more