Member since
09-24-2015
47
Posts
21
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
14889 | 06-07-2017 09:09 PM | |
1055 | 03-28-2017 04:46 PM | |
1161 | 12-08-2016 10:33 PM | |
1177 | 11-15-2016 05:41 PM | |
3444 | 09-23-2016 04:26 PM |
06-07-2017
09:09 PM
1 Kudo
@bigdata.neophyte Here are a few answers for you: NiFi can be interacted with via the UI as well as its REST API. The API is documented at https://nifi.apache.org/docs/nifi-docs/rest-api/index.html. NiFi is primarily a data flow tool whereas Kafka is a broker for a pub/sub type of use pattern. Kafka is frequently used as the backing mechanism for NiFi flows in a pub/sub architecture, so while they work well together they provide two different functions in a given solution. NiFi does have a visual command and control mechanism, while Kafka does not have a native command and control GUI Apache Atlas, Kafka, and NiFi all can work together to provide a comprehensive lineage / governance solution. There is a high level architecture slide at https://hortonworks.com/apache/atlas/#section_2 as well as a tutorial that might help this make more sense at https://hortonworks.com/hadoop-tutorial/cross-component-lineage-apache-atlas/. Data prioritization, back pressure, and balancing latency and throughput are all within NiFi's many strong points and these can be leveraged easily. Kafka does really not provide data prioritization. Security aspects of both Kafka and NiFi are tightly integrated with Apache Ranger, take a look at https://hortonworks.com/apache/ranger/ for additional details. Hope this helps, and please accept the answer if this was helpful.
... View more
05-31-2017
08:38 PM
1 Kudo
@Saikrishna Tarapareddy Yes, you should be able to. Take a look at this HCC article and see if it helps: https://community.hortonworks.com/articles/98394/accessing-data-from-osi-softs-pi-system.html.
... View more
05-24-2017
09:45 PM
@Vishal Prakash Shah See if this blog post helps - https://sharebigdata.wordpress.com/2016/06/12/hive-metastore-internal-tables/. Keep in mind that these are not HiveQL queries but rather queries to the underlying database.
... View more
05-24-2017
09:33 PM
@MB If you're getting a DNS error, that needs to be resolved either by configuring DNS for the hosts or by manually adding the host info to /etc/hosts on each node before you retry the cluster installation. Same goes for the repositories if you're using local repos.
... View more
03-28-2017
05:00 PM
@Jonathan T If you have the VM up and running properly now, there are several tutorials at https://hortonworks.com/tutorials/. They cover all kinds of scenarios from data ingest to processing and visualization, so they are very helpful for learning Hadoop basics.
... View more
03-28-2017
04:55 PM
One additional data point - although Safari does see the file and downloads it, when you open the CSV file (it's only 15 bytes) it just has a "404 not found" and no data in it.
... View more
03-28-2017
04:46 PM
2 Kudos
@Anishkumar Valsalam There is a good tutorial with sample flow templates available at https://hortonworks.com/hadoop-tutorial/learning-ropes-apache-nifi/. There are also several workflow templates available at https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates that can be leveraged as a base for testing and building your own flows.
... View more
03-15-2017
08:53 PM
@ Srinivas Santhanam Not sure if this will help, but have you tried using the --files option to pass the Python script? See the answer here for more details: https://community.hortonworks.com/comments/41935/view.html.
... View more
03-02-2017
10:51 PM
@Sachin Ambardekar There is documentation at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_cluster-planning/content/ch_hardware-recommendations_chapter.html that discusses overall cluster planning. Things like memory sizing, configurations for different types of nodes (masters vs. workers), and other hardware considerations are detailed at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_cluster-planning/content/server-node.1.html.
... View more
12-08-2016
10:33 PM
@justlearning There are a handful of documents and examples to get you started using Oozie, here are a few: Hortonworks Oozie documentation - http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_data-movement-and-integration/content/ch_data_movement_using_oozie.html Apache Oozie documentation - http://oozie.apache.org/docs/4.2.0/ Oozie Quick Start - http://oozie.apache.org/docs/4.2.0/DG_QuickStart.html Oozie examples - http://oozie.apache.org/docs/4.2.0/DG_Examples.html
... View more