Member since
10-06-2015
273
Posts
202
Kudos Received
81
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3132 | 10-11-2017 09:33 PM | |
2664 | 10-11-2017 07:46 PM | |
2025 | 08-04-2017 01:37 PM | |
1788 | 08-03-2017 03:36 PM | |
1679 | 08-03-2017 12:52 PM |
06-09-2017
07:40 PM
1 Kudo
@Muhammad Umar PutHiveQL is the correct processor to use. However, the SQL/HiveQL has to be formatted and passed to it by the previous processor. For example, you can use the ReplaceText processor to build a HiveQL statement (e.g. INSERT INTO nifitest ...), and then pass the output to the PutHiveQL processor. The above is the correct way to do it. However, there is also a workaround; by using the SelectHiveQL processor, you can write an insert statement as below. Take a look at the below two links that will clarify everything with examples: https://community.hortonworks.com/questions/52328/how-to-insert-data-into-hive-using-nifi.html https://community.hortonworks.com/articles/45706/using-the-new-hiveql-processors-in-apache-nifi-070.html
... View more
06-06-2017
09:41 PM
Adding to the above: Workflow Manager has become GA with Ambari 2.5+. Prior to that it was tech preview in Ambari 2.4.x. If you are using a previous version, either upgrade your Ambari (which may be done independantly of HDP) or use Oozie directly.
... View more
06-06-2017
08:56 PM
@JT Ng Also, take a look at this series of step-by-steps to using Workflow Manager: https://community.hortonworks.com/articles/82964/getting-started-with-apache-ambari-workflow-design.html
... View more
06-06-2017
08:44 PM
1 Kudo
@JT Ng Yes it does. https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-views/content/ch_workflow_designer_view.html Look under Ambari Views
... View more
06-06-2017
01:46 PM
@JT Ng Yes, you can upgrade Ambari independently of HDP. The latest version of Ambari is 2.5.1 so I recommend you move to that. Below is a link to the documentation where you will find more information about it as well as upgrade instructions (just skip the HDP upgrade parts). https://docs.hortonworks.com/HDPDocuments/Ambari/Ambari-2.5.1.0/index.html
... View more
06-06-2017
01:59 AM
@frank chen There's no specific processor for tensorflow. However, you can use the ExecuteStreamCommand to call shell scripts that call TensorFlow binaries. There are other ways to do this, including building a custom processor, but this is the easiest. To get an idea of how this would work, take a look at the below demo by @Timothy Spann that downloads linked images from twitter messages and runs Tensorflow image recognition on them. https://community.hortonworks.com/articles/59349/hdf-20-flow-for-ingesting-real-time-tweets-from-st.html
... View more
06-05-2017
07:42 PM
1 Kudo
@JT Ng Unfortunately, one oozie coordinator can only have one action/workflow. Therefore, you would create a Coordinator for each of your workflows (Sqoop, Hive, Spark, etc...), and then link them with a bundle. A Bundle is a higher-level oozie abstraction that will batch a set of coordinator applications, creating a data pipeline. In bundle.xml you can set multiple coordinator jobs by which multiple workflows can be launched. If you're using Ambari 2.5+, you can use Workflow Manager to do this. Take a look at the link below: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_workflow-management/content/sect_create_coord_bundle.html (Workflow Manager is also available as tech preview in Ambari 2.4.x)
... View more
06-02-2017
04:03 PM
Adding to the response. Take a look at the step-by-step instructions by @S Mahapatra in the link that Sonu provided.
... View more
06-02-2017
03:06 PM
@Phaneendra S Like debugging, this is a broad topic that is more an art than a science. To get you started, below are some links that provide insights on what you can look for. This one is a presentation showcasing the "new" version of Explain and walks through a debugging example: https://www.slideshare.net/HadoopSummit/how-to-understand-and-analyze-apache-hive-query-execution-plan-for-performance-debugging This one explains how Hive Joins would appear in Explains: http://www.openkb.info/2014/11/understanding-hive-joins-in-explain.html As always, if you find this post helpful, don't forget to "accept" answer.
... View more
06-02-2017
01:28 PM
1 Kudo
@Pulak Nath The practice exam on AWS is only meant to give you an idea of the format of the actual test rather than fully prepare you for it. As such, the tasks/questions will not change every time you complete the practice tasks. As always, if you find this post helpful, don't forget to "accept" answer.
... View more