Member since
10-06-2015
273
Posts
202
Kudos Received
81
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4044 | 10-11-2017 09:33 PM | |
3566 | 10-11-2017 07:46 PM | |
2571 | 08-04-2017 01:37 PM | |
2214 | 08-03-2017 03:36 PM | |
2242 | 08-03-2017 12:52 PM |
05-25-2017
03:46 PM
1 Kudo
The former. You are expected to write the code to run in the Spark shell. Also, take a look at this link with other good questions/answers regarding the exam: https://community.hortonworks.com/questions/70180/hdpcd-spark-exam.html
... View more
05-25-2017
03:36 PM
Response(s) provided here: https://community.hortonworks.com/questions/104805/hdp-sandbox-26-hivehive-2-select-count-or-select-d-2.html
... View more
05-25-2017
03:36 PM
Response(s) provided here: https://community.hortonworks.com/questions/104805/hdp-sandbox-26-hivehive-2-select-count-or-select-d-2.html
... View more
05-25-2017
03:34 PM
1 Kudo
@Lei Yin I faced a similar issue in the past and solved it by adjusting the Hive settings in the sandbox as below: 1) In Ambari, select "Hive" from the left menu then the "Configs" tab and the "Settings" sub-tab 2) Scroll down to the bottom of the page and modify the "HeveServer2 Heap Size", "Metastore Heap Size" as well as any other flagged items (possibly "Memory for Map Join"). If you hover next to each item, Ambari will make recommendations for the values to set, so feel free to use those by selecting the "set recommended" icon that appears. 3) Save and click "restart affected" services near the top of the page. Try the above and let us know if it works or not. As always, if you find any post here helpful, don't forget to "accept" an answer.
... View more
05-25-2017
02:22 PM
Please take a look at @Matt Clarke's response above on how to extract csv files only. It is the most straight forward way.
... View more
05-24-2017
06:18 PM
You can't. Sqoop can only be used to import from RDBMS to HDFS (and vice versa). It does not work with other file system interfaces.
... View more
05-24-2017
06:04 PM
@Andres Urrego Neither. Just use the "--warehouse-dir" flag with "import-all-tables". The directory you specify does not need to be a Hive warehouse directory. It can be anything and anywhere you specify in HDFS. The reason you're unable to use "--target-dir" is because that option is only available when all the imported data is to be placed in the one particular folder, whereas "import-all-tables" needs to create subfolders for each table. The "--warehouse-dir" flag only indicates the parent folder where you want all the data to go, and "import-all-tables" would be able to create subdirectories for each table brought in. I've assumed with the above that you want to import all tables. However, if you only want to import a few tables then your best bet is to write a (shell/perl/python/etc...) script that runs multiple Sqoop commands, with each one importing a table. Does that clarify things?
... View more
05-24-2017
04:00 PM
1 Kudo
@Andres Urrego "import-all-tables" does not support "--target-dir". As you've discovered, "--warehouse-dir" should be used instead. Data for each table will be put in a subfolder in the designated warehouse-dir path. As always, if you find this post helpful, don't forget to "accept" answer.
... View more
05-24-2017
12:58 PM
1 Kudo
@Tinkle Mahendru Take a look at the example Nifi workflow template in the link below (SplitRouteMerge.xml): https://cwiki.apache.org/confluence/download/attachments/57904847/SplitRouteMerge.xml?version=1&modificationDate=1441745127000&api=v2 This flow demonstrates splitting a file(s) on line boundaries, routing the splits based on a regex in the content, and then merging the files together for storage somewhere. It will give you a good idea on how to process and merge your files.
... View more
05-24-2017
12:33 PM
1 Kudo
@Narasimma varman Use the PutSQL and ExecuteSQL processors. You can read more about them and their usage at the below link. https://nifi.apache.org/docs.html You can configure them as below: Also, to get a better idea of how to chain the processors, take a look at the below article for an example flow that ingests data into a relational database using Nifi. https://www.batchiq.com/database-ingest-with-nifi.html As always, if you find this post helpful, don't forget to "accept" answer
... View more