Hi @Sami Ahmad Check out the parameters --target-dir and --warehouse-dir that can be used with the sqoop-import command. There are more details here: https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_controlling_the_import_process_2 ... View more

Hi @Manoj Dixit The natural progression with your background would be to look at technologies such as HDFS and Hive, and the associated governance and security tools like Atlas and Ranger. From there, you can branch out to NoSQL solutions such as HBase and then look at streaming technologies like Hortonworks Data Flow (Apache NiFi). ... View more

Hi @Manoj Dixit I just downloaded a sample TSV file with public data as well, and am able to reproduce your issue when using Hive View 2.0 I tried the same thing with the original hive view (not 2.0), and it does appear to work correctly. - see the screenshot included. Using the older hive view should get you around the issue for now. I'll be submitting this in case it is a bug, and will update if if is not. ... View more

Hi @Geoff Holmes The block report portion of the pre-upgrade steps is, as you've suspected, to help administrations identify/locate/handle corrupt blocks prior to ugprading and also to validate any namespace issues. For manual upgrades, there is a step in the docs to manually compare the before and after block reports to identify any issues. I'm not sure whether Ambari based upgrades do the same check behind the scenes. It is not a step I would skip though https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_command-line-upgrade/content/verify-hdfs-24.html ... View more

Hi @MPH You might be safer configuring the timeline properties in the YARN configs (and ensuring the ttl service is enabled), and letting YARN clean them up from the leveldb vs brute force deletion. You can reduce the time to live, and restart the ATS service to have it kick in and free up the space. See the following links in case they help further: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_yarn-resource-management/content/ref-ca0dc265-903c-418a-9145-2f39fb3f6dd4.1.html https://community.hortonworks.com/questions/46385/yarn-timeline-db-consuming-466gb-space.html the /app-logs can also be managed by the yarn configs for time to retain, I think its: yarn.nodemanager.log.retain-seconds ... View more

Hi @Alvin Jin I haven't seen a special processor to do this yet, but, I've seen a few examples where this can be done using the NiFi Expression Language. These two links should get you going. https://community.hortonworks.com/questions/55610/converting-datetime-to-unixepoch-time-milliseconds.html https://community.hortonworks.com/questions/48905/date-problems-with-convertjsontosql-or-putsql-in-n.html ... View more

hi @John Cleveland If you are using the HDF VirtualBox sandbox, you should be able to boot the machine and follow the Getting To Know the Sandbox guide here: https://hortonworks.com/tutorial/getting-started-with-hdf-sandbox/ I don't recall having to do any setup via scripts, just point my browser to the NiFi UI (http://sandbox.hortonworks.com:19090/nifi) Can you reach the NiFi URL after booting the machine? ... View more