Member since
02-02-2016
31
Posts
41
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1822 | 03-03-2016 06:05 AM | |
928 | 03-01-2016 12:30 PM | |
14557 | 02-23-2016 09:19 AM | |
540 | 02-18-2016 09:12 AM | |
6797 | 02-15-2016 09:49 AM |
08-24-2016
03:39 PM
You need to create the user rangerdba with password manager to get the same working. For more information on how to create the user please follow the article: Documentation
... View more
07-27-2016
09:50 AM
To add a new keytab you can use ktpass in AD-Kerberos with the following syntax: ktpass -out <keytabname>.keytab -princ <principal name> -pass <password> -mapuser <user_to_map_in_AD> -mapop set -crypto All -ptype KRB5_NT_PRINCIPAL For reference: https://technet.microsoft.com/en-us/library/cc753771(v=ws.11).aspx
... View more
03-03-2016
11:03 AM
1 Kudo
@glupu Thank you for this information
... View more
03-03-2016
08:24 AM
1 Kudo
Default username is admin and default password is also admin for Ambari UI
... View more
03-03-2016
08:23 AM
2 Kudos
Hi Jzhang, You will have to login as admin in Ambari, then click on Admin > Kerberos. More info: https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_Ambari_Security_Guide/content/_enabling_kerberos_security_in_ambari.html Regards, Karthik Gopal
... View more
03-03-2016
06:14 AM
You can also use the link below for tutorials using sandbox mentioned by @Divakar Annapureddy http://hortonworks.com/tutorials/ Regards, Karthik Gopal
... View more
03-03-2016
06:05 AM
4 Kudos
Hi Mahesh, exec - Use the exec command to run a Pig script with no interaction between the script and the Grunt shell (batch mode). Aliases defined in the script are not available to the shell; however, the files produced as the output of the script and stored on the system are visible after the script is run. Aliases defined via the shell are not available to the script. With the exec command, store statements will not trigger execution; rather, the entire script is parsed before execution starts. Unlike the run command, exec does not change the command history or remembers the handles used inside the script. Exec without any parameters can be used in scripts to force execution up to the point in the script where the exec occurs. run command. Both the exec and run commands are useful for debugging because you can modify a Pig script in an editor and then rerun the script in the Grunt shell without leaving the shell. Also, both commands promote Pig script modularity as they allow you to reuse existing components. run - Use the run command to run a Pig script that can interact with the Grunt shell (interactive mode). The script has access to aliases defined externally via the Grunt shell. The Grunt shell has access to aliases defined within the script. All commands from the script are visible in the command history. With the run command, every store triggers execution. The statements from the script are put into the command history and all the aliases defined in the script can be referenced in subsequent statements after the run command has completed. Issuing a run command on the grunt command line has basically the same effect as typing the statements manually. For comparison, see the exec command. Both the run and exec commands are useful for debugging because you can modify a Pig script in an editor and then rerun the script in the Grunt shell without leaving the shell. Also, both commands promote Pig script modularity as they allow you to reuse existing components. For more: https://pig.apache.org/docs/r0.9.1/cmds.html#exec https://pig.apache.org/docs/r0.9.1/cmds.html#run Regards, Karthik Gopal
... View more
03-03-2016
04:24 AM
3 Kudos
Hi Musheer, You can refer to the link - http://hortonworks.com/training/ Under developer section you will find the course objective which you can use to get started. Regards, Karthik Gopal
... View more
03-01-2016
12:30 PM
1 Kudo
For more information - http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_Installing_HDP_AMB/content/_start_the_ambari_server.html
... View more
03-01-2016
10:08 AM
1 Kudo
@Neeraj Sabharwal I guess there is an article which shows that both Spark and TensorFlow can work together - https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html So wanted to know if there are any recommended ways of installing TensorFlow with HDP
... View more
03-01-2016
06:58 AM
4 Kudos
Here are the installation modes for TensorFlow: https://www.tensorflow.org/versions/r0.7/get_started/os_setup.html Can any of these installations be automated through Cloudbreak? If not, do you already have a recommended way of installing TensorFlow on HDP?
... View more
Labels:
- Labels:
-
Apache Spark
02-23-2016
09:28 AM
1 Kudo
More information: http://stackoverflow.com/questions/19205057/how-to-fix-corrupt-hadoop-hdfs
... View more
02-23-2016
09:19 AM
2 Kudos
Hi Pranshu, You can follow the instructions in the link below: https://community.hortonworks.com/articles/4427/fix-under-replicated-blocks-in-hdfs-manually.html Regards, Karthik Gopal
... View more
02-18-2016
12:46 PM
2 Kudos
You can use the command - hdfs fsck / -delete to list corrupt of missing blocks and then follow the article above to fix the same.
... View more
02-18-2016
09:12 AM
2 Kudos
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_hdfs_admin_tools/content/ch04.html
... View more
02-18-2016
09:12 AM
2 Kudos
Hi Rushikesh, Hadoop jobs are data intensive, compressing data can speed up the I/O operations - MapReduce jobs are almost always I/O bound Compressed data can save storage space and speed up data transfers across the network - Capital allocation for hardware can go further Reduced I/O and network load can bring significant performance improvements - MapReduce jobs can finish faster overall On the other hand, CPU utilization and processing time increases during compression and decompression - Understanding the tradeoff is important for MapReduce pipeline's overall performance
... View more
02-18-2016
08:04 AM
1 Kudo
HBase Snapshots allow you to take a snapshot of a table without much impact on Region Servers. Snapshot, clone, and restore operations don't involve data copying. In addition, exporting a snapshot to another cluster has no impact on region servers.
... View more
02-18-2016
08:02 AM
1 Kudo
Hi Rushikesh, Suggest you to go through this article might be helpful: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_hbase_snapshots_guide/content/ch_hbase_snapshots_chapter.html Regards, Karthik Gopal
... View more
02-15-2016
09:49 AM
2 Kudos
Hi Rushikesh, You can create a daily script and use the following option of hadoop fs command to append the an existing file on HDFS. Usage: hadoop fs -appendToFile <localsrc> ... <dst> Append single src, or multiple srcs from local file system to the destination file system. Also reads input from stdin and appends to destination file system.
hadoop fs -appendToFile localfile /user/hadoop/hadoopfile hadoop fs -appendToFile localfile1 localfile2 /user/hadoop/hadoopfile hadoop fs -appendToFile localfile hdfs://nn.example.com/hadoop/hadoopfile hadoop fs -appendToFile - hdfs://nn.example.com/hadoop/hadoopfile Reads the input from stdin. Exit Code: Returns 0 on success and 1 on error.
... View more
02-15-2016
09:29 AM
3 Kudos
Hi Rushikesh, The command used to compress the file to tar.gz should be tar -zcvf to extract you can use tar -zxvf If you wish to move the compressed file you can use the command - hadoop fs -put /myHDFSFolder/myFolder.tar.gz The complete command would be: SYNTAX: tar zcvf <compressed file name>.tar.gz <source file 1> <source file 2> | hadoop fs -put <source file> <destination location> tar zcvf myFolder.tar.gz a.txt b.txt | hadoop fs -put myFolder.tar.gz /myHDFSFolder/ Hope this help!
... View more