Member since
06-05-2019
126
Posts
133
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1387 | 12-17-2016 08:30 PM | |
988 | 08-08-2016 07:20 PM | |
2006 | 08-08-2016 03:13 PM | |
2003 | 08-04-2016 02:49 PM | |
1830 | 08-03-2016 06:29 PM |
06-17-2016
03:31 PM
Hi @Ryan Cicak The best practice is to configure Ranger audits to both Solr and HDFS. HDFS is used for long term audit storage so you won't want to delete audit data. Solr should be used for short term storage. By using Solr you have data indexed and you can query it quickly from Ranger UI. I am not aware of any setting or property in Ranger to set a TTL and automatically delete data. You may leverage Solr TTL feature to purge data (link) or schedule a job to issue a delete query periodically.
... View more
06-10-2016
11:35 PM
2 Kudos
A remote Linux system can use NFS (Network File System) to mount an HDFS file system and interact with the file system. Before proceeding, it's important to understand that your linux instance is directly accessing your HDFS system through the network, therefore you will incur network latency. Depending on your dataset size, you have to remember you could be potentially processing gigabytes or more of data on a single machine therefore this is not the best approach for large datasets. These steps will show you how to mount and interact with a remote HDFS node within your Linux system: 1) The linux system must have NFS installed (CentOS for demo) yum install nfs-utils nfs-utils-lib 2) Your HDP cluster must have an NFS Gateway installed (Ambari allows this option with one click) * Keep track of either the FQDN or IP address of the NFSGateway 3) In Ambari, under HDFS > Advanced > General set Access time precision = 3600000 3) Mount the NFS Gateway on your linux system (must be root) mount -t nfs -o vers=3,proto=tcp,nolock myipaddressorfqdnofnfsgateway:/ /opt/remotedirectory
4) On both your HDFS node & remote Linux system add the same user with the same uid (making sure neither already exist) useradd -u 1234 testuser * If your user/uid doesn't match between HDFS node and your remote Linux system - whatever uid you are logged in as on your remote Linux system will be passed and interpreted by the NFS Gateway. For example if your Linux system has usertest (uid = 501) and you write a file to HDFS's /tmp, the file owner of the file will be whichever user on the HDFS node matches uid=501 - therefore it is good practice to match both the username and the uid across both systems. 5) On your remote Linux system, login as your "testuser" and go-to your mounted NFS directory cd /opt/remotedirectory You will now be able to interact with HDFS with native linux command such as cp, less, more, etc:.
... View more
Labels:
06-25-2016
01:53 AM
1 Kudo
As mentioned in https://community.hortonworks.com/questions/37192/error-no-package-python27-available-while-setting.html, the tutorial has been corrected.
... View more
06-12-2016
01:59 AM
1 Kudo
Updated tutorial:
1) using centos-release-scl
2) wget https://bootstrap.pypa.io/ez_setup.py
Thanks!
... View more
06-07-2016
07:09 AM
1 Kudo
If you want to see dates and update history for tutorials then I would suggest looking at the source in github. https://github.com/hortonworks/tutorials Tutorials are updated on Sandbox update release schedule which tend to correspond to major HDP releases. Here you can see the latest version of the HDP tutorials for HDP 2.4: https://github.com/hortonworks/tutorials/tree/hdp/tutorials/hortonworks For example, I believe earlier you had a question on Ipython Notebook with Spar tutorial and here you can see the history of updates for this tutorial: https://github.com/hortonworks/tutorials/commits/hdp/tutorials/hortonworks/ipython-notebook-with-spark/tutorial.md The tutorials list what is the prerequisite to do the tutorial. If you want to learn more about the tutorials or make a contribution then at the bottom of each tutorial there is paragraph that talks about the github repo and contribution guide. I am happy to chat with you to see how we can make this template more descriptive.
... View more
06-01-2016
07:05 PM
Lockout is not at DB level here since we are not authenticating with DB username/password but ambari username/password. So, I don't think there will be a way to lockout at DB level. It has to be implemented at ambari application level, and as @jeff pointed out, can be an enhancement
... View more
05-31-2016
06:42 PM
2 Kudos
The design was to leave space to handle more than one cluster at a time in a single Ambari Server instance. Maybe Ambari could have a convenient resource /api/v1/cluster (like you mention) that will convert to the [0] cluster automatically? If that makes sense, maybe file that JIRA in Ambari project?
... View more
06-28-2016
12:57 AM
@Ryan Cicak Hi,this is the demo that help me well. But when I execeted the command: insert into brancha(full_name,ssn,location) values ('ryan', '111-222-333', 'chicago'); It report error like this: java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/usr/local/data-governance/apache-atlas-0.7-incubating-SNAPSHOT/hook/hive/atlas-client-0.7-incubating-SNAPSHOT.ja The detail of this issue is posted on my another thread: https://community.hortonworks.com/questions/41898/using-hive-hook-file-does-not-exist-atlas-client-0.html Please check. I hope you can help me. Thank you very much.
... View more
06-01-2016
07:43 PM
3 Kudos
Since there have been no responses to this question for over a week, I've done research. Cloud providers have solutions to active directory in their offering: Azure: Azure Active Directory AWS: AWS Directory Service Both cloud providers have a way to sync your on-premise Active Directory with their cloud-based service (Azure Active Directory or AWS Directory Service). These solutions both include Kerberos authentication support.
... View more
05-21-2016
10:22 PM
Hi @Ryan Cicak, I just tested your scenario and I'm getting people table in Atlas search. Are you sure that your table is created ?
... View more
- « Previous
- Next »