Member since
10-01-2015
3933
Posts
1150
Kudos Received
374
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2323 | 05-03-2017 05:13 PM | |
1966 | 05-02-2017 08:38 AM | |
2110 | 05-02-2017 08:13 AM | |
2122 | 04-10-2017 10:51 PM | |
863 | 03-28-2017 02:27 AM |
05-05-2016
07:14 AM
hbase is great for random key lookups, I've worked on a project where wordcloud powered by HBase worked just fine. If you have a dashboard, HBase or perhaps Phoenix works pretty well behind it.
... View more
05-05-2016
12:26 AM
Unexpected Exception: (setuptools 1.1.6 (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), Requirement.parse('setuptools>=11.3'))
the full traceback was:
Traceback (most recent call last):
File "/usr/local/bin/ansible-playbook", line 72, in <module>
mycli = getattr(__import__("ansible.cli.%s" % sub, fromlist=[myclass]), myclass)
File "/Library/Python/2.7/site-packages/ansible/cli/playbook.py", line 30, in <module>
from ansible.executor.playbook_executor import PlaybookExecutor
File "/Library/Python/2.7/site-packages/ansible/executor/playbook_executor.py", line 30, in <module>
from ansible.executor.task_queue_manager import TaskQueueManager
File "/Library/Python/2.7/site-packages/ansible/executor/task_queue_manager.py", line 29, in <module>
from ansible.executor.play_iterator import PlayIterator
File "/Library/Python/2.7/site-packages/ansible/executor/play_iterator.py", line 29, in <module>
from ansible.playbook.block import Block
File "/Library/Python/2.7/site-packages/ansible/playbook/__init__.py", line 25, in <module>
from ansible.playbook.play import Play
File "/Library/Python/2.7/site-packages/ansible/playbook/play.py", line 27, in <module>
from ansible.playbook.base import Base
File "/Library/Python/2.7/site-packages/ansible/playbook/base.py", line 35, in <module>
from ansible.parsing.dataloader import DataLoader
File "/Library/Python/2.7/site-packages/ansible/parsing/dataloader.py", line 33, in <module>
from ansible.parsing.vault import VaultLib
File "/Library/Python/2.7/site-packages/ansible/parsing/vault/__init__.py", line 66, in <module>
from cryptography.hazmat.primitives.hashes import SHA256 as c_SHA256
File "/Library/Python/2.7/site-packages/cryptography/hazmat/primitives/hashes.py", line 15, in <module>
from cryptography.hazmat.backends.interfaces import HashBackend
File "/Library/Python/2.7/site-packages/cryptography/hazmat/backends/__init__.py", line 7, in <module>
import pkg_resources
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2797, in <module>
parse_requirements(__requires__), Environment()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 580, in resolve
raise VersionConflict(dist,req) # XXX put more info here
VersionConflict: (setuptools 1.1.6 (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), Requirement.parse('setuptools>=11.3'))
... View more
Labels:
- Labels:
-
Apache Metron
05-04-2016
05:57 PM
2 Kudos
In absence of a secured cluster, I enabled Linux Secured Containers and white-listed yarn user. In a production environment, what are the security risks with whitelisting yarn user and having regular users execute Oozie workflows on behalf of hbase user.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache YARN
05-04-2016
08:01 AM
Not quite, I was able to make it work using https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/SecureContainer.html tested on standard sandbox
... View more
05-04-2016
06:44 AM
You can create composite keys a combination of fields, for example ID|timestamp|another-field then you can use built in pig functions to create that row key. You can also presplit a table and then your non-unique key can possibly be OK, it all depends how often you insert too much data, for one time is fine, you can move regions around after insert. Take a look at key design section of HBase, link is to the old version but concepts are the same https://hbase.apache.org/0.94/book/rowkey.design.html
... View more
05-04-2016
01:21 AM
changing min_user_id in Ambari requires restarting the following: App Timeline Server, NodeManager, ResourceManager, YARN Client and since I had Oozie enabled, that too had stale configs.
... View more
05-03-2016
11:44 PM
I see you asked this question in a separate thread, that's the right approach to keep them separate.
... View more
05-03-2016
11:30 PM
1 Kudo
Autoincrementing rowkeys will cause hotspotting, you want to create rowkeys that are not sequential, in fact they should be as random as possible. HBase does not like monotonously increasing rowkeys.
... View more
05-03-2016
08:59 PM
According to our documentation any user ID with UID 1000 and up can execute jobs on YARN. In my tests, having a user below the UID min number defined for a cluster still executes a job and I cannot prevent it from executing jobs. Can someone confirm it works with HDP 2.3.4, also does it only take effect in a secure cluster?
... View more
Labels:
05-03-2016
06:11 PM
more specifically, we'd like to change min.user.id in container-executor.cfg
... View more
Labels:
- Labels:
-
Apache YARN