Member since
10-22-2015
55
Posts
41
Kudos Received
16
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
738 | 01-11-2019 03:06 PM | |
418 | 08-27-2018 04:10 PM | |
1020 | 10-10-2017 05:38 PM | |
697 | 06-22-2017 05:31 PM | |
767 | 05-30-2017 07:59 PM |
07-11-2019
01:11 PM
Slider provides variables here and here that will give you unique HDFS and ZK paths for each application, so if you're using those it will be fine.
... View more
07-03-2019
03:25 PM
2 Kudos
You should be able to create a new application with a different name using the same templates. The main things to watch out for are port conflicts and ensuring different applications are using different HDFS and ZK paths. These should already be taken care of if you copied the hbase and kafka templates from the slider app-packages.
... View more
01-11-2019
03:06 PM
The --ipc flag is not configurable through YARN at this time. There is a ticket open for adding this feature, YARN-8447. No one is currently working on this, so it could be helpful for you to add a comment to the ticket describing your use case.
... View more
11-12-2018
03:53 PM
YARN-7787 is open to discuss the issue, but there is no clear solution.
... View more
10-26-2018
02:16 PM
You only need Registry DNS if you are launching applications using the YARN service framework and Registry DNS is needed by those applications.
... View more
10-10-2018
01:35 PM
Just delete the table testtable. It must have errored out in the middle of a previous service check.
... View more
08-29-2018
01:49 PM
For Accumulo it is recommended to set the max number of open files (ulimit -n) to at least 32768.
... View more
08-27-2018
04:10 PM
1 Kudo
You can configure a restart policy for each component. Add a field like the following: "restart_policy": "ON_FAILURE" The available restart policies are ALWAYS, ON_FAILURE, and NEVER. The default policy is ALWAYS.
... View more
03-26-2018
07:45 PM
Please accept the answer if it fixes your problem.
... View more
03-23-2018
03:51 PM
Another command that might be informative is checking the last PID assigned: sysctl kernel.ns_last_pid
... View more
03-23-2018
03:22 PM
1 Kudo
It's likely that the host has run out of PIDs, and that's why the RM can't create a new thread. Here are some commands that can help you identify whether this is the issue and increase the maximum number of PIDs allowed. Check the number of threads running: ps -elfT | wc -l Check the current pid_max: sysctl kernel.pid_max Increase the pid_max: sysctl -w kernel.pid_max=4194304
... View more
03-06-2018
04:40 PM
The AMBARI-22947 patch will be needed to configure the container-executor.cfg file through Ambari. Prior to this change, only a few properties were configurable for this file.
... View more
11-10-2017
03:03 PM
Did you reset-security back to the accumulo principal? Also, you should kinit as accumulo@EXAMPLE.COM before issuing the reset-security command, so I'm not sure it will work using sudo -u accumulo unless you include kinit and reset-security in a single command.
... View more
11-08-2017
05:52 PM
That's right, use the original admin user to grant permissions to your custom user. To find out which permissions need to be granted for full admin access, check the permissions of accumulo@EXAMPLE.COM in the shell or take a look at the example in the Apache Accumulo user manual.
... View more
11-07-2017
05:47 PM
The yarn.nodemanager.delete.debug-delay-sec property controls how soon the NM will delete the local logs after the application finishes.
... View more
10-10-2017
05:38 PM
1 Kudo
The accumulo shell will only work from a host where an Accumulo Client is installed. You can check in the Ambari UI to find out where clients are installed.
... View more
09-20-2017
04:03 PM
Docker has a fixed amount of space for images and containers. It may help to clean up exited containers and dangling images with commands like the following: docker rm $(docker ps -a -q -f "status=exited")
docker rmi $(docker images -q -f "dangling=true") You should also try to remove images you are no longer using and to stop and remove running containers.
... View more
09-20-2017
02:30 PM
Is this occurring on a Mac?
... View more
09-08-2017
05:45 PM
You should set the site.global.slider.allowed.ports property in the global section of the appConfig.json. LLAP appears to have a CLI option slider-appconfig-global that will add custom properties.
... View more
06-22-2017
05:31 PM
The client software does not need to be installed on all the nodes, only on the node(s) where client / CLI commands will be issued. In Slider's case, Slider jars will be uploaded in a tarball to HDFS from the client and YARN will localize the tarball for Slider's containers.
... View more
06-07-2017
03:53 PM
1 Kudo
The service check is looking for a particular string (appTrackingUrl=.*) in the output of the command 'yarn org.apache.hadoop.yarn.applications.distributedshell.Client -shell_command ls -num_containers 1 -jar /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar -timeout 300000 --queue default' and this error means that the string was not present. This probably means that the command failed. I suggest looking into why the command failed. You could see if the app appears in the RM UI. You could try to locate the stdout/stderr for the command, and/or try running the command manually as the ambari-qa user to see what the error is.
... View more
06-05-2017
01:42 PM
Try using the default "yarn" as the slider.user. It may be that additional configuration would be needed to be able to run Slider as the "admin" user.
... View more
05-30-2017
07:59 PM
1 Kudo
It should be possible to run Solr on YARN using Slider in HDP 2.4.2: https://github.com/hortonworks/slider-release/tree/HDP-2.4.2.0-tag/app-packages/solr Theoretically you would be able to launch a Solr app on YARN through the Slider View in Ambari, but I am not sure if anyone has tried this out for Solr. The Slider View for Ambari says it's marked as deprecated, so I am not sure of its status: https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.18/bk_ambari-views/content/ch_using_slider_view.html
... View more
02-21-2017
04:24 PM
These are all linked to a single directory, so any of them would be fine. I typically use the client directories for HDP services, e.g. accumulo-client. The ACCUMULO_HOME variable is often not set explicitly and is determined by the accumulo scripts to be the parent directory of bin/accumulo.
... View more
12-16-2016
02:50 PM
1 Kudo
With YARN applications, you specify the resources you want the containers to use when the app is launched. With HBase on YARN using Slider, you specify this in the resources.json file passed to the slider create command.
... View more
12-15-2016
03:56 PM
1 Kudo
HBase can run on YARN or without YARN. MapReduce jobs always run on YARN, so whenever there is an HBase operation that performs a MapReduce, that will be run on YARN.
... View more
08-30-2016
12:23 AM
1 Kudo
Slider uses its own Hadoop jars in the Slider lib directory. I suspect that the Hadoop client jars Slider is using are not compatible with the version of Hadoop running on the cluster. If you built Slider yourself, you could try rebuilding it with a hadoop.version that matches the version running on your cluster (you may need to adjust some other dependency versions in this case as well). Alternatively, you could try replacing the hadoop jars in the Slider lib directory, but if the versions are too far apart this will not work and you'll have to rebuild Slider.
... View more
08-03-2016
03:06 PM
Thanks for trying it out! It does sound like a Slider bug if it isn't working for multiple containers on the same host. You're welcome to open a ticket at https://issues.apache.org/jira/browse/SLIDER, or I can do that for you.
... View more
08-01-2016
08:42 PM
1 Kudo
I don't think the default hbase app package will allow you to get the region server ports through the REST API because Slider is not allocating those ports. The port is set to 0 in the hbase-site config, so hbase will pick a random port itself. If you wanted Slider to allocate the port instead, you might be able to change the hbase.regionserver.port property to the following in the appConfig: "site.hbase-site.hbase.regionserver.port": "${HBASE_REGIONSERVER.ALLOCATED_PORT}{PER_CONTAINER}", You would also have to add componentExports to the HBASE_REGIONSERVER component in the metainfo.xml file: <component>
<name>HBASE_REGIONSERVER</name>
<componentExports>
<componentExport>
<name>regionservers</name>
<value>${THIS_HOST}:${site.hbase-site.hbase.regionserver.port}</value>
</componentExport>
</componentExports>
... You should then be able to access this information through /ws/v1/slider/publisher/slider/componentinstancedata
... View more
06-22-2016
06:41 PM
As described in the answer to this question: https://community.hortonworks.com/questions/35138/target-replicas-is-5-but-found-3-live-replicas.html Start the accumulo shell on a host where the Accumulo client is installed and run the following: config -t accumulo.root -s table.file.replication=3
config -t accumulo.metadata -s table.file.replication=3 You'll have to change these properties AND run the hdfs setrep command so that both new and existing files have their settings changed.
... View more