Member since
05-12-2016
6
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5562 | 05-17-2016 07:00 PM |
05-17-2016
07:00 PM
1 Kudo
Apparently, this is a known bug... I replaced the rewrite.xml with the new one and now it's good. http://www-01.ibm.com/support/docview.wss?uid=swg21980169 I hope it will help others!
... View more
05-17-2016
05:24 PM
OK, so I found something that looks like an internal mapping error in YARNUI's rewrite.xml file: <rule dir="OUT" name="YARNUI/yarn/inbound/jobhistory/logs" pattern="*1; url=*://*:*/jobhistory/logs/{**}">
<rewrite template="1; url=/gateway/jobstoryui/jobstory/jobhistory/logs/{**}"/>
</rule> Is it just me or the mapping is not right? this is the exact URL that
doesn't generate the gateway path when clicking on the logs from the
YARN webpage that should pass it to MapReduce. It also looks like a mix of the IN-OUT syntax. Those are some other rules which look better: <rule dir="IN" name="YARNUI/yarn/inbound/resourcemanager/home" pattern="*://*:*/**/yarn/resourcemanager?{scheme}?{host}?{port}">
<rewrite template="{scheme}://{host}:{port}/cluster"/>
</rule>
<rule dir="IN" name="YARNUI/yarn/inbound/logs" pattern="*://*:*/**/yarn/logs?{scheme}?{host}?{port}?{**}">
<rewrite template="{scheme}://{host}:{port}/logs/?{**}"/>
</rule>
<rule dir="IN" name="YARNUI/yarn/inbound/jobhistory/job" pattern="*://*:*/**/yarn/jobhistory/job/{**}?{scheme}?{host}?{port}">
<rewrite template="{scheme}://{host}:{port}/jobhistory/job/{**}"/>
</rule>
<rule dir="IN" name="YARNUI/yarn/inbound/sparkhistory/job" pattern="*://*:*/**/yarn/history/{**}?{**}?{scheme}?{host}?{port}">
<rewrite template="{scheme}://{host}:{port}/history/{**}?{**}"/>
</rule> <rule dir="OUT" name="YARNUI/yarn/outbound/headers/logs/location">
<match pattern="{scheme}://{host}:{port}/logs/?{**}"/>
<rewrite template="{$frontend[url]}/yarn/logs?{scheme}?host={$hostmap(host)}?{port}?{**}"/>
</rule>
<rule dir="OUT" name="YARNUI/yarn/outbound/logs/files" pattern="/logs/{**}">
<rewrite template="{$frontend[url]}/yarn/logs/{**}"/>
</rule>
<rule dir="OUT" name="YARNUI/yarn/outbound/static" pattern="/static/{**}">
<rewrite template="{$frontend[url]}/yarn/static/{**}"/>
Can anyone let me know what should be the correct mapping for that one?
Thanks.
... View more
05-16-2016
06:09 PM
thanks, sure.. I checked the XMLs already. couldn't spot an issue. </gateway><service>
<role>YARNUI</role>
<url>http://my-yarn-server.com:8088</url>
</service></topology> <role>RESOURCEMANAGER</role>
<url>http://my-rm-server.com:8088/ws</url>
</service> </gateway><service>
<role>JOBSTORYUI</role>
<url>http://my-job-history-server.com.ibm.com:19888</url>
</service></topology> The issues seems to be around those 3, when the root issue looks like coming from the Resource Manager. I tend to believe that when the Resource Manager will act and look as it should, the other will aline as well. I have the following authorization entries: <provider>
<role>authorization</role>
<name>AclsAuthz</name>
<enabled>true</enabled> <param>
<name>resourcemanager.acl</name>
<value>*;MYPROJECT;*</value>
</param> <param>
<name>yarnui.acl</name>
<value>*;MYPROJECT;*</value>
</param> <param>
<name>jobstoryui.acl</name>
<value>*;MYPROJECT;*</value>
</param> ... ... (I have it for all the services). I can try to remove authorization from those 3 and see if it changes anything. Any other thoughts? do you see any issue with the mapping? Thanks.
... View more
05-14-2016
12:04 AM
One more important thing: about the error (pic with HTTP 404) - I can see that when I manually add the gateway path to the URL in the error, it works (partially, still with Java error, but it directs to the right place) So if this is a job URL with an error as in picture: (it launches it when I click on the Job link) https://myhost:8443/proxy/application_1463172884925_0003/mapreduce/job/job_1463172884925_0003 adding "/gateway/yarnui/yarn" before proxy brings me to the right place. What am I missing?
... View more
05-13-2016
11:38 PM
Hi thanks for your quick reply, those are the versions I'm using: # ambari-server --version
2.1.0 # hadoop version
Hadoop 2.7.1-IBM-8 Just reminding, when I use the direct link to the ResourceManager (not through Knox) it's good, so it must be something with Knox, either mapping or a Java rendering issue that I'm not aware of. One more important thing that I noticed today - When I go to the quick link from the MapReduce service I'm able to get to the history links (using Knox), so it seems like an issue with Yarn since I see it on the Resource Manager page. Ideas are welcome. Thanks!
... View more
05-12-2016
05:32 AM
1 Kudo
Hello, I'm running a BigInsights cluster and I have Knox configured with LDAP Authentication and Authorization. I've noticed that when I go to the ResourceManager UI using the Yarn quick link through Ambari UI and click on the applicationMaster link of a running job, the page doesn't look so good (like a rendering issue) and it states the following at the top: "This page will not function without javascript enabled. Please enable javascript
on your browser." Javascript is enabled on the browser. I actually saw this in another place which turned out to be mapped wrong, so it seems to be a sympton and not the problem. The thing is that when I try to view the running application details (using the internal link in the application manager) I get a server error, which seems like a mapping issue. When clicking on the JobID: Using direct link with port 8088 works well. This is the Knox mapping I'm using:
<service>
<role>NAMENODE</role>
<url>hdfs://{{namenode_host}}:{{namenode_rpc_port}}</url>
</service> <service>
<role>JOBTRACKER</role>
<url>rpc://{{rm_host}}:{{jt_rpc_port}}</url>
</service> <service>
<role>WEBHDFS</role>
<url>http://{{namenode_host}}:50070/webhdfs</url>
</service> <service>
<role>WEBHCAT</role>
<url>http://{{webhcat_server_host}}:{{templeton_port}}/templeton</url>
</service> <service>
<role>OOZIE</role>
<url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie</url>
</service> <service>
<role>WEBHBASE</role>
<url>http://{{hbase_master_host}}:60010</url>
</service> <service>
<role>HIVE</role>
<url>http://{{hive_server_host}}:{{hive_http_port}}/{{hive_http_path}}</url>
</service> <service>
<role>RESOURCEMANAGER</role>
<url>http://{{rm_host}}:{{rm_port}}/ws</url>
</service> <service>
<role>HDFSUI</role>
<url>http://{{namenode_host}}:50070</url>
</service>
<service>
<role>HBASEUI</role>
<url>http://{{hbase_master_host}}:{{hbase_master_ui_port}}</url>
</service> <service>
<role>SOLR</role>
<url>http://{{solr_host}}:{{solr_port}}/solr</url>
</service>
<service>
<role>SPARKUI</role>
<url>http://{{spark_historyserver_host}}:{{spark_historyserver_ui_port}}</url>
</service>
<service>
<role>OOZIEUI</role>
<url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie</url>
</service>
<service>
<role>YARNUI</role>
<url>http://<my yarn server>:8088</url>
</service>
<service>
<role>JOBSTORYUI</role>
<url>http://{{mr_historyserver_address}}</url>
</service>
Any ideas? Thanks, AT
... View more
Labels: