question phoenix from spark in Archives of Support Questions (Read Only)

phoenix from spark

arunpoy — Tue, 23 May 2017 19:56:21 GMT

I am trying to access phoenix table from spark and have enabled pheonix namespace on the server. when i try to access using the phoenix client (sqlline.py), i am able to do without issue. But when i try to access usign spark-shell i get the foll0wing error.

java.sql.SQLException: ERROR 726 (43M10):  Inconsistent namespace mapping properites.. Cannot initiate connection as SYSTEM:CATALOG is found but client does not have phoenix.schema.isNamespaceMappingEnabled enabled

Even i tried printing the classpath of spark-shell and i do see the hbase-site.xml in the classpath.

Spark Command: /usr/java/jdk1.8.0_60//bin/java -Dhdp.version=2.6.0.3-8 -cp /usr/hdp/current/phoenix-client/phoenix-4.8.1-HBase-1.1-client.jar:/etc/hbase/conf/hbase-site.xml:/usr/hdp/current/spark-client/conf/:/usr/hdp/current/spark-client/lib/spark-assembly-1.6.3.2.6.0.3-8-hadoop2.7.3.2.6.0.3-8.jar:/usr/hdp/current/spark-client/lib/datanucleus-api-jdo-3.2.6.jar:/usr/hdp/current/spark-client/lib/datanucleus-core-3.2.10.jar:/usr/hdp/current/spark-client/lib/datanucleus-rdbms-3.2.9.jar:/usr/hdp/current/hadoop-client/conf/:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-s3-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-core-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-kms-1.10.6.jar -Dscala.usejavacp=true -Xms1g -Xmx1g org.apache.spark.deploy.SparkSubmit --class org.apache.spark.repl.Main --name Spark shell

What could be the issue. How do i pass the the phoenix namespace enabled parameter from spark-shell?

Re: phoenix from spark

jsensharma — Tue, 23 May 2017 20:13:17 GMT

@ARUN

Can you try putting the correct "hbase-site.xml" in spark conf directory.

Also check if it is picking up the right file which has the mentioned property? Or incorrect file?

phoenix.schema.isNamespaceMappingEnabled

Re: phoenix from spark

arunpoy — Tue, 23 May 2017 23:05:02 GMT

@Jay SenSharma

Thanks

It works fine now. But is there a way to do this setting in ambari. copying hbase-site.xml to more than 30 nodes is not the right way as there is a chance to miss.

Re: phoenix from spark

jsensharma — Tue, 23 May 2017 23:47:03 GMT

@ARUN

Good to know that the issue is resolved. I am not aware of any such operation from ambari side to paste the "hbase-site.xml" as it depends on customer's usecase. Like some customers might want to use spark but not HBase.