Member since
01-20-2017
8
Posts
1
Kudos Received
0
Solutions
02-17-2017
03:13 PM
1 Kudo
Greetings, I Have this class below. When I call it the connection works but I get warning and info messages. import jaydebeapi
import re
import sys
class Hive(object):
"""Class to issue beeline commands to specified server and database"""
def __init__(self, principal, database, server, port):
if server is None:
server='localhost'
if port is None:
port=10000
if not isinstance(port, (int, long)):
raise TypeError("Error, port must be an integer, not ["+port+"]")
self.principal = principal
self.database = database
self.server = server
self.port = port
self.valid_name=re.compile('(?x) \A \w+ \Z')
if not self.valid_name.match(self.database):
raise TypeError("Invalid database name [" + self.database + "]")
url=("jdbc:hive2://" + server + ":" + str(port) + "/default;principal="
+ principal + ";")
# TODO suppress stdout from the driver. How?
conn=jaydebeapi.connect("org.apache.hive.jdbc.HiveDriver", url)
Messages: "2017-02-17 10:05:43,190 INFO [main] jdbc.Utils (Utils.java:parseURL(312)) - Supplied authorities: localhost:10000",
"2017-02-17 10:05:43,193 INFO [main] jdbc.Utils (Utils.java:parseURL(428)) - Resolved authority: localhost:10000",
"2017-02-17 10:05:43,526 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable" I don't have access to the server to alter its logging config. How can I suppress these warnings in the python class?
... View more
02-08-2017
08:20 PM
Phys2 is EOL. https://github.com/BradRuderman/pyhs2/blob/1094d4b3a1e9032ee17eeb41f3381bbbd95862c1/README.md
... View more
02-08-2017
07:00 PM
I'm creating some python code to execute some beeline commands. The command looks ok, but I get an error:
WARNING: Use "yarn jar" to launch YARN applications.
Error: Error while compiling statement: FAILED: ParseException line 1:8
character '<EOF>' not supported here (state=42000,code=40000)
Error: Error while compiling statement: FAILED: ParseException line 1:12
character '<EOF>' not supported here (state=42000,code=40000)
['beeline', "-u 'jdbc:hive2://localhost:10000/default;principal=hive/dev15-namenode-01.example.com@example.COM'", '--fastConnect=true', '--showHeader=false', '--verbose=false', '--showWarnings=false', '--silent=true', '--outputFormat=csv2', "-e 'use tsz;'", "-e 'show tables;'"]
Traceback (most recent call last):
File "./beeline.py", line 55, in <module>
tables=beeline.show_tables()
File "./beeline.py", line 43, in show_tables
return self._run_cmd(cmd)
File "./beeline.py", line 48, in _run_cmd
stdout = subprocess.check_call(cmd)
File "/usr/lib64/python2.6/subprocess.py", line 505, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['beeline', "-u 'jdbc:hive2://localhost:10000/default;principal=hive/dev15-namenode-01.example.com@example.COM'", '--fastConnect=true', '--showHeader=false', '--verbose=false', '--showWarnings=false', '--silent=true', '--outputFormat=csv2', "-e 'use tsz;'", "-e 'show tables;'"]' returned non-zero exit status 2
The errors seem to correspond to the semicolons in the -e blocks. I know the command via the shell works. Any ideas on the cause? import os
import subprocess
class BeeLine(object):
"""Make empoyee person class with names, and title required."""
def __init__(self, principal, database, server='localhost', port=10000):
self.principal = principal
self.database = database
self.server = server
self.port = port
if not isinstance(port, (int, long)):
raise TypeError("Error, port must be an integer")
url=("'jdbc:hive2://" + server + ":" + str(port) + "/default;principal="
+ principal + "'")
self.options=[ "beeline"
, "-u " + url
, "--fastConnect=true"
, "--showHeader=false"
, "--verbose=false"
, "--showWarnings=false"
, "--silent=true"
]
def show_tables(self):
"""Show tables of object's database"""
cmd=[]
cmd.extend(self.options)
cmd.append("--outputFormat=csv2")
cmd.append("-e 'use " + self.database + ";'")
cmd.append("-e 'show tables;'")
return self._run_cmd(cmd)
def _run_cmd(self, cmd):
"""Private function. Run command, from array, and return stdout"""
print cmd
stdout = subprocess.check_call(cmd) <<<< Command run HERE
return stdout
#
# Main
#
beeline=BeeLine(
principal='hive/dev15-namenode-01.example.com@example.COM'
, database='tsz')
tables=beeline.show_tables()
print tables
... View more
Labels:
01-23-2017
05:12 PM
I tried that, but specifying a service_name of hive-site results in a 404.
... View more
01-23-2017
04:42 PM
If I know the tag for a previous hive-site configuration how can instruct REST to restore to it? The only thing I've found is for restoring a level higher (HIVE), but I don't know that version. And is there a document that describes these types of task?
... View more
Labels:
01-20-2017
08:56 PM
What are sub configuration types?
... View more
01-20-2017
07:29 PM
REST list of configurations ( /api/v1/clusters/dev15/configurations?type=hive-site ) returns 13 version, numbered 1-13. But the web UI shows 17 versions. A REST query for v17 returns nothing. Why do REST and web UI service config versions not match? How can versions be reconciled?
... View more
01-20-2017
06:36 PM
I know how to get service configurations (e.g. /api/v1/clusters/dev15/configurations?type=<service>), but how can I instruct REST to restore an older config to the current config if I know the old config's tag or version?
... View more
Labels: