executing a file with multiple queries in it should work. But each statement is executed individually and is non-atomic (they are not executed within a single transaction). Please post exceptions or errors if this has not worked for you. ... View more

Yes, that configuration should work. But you should we aware that you now have 2 instances that can modify the hive metadata without locking at the hive level. Meaning, the database is the one that is in-charge of concurrency. Say you have a HMS client on one reading the full HMS snapshot via one instance, and at the same time, another client is modifying the metadata via the other instance, you may (or may not) see exceptions in the log about certain cached objects (in datanucleus) were no longer found. The HMS handler has retrying logic in it that will retry the transaction and succeed. It is safe to ignore the exceptions as long as the queries are succeeding. Hope this helps. Thanks ... View more

The JsonSerDe class is in hive-hcatalog-xxx.jar.  So you will have to run "ADD JAR" on this file prior.   JsonSerDe has been made a first class citizen recently via https://jira.apache.org/jira/browse/HIVE-18785. With this fix you should be able to just use "STORED AS JSONFILE" instead of having to specify the ROW FORMAT SERDE   This fix is available in CDH6.1 ( I believe, dont quote me on this)   Hope this helps. Thanks ... View more

There is not information for us to provide any guidance on what could be wrong. Do you believe that the hive results are inaccurate or the impala results? Could you provide the following items for us to look at? 1) full table definition in hive 2) Full or Sample data to help reproduce the issue. 3) Queries results from both Hive and impala for the sample data above. Thanks ... View more

Could you please post the HMS log snippet please showing the full exception stack? Could you also post the output of select * from SEQUENCE_TABLE from your HMS metastore DB? Thanks ... View more

the principal is the username that will be passed thru to HS2 for authentication (in your case, user to be authenticated against LDAP). This will be the same user that the jobs will run as in yarn/MR (if impersonation is turned on in Hive). It is not the kerberos-related. It is equivalent to -n command option for beeline. Hope this helps.  ... View more

Its not quite clear what the issue is. We will probably need HS2 and spark logs to do understand the issue.   However, I am curious if the second step succeeded. "load data inpath '/tmp/new.txt' into table new_tmp;"   This appears to be a local path but there is no "LOCAL" keyword in the command. Have you verfied that the data was actually inserted into new_tmp table after this step?   Also what version of CDH is this? Thanks ... View more

Hello, I am a Hive engineer so I am not quite certain of the HUE side settings but like you said the first step should be to narrow down if it is a hive-side or a hue-side issue. Testing from Beeline makes sense to ensure that hive-side settings are good.   To beeline JDBC URL, you can add the TrustStore localtion and credentials for it to use SSL connection. Something like this   beeline -u "jdbc:hive2://<HS2HostFQN>:10000/default;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/signer/truststore.jks;sslTrustPassword=<pwd>;principal=<user>"   Let me know what you find. Thanks ... View more

I would say this is a bug. If the user isnt supposed to be performing a certain action (updating the name node URIs in this case), then either the UI should have prevented the user from performing the action or should have done nothing during the action if the cluster was not HA-enabled for NN. Manipulating the metadata is bad. I will file an internal jira. Thank you for reporting. ... View more

The bold text is used to tell hive how to read/interpret the data for the hive table (located at '/user/hive/warehouse/original_access_logs' in this case). 1) ROW FORMAT SERDE "org.apache.hadoop.hive.contrib.serde2.RegexSerDe" -- tells hive to use this class to serialize and deserialize the rows to/from the file. 2) input.regex is a property used by this class (RegexSerde) to deserialize the rows read from the table data. So this regex pattern is applied to the row value read from the file to split up into different columns defined in the meta data for this hive table. 3) output.format.string is a property used by this class (RegexSerde) to serialize the rows being written out to this table data. This value is used as a format to generate a row value (from its column values) that is to be written back to the output file for this hive table. Hope this helps. Thanks ... View more