Member since
12-15-2015
16
Posts
3
Kudos Received
0
Solutions
11-15-2017
08:03 PM
1 Kudo
We have a use case to stream data from Oracle tho HDFS/Hive (Ideally in Real Time) from Oracle DB 1.The Oracle DB Cannot be modified as it is a proprietary Product. We will have access and read privileges to it 2.Data needs to be landed in HDFS and also loaded into Hive 3.Data may be made avail in Kafka as well Is niFI a good solution to accomplish this. When we looked at NifI a few releases ago, there were some limitations. Any one has experience doing this , please chime in.
... View more
Labels:
- Labels:
-
Apache NiFi
05-22-2017
01:26 PM
@Timothy Spann Were you able to configure TOAD with a kerberized cluster?
... View more
10-21-2016
03:34 PM
Hi Ali - With HDP2.5 Ambari can be used for this installation right? I am confused between : https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_solr-search-installation/content/ch_hdp-search-install-ambari.html and https://doc.lucidworks.com/lucidworks-hdpsearch/2.5/Guide-Install-Ambari.html Do i have to do the first and then do the second ?
... View more
10-21-2016
02:15 PM
1 Kudo
With HDP 2.5 we need to use the jdbc connectors instead of the %sql or hive. However we had to do the following prior to getting this to work. in /usr/hdp/current/zeppelin-server/interpreter/jdbc either copy the hive, hadoop and phoenix jars or create soft links for the same Since all the sample notebooks use the %sql or %hive they would not work. you will need to use %jdbc(phoenix) or %jdbc(hive) as appropriate. Once you do this restart the jdbc interpreter >pwd
/usr/hdp/current/zeppelin-server/interpreter/jdbc
> ls -l
hive-jdbc.jar -> /usr/hdp/current/hive-client/lib/hive-jdbc-1.2.1000.2.5.0.0-1245-standalone.jar
hadoop-common.jar -> /usr/hdp/current/hadoop-client/hadoop-common-2.7.3.2.5.0.0-1245.jar
phoenix-core-4.7.0.2.5.0.0-1245.jar /usr/hdp/current/phoenix-client/lib/phoenix-core-4.7.0.2.5.0.0-1245.jar
... View more
07-19-2016
05:51 PM
PDFs are 50KB Max. and each rowkey can have upto a max of 5 PDFs associated with it. And the total volume of records would be around 500K range. Like you suggest we have 2 column Families, one for the metadata and one for the documents. your suggestion actually gives a vote of confidence to our thought process.
... View more
07-18-2016
07:36 PM
1 Kudo
We are planning to store PDF and Word Documents in Hbase. Storing part is fine. Retrieval is part i have questions on. 1. If we need to query this - Is there a way to do it using any Reporting tools ? Hbase --> Hive External Table -->JDBC/ODBC --> Excel or any BI Tool. However how will the consumer app know that the field is a PDF FIle and not just a text field. 2. Is there a way for HBASE REST to handle this ? Thanks in advance.
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Phoenix
05-18-2016
10:23 AM
^ Great Article. 1.If you have a secured cluster managed by ranger ensure ambari-qa user has access to root.default queue. Many of our validations during the upgrade failed due to this. The SQL mentioned in the above article helped when we could not finalized the upgrade. Couple of components were stuck in State = 'UPGRADING'. The hdp-select did show the right versions. A Restart of ambari-agent on those nodes cleared this up. all the records were updated to upgraded and wer were able to proceed. SELECT rv.version, h.host_name, hv.state
FROM repo_version rv
JOIN host_version hv ON rv.repo_version_id = hv.repo_version_id
JOIN hosts h ON hv.host_id = h.host_id
where (STATE NOT IN ('CURRENT','UPGRADED') OR STATE='UPGRADING')
ORDER BY rv.version ASC, h.host_name;
... View more
05-04-2016
08:46 PM
Configured Ranger and KMS on a Kerberized cluster. Able to create zones and keys.However unable to cat any file put on the directory. Have given the user access to directory in ranger and ability to decrypt eeks. Any ideas? hdfs dfs -cat /zone_encr3/abc1.txt
cat: org.apache.hadoop.security.authentication.client.AuthenticationException: Authentication failed, status: 500, message: Internal Server Error Nothing much in kms.log either.
... View more
Labels:
05-04-2016
06:15 PM
In case of oracle metastore , the query would to run from SQL Developer or TOAD would be : SELECT 'show create table' || dbs.name || '.' || tbls.tbl_name || ';'
FROM tbls LEFT JOIN dbs ON tbls.db_id = dbs.db_id
... View more
03-14-2016
06:41 PM
Hi Neeraj - Thanks for the post. Since current HDP doesn't ship with Drill , would it be reasonable to expect using the Drill ODBC Driver for HBASE Connectivity from Excel and other BI tools ? ( though its more of a MAPR Focus?) I would rather maintain one ODBC Driver vs having Drill for HBASE and HDP ODBC for Hive.
... View more