Member since
01-02-2017
18
Posts
3
Kudos Received
0
Solutions
03-24-2018
08:56 PM
Hello, I thought that NiFi 1.5 can report lineage to Atlas but I couldn't find how to it. I've checked the reporting tasks but couldn't find Atlas reporting task. Does anyone succeeded in using this new feature? Thanks Joe
... View more
11-07-2017
09:56 AM
1 Kudo
Hello, Is there any recommendation on number of concurrent users with Hive LLAP and Druid ? is it acceptable to have 100 or 1000 users with these solutions? If no, is their any other solution to achieve this scale? Thanks Joe
... View more
Labels:
10-01-2017
03:47 PM
Hello, There are several tools in HDP that doesn't use Yarn (Storm, HBase, etc). If I have OS, HBase, Storm and other tools taking resources on my cluster, how does Yarn know how much resources he owns for its application? What are the best practices for multi-tenancy and isolation in this case? How can I isolate IO with Yarn? is this something coming in future versions? Thanks
... View more
Labels:
06-13-2017
06:39 AM
Hi @rkovacs Thank you for your reply. I understand that I can not upgrade a Cloudbreak cluster since Ambari upgrade is not supported. What do you mean by HDP upgrade is supported? Thanks for the clarification
... View more
06-12-2017
05:12 PM
Hi, How can I upgrade a cluster deployed by Cloudbreak? If I upgrade through Ambari, what's the impact on Cloudbreak relation? (API, version, Autoscaling, etc) Thanks
... View more
Labels:
03-12-2017
04:25 PM
1 Kudo
Hi, From my understanding, enabling SSL in NiFi and connecting to LDAP is enough for security. What's the advantage of HDF Kerberisation ? is it only for Kafka and Storm ? Can I have a non-Kerberised HDF cluster talking with a Kerberised HDF cluster Thanks
... View more
Labels:
03-12-2017
04:21 PM
Hi I have two HDF cluster, one secured (with SSL enabled) and the other without security. Can I have the cluster 2 (unsecure) send data to cluster 1 (secured) with S2S ? if yes, what's the required configuration? Thanks
... View more
Labels:
01-29-2017
11:03 AM
Hi @Matt Burgess Thanks for your detailed answer. Your first suggestion looks interesting. I'll give it a try. I still have a question on ConvertJsonToSQL if you can help https://community.hortonworks.com/questions/80362/jdbc-connection-pool-for-convertjsontosql.html
... View more
01-29-2017
11:02 AM
Hi, What's the role of the JDBC Connection Pool property of the ConvertJsonToSQL processor ? How to use this processor with Hive ? I noticed that HiveConnectionPool was possible choice for this property in previous NiFi version but it's no more the case ? Thanks
... View more
Labels:
01-27-2017
07:24 PM
@Binu Mathew
Thanks for your reply. No I don't need data in JSON, I just need to ingest directly into Hive. How do you ingest data in Hive in your suggestion ? PutHiveQL waits for an SQL statement
... View more
01-27-2017
06:52 PM
Hi @mqureshi
I don't agree with your statement. If it's the case, why there's processors like getFile, getHDFS, QueyTable etc ?
... View more
01-27-2017
06:23 PM
I want to use NiFi to ingest CSV file (getFile) to Hive (PutHiveQL). Since HiveQL need the SQL statement, how can I generate it ? A solution would be getFile -> InferAvroSchema -> ConvertCSVToAvro -> ConvertAvroToJSon -> ConvertJSONtoSQL -> PutHiveQL. This looks so complex and resource consuming. Any suggestions ?
... View more
Labels:
01-27-2017
05:45 PM
Thanks @Eugene Koifman
Can you point to an updated complete and updated documentation/book on Hive features ? (ACID, LLAP, etc)
... View more
01-27-2017
05:03 PM
Hi, I created a simple Hive table (everything default) and were surprised that I could insert data into it doing INSERT INTO TABLE etc. The Hive manual claims that insert is available only for tables that support ACID. However, I didn't set anything to enable ACID support for my table. Also, data is not in ORC. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-InsertingvaluesintotablesfromSQL Can you please explain this behavior ? Thanks
... View more
01-17-2017
05:54 PM
1 Kudo
Hi, I am able to backup all my hive tables by backuping the HDFS folder /apps/hive/warehouse and the MySQL database (mysql dump and restore). How can I backup some Hive tables only? for HDFS files it's easy. But how can I backup/restore the metadata from MySQL for only those tables ? Cheers! Joe
... View more
Labels:
01-02-2017
10:42 PM
Hello Does the SHC supports write operations (https://github.com/hortonworks-spark/shc) ? What are the difference between this connector and the one provided by Cloudera Labs ? Which one of these connectors is referenced in HBase doc here ? https://hbase.apache.org/book.html#spark
... View more
Labels:
01-02-2017
03:33 PM
Hi, I am implementing DR for our HBase cluster. Can you share any best practices/gotchas around this ? I know that question is general, but any information on tuning, monitoring or architecture are welcome. I wanted to check, if someone has any lesson learned with some dos and don'ts. Joe
... View more
Labels: