About shishir_saxena4

shishir_saxena4 · ‎03-14-2016

Does nifi HBase ClientService support connecting to Kerberized HBase ? Nifi - 0.4.0 HBase - 1.1.2

shishir_saxena4 · ‎03-14-2016

Thank You @Artem Ervits for all the links. That was useful information. I am still struggling with getting sizing for my requirements. I will be moving less than 100Gb of data on a daily basis so volume is not a whole lot. But my data comes in spurts. 20Mbps rate at peak times will suffice. I am wondering if I can run my Nifi client from a VM with 8 Gb of RAM and 200Gb of disk space instead of investing in a server.

shishir_saxena4 · ‎03-12-2016

Is there any guide available for NiFi capacity planning ? What load characteristic should be looked at ? - Number of concurrent processors running - Total throughput Any help is greatly appreciated.

shishir_saxena4 · ‎03-10-2016

@Gerd Koenig In Ambari 2.0.1 port is hardcoded to 6080. Newer release of Ambari has a fix. https://github.com/apache/ambari/commit/f9e73665b48c44cb6e8118bb613d81584fddc497

shishir_saxena4 · ‎03-10-2016

@Nelson KA Rajendran You need to create a dummy table with one row create table dummy(a string); insert into table dummy values ('a'); Then you can insert to your test_array table using INSERT INTO table test_array SELECT 1, array('a','b') from dummy; You can't insert a complex type directly to Hive table.

shishir_saxena4 · ‎03-10-2016

@Kuldeep Kulkarni Interesting question. You can probably enable Ranger Admin HA, then bring another node down. But this will still leave Usersync processes needing to be moved.

shishir_saxena4 · ‎03-09-2016

Thank You @Artem Ervits @Sunile Manjee @Ancil McBarnett. I was able to get my solution using a combination of commands. hdfs dfs -cat $1 | head --bytes 10K > $SAMPLE_FILE java -jar $AVRO_TOOLS_PATH/avro-tools-1.7.7.jar getschema $SAMPLE_FILE > $AVRO_SCHEMA_FILE hdfs dfs -put $AVRO_SCHEMA_FILE $AVRO_SCHEMA_DIR head command needs to be used with --bytes option to get first 10K bytes. Then I used Avro tools to retrieve schema and copied schema back to HDFS.

shishir_saxena4 · ‎03-09-2016

Thanks @Ancil McBarnett @Sunile Manjee. I don't have .avsc file for schema. How can I extract Avro schema for this data ?

shishir_saxena4 · ‎03-09-2016

I have a dataset that is almost 600GB in Avro format in HDFS. Whay is the most efficient way to create a Hive table directly on this dataset ? For smaller datasets, I can move my data to disk, use Avro tools to extract schema, upload schema to HDFS and create Hive table based on that schema. Is there a way to directly extract Avro schema from a dataset in HDFS without writing java code ?

shishir_saxena4 · ‎03-07-2016

@Michael Dennis Uanang Can you please close this issue if it is resolved and open another one ? I am not clear on new issue. Are you able to write messages to topics but suddenly lose access because brokers are not registered with zookeepers ?

Online	Offline
Last Visited	‎02-14-2022 09:06 AM

Member Since	‎02-16-2016 01:09 PM
Last Visited	‎02-14-2022 09:06 AM
Posts	176
Kudos received	196

Cloudera Community

Re: Migrating HDF 2.0 node from standalone to clus...

Re: How to download JSON files from live feed?

Re: Newly added DataNodes won't joining the party

Re: Is there a list of available metrics in Ambari...

Re: HIve not loading escape character (\)

Does nifi HBase ClientService support connecting t...

Re: Capacity planning for NiFi cluster

Capacity planning for NiFi cluster

Re: how to configure Ambari Alert for Ranger Admin

Re: Insert values in Array data type - Hive

Re: Ranger host migration

Re: Is there a way to create Hive table based on A...

Re: Is there a way to create Hive table based on A...

Is there a way to create Hive table based on Avro ...

Re: After upgrading from HDP 2.2 to HDP 2.3, all k...