About aervits

aervits · ‎02-13-2016

@Rushikesh Deshmukh please provide load statement

aervits · ‎02-12-2016

sc.parallelize(records).toDF().write.format("orc").save("people") that method was refactored. There's a new way of writing ORC files. Convert your RDD to DataFrame with toDF() and then write it out as above.Try to use later versions of Spark. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_spark-guide/content/ch_orc-spark.html

aervits · ‎02-12-2016

@Michel Sumbul the feature is called rquest throttling. It is available in HBase 1.1 and thus in HDP 2.3. More info https://blogs.apache.org/hbase/entry/the_hbase_request_throttling_feature The throttle can then be set from the HBase shell, like so: hbase> set_quota TYPE => THROTTLE, USER => 'uname', LIMIT => '100req/sec' hbase> set_quota TYPE => THROTTLE, TABLE => 'tbl', LIMIT => '10M/sec' hbase> set_quota TYPE => THROTTLE, NAMESPACE => 'ns', LIMIT => 'NONE' our in-house engineer responded in a similar post recently https://community.hortonworks.com/questions/1821/hbase-quota-management.html

aervits · ‎02-12-2016

@Satish S your issue is all the \ in your query, you use \ outside of quotes, inside the quotes it is interpreted as SQL. That's why your MySQL is complaining. Remove all that and do use '$CONDITIONS', make sure to read the note about wrapping $CONDITIONS with single quotes or double because it does make a difference.

aervits · ‎02-12-2016

@Saurabh Kumar https://github.com/evidens/json2csv

aervits · ‎02-12-2016

@Darpan Patel flume is one directional, you can't push data out of hdfs to fs. We have Apache NiFi that you can leverage for moving logs to and from hdfs.

aervits · ‎02-12-2016

@bsainiIn the first phase, we have enabled NFSv3 interface access to HDFS. This is done using NFS Gateway, a stateless daemon, that translates NFS protocol to HDFS access protocols as shown in the following diagram. Many instances of such daemon can be run to provide high throughput read/write access to HDFS from multiple clients. As a part of this work, HDFS now has a significant functionality that supports inode ID or file handles, that was done in Apache JIRA HDFS-4489. Source: http://hortonworks.com/blog/simplifying-data-management-nfs-access-to-hdfs/

aervits · ‎02-11-2016

@PJ Moutrie look at git page last source code check-in is from 7 days ago, project is chugging along. Wait for new version https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git

aervits · ‎02-11-2016

@Roberto Sancho https://github.com/twitter/elephant-bird/wiki/Hadoop-2.x-Support So you might have an old jar? Still try built-in jsonloader instead but confirm my answer.

aervits · ‎02-11-2016

@Roberto Sancho I'm guessing your elephant bird library is compiled for Hadoop 1.x line. See if they offer Hadoop 2 compiled jar. Why not use https://pig.apache.org/docs/r0.15.0/func.html#jsonloadstore ?

Online	Offline
Last Visited	‎08-15-2019 06:35 AM

Member Since	‎10-01-2015 11:46 AM
Last Visited	‎08-15-2019 06:35 AM
Posts	3,933
Kudos received	1074

Cloudera Community

Re: Where can I get latest resource_management.c...

Re: How to Kerberize Flume?

Re: Load Hive Table form Pig Output File.

Re: HDP 2.6 Cluster Issues with Hive Metastore

Re: which HDP release will storm 1.1.0 be packaged...

Re: Single records of a file split into multiple?

Re: saveAsOrcFile is not a member of org.apache.sp...

Re: Limit ressource allocate to HBase query based ...

Re: Free form query in Sqoop Import with WHERE cla...

Re: Is there a good way to convert curl api json o...

Re: How to redirect the access logs and other logs...

Re: Multiple NFS Gateways for HDFS

Re: What is the current status of Hortonworks' Dat...

Re: PIG ERRO execution

Re: PIG ERRO execution