Member since
06-07-2016
923
Posts
322
Kudos Received
115
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2764 | 10-18-2017 10:19 PM | |
3123 | 10-18-2017 09:51 PM | |
12307 | 09-21-2017 01:35 PM | |
1022 | 08-04-2017 02:00 PM | |
1333 | 07-31-2017 03:02 PM |
11-01-2016
04:29 PM
2 Kudos
@Hanife Shaik I agree with @Constantin Stanca. This is an HBase use case. And yes HBase is created for transactional use cases and exactly for use cases where you will scale to 10's of TBs. 30 TB to begin with is large for traditional systems but sclaing makes it even more compelling on why you should go with HBase. Scaling Oracle, MySQL or any other traditional system will bring in traditional challenges or manual sharding and increase the complexity for operations team to manage. HBase on the other hand will provide you automatic sharding, automatic failover to new nodes and scaling by simply adding new nodes and easy online maintenance.
... View more
10-31-2016
01:42 PM
@AnjiReddy Anumolu In that case, what's the value of oozie.service.WorkflowAppService.system.libpath? https://oozie.apache.org/docs/3.2.0-incubating/oozie-default.xml
... View more
10-31-2016
04:50 AM
@AnjiReddy Anumolu I think you need the following (you are missing "use"). oozie.use.system.lib.path =true
... View more
10-31-2016
04:24 AM
1 Kudo
@Hoang Le I am assuming this is just some sandbox type deployment where you are just trying to experiment and learn Hadoop. Feel free to distribute process evenly on both nodes and set replication factor to 2, making both nodes data nodes. This means, one of your node wil serve as both data node as well as namenode. You cannot have any meaningful Hadoop deployment on two nodes. Not even for POC/evaluation purposes. You might use these two nodes to install five VMs on it and in that case have two masters and three data nodes.
... View more
10-27-2016
04:28 PM
@Sami Ahmad Flume is simply moving your data from source to target. In this case from twitter to HDFS. I believe twitter sends JSON records. This means file being written is JSON format. Flume is not altering your file format. It is only moving data.
... View more
10-27-2016
04:04 PM
@Michael Belostoky Do you have HDP or you just have downloaded HDFS and Hive from Apache's website? If you are using individual Apache components, then you can download and upgrade Hive by following instructions on Hive Website. You will have to first upgrade Hive Metastore by running scripts to upgrade schema and then upgrade the Hive installation. https://github.com/apache/hive If you have HDP, then you have to upgrade HDP, not just Hive. Think about how Hortonworks provides value to it's customers. One of the main benefits that HDP customers get is that all individual Apache Hadoop Ecosystem projects are tested together and packaged in each HDP version. This ensures that Hive 1.x that's bundled in HDP x.x is thoroughly tested with every component it is expected to work with. So, Hive in each HDP version is tested with HDFS, HBase, Spark, Pig, Sqoop and other projects. If you upgrade only Hive and nothing else then Hortonworks cannot support it because our support and engineering rely on all the integration testing that went into each supported release. So, if you have HDP then the way to upgrade is to upgrade HDP.
... View more
10-18-2016
11:50 PM
Those are Ambari server logs. If your namenode is down, then can you please share namenode logs? Should be under /var/log/hadoop
... View more
10-18-2016
11:07 PM
@kiran thella Which file is this log from? Is this the file under /var/log/hadoop/ folder? Thanks
... View more
10-18-2016
08:53 PM
1 Kudo
@RajuKV Can you please elaborate on "when we create queue it's creating below default value"? Creating what below default value? I think the property you are looking for is yarn.scheduler.capacity.root.support.services.minimum-user-limit-percent But please confirm and see the following link. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_yarn_resource_mgt/content/setting_user_limits.html
... View more
10-14-2016
02:13 AM
@Ashish Vishnoi Try the following sqoop export --connect jdbc:teradata://xxxxx/database=xx --connection-manager org.apache.sqoop.teradata.TeradataConnManager --username xx --password xx --export-dir /test/xxx/xx/temp/teradata1 --table test --input-fields-terminated-by ',' -m 2 --input-null-string '\n' --input-null-non-string '\n'
... View more