Member since
05-07-2018
32
Posts
0
Kudos Received
0
Solutions
08-21-2020
02:27 PM
@BohdanSemchuk You can use the attached modified jar which helps overcome the issue that you have come across. Download the jar and copy it to the hdfs location hdfs://user/hive/ and provide hive:hadoop permission with 777 permission. Download Link: https://drive.google.com/file/d/1o-DtrY4bNLaUkDJfOJAjchNtVcL4P64-/view?usp=sharing Steps to add the jar: 1. Login to the beeline-shell 2. execute the below command, add jar hdfs://com:8020/user/hive/hivexmlserde-1.0.5.3_updated.jar; 3. Create the table as in below, Sample table: ========== CREATE EXTERNAL TABLE `ssp`( `name` array<string> COMMENT 'from deserializer', `value` array<string> COMMENT 'from deserializer', `numvalue` array<double> COMMENT 'from deserializer', `id` array<int> COMMENT 'from deserializer', `start_date` array<string> COMMENT 'from deserializer', `end_date` array<string> COMMENT 'from deserializer') ROW FORMAT SERDE 'com.ibm.spss.hive.serde2.xml.XmlSerDe' WITH SERDEPROPERTIES ( 'column.xpath.end_date'='/row/end/text()', 'column.xpath.id'='/row/id/text()', 'column.xpath.name'='/row/column/name/text()', 'column.xpath.numvalue'='/row/column/number/text()', 'column.xpath.start_date'='/row/start/text()', 'column.xpath.value'='/row/column/string/text()') STORED AS INPUTFORMAT 'com.ibm.spss.hive.serde2.xml.XmlInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION 'hdfs://com:8020/user/hive' TBLPROPERTIES ( 'transient_lastDdlTime'='1566907280', 'xmlinput.end'='</row>', 'xmlinput.start'='<row'); 4. Running the `show tables` populates this table without any issue. Try implementing the above recommendation and help us know if you were able to alleviate the issue.
... View more
08-17-2020
01:48 AM
The issue could be associated with the corrupt permissions in NM local-dir (usercache) are causing the shuffle fetch failure which may occur due to the below reasons, 1.Disk failure cause the fetch failure 2. Network issue causes the fetch failure This can be overcome by carrying out the below steps in the NodeManger installed nodes. * Stop the NM instance on the node * Remove YARN NM local-dir path - sudo rm -rf /JBOD_D${i}/hadoop/cdh/yarn/nm/*" where ${i} is the iteration over the disks that each participate as NM local-dir storage - the idea is to leave /JBOD_D${i}/hadoop/cdh/yarn/nm in place on each disk, but remove everything below that point. * Restart the NM instance and it will recreate the structure with the appropriate permissions. Once you have implemented the above solution and if it continues to working well, then please spare few minutes to hit the "Accept as Solution"
... View more
08-13-2020
09:08 AM
@torafca5 Could you please try downloading the jar from the below link, http://www.congiu.net/hive-json-serde/1.3.8/hdp23/json-serde-1.3.8-jar-with-dependencies.jar Once the jar is downloaded, move the jar to the location /usr/hdp/3.0.1.0-187/hive/lib. Please place the jar on all the nodes hosting Hive services. Also, please make sure you are not using LLAP(HiveserverInteractive) to connect to the hive. add jar command does not work with LLAP. implementing the above recommendation should help overcome this issue.
... View more
08-12-2020
03:36 AM
@Yukti You can get the mount point related information in the below doc under the topic "File System Requirements", https://docs.cloudera.com/cdp/latest/release-guide/topics/cdpdc-os-requirements.html
... View more
08-02-2019
08:17 PM
@sivakkannan muthukumar Unzip this file , you will get a folder named deploy-scripts, navigate to the folder and look for a script named ‘docker-deploy-hdp265.sh’. Execute the script as ‘sh docker-deploy-hdp265.sh’. This would take around < 15 mins. Once the scripts completes successfully , you could login to ambari UI. Before that map the localhost to sandbox-hdp.hortonworks.com , so that once the script completes ambari can be accessed from sandbox-hdp.hortonworks.com:8080
... View more
05-30-2018
12:56 PM
@GintaThis actually helped to resolve the issue.
... View more