Member since
06-13-2017
49
Posts
3
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
645 | 12-07-2020 10:18 AM | |
729 | 07-29-2017 10:23 AM |
01-28-2022
12:54 AM
Hello, I'm a learner & i would like to use the method you made mentioned here to collect logs in a remote server & send to Nifi. Please, can you put me through because i have been battling with how to build a msi before the real implementation. Thank you so much.
... View more
01-15-2021
01:45 AM
It is recommended to check if your cluster computing power is insufficient. Often, such errors occur when memory or CPU resources are insufficient.
... View more
12-13-2020
04:32 AM
Hi As per the link, https://zeppelin.apache.org/docs/0.8.0/usage/rest_api/interpreter.html#restart-an-interpreter To restart zeppelin interpreter we need to know interpreter id , which we are unable to find . https://zeppelin.apache.org/docs/0.8.0/usage/rest_api/interpreter.html#restart-an-interpreter Also interpreter restart can only be done with admin user, hence that also needs to be passed via script. Thanks ASIF.
... View more
12-07-2020
12:47 PM
Thank you but both version support java 8, no problem on this side. See : https://supportmatrix.hortonworks.com/ The main problems are the new HBase version with HDP 3.1.5 and the new Hive version with the addition of the Hive Warehouse Connector on HDP 3.1.5.
... View more
12-07-2020
11:28 AM
I recommend not using nifi and working with the console. Using nifi is not recommended because additional logs are generated. It is recommended to divide and compress the files that you want to move from the console into appropriate sizes and send them. For HDFS, you can use the distcp command. https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html
... View more
12-07-2020
11:17 AM
In my experience, lack of computer resources was the biggest cause. It is always necessary to prepare disks and memory with sufficient space.
... View more
12-07-2020
10:18 AM
Check following links~ https://db.rstudio.com/databases/hive/ https://community.rstudio.com/t/how-to-connect-rstudio-server-pro-to-hdp-hive/30804/3
... View more
12-07-2020
10:14 AM
"jps" is the "ps" java version. Most Hadoop processes are developed as Java programs, and the command to view the Java process status is jps. https://docs.oracle.com/javase/7/docs/technotes/tools/share/jps.html
... View more
08-13-2020
12:42 AM
While starting Hortonworks sandbox it gets stuck on "extracting and loading the hortonworks sandbox..." And after some times it shows the message of critical error or sometimes it says "your system has ran into an error we'll restart it"
... View more
05-28-2020
12:55 AM
Works for me. +1 on enabling SolrCloud. (Ambari -> Ranger -> Configs -> Ranger Audit -> Audit to SolrCloud: ON)
... View more
08-20-2019
11:20 PM
Thanks, youngick I set dfs . permissions . enabled=false in ambari and restarted hdfs, then closed Spoon, cleared PDI caches in ...\data-integration\system\karaf\caches\ and in ~\.pentaho\caches, started Spoon back and try job again with the same error. Nothing changed. Also I'm tried such hard thing as namenode format as described here: https://stackoverflow.com/questions/15571584/writing-to-hdfs-could-only-be-replicated-to-0-nodes-instead-of-minreplication?rq=1 This method didn't help either. As this is all a study case only, I'll come to it back later.
... View more
03-26-2018
11:07 AM
and Even though enough mem resource in my server(128GB Ram),
We see the message "Output is truncated to 1000 rows. Learn more about common.max_count" because of zeppelin configuration.
... View more
08-08-2017
10:32 AM
Finally Worked for Me and did some work around. Steps as below. 1) Create Temp table with same columns. 2) Overwrite table with required row data. 3)Drop Hive partitions and HDFS directory. 4)Insert records for respective partitions and rows. 5) verify the counts. 1) hive> select count(*) from emptable where od='17_06_30' and ccodee=!'123';
OK
27
hive> select count(*) from emptable where od='17_06_30' and ccodee='123';
OK
7
hive>show create table emptable_tmp; :- Note hdfs location
2)Create table and overwrite with required partitioned data
hive> CREATE TABLE `emptable_tmp`(
'rowid` string,PARTITIONED BY (`od` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.SequenceFileInputFormat';
hive> insert into emptable_tmp partition(od) select * from emptable where od='17_06_30' and ccodee!='123';
Time taken for adding to write entity : 1
Partition database.emptable_tmp{od=17_06_30} stats: [numFiles=20, numRows=27,totalSize=6216,rawDataSize=5502]
OK
3) Drop Partitions from Hive and HDFS directory as well, as this is External table.
hive> alter table emptable drop partition(od='17_06_30');
Dropped the partition od=17_06_30
OK
Time taken: 0.291 seconds
HDFS partition deletion
#hdfs dfs -rm -r /hdfs/location/emptable/ods='17_06_30'
4) Insert data for that partition only.
hive> insert into emptable partition(od) select * from emptable_tmp;
Partition database.emptable{ds=17_06_30} stats: [numFiles=66, numRows=20, totalSize=5441469982, rawDataSize=]
OK
Time taken: 27.282 seconds
5) Verifying the counts on partitions and respective rows data
1) hive> select count(*) from emptable where od='17_06_30' and ccodee=!'123';
OK
27
hive> select count(*) from emptable where od='17_06_30' and ccodee='123';
OK
0
... View more
07-31-2017
07:53 PM
http://127.0.0.1:18888 --> http://127.0.0.1:8888 and ambari url is http://127.0.0.1:8080
... View more
07-31-2017
01:53 PM
HDP 2.6 sandbox file size is so big. I recommend you should use download manager apps(ex. Internet Download Manager) with this link 'https://downloads-hortonworks.akamaized.net/sandbox-hdp-2.6/HDP_2.6_virtualbox_05_05_2017_14_46_00_hdp.ova'.
... View more