Member since
06-13-2017
49
Posts
3
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
644 | 12-07-2020 10:18 AM | |
726 | 07-29-2017 10:23 AM |
01-15-2021
01:45 AM
It is recommended to check if your cluster computing power is insufficient. Often, such errors occur when memory or CPU resources are insufficient.
... View more
01-15-2021
01:39 AM
Did you know that Nifi was developed in the Java language? Pure Java cannot handle Windows event logs. The method using JNA is recommended. Please refer to the following link. https://code.dblock.org/2010/09/30/jna-reading-windows-event-log-entries-in-java.html
... View more
12-24-2020
04:32 PM
Ambari installation requires either MySQL or postgresql. In general, you only need to install MySQL on the Ambari server host. https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/administering-ambari/content/amb_using_ambari_with_mysql_or_mariadb.html
... View more
12-11-2020
08:36 AM
Please visit the following link. Good luck. https://zeppelin.apache.org/docs/0.8.0/usage/rest_api/interpreter.html#restart-an-interpreter
... View more
12-08-2020
11:38 PM
In my opinion, there seems to be a problem running the interpreter due to insufficient memory. First check if there is insufficient memory by monitoring the system.
... View more
12-07-2020
11:28 AM
I recommend not using nifi and working with the console. Using nifi is not recommended because additional logs are generated. It is recommended to divide and compress the files that you want to move from the console into appropriate sizes and send them. For HDFS, you can use the distcp command. https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html
... View more
12-07-2020
11:17 AM
In my experience, lack of computer resources was the biggest cause. It is always necessary to prepare disks and memory with sufficient space.
... View more
12-07-2020
10:27 AM
HDP2.6 uses java 7 and HDP 3.1.5 uses java 8. If you want to write a program that works for both versions, you can use the compile option to run on java 7.
... View more
12-07-2020
10:18 AM
Check following links~ https://db.rstudio.com/databases/hive/ https://community.rstudio.com/t/how-to-connect-rstudio-server-pro-to-hdp-hive/30804/3
... View more
12-07-2020
10:14 AM
"jps" is the "ps" java version. Most Hadoop processes are developed as Java programs, and the command to view the Java process status is jps. https://docs.oracle.com/javase/7/docs/technotes/tools/share/jps.html
... View more
09-20-2019
10:03 PM
Why don't you use the row_number function? Just CTAS ! ( Create new_Table As Select row_number and other columns ) http://dwgeek.com/apache-hive-rownum-pseudo-column-equivalent.html/
... View more
08-18-2019
03:44 AM
If you want to just study, I recommend the config dfs . permissions . enabled=false . And in my experience, Pentaho Spoon store some info to it's cache dir. You can try like this... - After Pentaho Spoon exit - then delete cache dir - and then restart your job.
... View more
06-20-2019
01:17 PM
I recommend Rest API than Java API, because Java API version problems can make you crazy. https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/HistoryServerRest.html
... View more
06-20-2019
01:08 PM
If fsck results contain missing block with "corrupt" , you'd better run fsck with more options. hdfs fsck [corrupted path] -files -locations -blocks
... View more
10-27-2018
12:24 AM
You can write your processes into script file. test.sh # Run the python to generate JSON file # hive -e "LOAD DATA INPATH '/Project/' OVERWRITE INTO TABLE tb1;" # hive -e "select * from your table" Run test.sh !
... View more
10-23-2018
04:49 AM
Why don't you try with string argument like this?
hive>select qp("hello");
... View more
10-23-2018
04:43 AM
Let me know how you generate the files.
... View more
03-26-2018
11:37 AM
You have to allocate more memory for sandbox. In my case, I allocated 18GB.
... View more
03-26-2018
11:07 AM
and Even though enough mem resource in my server(128GB Ram),
We see the message "Output is truncated to 1000 rows. Learn more about common.max_count" because of zeppelin configuration.
... View more
03-26-2018
11:01 AM
I expect an java out of memory error.
... View more
03-21-2018
12:47 PM
Writing flowfiles to disk will be costly operation as the size of the flow files is huge and there are lots of such flow files. Any alternative solution apart from writing contents of flowfiles to disk? I think you should select the better solution(as I know, there is no best) because both of them have the trade-off. Your questioned solution has no file operation and no writing contents of flowfiles to disk. But, very slo~~~w ! My solution has costly file operation and writing contents of flowfiles to disk. But, Fast !
... View more
03-21-2018
11:51 AM
Yes, You did ! ^^
... View more
03-21-2018
10:30 AM
You can use ExcecuteProcess for SCP or GetSFTP...
... View more
03-21-2018
09:55 AM
I mean ... ListDataBaseTables -> GenerateTableFecth -> Execute SQL -> ConvertRecord(Avro to CSV) -> CSV to MySQL server local file -> ExcecuteProcess(MySQL LOAD DATA... command)
... View more
03-21-2018
08:42 AM
If possible, I recommend MySQL 'LOAD DATA' command instead of putSQL and PutDatabaseRecord. NiFi save csv --> MySQL local CSV --> MySQL LOAD DATA ... MySQL LOAD DATA is dramatically fast !
... View more
01-31-2018
10:40 AM
SolrCloud already enabled. Sometimes rager_audit collection created, but not for me.
... View more
01-31-2018
10:16 AM
A few days ago, I have suffered from exactly same issue. ZK, Solr, Ranger all restarted but there is no collection. Anybody knows the solution?
... View more
01-31-2018
10:02 AM
I'm sorry about my missunderstand. Zeppelin does not support HA, so you need to manually setup like knox. https://community.hortonworks.com/questions/24961/how-to-configure-ha-for-knox-gateway-using-any-loa.html
... View more