Member since
02-04-2016
189
Posts
70
Kudos Received
9
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3466 | 07-12-2018 01:58 PM | |
7227 | 03-08-2018 10:44 AM | |
3323 | 06-24-2017 11:18 AM | |
22315 | 02-10-2017 04:54 PM | |
2081 | 01-19-2017 01:41 PM |
12-05-2022
08:41 AM
@hargav Please create a new community question for your queries around MergeRecord processor. This is the best way to get attention and best for community to have a separate thread for each specific query. I am not clear on your use case for using "cron driven" scheduling with the MergeRecord. This would not be a common thing to do. Best to explain your use case in a new community thread along with sharing your MergeRecord processor configuration. Feel free to @MattWho in the new community post to notify me. Thanks, Matt
... View more
06-09-2021
06:42 AM
you can try below set parameters set hive.vectorized.execution.reduce.enabled=false; and set hive.vectorized.execution.enabled=true;
... View more
01-21-2021
01:16 AM
Hello, can you please help me with a similar script for batch renaming Hadoop files? Thanks!
... View more
07-29-2020
04:00 PM
Did you get a solution to this . I am also getting a communication error . My Nifi Instance and Mysql are on the same linux server .
... View more
05-28-2019
06:08 PM
It took me a while to look in /var/log/messages, but I found a ton of ntpd errors. It turns out that our nodes were having issues getting out to the servers they were configured to use for sync. I switched all the configurations to use a local premise server and restarted everything. I'm hoping that will be the full solution to our issue.
... View more
07-12-2018
01:58 PM
I was able to get this to work by using the insertInto() function, rather than the saveAsTable() function.
... View more
05-24-2018
12:04 PM
Thanks Matt, My issue was firewall related. I'm all set now. Thanks for your help!
... View more
04-10-2018
08:31 PM
Here's what I ended up with: spark.udf.register("getOnlyFileName", (fullPath: String) => fullPath.split("/").last) val df2= df1.withColumn("source_file_name2", callUDF("getOnlyFileName", input_file_name()))
... View more
08-01-2017
11:35 AM
Using your sed approach, this should replace all NULL with empty character sed 's/[\t]/,/g; s/NULL//g' > myfile.csv If there is a chance that NULL is a substring of a value you will need to do the following where ^ is beginning of line and $ is end of line and , is your field delimiter sed 's/[\t]/,/g; s/^NULL,/,/g; s/,NULL,/,,/g; s/,NULL$/,/g;' > myfile.csv Note that if your resultset is large, it is probably best to use Pig on HDFS and not sed (to leverage the parallel processing of hadoop and save yourself a lot of time. Note also: To use empty character as nulls in the actual hive table, use the following in the DDL TBLPROPERTIES('serialization.null.format'='');
... View more