Member since
04-29-2021
35
Posts
0
Kudos Received
0
Solutions
09-02-2021
02:28 AM
Hi @MattWho What would be the regular expression if I have to put the selection condition on field three of the data. the field I put in bold. I want to select the lines with the 1995 only. |226789|23-Feb-1996|1995|0|1|1|0|0|0|1|0|0|0|0|0|0|0|0|0|0|0|0 |226780|08-Mar-1996|1996|0|0|0|0|0|1|0|0|0|0|0|0|0|0|0|0|0|0|0 |222507|01-Jan-1995|1995|0|0|0|0|0|1|0|0|0|0|0|0|0|0|0|0|0|0|0 |22308|01-Jan-1995|1995|0|1|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0 |222707|01-Jan-1995|1995|0|1|0|0|0|0|0|0|1|0|0|0|0|0|0|0|1|0|0
... View more
09-01-2021
08:50 AM
Hi Everyone, I use ListSFTP and FetchSFTP to collect the files that lines. I want to filter the files based on the third field. I want to collect the files that have the year 1995 only in the lines.
|226789|23-Feb-1996|1995|0|1|1|0|0|0|1|0|0|0|0|0|0|0|0|0|0|0|0
|226780|08-Mar-1996|1996|0|0|0|0|0|1|0|0|0|0|0|0|0|0|0|0|0|0|0
|222507|01-Jan-1995|1995|0|0|0|0|0|1|0|0|0|0|0|0|0|0|0|0|0|0|0
|22308|01-Jan-1995|1995|0|1|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0
|222707|01-Jan-1995|1995|0|1|0|0|0|0|0|0|1|0|0|0|0|0|0|0|1|0|0
... View more
Labels:
- Labels:
-
Apache NiFi
08-17-2021
12:52 AM
Hi Everyone, I have collected data in ASN1 format, I want to convert them using Apache Nifi. Which Nifi processors can help me?
... View more
Labels:
- Labels:
-
Apache NiFi
07-16-2021
07:12 AM
Hi @stevenmatison, I don't see any data in the cassandra database. I don't see any errors either. Here are the screenshots.
... View more
07-15-2021
09:20 AM
Hi @stevenmatison, The goal is to be able to extend to several servers, in order to collect the files in a parallel way to count the number of lines and the number of files to store in a database each day.the type of data contained in the data file |226789|23-Feb-1996|0|1|1|0|0|0|1|0|3|0|0|6|0|2|0|0|6|9|7 |226780|08-Mar-1996|4|0|2|0|0|1|0|0|0|3|0|0|0|0|0|0|0|0|8 |222507|01-Jan-1995|0|0|5|0|0|1|0|0|0|0|6|0|0|0|0|0|0|0|5 |22308|01-Jan-1995|0|1|8|0|0|6|0|0|0|2|0|4|0|0|0|6|0|0|4 |222707|01-Jan-1995|0|1|0|0|5|0|0|0|1|0|0|6|0|0|7|0|1|0|2 I collect files I count the number of files and the number of lines in the files. I want to store these values in a database. I installed apache cassandra and created a database and a table. when I insert the number of lines and documents, I check my table and I don't see any data. GetFile-->MergeContent-->CountText-->ReplaceText-->PutCassandraRecord. The processors that make up the nifi flowfile I set up. flowfile (all processors) PutCassandraRecord I want to store the number of files obtained and the number of lines obtained in cassandra if possible. So concerning the configuration of PutcassandraRecord you can make me a proposal.
... View more
07-14-2021
11:38 AM
Hi Everyone, The goal is to be able to extend to several servers, in order to collect the files in a parallel way to count the number of lines and the number of files to store in a database each day. I collect files I count the number of files and the number of lines in the files. I want to store these values in a database. I installed apache cassandra and created a database and a table. when I insert the number of lines and documents, I check my table and I don't see any data. GetFile-->MergeContent-->CountText-->ReplaceText-->PutCassandraRecord I want to check the data entered in the database table but I don't see any data. here is the configuration of the PutcassandraRecord processor. PutCassandraRecord
... View more
Labels:
- Labels:
-
Apache MiNiFi
-
Apache NiFi
06-21-2021
11:03 PM
Hi @ChethanYM, I just use hadoop. I followed This tutorial , but I did the installation steps of hadoop only. https://www.youtube.com/watch?v=71EQblrUPRM&t=1375s
... View more
06-21-2021
10:23 AM
Hi @ChethanYM, I installed hadoop-2.7.1 and it's during the installation that I created a hdfs user. but the hdfs command started working when I finished installing hadoop. here is the output of the command #ls -lrth | grep hdfs. the output does not give anything. Yes I can reinstall hadoop. [root@MASTER alternatives]# ls -lrth | grep hdfs [root@MASTER alternatives]# [root@MASTER alternatives]# hadoop version Hadoop 2.7.1 Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r 20fe5304904fc2f5a18053c389e43cd26f7a70fe Compiled by vinodkv on 2017-06-02T06:14Z Compiled with protoc 2.5.0 From source with checksum 60125541c2b3e266cbf3becc5bda666 This command was run using /usr/local/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar [root@MASTER alternatives]#
... View more
06-21-2021
09:05 AM
Hi @ChethanYM, I set the PATH like below and try: When I run this command, I get the same error. ## PATH=$PATH:$HADOOP_HOME/bin ## source ~/.bash_profile [root@MASTER ~]# which hdfs /usr/local/hadoop-2.7.1/bin/hdfs [root@MASTER ~]# sudo -u hdfs hdfs dfsadmin -safemode leave sudo: hdfs: command not found [root@MASTER ~]# id hdfs uid=30008(hdfs) gid=30009(hdfs) groups=30009(hdfs) [root@MASTER ~]#
... View more
06-21-2021
01:59 AM
Hi @RohitPathak, I tried with root/sudo user but still error, [root@MASTER ~]# sudo -u hdfs hdfs dfsadmin -safemode leave sudo: hdfs: command not found [root@MASTER ~]#
... View more