Skip to main content
NafeesShaikh93
NafeesShaikh93
Silver ⭐️⭐️
March 27, 2023
Question

how to send logs from nifi to graylog

Hello Team,

 

I am trying to send logs to the gaylog server from Nifi for the transfer of each file log.

could you please help me with the stepwise procedure to do so?

Also i have been trying to copy few files from remote server 1 to 2 and then the same files from 1 will be moved to 3 with the logs of each files in getting pushed to graylog.

 

can you please help me with the step wise procedure to perform this action.

 

i have been struggling from weeks to do above two task. 

7 replies

VidyaSargur
Community Manager
VidyaSargur
Community Manager
March 27, 2023

@NafeesShaikh93, Welcome to our community! To help you get the best possible answer, I have tagged in our Nifi experts @cotopaul @SAMSAL @BennyZ @MattWho @steven-matison who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
NafeesShaikh93
NafeesShaikh93Author
Silver ⭐️⭐️
March 27, 2023

Thank you so much vidyasagar for responding back.

As a latro member, we are planning to use nifi for file transmission between one server to another remote server.
i have done the testing of file transmission using getsftp and putsftp processors. Now we are willing to get the logs of these files those have travelled from source to destination server. We want to store those logs into graylog application so that we can observe it over there. and we have tried Alot to push the logs to there, but so far we are not successful in this task.
could you please help us with the proper stepwise procedure to complete this job 

Translate
cotopaul
cotopaul
Platinum ⭐️
March 27, 2023

hi @NafeesShaikh93,

What logs are you trying to push into Graylog? The logs generated by NiFi when transferring the files from Source to Target, or are you talking about other logs?

 

What have you tried and why did it not succeed? Let's start with that, as your Flow Logic was built specific for your use case and somebody else's logic might not suite your needs.

Translate
NafeesShaikh93
NafeesShaikh93Author
Silver ⭐️⭐️
March 27, 2023

Hello Cotopaul,

 

i am trying to push the logs of file information, like:

1- filename:

2- file type

3- size

4- source

5- destination

6- size at source

7- size at destination.

 

general category logs.

 

would it be fine with you to hop in a call to have more clarification on this topic.

i may be able to explain you in a better way.

 

NafeesShaikh93_0-1679912391442.png

 

 

 

 

 

Translate
cotopaul
cotopaul
Platinum ⭐️
March 27, 2023

Now, based on your Flow Logic, where do you encounter problems and what are those problems?


In addition, how do you retrieve dynamically the destination and the size at destination? As far as I know, PutFile does not write those attributes so you will have to manually define them in your Flow and I do not see any UpdateAttribute or UpdateContent on your canvas. The size, the file type, the filename and the source are written by default when the file is extracted and your can further use them with the help of the NiFi Expression Language.

Translate
NafeesShaikh93
NafeesShaikh93Author
Silver ⭐️⭐️
March 27, 2023
  • well this is certainly new for me. I think it will be better if we could connect for 30 mints in our free time to have a clear cut understanding of what needs to be done to fix it.
Translate
Terms & ConditionsAccessibility statement