Skip to main content
janvit04
janvit04
Silver ⭐️⭐️
April 20, 2023
Question

Write /Prepare Fixed width length file in Nifi

I have an input JSON file and I want to convert it to a fixed-width length type file. If the length is less than the required length, trailing spaces should be added to the field.

Here is the sample Input JSON:

 

[
  {
    "orderId": "1234567890",
    "orderName": "Test1"
  },
  {
    "orderId": "12235",
    "orderName": "Test2"
  },
  {
    "orderId": "12236",
    "orderName": "Test3"
  }
]

 


Here is the desired Output:

 

1234567890 Test1     
12235      Test2     
12236      Test3

 

 

 


The condition here is if the length of orderId(in input JSON) is less than 10 then add the required spaces.

Can this be done in Nifi?


I want to have fixed-width length files.

Thanks in advance!







6 replies

DianaTorres
Community Manager
DianaTorres
Community Manager
April 20, 2023

@janvit04 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @cotopaul @SAMSAL @steven-matison  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
cotopaul
cotopaul
Platinum ⭐️
April 21, 2023

hi @janvit04,

Your request is not 100% clear for me. Maybe you could provide some additional information maybe? The output should be a JSON as well or how do you want your output? Are we going to talk about a maximum length of 10 characters all the time? What happens if you encounter 11 characters? All these questions are relevant to fully understand your use case and provide you with some guidance on what to do next in order to achieve what you are trying to achieve. Without any further information, I can tell you that what you are trying is possible but it is not easy at all: you would have to split your json, evaluate each value and add it as an attribute to your flowfile, measure the length of those attributes, route them based on side, modify them and save them afterwards in your desired format. Another solution would be a script (python for example), executed in ExecuteStreamCommand, which reads the content of this file and applies all your necessary rules. Once the processing is done, the file gets sent downstream to the next processor, whatever it would be.

Translate
janvit04
janvit04Author
Silver ⭐️⭐️
April 26, 2023

Hi @cotopaul 
Thanks for your reply.
1. Yes we have a maximum length of 10 characters every time. The characters will be 10 or less than 10 always.
2. I have an incoming json (a list of maps), and I want each of the maps to be written as:
    1. text format.

    2. have fixed width length. If there are 6 characters in the value of that field and the maximum length is 10 then the remaining characters should be filled with spaces.
    3. There is no delimiter between the fields.
3. I have tried using the ConvertRecord processor to read the JSON and convert it to CSV. But there is a problem with CSV, we have to define a delimiter and I don't need any delimiter.

I hope the above information helps you.

Is there any way we can prepare the fixed-width length file considering the above points?

Thanks.

Translate
cotopaul
cotopaul
Platinum ⭐️
April 27, 2023

Well in this case, I see two possible options:
- You write a custom Script (Groovy, Bash, Python, Java, whatever you want) and you execute it in a ExecuteStreamCommand processor. For example in Python you read your data with sys.stdin and write it down the stream with sys.stdout, after doing all the processing your require.
- Another option would be to split each line of your JSON, evaluate the length of each element using NiFi Expression Language and create 10 UpdateAttribute/UpdateRecord to append some white spaced to your data. If you have a large JSON file you might get into trouble, with to many open files and so on.

Translate
DianaTorres
Community Manager
DianaTorres
Community Manager
May 2, 2023

@janvit04 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks

 

Translate
Regards, Diana Torres, Senior Community Moderator
Terms & ConditionsAccessibility statement