Write /Prepare Fixed width length file in Nifi

[#1]

I have an input JSON file and I want to convert it to a fixed-width length type file. If the length is less than the required length, trailing spaces should be added to the field.

Here is the sample Input JSON:

[
  {
    "orderId": "1234567890",
    "orderName": "Test1"
  },
  {
    "orderId": "12235",
    "orderName": "Test2"
  },
  {
    "orderId": "12236",
    "orderName": "Test3"
  }
]

Here is the desired Output:

1234567890 Test1     
12235      Test2     
12236      Test3     

The condition here is if the length of orderId(in input JSON) is less than 10 then add the required spaces.

Can this be done in Nifi?

I want to have fixed-width length files.

Thanks in advance!

[#2]

@janvit04 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @cotopaul @SAMSAL @steven-matison  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

hi @janvit04,

Your request is not 100% clear for me. Maybe you could provide some additional information maybe? The output should be a JSON as well or how do you want your output? Are we going to talk about a maximum length of 10 characters all the time? What happens if you encounter 11 characters? All these questions are relevant to fully understand your use case and provide you with some guidance on what to do next in order to achieve what you are trying to achieve. Without any further information, I can tell you that what you are trying is possible but it is not easy at all: you would have to split your json, evaluate each value and add it as an attribute to your flowfile, measure the length of those attributes, route them based on side, modify them and save them afterwards in your desired format. Another solution would be a script (python for example), executed in ExecuteStreamCommand, which reads the content of this file and applies all your necessary rules. Once the processing is done, the file gets sent downstream to the next processor, whatever it would be.

[#4]

Hi @cotopaul 
Thanks for your reply.
1. Yes we have a maximum length of 10 characters every time. The characters will be 10 or less than 10 always.
2. I have an incoming json (a list of maps), and I want each of the maps to be written as:
    1. text format.

    2. have fixed width length. If there are 6 characters in the value of that field and the maximum length is 10 then the remaining characters should be filled with spaces.
    3. There is no delimiter between the fields.
3. I have tried using the ConvertRecord processor to read the JSON and convert it to CSV. But there is a problem with CSV, we have to define a delimiter and I don't need any delimiter.

I hope the above information helps you.

Is there any way we can prepare the fixed-width length file considering the above points?

Thanks.

[#5]

Well in this case, I see two possible options:
- You write a custom Script (Groovy, Bash, Python, Java, whatever you want) and you execute it in a ExecuteStreamCommand processor. For example in Python you read your data with sys.stdin and write it down the stream with sys.stdout, after doing all the processing your require.
- Another option would be to split each line of your JSON, evaluate the length of each element using NiFi Expression Language and create 10 UpdateAttribute/UpdateRecord to append some white spaced to your data. If you have a large JSON file you might get into trouble, with to many open files and so on.

[#6]

Hi @cotopaul 
Thanks for your reply. I am familiar with Jolt (a json to json transformation library). I have been thinking to add the required padding using the funmctions in Jolt and then use the FreeFormTextRecordSetWriter controller service.
This service take the name of the key in Json and prepare the file containing only the value.
It also keeps the padding added in the previous Jolt Transform.

I think using 10 UpdateAttribute will be tough and I have multiple fields that need the required padding/empty spaces.

Thank you for your answers!

[#7]

@janvit04 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks