Skip to main content
Techie123
Techie123
Silver ⭐️⭐️⭐️
November 8, 2022
Solved

Attribute Value length Limit In Nifi

Hi, 

 

I have the requirement where I am converting the base64 Image content to attribute and after that I am preparing the payload including the base64 Image content. But the problem is while converting the base64 Image content (approx. 6-7 MB size ) it is getting truncated in the attribute Value. 

 

Can you please tell me what is the length limit of attribute value in NIFI.

Appreciate your help. Thank you.

Faerballert
FaerballertAnswer
Gold ⭐️
November 8, 2022

Ok, there are 2 cases.

 

1. 

You use Base64EncodeContent to encode your image and after that with ExtractText there you write the content of your flowfile to an attribute.

There you have to set the property Maximum Capture Group Length to your desired length. I would prefer that you also set Maximum Buffer Size to ~10 MB that the process gets fully buffered.

 

2. 

Use Processor ExecuteStreamCommand 

Properties:

Command Arguments: -w 0

Command Path: base64

Output Destination Attribute: attribute.base64

Max Attribute Length: your desired length

 

If you dont work with one of those 2 cases, then please describe your flow 🙂  

 

View in thread

7 replies

Faerballert
Faerballert
Gold ⭐️
November 8, 2022

Hello,

 

the maximal value length of an attribute is 2147483647 (max Integer)

 

Greetings

Translate
    Techie123
    Techie123Author
    Silver ⭐️⭐️⭐️
    November 8, 2022

    Hi @Faerballert ,

     

    Thanks for the reply. Can we increase the length of an attribute. If so what needs to be done here?

     

     

    Translate
    Faerballert
    FaerballertAnswer
    Gold ⭐️
    November 8, 2022

    Ok, there are 2 cases.

     

    1. 

    You use Base64EncodeContent to encode your image and after that with ExtractText there you write the content of your flowfile to an attribute.

    There you have to set the property Maximum Capture Group Length to your desired length. I would prefer that you also set Maximum Buffer Size to ~10 MB that the process gets fully buffered.

     

    2. 

    Use Processor ExecuteStreamCommand 

    Properties:

    Command Arguments: -w 0

    Command Path: base64

    Output Destination Attribute: attribute.base64

    Max Attribute Length: your desired length

     

    If you dont work with one of those 2 cases, then please describe your flow 🙂  

     

    Translate
      Techie123
      Techie123Author
      Silver ⭐️⭐️⭐️
      November 9, 2022

      Hi @Faerballert , I am following the first flow but while converting content of flowfile to an attribute using ExtractText the value of the attribute is getting truncated . Is there any way that we can resize the image in NiFi? I checked for Resize image processor but its not available.

      Can you please suggest.    

      Translate
      Faerballert
      Faerballert
      Gold ⭐️
      November 9, 2022

      @Techie123 can you show the properties of the processor?

      Can you maybe check if the base64 string length is for real longer than the max integer value?

       

      With ExecuteStreamCommand you can resize your image.

       

      Attributes should actually be kept small, so it's not the optimal solution either.
      Maybe you can also define the workflow a bit, for which you need the string in an attribute - maybe you can work around that?!

      Translate
        Techie123
        Techie123Author
        Silver ⭐️⭐️⭐️
        November 9, 2022

        Hi @Faerballert ,

         

        I am converting the base64 image content to attribute because i am preparing the payload for the same.

        Techie123_0-1667982349471.png

        I have attached the snapshot of the payload. file_content.0 is nothing but the content of the file that i have converted into the attribute using ExtractText processor and using it into the payload 

         

        Translate
        Terms & ConditionsAccessibility statement