In Nifi custom processor configuring some attributes as expressions to read values from variable registry, but it is reading expression itself instead of reading its value. Example Code as below. protected static final PropertyDescriptor HOST = new PropertyDescriptor.Builder() .name("Feed URL") .description("Feed Service Host") .required(true) .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY) .addValidator(StandardValidators.NON_EMPTY_VALIDATOR) .build();   @Override public void onTrigger(final ProcessContext context, ProcessSessionFactory sessionFactory) throws ProcessException { final ComponentLog logger = getLogger(); try { sessionFactoryReference.compareAndSet(null, sessionFactory); String hostValue = context.getProperty(HOST).toString(); // } } So from the above code if we print 'hostValue' its printing the expression as ${Feed-URL} instead of printing value of expression. ... View more

Unable to persist merged avro fiels to HDFS using PutHDFS processor in nifi with compression codec as 'SNAPPY'. Getting below exception.   o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=cc59cc35-87b1-3868-8e40-a311596c48d2] Failed to write to HDFS due to java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63) at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150) at org.apache.hadoop.io.compress.CompressionCodec$Util.createOutputStreamWithCodecPool(CompressionCodec.java:131) at org.apache.hadoop.io.compress.SnappyCodec.createOutputStream(SnappyCodec.java:102) at org.apache.nifi.processors.hadoop.PutHDFS$1$1.process(PutHDFS.java:318) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2235) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2203) at org.apache.nifi.processors.hadoop.PutHDFS$1.run(PutHDFS.java:305) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1942) at org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:235) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ... View more

Hi All, I am trying to persist data into an S3 bucket through Nifi PutS3Object processor, and sometimes I'm getting the below error Please help me with the possibilities to fix this issue. ERROR [Timer-Driven Process Thread-3967] o.a.nifi.processors.aws.s3.PutS3Object PutS3Object[id=e1ed3d31-f12a-1572-b913-4595cd91cf49] Failed to put StandardFlowFileRecord[uuid=2724806e-3b0d-48d5-a806-eb02e97d40c7,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1536017934845-69432, container=default, section=824], offset=1001138, length=827387],offset=366284,name=5410310069030717,size=411944] to Amazon S3 due to com.amazonaws.SdkClientException: Unable to execute HTTP request: Socket is closed: {} com.amazonaws.SdkClientException: Unable to execute HTTP request: Socket is closed at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:972) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:676) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:650) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:633) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$300(AmazonHttpClient.java:601) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:583) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:447) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4137) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1685) at org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:474) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2175) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2145) at org.apache.nifi.controller.repository.BatchingSessionFactory$HighThroughputSession.read(BatchingSessionFactory.java:192) at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:417) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.SocketException: Socket is closed ... View more