Member since 
    
	
		
		
		06-16-2020
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                55
            
            
                Posts
            
        
                14
            
            
                Kudos Received
            
        
                5
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 1814 | 10-23-2024 11:21 AM | |
| 1649 | 10-22-2024 07:59 AM | |
| 1639 | 10-22-2024 07:37 AM | |
| 1029 | 10-21-2024 09:25 AM | |
| 3024 | 06-16-2023 07:23 AM | 
			
    
	
		
		
		02-27-2025
	
		
		07:38 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @upadhyayk04 So for Ranger Admin, I configured it to use LDAP and it wasn't enough to login to the UI in version 2.5.0. Once I configured both Ranger Admin and Ranger Usersync it worked.     I am under the same assumption as you. I thought just configuring the Ranger Admin to use LDAP was enough but in this case it wasn't. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-20-2025
	
		
		06:11 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @steven-matison - Sorry but where do you see that it is required?    To me, setting the LDAP configurations for Ranger Admin would allow logging into the Ranger Admin UI via an LDAP source and Ranger Usersync was primarily for syncing Users/Groups to be used in policies.   So why would Ranger Usersync be needed to login to the Ranger Admin UI?  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-12-2025
	
		
		01:20 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I am running Ranger Admin 2.4.0. Right now I have it synced with LDAP and am able to login to the Ranger Admin UI via my LDAP username and password. When I try upgrading to Ranger Admin 2.5.0 using the same configurations I am getting these errors in the logs.  catalina.out -   java.lang.RuntimeException: Failed to create user drew.nicolette in x_portal_user table. retrying  at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3314)  at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.run(XUserMgr.java:3288)  at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.addRunnable(RangerTransactionSynchronizationAdapter.java:136)  at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.executeOnTransactionCommit(RangerTransactionSynchronizationAdapter.java:82)  at org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2601)  at org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(<generated>)  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)  at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386)  at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85)  at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)  at org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$c3461920.createServiceConfigUser(<generated>)  at org.apache.ranger.security.web.authentication.RangerAuthSuccessHandler.onAuthenticationSuccess(RangerAuthSuccessHandler.java:96)  at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.successfulAuthentication(AbstractAuthenticationProcessingFilter.java:329)  at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:237)  at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)  at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)  at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)   at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112)  at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.session.ForceEagerSessionCreationFilter.doFilterInternal(ForceEagerSessionCreationFilter.java:45)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221)  at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186)  at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)  at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:181)  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:156)  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483)  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)  at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:679)  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)  at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:617)  at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)  at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:934)  at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1698)  at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)  at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)  at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)  at java.lang.Thread.run(Thread.java:750)  Caused by: javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'persist' call  at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:300)  at com.sun.proxy.$Proxy33.persist(Unknown Source)  at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:110)  at org.apache.ranger.biz.UserMgr.createUser(UserMgr.java:161)  at org.apache.ranger.biz.UserMgr$$FastClassBySpringCGLIB$$3bbcf0cf.invoke(<generated>)  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)  at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386)  at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85)  at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)  at org.apache.ranger.biz.UserMgr$$EnhancerBySpringCGLIB$$a7bf07b4.createUser(<generated>)  at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3309)     ranger-admin.log  2025-02-12 20:48:12,287 [https-jsse-nio-6182-exec-5] INFO [SpringEventListener.java:76] Login Successful:drew.nicolette | Ip Address:172.34.100.196 | sessionId=0A3BABC02CC58F33F6A7055DAD1FAE26 | Epoch=1739393292287  2025-02-12 20:48:12,293 [https-jsse-nio-6182-exec-5] ERROR [SessionMgr.java:486] Error getting user for loginId=drew.nicolette     However, the fix I noticed is to run usersync along side of ranger admin. This allowed me to login to the Ranger Admin UI.  Does anyone know if this is required? Any help would be greatly appreciated!  @Shelton  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
	
					
			
		
	
	
	
	
				
		
	
	
			
    
	
		
		
		02-12-2025
	
		
		12:35 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Shelton   My team and I have been running Apache Ranger Admin v2.4.0 in Docker. We recently started an effort to try and upgrade Apache Ranger Admin v2.5.0. In v2.4.0, we set up the install.properties file to correctly point to our LDAP server for logging in to the Ranger Admin UI and everything was working well. One thing to note, is that in v2.4.0 we didn't have usersync running, and we were still able to login.    After upgrading to Apache Ranger Admin v2.5.0, we ran into an issue trying to login to the Ranger Admin UI. In the logs this is what it showed...  ranger-admin.log -  2025-02-12 20:14:42,853 [https-jsse-nio-6182-exec-8] INFO [SpringEventListener.java:76] Login Successful:drew.nicolette | Ip Address:xxxxxxxxx | sessionId=5F137022A5347056C441709DDC19A26F | Epoch=1739391282852  2025-02-12 20:14:42,890 [https-jsse-nio-6182-exec-8] ERROR [SessionMgr.java:486] Error getting user for loginId=drew.xxxxxx  catalina.out -  java.lang.RuntimeException: Failed to create user drew.xxxxxxx in x_portal_user table. retrying  at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3314)  at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.run(XUserMgr.java:3288)  at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.addRunnable(RangerTransactionSynchronizationAdapter.java:136)  at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.executeOnTransactionCommit(RangerTransactionSynchronizationAdapter.java:82)  at org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2601)  at org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(<generated>)  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)  at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386)  at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85)  at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)  at org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$c3461920.createServiceConfigUser(<generated>)  at org.apache.ranger.security.web.authentication.RangerAuthSuccessHandler.onAuthenticationSuccess(RangerAuthSuccessHandler.java:96)  at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.successfulAuthentication(AbstractAuthenticationProcessingFilter.java:329)  at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:237)  at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)  at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)  at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112)  at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.session.ForceEagerSessionCreationFilter.doFilterInternal(ForceEagerSessionCreationFilter.java:45)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)  at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221)  at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186)  at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)  at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:181)  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:156)  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483)  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)  at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:679)  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)  at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:617)  at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)  at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:934)  at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1698)  at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)  at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)  at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)  at java.lang.Thread.run(Thread.java:750)  Caused by: javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'persist' call  at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:300)  at com.sun.proxy.$Proxy33.persist(Unknown Source)  at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:110)  at org.apache.ranger.biz.UserMgr.createUser(UserMgr.java:161)  at org.apache.ranger.biz.UserMgr$$FastClassBySpringCGLIB$$3bbcf0cf.invoke(<generated>)  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)  at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386)  at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85)  at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)  at org.apache.ranger.biz.UserMgr$$EnhancerBySpringCGLIB$$a7bf07b4.createUser(<generated>)  at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3309)  ... 57 more   However, once we added ranger usersync into the process. We were able to login successfully.     Is Apache Ranger Admin v2.5.0 dependent on Usersync to run for login to the Admin UI or am I missing something obvious?  Thanks! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache Ranger
 
			
    
	
		
		
		12-10-2024
	
		
		06:02 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I need to authenticate to a Kafka Broker using OAuth.   I am looking at the Apache NiFi issues pages and see this URL to add support for SASL/OAUTHBEARER to the Kafka 3 processors which now have an allowable values list of SASL mechanisms - https://issues.apache.org/jira/browse/NIFI-7421    Does anyone know if this will ever be implemented?  Otherwise, does anyone have any list of ideas I could use to achieve this requirement?  @MattWho   @SAMSAL  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache NiFi
 
			
    
	
		
		
		11-04-2024
	
		
		06:02 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 The problem was my StandardRestrictedSSLContextService didn't include a keystore. For some reason, I thought it was just one way SSL communication. Once I added the keystore to the ContextService it was authenticating correctly! Thanks @MattWho !   
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-30-2024
	
		
		06:30 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 All,     I have two separate NiFi instance and I want to push provenance events from one instance to another. Source NiFi's hostname is nifi8443 and Target NiFi's hostname is nifi8444.  Both of these instances are running over HTTPS. On the "Target NiFi" I set up an input port called Test.       On source NiFi, here's what my SiteToSiteProvenanceReportingTask looks like.       Here's a couple things I did.     I realized that the reporting task on NiFi8443 is essentially acting like an client and NiFi8444 is acting as the server. I added the NiFi8444 server certificate into a truststore.p12 file on NiFi8443 and created the corresponding SSLContextService to point to that Truststore file.     I resolved the SSL issue and then I was reading on the target node (ie. NiFi8444) I had to make an identity for the full DN of the source (ie NiFi8443) certificate. So on NiFI8444, I created a new user called this - CN=nifi8443, O=NiFi Server, L=San Francisco, ST=California, C=US      Then I added that identity to two policies. The first one is retrieve-site-to-site -      And the second one is the input port -      I thought that was all the requirements however, I am currently getting this error when I start the reporting task.    SiteToSiteProvenanceReportingTask[id=d9ece17e-0192-1000-9050-2a4a5a2f9e02] Unable to refresh remote group peers due to: response code 401:Unauthorized with explanation: null  It seems like it has to do with the identity from NiFi8443 not properly being authenticated correctly. But when I run keytool -list -v -keystore nifi8443.p12 on the server certificate I get this -     @MattWho @SAMSAL - Do you know why I getting a 401 unauthorized error?  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache NiFi
 
			
    
	
		
		
		10-23-2024
	
		
		07:00 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 @Mikkkk   Please check my screenshot below. I changed the "Replacement Value Strategy" property to "Literal". I also updated the dynamic user property to this  - ${field.value:replaceAll('[\\p{Punct}]', ' ')}    Here's the screenshot -          ${field.value}: In NiFi’s Expression Language, field.value refers to the current value of the field being processed in a record (for example, the value of the ENTITY_NAME field if that’s what you’re working on).  :replaceAll('[\\p{Punct}]', ' '): This function searches the field.value for any punctuation characters (using the [\\p{Punct}] regex) and replaces each occurrence with a space.     Please "Accept the solution" if it helped! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-23-2024
	
		
		11:32 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @Shalexey   Can you provide an example of what you mean by filter?  The LookupRecord processor primary function is to extract one or more fields from a Record and looks up a value for those fields in a LookupService.    If you are trying to do a filter you can use another NiFi processor prior to the LookupRecord processor. Here are some examples below...    1. RouteOnAttribute  Routes flow files based on attribute values by evaluating expressions. It allows you to define conditions (e.g., ${status:equals('SUCCESS')}) to send data to different relationships, like success or unmatched.  2. RouteOnContent  Filters flow files based on patterns within the content. This processor is ideal for detecting specific keywords, tags, or values inside text, XML, or JSON data and routing them accordingly.  3. FilterCSV  Designed to filter rows in CSV files based on specific column values or positions. It removes unwanted data from the flow, only passing records that meet the defined criteria (e.g., status = 'active').  4. QueryRecord  Uses SQL queries to filter structured data (JSON, CSV, Avro) within flow files. You can select or discard records by writing SQL-like queries (e.g., SELECT * FROM FLOWFILE WHERE type = 'transaction').    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-23-2024
	
		
		11:21 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		3 Kudos
		
	
				
		
	
		
					
							 @MDTechie - Here is a provided solution for the question you asked. I usually like to Jolt Transforms step by step. Please see the Jolt Spec below along with explanations for each step.     [
  {
    "operation": "shift",
    "spec": {
      "id": "id",
      "categories": {
        "*": {
          "@": "categories[]",
          "subcategories": {
            "*": {
              "@": "categories[]"
            }
          }
        }
      }
    }
  },
  {
    "operation": "remove",
    "spec": {
      "categories": {
        "*": {
          "subcategories": ""
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "categories": {
        "*": {
          "code": "categories[&1].categoryCode",
          "name": "categories[&1].categoryName",
          "@(2,id)": "categories[&1].individualId"
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "categories": {
        "*": ""
      }
    }
  }
]     The first operation   Copies the id field from the original input to the top level of the output.  Iterates over the categories array and:  Copies each category object to a new array called categories[].  If a category contains a subcategories array, each subcategory object is also copied into the same categories[] array, effectively flattening the nested structure.     The second operation   Searches within each category object in the categories[] array.  Removes the subcategories field if it exists.   The third operation      Iterates over the categories[] array, processing each category object.  Renames the code field to categoryCode and the name field to categoryName for consistency with the new output schema.  Adds a new field called individualId inside each category object, using the id value from two levels up (the root of the original input).   The fourth operation:      Takes each object from the categories[] array and moves it to the root level.  As a result, the categories array wrapper is removed, leaving a flat array of individual objects.   I suggest piecing it out in a JOLT tester to understand it a little better. Hope this helps! 🙂     If you found the solution helpful, please "Accept as Solution"             
						
					
					... View more