Member since
06-16-2020
55
Posts
14
Kudos Received
5
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 885 | 10-23-2024 11:21 AM | |
| 852 | 10-22-2024 07:59 AM | |
| 840 | 10-22-2024 07:37 AM | |
| 499 | 10-21-2024 09:25 AM | |
| 2277 | 06-16-2023 07:23 AM |
02-27-2025
07:38 AM
@upadhyayk04 So for Ranger Admin, I configured it to use LDAP and it wasn't enough to login to the UI in version 2.5.0. Once I configured both Ranger Admin and Ranger Usersync it worked. I am under the same assumption as you. I thought just configuring the Ranger Admin to use LDAP was enough but in this case it wasn't.
... View more
02-20-2025
06:11 AM
@steven-matison - Sorry but where do you see that it is required? To me, setting the LDAP configurations for Ranger Admin would allow logging into the Ranger Admin UI via an LDAP source and Ranger Usersync was primarily for syncing Users/Groups to be used in policies. So why would Ranger Usersync be needed to login to the Ranger Admin UI?
... View more
02-12-2025
01:20 PM
I am running Ranger Admin 2.4.0. Right now I have it synced with LDAP and am able to login to the Ranger Admin UI via my LDAP username and password. When I try upgrading to Ranger Admin 2.5.0 using the same configurations I am getting these errors in the logs. catalina.out - java.lang.RuntimeException: Failed to create user drew.nicolette in x_portal_user table. retrying at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3314) at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.run(XUserMgr.java:3288) at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.addRunnable(RangerTransactionSynchronizationAdapter.java:136) at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.executeOnTransactionCommit(RangerTransactionSynchronizationAdapter.java:82) at org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2601) at org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703) at org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$c3461920.createServiceConfigUser(<generated>) at org.apache.ranger.security.web.authentication.RangerAuthSuccessHandler.onAuthenticationSuccess(RangerAuthSuccessHandler.java:96) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.successfulAuthentication(AbstractAuthenticationProcessingFilter.java:329) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:237) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.session.ForceEagerSessionCreationFilter.doFilterInternal(ForceEagerSessionCreationFilter.java:45) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:181) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:156) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:679) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:617) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:934) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1698) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750) Caused by: javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'persist' call at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:300) at com.sun.proxy.$Proxy33.persist(Unknown Source) at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:110) at org.apache.ranger.biz.UserMgr.createUser(UserMgr.java:161) at org.apache.ranger.biz.UserMgr$$FastClassBySpringCGLIB$$3bbcf0cf.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703) at org.apache.ranger.biz.UserMgr$$EnhancerBySpringCGLIB$$a7bf07b4.createUser(<generated>) at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3309) ranger-admin.log 2025-02-12 20:48:12,287 [https-jsse-nio-6182-exec-5] INFO [SpringEventListener.java:76] Login Successful:drew.nicolette | Ip Address:172.34.100.196 | sessionId=0A3BABC02CC58F33F6A7055DAD1FAE26 | Epoch=1739393292287 2025-02-12 20:48:12,293 [https-jsse-nio-6182-exec-5] ERROR [SessionMgr.java:486] Error getting user for loginId=drew.nicolette However, the fix I noticed is to run usersync along side of ranger admin. This allowed me to login to the Ranger Admin UI. Does anyone know if this is required? Any help would be greatly appreciated! @Shelton
... View more
Labels:
02-12-2025
12:35 PM
@Shelton My team and I have been running Apache Ranger Admin v2.4.0 in Docker. We recently started an effort to try and upgrade Apache Ranger Admin v2.5.0. In v2.4.0, we set up the install.properties file to correctly point to our LDAP server for logging in to the Ranger Admin UI and everything was working well. One thing to note, is that in v2.4.0 we didn't have usersync running, and we were still able to login. After upgrading to Apache Ranger Admin v2.5.0, we ran into an issue trying to login to the Ranger Admin UI. In the logs this is what it showed... ranger-admin.log - 2025-02-12 20:14:42,853 [https-jsse-nio-6182-exec-8] INFO [SpringEventListener.java:76] Login Successful:drew.nicolette | Ip Address:xxxxxxxxx | sessionId=5F137022A5347056C441709DDC19A26F | Epoch=1739391282852 2025-02-12 20:14:42,890 [https-jsse-nio-6182-exec-8] ERROR [SessionMgr.java:486] Error getting user for loginId=drew.xxxxxx catalina.out - java.lang.RuntimeException: Failed to create user drew.xxxxxxx in x_portal_user table. retrying at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3314) at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.run(XUserMgr.java:3288) at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.addRunnable(RangerTransactionSynchronizationAdapter.java:136) at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.executeOnTransactionCommit(RangerTransactionSynchronizationAdapter.java:82) at org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2601) at org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703) at org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$c3461920.createServiceConfigUser(<generated>) at org.apache.ranger.security.web.authentication.RangerAuthSuccessHandler.onAuthenticationSuccess(RangerAuthSuccessHandler.java:96) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.successfulAuthentication(AbstractAuthenticationProcessingFilter.java:329) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:237) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.session.ForceEagerSessionCreationFilter.doFilterInternal(ForceEagerSessionCreationFilter.java:45) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:181) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:156) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:679) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:617) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:934) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1698) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750) Caused by: javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'persist' call at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:300) at com.sun.proxy.$Proxy33.persist(Unknown Source) at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:110) at org.apache.ranger.biz.UserMgr.createUser(UserMgr.java:161) at org.apache.ranger.biz.UserMgr$$FastClassBySpringCGLIB$$3bbcf0cf.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703) at org.apache.ranger.biz.UserMgr$$EnhancerBySpringCGLIB$$a7bf07b4.createUser(<generated>) at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3309) ... 57 more However, once we added ranger usersync into the process. We were able to login successfully. Is Apache Ranger Admin v2.5.0 dependent on Usersync to run for login to the Admin UI or am I missing something obvious? Thanks!
... View more
Labels:
- Labels:
-
Apache Ranger
12-10-2024
06:02 PM
I need to authenticate to a Kafka Broker using OAuth. I am looking at the Apache NiFi issues pages and see this URL to add support for SASL/OAUTHBEARER to the Kafka 3 processors which now have an allowable values list of SASL mechanisms - https://issues.apache.org/jira/browse/NIFI-7421 Does anyone know if this will ever be implemented? Otherwise, does anyone have any list of ideas I could use to achieve this requirement? @MattWho @SAMSAL
... View more
Labels:
- Labels:
-
Apache NiFi
11-04-2024
06:02 AM
The problem was my StandardRestrictedSSLContextService didn't include a keystore. For some reason, I thought it was just one way SSL communication. Once I added the keystore to the ContextService it was authenticating correctly! Thanks @MattWho !
... View more
10-30-2024
06:30 PM
All, I have two separate NiFi instance and I want to push provenance events from one instance to another. Source NiFi's hostname is nifi8443 and Target NiFi's hostname is nifi8444. Both of these instances are running over HTTPS. On the "Target NiFi" I set up an input port called Test. On source NiFi, here's what my SiteToSiteProvenanceReportingTask looks like. Here's a couple things I did. I realized that the reporting task on NiFi8443 is essentially acting like an client and NiFi8444 is acting as the server. I added the NiFi8444 server certificate into a truststore.p12 file on NiFi8443 and created the corresponding SSLContextService to point to that Truststore file. I resolved the SSL issue and then I was reading on the target node (ie. NiFi8444) I had to make an identity for the full DN of the source (ie NiFi8443) certificate. So on NiFI8444, I created a new user called this - CN=nifi8443, O=NiFi Server, L=San Francisco, ST=California, C=US Then I added that identity to two policies. The first one is retrieve-site-to-site - And the second one is the input port - I thought that was all the requirements however, I am currently getting this error when I start the reporting task. SiteToSiteProvenanceReportingTask[id=d9ece17e-0192-1000-9050-2a4a5a2f9e02] Unable to refresh remote group peers due to: response code 401:Unauthorized with explanation: null It seems like it has to do with the identity from NiFi8443 not properly being authenticated correctly. But when I run keytool -list -v -keystore nifi8443.p12 on the server certificate I get this - @MattWho @SAMSAL - Do you know why I getting a 401 unauthorized error?
... View more
Labels:
- Labels:
-
Apache NiFi
10-23-2024
07:00 PM
2 Kudos
@Mikkkk Please check my screenshot below. I changed the "Replacement Value Strategy" property to "Literal". I also updated the dynamic user property to this - ${field.value:replaceAll('[\\p{Punct}]', ' ')} Here's the screenshot - ${field.value}: In NiFi’s Expression Language, field.value refers to the current value of the field being processed in a record (for example, the value of the ENTITY_NAME field if that’s what you’re working on). :replaceAll('[\\p{Punct}]', ' '): This function searches the field.value for any punctuation characters (using the [\\p{Punct}] regex) and replaces each occurrence with a space. Please "Accept the solution" if it helped!
... View more
10-23-2024
11:32 AM
1 Kudo
@Shalexey Can you provide an example of what you mean by filter? The LookupRecord processor primary function is to extract one or more fields from a Record and looks up a value for those fields in a LookupService. If you are trying to do a filter you can use another NiFi processor prior to the LookupRecord processor. Here are some examples below... 1. RouteOnAttribute Routes flow files based on attribute values by evaluating expressions. It allows you to define conditions (e.g., ${status:equals('SUCCESS')}) to send data to different relationships, like success or unmatched. 2. RouteOnContent Filters flow files based on patterns within the content. This processor is ideal for detecting specific keywords, tags, or values inside text, XML, or JSON data and routing them accordingly. 3. FilterCSV Designed to filter rows in CSV files based on specific column values or positions. It removes unwanted data from the flow, only passing records that meet the defined criteria (e.g., status = 'active'). 4. QueryRecord Uses SQL queries to filter structured data (JSON, CSV, Avro) within flow files. You can select or discard records by writing SQL-like queries (e.g., SELECT * FROM FLOWFILE WHERE type = 'transaction').
... View more
10-23-2024
11:21 AM
3 Kudos
@MDTechie - Here is a provided solution for the question you asked. I usually like to Jolt Transforms step by step. Please see the Jolt Spec below along with explanations for each step. [
{
"operation": "shift",
"spec": {
"id": "id",
"categories": {
"*": {
"@": "categories[]",
"subcategories": {
"*": {
"@": "categories[]"
}
}
}
}
}
},
{
"operation": "remove",
"spec": {
"categories": {
"*": {
"subcategories": ""
}
}
}
},
{
"operation": "shift",
"spec": {
"categories": {
"*": {
"code": "categories[&1].categoryCode",
"name": "categories[&1].categoryName",
"@(2,id)": "categories[&1].individualId"
}
}
}
},
{
"operation": "shift",
"spec": {
"categories": {
"*": ""
}
}
}
] The first operation Copies the id field from the original input to the top level of the output. Iterates over the categories array and: Copies each category object to a new array called categories[]. If a category contains a subcategories array, each subcategory object is also copied into the same categories[] array, effectively flattening the nested structure. The second operation Searches within each category object in the categories[] array. Removes the subcategories field if it exists. The third operation Iterates over the categories[] array, processing each category object. Renames the code field to categoryCode and the name field to categoryName for consistency with the new output schema. Adds a new field called individualId inside each category object, using the id value from two levels up (the root of the original input). The fourth operation: Takes each object from the categories[] array and moves it to the root level. As a result, the categories array wrapper is removed, leaving a flat array of individual objects. I suggest piecing it out in a JOLT tester to understand it a little better. Hope this helps! 🙂 If you found the solution helpful, please "Accept as Solution"
... View more
