About azure

azure · ‎07-30-2019

Turns out , the following needs to be executed on mySQL: INSERT INTO schema_lock SELECT CONCAT('schema_metadata_info-', schema_metadata_info.name) AS name, schema_metadata_info.timestamp FROM schema_metadata_info WHERE CONCAT('schema_metadata_info-', schema_metadata_info.name) NOT IN (SELECT name FROM schema_lock);

azure · ‎07-30-2019

Hi , We recently upgrade our DEV and PROD environments from HDF 3.3 to HDF 3.4.1.1 after following upgrade instructions on a HDF-only cluster. After the upgrade, schema registry is giving the following exceptions in both environments for clients using the confluent API: ERROR [2019-07-30 11:17:42.014] [dw-94 - POST /api/v1/confluent/subjects/EgressLiveAllCXXX1-value/versions] c.h.r.s.w.ConfluentSchemaRegistryCompatibleResource - Encountered error while adding subject [EgressLiveAllCowBehavior1-value] java.lang.RuntimeException: Failed to obtain write lock : schema_metadata_info-EgressLiveAllCXXXX1-value in 120 sec at com.hortonworks.registries.schemaregistry.DefaultSchemaRegistry.lockSchemaMetadata(DefaultSchemaRegistry.java:520) at com.hortonworks.registries.schemaregistry.DefaultSchemaRegistry.addSchemaVersion(DefaultSchemaRegistry.java:489) at com.hortonworks.registries.schemaregistry.webservice.ConfluentSchemaRegistryCompatibleResource.lambda$registerSchemaVersion$1(ConfluentSchemaRegistryCompatibleResource.java:322) at com.hortonworks.registries.schemaregistry.webservice.BaseRegistryResource.handleLeaderAction(BaseRegistryResource.java:77) at com.hortonworks.registries.schemaregistry.webservice.ConfluentSchemaRegistryCompatibleResource.registerSchemaVersion(ConfluentSchemaRegistryCompatibleResource.java:307) at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ...... .... I've tried flusing the mySQL tables, restarted the database and the SR services but no luck As this is impacting production, any help would be highly appreciated.

azure · ‎01-21-2019

The solution worked! I had some strange error my side that disappeared after a clean setup of the flow. Thanks @Shu!

azure · ‎01-13-2019

This really helps--and its the solution that should work. However, every time i give the SQL SELECT statement in the flowfile content (using generateflowfile processor), the ExecuteSQL processor throws an error. The database.name attribute is set and the select statement is a simple select a from table the error is below. any idea what i'm missing? [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.ExecuteSQL ExecuteSQL[id=15613b27-66e0-101a-ab78-86906b96182b] ExecuteSQL[id=15613b27-66e0-101a-ab78-86906b96182b] faile d to process session due to org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=ed4af901-f621-4aab-b7db-aad8c72e41ca,claim=StandardContentClaim [resourceClaim=StandardResou rceClaim[id=1547378310570-136, container=default, section=136], offset=890898, length=10357],offset=9267,name=ed4328e2-1284-4ff3-b252-f7de7c92a94e,size=1090] is not known in this session (StandardProcessSession[ id=315712]); Processor Administratively Yielded for 1 sec: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=ed4af901-f621-4aab-b7db-aad8c72e41ca,claim=StandardContentCla im [resourceClaim=StandardResourceClaim[id=1547378310570-136, container=default, section=136], offset=890898, length=10357],offset=9267,name=ed4328e2-1284-4ff3-b252-f7de7c92a94e,size=1090] is not known in this session (StandardProcessSession[id=315712])

azure · ‎01-12-2019

@shu Many thanks for responding--however, with this solution I will need multiple ExecuteSQL processors each specifying the database.name attribute value, correct? I'll end up with dozens of ExecuteSQL processors that have the same SQL but different Database.name. Is my understanding correct or am i missing a trick? I guess, the question is, how/where do i set the database.name attribute? I've configured the service, but how is the execute SQL processor configured?

azure · ‎01-11-2019

Hi, I'm trying to connect to different databases that all have the same structure and execute the same query on them. However, i can't figure out how to give this to nifi execute SQL processor and i'm having to create a processor for every connection to the database. IS there a way for me to create a list of connection strings and then simply loop through them? i.e. i need to dynamically assign the connection pooling service to the processor. How?

Online	Offline
Last Visited	‎07-31-2019 10:48 PM

Member Since	‎01-11-2019 06:44 PM
Last Visited	‎07-31-2019 10:48 PM
Posts	10
Kudos received	2

Cloudera Community

Re: Schema registry breaks after upgrade to HDF 3....

Re: Schema registry breaks after upgrade to HDF 3....

Schema registry breaks after upgrade to HDF 3.4.1....

Re: Nifi: Multiple databases , same query, how do ...

Re: Nifi: Multiple databases , same query, how do ...

Re: Nifi: Multiple databases , same query, how do ...

Nifi: Multiple databases , same query, how do pass...