Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

falcon - oozie DB connection error (derby read-only)

falcon - oozie DB connection error (derby read-only)

Explorer

I tried to run schedule falcon process..but still I have a problem.

on Ambari, I added oozie configuration for falcon, and then I started server.

(I referred http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/configur...

On Falcon web UI, I scheduled the process entity, unexpected error occurred.

I think oozie derby database is read-only mode, so it can't write/update data to database.

please see attached log, and help me!

-------------------------------------------------------------------------------------

<falcon application log>

2016-03-13 05:33:22,193 INFO  - [2089900275@qtp-954636025-27 - 4fa9cf31-2f72-4091-9095-c1b3049c5c2e:] ~ HttpServletRequest RemoteUser is falcon (Servlets:47)
2016-03-13 05:33:22,196 INFO  - [2089900275@qtp-954636025-27 - 4fa9cf31-2f72-4091-9095-c1b3049c5c2e:falcon:GET//admin/clearuser] ~ Logging in falcon (CurrentUser:63)
2016-03-13 05:33:22,196 INFO  - [2089900275@qtp-954636025-27 - 4fa9cf31-2f72-4091-9095-c1b3049c5c2e:falcon:GET//admin/clearuser] ~ Request from authenticated user: falcon, URL=/api/admin/clearuser?user.name=falcon (FalconAuthenticationFilter:181)
2016-03-13 05:33:22,232 DEBUG - [2089900275@qtp-954636025-27 - 4fa9cf31-2f72-4091-9095-c1b3049c5c2e:] ~ Audit: falcon/10.0.2.2 performed request http://127.0.0.1:15000/api/admin/clearuser?user.name=falcon (10.0.2.15) at time 2016-03-13T05:33Z (FalconAuditFilter:86)
2016-03-13 05:33:22,379 INFO  - [2089900275@qtp-954636025-27 - af37b407-f18d-400a-b3ed-924be32017a9:] ~ HttpServletRequest RemoteUser is falcon (Servlets:47)
2016-03-13 05:33:22,380 INFO  - [2089900275@qtp-954636025-27 - af37b407-f18d-400a-b3ed-924be32017a9:falcon:GET//admin/version] ~ Logging in falcon (CurrentUser:63)
2016-03-13 05:33:22,381 INFO  - [2089900275@qtp-954636025-27 - af37b407-f18d-400a-b3ed-924be32017a9:falcon:GET//admin/version] ~ Request from authenticated user: falcon, URL=/api/admin/version?user.name=falcon (FalconAuthenticationFilter:181)
2016-03-13 05:33:22,399 DEBUG - [2089900275@qtp-954636025-27 - af37b407-f18d-400a-b3ed-924be32017a9:] ~ Audit: falcon/10.0.2.2 performed request http://127.0.0.1:15000/api/admin/version?user.name=falcon (10.0.2.15) at time 2016-03-13T05:33Z (FalconAuditFilter:86)
2016-03-13 05:33:25,028 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:] ~ HttpServletRequest RemoteUser is ambari-qa (Servlets:47)
2016-03-13 05:33:25,028 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Logging in ambari-qa (CurrentUser:63)
2016-03-13 05:33:25,028 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Request from authenticated user: ambari-qa, URL=/api/entities/list/feed,process?fields=clusters,tags,status&nameseq=process&offset=0&numResults=10&user.name=ambari-qa (FalconAuthenticationFilter:181)
2016-03-13 05:33:25,034 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:25,521 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:25,592 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:25,651 DEBUG - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000184-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/cleanseEmailProcess/e6f93a6ff285be3cff8b6676371f48f5_1454389189084 (OozieWorkflowEngine:298)
2016-03-13 05:33:25,652 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:25,995 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:25,998 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:26,025 DEBUG - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000184-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/cleanseEmailProcess/e6f93a6ff285be3cff8b6676371f48f5_1454389189084 (OozieWorkflowEngine:298)
2016-03-13 05:33:26,027 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:26,434 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:26,437 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:26,555 DEBUG - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000181-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/rawEmailIngestProcess/5cecdb80f632bbb84c35a88f41246f3e_1454389162953 (OozieWorkflowEngine:298)
2016-03-13 05:33:26,556 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:26,799 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:26,803 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:26,824 DEBUG - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000181-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/rawEmailIngestProcess/5cecdb80f632bbb84c35a88f41246f3e_1454389162953 (OozieWorkflowEngine:298)
2016-03-13 05:33:26,825 INFO  - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:ambari-qa:GET//entities/list/feed,process] ~ {Action:list, Dimensions:{}, Status: SUCCEEDED, Time-taken:1791932353 ns} (METRIC:38)
2016-03-13 05:33:26,830 DEBUG - [2089900275@qtp-954636025-27 - e9ac11d9-2d79-430c-9215-c7619705e21f:] ~ Audit: ambari-qa/10.0.2.2 performed request http://127.0.0.1:15000/api/entities/list/feed,process?fields=clusters,tags,status&nameseq=process&of... (10.0.2.15) at time 2016-03-13T05:33Z (FalconAuditFilter:86)
2016-03-13 05:33:29,743 INFO  - [2089900275@qtp-954636025-27 - e1e57718-dd7b-41df-8f15-794d323bdd43:] ~ HttpServletRequest RemoteUser is ambari-qa (Servlets:47)
2016-03-13 05:33:29,743 INFO  - [2089900275@qtp-954636025-27 - e1e57718-dd7b-41df-8f15-794d323bdd43:ambari-qa:GET//entities/definition/process/rawEmailIngestProcess] ~ Logging in ambari-qa (CurrentUser:63)
2016-03-13 05:33:29,743 INFO  - [2089900275@qtp-954636025-27 - e1e57718-dd7b-41df-8f15-794d323bdd43:ambari-qa:GET//entities/definition/process/rawEmailIngestProcess] ~ Request from authenticated user: ambari-qa, URL=/api/entities/definition/process/rawEmailIngestProcess?user.name=ambari-qa (FalconAuthenticationFilter:181)
2016-03-13 05:33:29,753 DEBUG - [2089900275@qtp-954636025-27 - e1e57718-dd7b-41df-8f15-794d323bdd43:] ~ Audit: ambari-qa/10.0.2.2 performed request http://127.0.0.1:15000/api/entities/definition/process/rawEmailIngestProcess?user.name=ambari-qa (10.0.2.15) at time 2016-03-13T05:33Z (FalconAuditFilter:86)
2016-03-13 05:33:33,886 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:] ~ HttpServletRequest RemoteUser is ambari-qa (Servlets:47)
2016-03-13 05:33:33,886 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Logging in ambari-qa (CurrentUser:63)
2016-03-13 05:33:33,886 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Request from authenticated user: ambari-qa, URL=/api/entities/list/feed,process?fields=clusters,tags,status&nameseq=process&offset=0&numResults=10&user.name=ambari-qa (FalconAuthenticationFilter:181)
2016-03-13 05:33:33,893 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,058 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:34,060 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,078 DEBUG - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000184-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/cleanseEmailProcess/e6f93a6ff285be3cff8b6676371f48f5_1454389189084 (OozieWorkflowEngine:298)
2016-03-13 05:33:34,079 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,268 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:34,271 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,286 DEBUG - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000184-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/cleanseEmailProcess/e6f93a6ff285be3cff8b6676371f48f5_1454389189084 (OozieWorkflowEngine:298)
2016-03-13 05:33:34,286 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,403 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:34,407 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,431 DEBUG - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000181-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/rawEmailIngestProcess/5cecdb80f632bbb84c35a88f41246f3e_1454389162953 (OozieWorkflowEngine:298)
2016-03-13 05:33:34,431 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,557 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:34,565 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:34,585 DEBUG - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ Found bundle 0000181-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/rawEmailIngestProcess/5cecdb80f632bbb84c35a88f41246f3e_1454389162953 (OozieWorkflowEngine:298)
2016-03-13 05:33:34,586 INFO  - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:ambari-qa:GET//entities/list/feed,process] ~ {Action:list, Dimensions:{}, Status: SUCCEEDED, Time-taken:695784130 ns} (METRIC:38)
2016-03-13 05:33:34,588 DEBUG - [2089900275@qtp-954636025-27 - 5ee98c82-768a-4f58-a239-6490990f1b14:] ~ Audit: ambari-qa/10.0.2.2 performed request http://127.0.0.1:15000/api/entities/list/feed,process?fields=clusters,tags,status&nameseq=process&of... (10.0.2.15) at time 2016-03-13T05:33Z (FalconAuditFilter:86)
2016-03-13 05:33:37,311 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:] ~ HttpServletRequest RemoteUser is ambari-qa (Servlets:47)
2016-03-13 05:33:37,312 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Logging in ambari-qa (CurrentUser:63)
2016-03-13 05:33:37,312 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Request from authenticated user: ambari-qa, URL=/api/entities/suspend/PROCESS/rawEmailIngestProcess?user.name=ambari-qa (FalconAuthenticationFilter:181)
2016-03-13 05:33:37,314 DEBUG - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Invoking method suspend on service org.apache.falcon.resource.SchedulableEntityManager (IPCChannel:45)
2016-03-13 05:33:37,315 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:37,418 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:37,422 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:37,437 DEBUG - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Found bundle 0000181-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/rawEmailIngestProcess/5cecdb80f632bbb84c35a88f41246f3e_1454389162953 (OozieWorkflowEngine:298)
2016-03-13 05:33:37,438 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:37,517 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Creating FS impersonating user ambari-qa (HadoopClientFactory:197)
2016-03-13 05:33:37,520 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:37,535 DEBUG - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Found bundle 0000181-160202025757387-oozie-root-B with app path hdfs://sandbox.hortonworks.com:8020/apps/falcon/primaryCluster/staging/falcon/workflows/process/rawEmailIngestProcess/5cecdb80f632bbb84c35a88f41246f3e_1454389162953 (OozieWorkflowEngine:298)
2016-03-13 05:33:37,536 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Creating Oozie client object for http://sandbox.hortonworks.com:11000/oozie/ (OozieClientFactory:50)
2016-03-13 05:33:37,582 ERROR - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Unable to suspend entity (AbstractSchedulableEntityManager:151)
org.apache.falcon.FalconException: E0603 : E0603: SQL error in operation, <openjpa-2.2.2-r422266:1468616 fatal general error> org.apache.openjpa.persistence.PersistenceException: An SQL data change is not permitted for a read-only connection, user or database. {prepstmnt 802408967 UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?))} [code=20000, state=25502] FailedObject: UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?)) [java.lang.String]
        at org.apache.falcon.workflow.engine.OozieWorkflowEngine.suspend(OozieWorkflowEngine.java:1455)
        at org.apache.falcon.workflow.engine.OozieWorkflowEngine.doBundleAction(OozieWorkflowEngine.java:385)
        at org.apache.falcon.workflow.engine.OozieWorkflowEngine.doBundleAction(OozieWorkflowEngine.java:371)
        at org.apache.falcon.workflow.engine.OozieWorkflowEngine.suspend(OozieWorkflowEngine.java:345)
        at org.apache.falcon.resource.AbstractSchedulableEntityManager.suspend(AbstractSchedulableEntityManager.java:145)
        at org.apache.falcon.resource.SchedulableEntityManager.suspend(SchedulableEntityManager.java:143)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:49)
        at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$10.doExecute(SchedulableEntityManagerProxy.java:445)
        at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:575)
        at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.suspend_aroundBody20(SchedulableEntityManagerProxy.java:447)
        at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure21.run(SchedulableEntityManagerProxy.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
        at org.apache.falcon.aspect.AbstractFalconAspect.logAroundMonitored(AbstractFalconAspect.java:51)
        at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.suspend(SchedulableEntityManagerProxy.java:431)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
        at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
        at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
        at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
        at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
        at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
        at org.apache.falcon.security.FalconAuthorizationFilter.doFilter(FalconAuthorizationFilter.java:106)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.falcon.security.FalconAuthenticationFilter$2.doFilter(FalconAuthenticationFilter.java:184)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:595)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:554)
        at org.apache.falcon.security.FalconAuthenticationFilter.doFilter(FalconAuthenticationFilter.java:193)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.falcon.security.FalconAuditFilter.doFilter(FalconAuditFilter.java:64)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: E0603 : E0603: SQL error in operation, <openjpa-2.2.2-r422266:1468616 fatal general error> org.apache.openjpa.persistence.PersistenceException: An SQL data change is not permitted for a read-only connection, user or database. {prepstmnt 802408967 UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?))} [code=20000, state=25502] FailedObject: UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?)) [java.lang.String]
        at org.apache.oozie.client.OozieClient.handleError(OozieClient.java:612)
        at org.apache.oozie.client.OozieClient$JobAction.call(OozieClient.java:726)
        at org.apache.oozie.client.OozieClient$JobAction.call(OozieClient.java:712)
        at org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:560)
        at org.apache.oozie.client.OozieClient.suspend(OozieClient.java:871)
        at org.apache.oozie.client.ProxyOozieClient.access$801(ProxyOozieClient.java:48)
        at org.apache.oozie.client.ProxyOozieClient$8.call(ProxyOozieClient.java:243)
        at org.apache.oozie.client.OozieClient.doAs(OozieClient.java:244)
        at org.apache.oozie.client.ProxyOozieClient.suspend(ProxyOozieClient.java:240)
        at org.apache.falcon.workflow.engine.OozieWorkflowEngine.suspend(OozieWorkflowEngine.java:1450)
        ... 62 more
2016-03-13 05:33:37,587 ERROR - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Action failed: Bad Request
Error: E0603 : E0603: SQL error in operation, <openjpa-2.2.2-r422266:1468616 fatal general error> org.apache.openjpa.persistence.PersistenceException: An SQL data change is not permitted for a read-only connection, user or database. {prepstmnt 802408967 UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?))} [code=20000, state=25502] FailedObject: UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?)) [java.lang.String] (FalconWebException:83)
2016-03-13 05:33:37,588 ERROR - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ Action failed: Bad Request
Error: default/org.apache.falcon.FalconWebException::org.apache.falcon.FalconException: E0603 : E0603: SQL error in operation, <openjpa-2.2.2-r422266:1468616 fatal general error> org.apache.openjpa.persistence.PersistenceException: An SQL data change is not permitted for a read-only connection, user or database. {prepstmnt 802408967 UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?))} [code=20000, state=25502] FailedObject: UPDATE BUNDLE_ACTIONS SET status = ?, last_modified_time = ?, pending = ? WHERE bundle_action_id IN (SELECT DISTINCT t0.bundle_action_id FROM BUNDLE_ACTIONS t0 WHERE (t0.bundle_action_id = ?)) [java.lang.String]
 (FalconWebException:83)
2016-03-13 05:33:37,589 INFO  - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:ambari-qa:POST//entities/suspend/PROCESS/rawEmailIngestProcess] ~ {Action:suspend, Dimensions:{entityType=PROCESS, colo=NULL, entityName=rawEmailIngestProcess}, Status: FAILED, Time-taken:274654474 ns} (METRIC:38)
2016-03-13 05:33:37,591 DEBUG - [2089900275@qtp-954636025-27 - f867eee0-6f5f-4947-8c73-20df66948767:] ~ Audit: ambari-qa/10.0.2.2 performed request http://127.0.0.1:15000/api/entities/suspend/PROCESS/rawEmailIngestProcess?user.name=ambari-qa (10.0.2.15) at time 2016-03-13T05:33Z (FalconAuditFilter:86)
2016-03-13 05:34:48,077 INFO  - [Thread-11:] ~ config.location is set, using: /usr/hdp/current/falcon-server/conf/runtime.properties (ApplicationProperties:108)
2016-03-13 05:34:48,081 INFO  - [Thread-11:] ~ Initializing org.apache.falcon.util.RuntimeProperties properties with domain falcon (ApplicationProperties:145)
2016-03-13 05:34:48,081 DEBUG - [Thread-11:] ~ log.cleanup.frequency.hours.retention=minutes(1) (ApplicationProperties:151)
2016-03-13 05:34:48,081 DEBUG - [Thread-11:] ~ log.cleanup.frequency.months.retention=months(3) (ApplicationProperties:151)
2016-03-13 05:34:48,081 DEBUG - [Thread-11:] ~ log.cleanup.frequency.minutes.retention=hours(6) (ApplicationProperties:151)
2016-03-13 05:34:48,081 DEBUG - [Thread-11:] ~ domain=falcon (ApplicationProperties:151)
2016-03-13 05:34:48,081 DEBUG - [Thread-11:] ~ log.cleanup.frequency.days.retention=days(7) (ApplicationProperties:151)
2016-03-13 05:34:48,081 INFO  - [Thread-11:] ~ config.location is set, using: /usr/hdp/current/falcon-server/conf/runtime.properties (ApplicationProperties:108)
2016-03-13 05:34:48,082 INFO  - [Thread-11:] ~ Initializing org.apache.falcon.util.RuntimeProperties properties with domain falcon (ApplicationProperties:145)
2016-03-13 05:34:48,082 DEBUG - [Thread-11:] ~ log.cleanup.frequency.hours.retention=minutes(1) (ApplicationProperties:151)
2016-03-13 05:34:48,082 DEBUG - [Thread-11:] ~ log.cleanup.frequency.months.retention=months(3) (ApplicationProperties:151)
2016-03-13 05:34:48,082 DEBUG - [Thread-11:] ~ log.cleanup.frequency.minutes.retention=hours(6) (ApplicationProperties:151)
2016-03-13 05:34:48,082 DEBUG - [Thread-11:] ~ domain=falcon (ApplicationProperties:151)
2016-03-13 05:34:48,082 DEBUG - [Thread-11:] ~ log.cleanup.frequency.days.retention=days(7) (ApplicationProperties:151)
5 REPLIES 5
Highlighted

Re: falcon - oozie DB connection error (derby read-only)

@Hanna Lee

First, Derby is not the recommended database option.

You are correct on "An SQL data change isnot permitted for a read-only connection, user or database"

See this link

These are the notes from previous posts.

1. Stop the oozie service from Ambari 
2. Get the location of the following property 'Oozie Data Dir' (defeult is /hadoop/oozie/data) 
3. Go to the oozie server machine via ssh 
4. Go to the 'Oozie Data Dir' folder and check to see that oozie-db folder and subdirectory and files is owned by oozie:hadoop 
5. If this is not the case, do a chown -R oozie:hadoop <OOZIE_DATA_DIR>/oozie_db 
Highlighted

Re: falcon - oozie DB connection error (derby read-only)

@Hanna Lee I would change the db to mysql

Highlighted

Re: falcon - oozie DB connection error (derby read-only)

New Contributor

It worked for me thanks

Highlighted

Re: falcon - oozie DB connection error (derby read-only)

Explorer

First, I downloaded the HDP sandbox 2.3, that is set to Derby database.

Should I change to mysql database?

Second, during following your guide, 'Oozie Data Dir' is owned by oozie user, but "db.lck" file is owned by root user, so I changed from root:hadoop to oozie:hadoop.

And then I restarted the oozie server, other problem is occurred.

--------------------------------------------------------------------------

<oozie log (ambari)>

Validate DB Connection
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/oozie/libserver/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/oozie/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

Error: Could not connect to the database: java.sql.SQLException: Failed to start database '/hadoop/oozie/data/oozie-db' with class loader sun.misc.Launcher$AppClassLoader@3b25bbd3, see the next exception for details.

Stack trace for the error was (for debug purposes):
--------------------------------------
java.lang.Exception: Could not connect to the database: java.sql.SQLException: Failed to start database '/hadoop/oozie/data/oozie-db' with class loader sun.misc.Launcher$AppClassLoader@3b25bbd3, see the next exception for details.
	at org.apache.oozie.tools.OozieDBCLI.validateConnection(OozieDBCLI.java:955)
	at org.apache.oozie.tools.OozieDBCLI.createDB(OozieDBCLI.java:190)
	at org.apache.oozie.tools.OozieDBCLI.run(OozieDBCLI.java:128)
	at org.apache.oozie.tools.OozieDBCLI.main(OozieDBCLI.java:79)
Caused by: java.sql.SQLException: Failed to start database '/hadoop/oozie/data/oozie-db' with class loader sun.misc.Launcher$AppClassLoader@3b25bbd3, see the next exception for details.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
	at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
	at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
	at org.apache.derby.jdbc.Driver20.connect(Unknown Source)
	at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
	at java.sql.DriverManager.getConnection(DriverManager.java:571)
	at java.sql.DriverManager.getConnection(DriverManager.java:215)
	at org.apache.oozie.tools.OozieDBCLI.createConnection(OozieDBCLI.java:945)
	at org.apache.oozie.tools.OozieDBCLI.validateConnection(OozieDBCLI.java:951)
	... 3 more
Caused by: java.sql.SQLException: Failed to start database '/hadoop/oozie/data/oozie-db' with class loader sun.misc.Launcher$AppClassLoader@3b25bbd3, see the next exception for details.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
	... 17 more
Caused by: java.sql.SQLException: A Read Only database was created with dirty data buffers.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
	... 14 more
Caused by: ERROR XSLAF: A Read Only database was created with dirty data buffers.
	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
	at org.apache.derby.impl.store.raw.log.LogToFile.recover(Unknown Source)
	at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
	at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
	at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
	at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
	at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
	at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
	at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
	... 14 more
--------------------------------------
Highlighted

Re: falcon - oozie DB connection error (derby read-only)

Explorer

@Hanna Lee It looks the same error: ERROR XSLAF: A ReadOnly database was created with dirty data buffers.

When you change the ownership of Oozie Data Dir, could you use "chown -R ..." to make sure all the subdirectories are owned by oozie:hadoop? You also need to restart Oozie after you change the ownership.

BTW. If you use clean HDP 2.3, it should work by default and you should not need extra configuration on derby db.

Don't have an account?
Coming from Hortonworks? Activate your account here