Cannot merge IDs

@Marine_Ingwe tried to merge 2 IDs and was unable to both through the merge functionality as well as by manually removing the old ID and adding the new ID.

ID being merged (i.e. removed): OTF_BCC_temp10

ID being merged into (i.e. new, correct ID): OTF_BCC_LF0003

We tested as per steps below and were able to replicate her findings – the 2 IDs are not able to be merged.

  1. Using the following URL to force the merge - https://africancarnivore.wildbook.org/merge.jsp?individualA=4478c40e-084d-4fca-be81-b89611e30040&individualB=cbe75a36-c6d6-401f-adb3-a4020faae4cd - generated a “Success” message but it was incorrect - the 2 IDs were un-merged:

  2. On the 2 encounters assigned to OTF_BCC_temp10, we removed the encounter from the marked individual using that functionality in the Identity section. This generated a very long error message and did not complete the assignment of the new ID (OTF_BCC_LF0003):

OTF_BCC_temp10 does not have any social associations (no social unit assigned and no social relationship).

OTF_BCC_LF0003 does have a social relationship, assigned as cub of OTF_BCC_LF0002. Since this is the ID that’s being retained, this shouldn’t have affected the merge but we tested removing the social relationship from OTF_BCC_LF0003 and then went through all of the same merge and remove-re-assign IDs steps as above and received the same results.

URL’s as follows:

Encounters of BCC_temp10 that don’t merge:

https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=a4794979-d190-443a-b7fb-7b1447bcc8b3

https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=5367b7b2-7e11-40a6-949b-34cc0fe69b43

Please investigate and provide solution if possible.

Thanks

Paul

pasting error message from manual encounter page identity update:

Execution of method "add" on field "encounters" caused an error : Add request failed : INSERT INTO "MARKEDINDIVIDUAL_ENCOUNTERS" ("INDIVIDUALID_OID","CATALOGNUMBER_EID","IDX") VALUES (?,?,?) Add request failed : INSERT INTO "MARKEDINDIVIDUAL_ENCOUNTERS" ("INDIVIDUALID_OID","CATALOGNUMBER_EID","IDX") VALUES (?,?,?) org.datanucleus.exceptions.NucleusDataStoreException: Add request failed : INSERT INTO "MARKEDINDIVIDUAL_ENCOUNTERS" ("INDIVIDUALID_OID","CATALOGNUMBER_EID","IDX") VALUES (?,?,?) at org.datanucleus.store.rdbms.scostore.JoinListStore.internalAdd(JoinListStore.java:317) at org.datanucleus.store.rdbms.scostore.AbstractListStore.add(AbstractListStore.java:114) at org.datanucleus.store.types.wrappers.backed.Vector.add(Vector.java:779) at org.datanucleus.state.RelationshipManagerImpl.processManyToOneBidirectionalRelation(RelationshipManagerImpl.java:880) at org.datanucleus.state.RelationshipManagerImpl.process(RelationshipManagerImpl.java:404) at org.datanucleus.ExecutionContextImpl.performManagedRelationships(ExecutionContextImpl.java:3934) at org.datanucleus.ExecutionContextImpl.flush(ExecutionContextImpl.java:3997) at org.datanucleus.ExecutionContextThreadedImpl.flush(ExecutionContextThreadedImpl.java:436) at org.datanucleus.ExecutionContextImpl.preCommit(ExecutionContextImpl.java:4165) at org.datanucleus.ExecutionContextImpl.transactionPreCommit(ExecutionContextImpl.java:770) at org.datanucleus.TransactionImpl.internalPreCommit(TransactionImpl.java:385) at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:275) at org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:107) at org.ecocean.Shepherd.commitDBTransaction(Shepherd.java:4168) at org.ecocean.Shepherd.updateDBTransaction(Shepherd.java:4216) at org.ecocean.servlet.IndividualAddEncounter.doPost(IndividualAddEncounter.java:109) at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:458) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:373) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:370) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.ecocean.servlet.StrutsUTF8Filter.doFilter(StrutsUTF8Filter.java:13) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "MARKEDINDIVIDUAL_ENCOUNTERS_pkey" Detail: Key ("INDIVIDUALID_OID", "IDX")=(4478c40e-084d-4fca-be81-b89611e30040, 26) already exists. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2155) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:288) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:430) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:356) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:168) at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:135) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeUpdate(ParamLoggingPreparedStatement.java:393) at org.datanucleus.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:431) at org.datanucleus.store.rdbms.scostore.JoinListStore.internalAdd(JoinListStore.java:298) ... 64 more Nested Throwables StackTrace: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "MARKEDINDIVIDUAL_ENCOUNTERS_pkey" Detail: Key ("INDIVIDUALID_OID", "IDX")=(4478c40e-084d-4fca-be81-b89611e30040, 26) already exists. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2155) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:288) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:430) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:356) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:168) at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:135) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeUpdate(ParamLoggingPreparedStatement.java:393) at org.datanucleus.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:431) at org.datanucleus.store.rdbms.scostore.JoinListStore.internalAdd(JoinListStore.java:298) at org.datanucleus.store.rdbms.scostore.AbstractListStore.add(AbstractListStore.java:114) at org.datanucleus.store.types.wrappers.backed.Vector.add(Vector.java:779) at org.datanucleus.state.RelationshipManagerImpl.processManyToOneBidirectionalRelation(RelationshipManagerImpl.java:880) at org.datanucleus.state.RelationshipManagerImpl.process(RelationshipManagerImpl.java:404) at org.datanucleus.ExecutionContextImpl.performManagedRelationships(ExecutionContextImpl.java:3934) at org.datanucleus.ExecutionContextImpl.flush(ExecutionContextImpl.java:3997) at org.datanucleus.ExecutionContextThreadedImpl.flush(ExecutionContextThreadedImpl.java:436) at org.datanucleus.ExecutionContextImpl.preCommit(ExecutionContextImpl.java:4165) at org.datanucleus.ExecutionContextImpl.transactionPreCommit(ExecutionContextImpl.java:770) at org.datanucleus.TransactionImpl.internalPreCommit(TransactionImpl.java:385) at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:275) at org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:107) at org.ecocean.Shepherd.commitDBTransaction(Shepherd.java:4168) at org.ecocean.Shepherd.updateDBTransaction(Shepherd.java:4216) at org.ecocean.servlet.IndividualAddEncounter.doPost(IndividualAddEncounter.java:109) at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:458) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:373) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:370) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.ecocean.servlet.StrutsUTF8Filter.doFilter(StrutsUTF8Filter.java:13) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829)

Thanks for the detailed troubleshooting steps you tried!

I don’t have a timeline for a resolution, but I did want to give you a heads up that this will likely take a bit of time for our devs to tackle. I’ll post a ticket link here when it’s ready.

Thanks Anastasia

I appreciate the update as to timing.

Paul

Quick update: The merge issue has been fixed in ACW and was related to some weird code associated with one of OTF_BCC_LF0003’s encounters. A check against the entire database showed that this individual was the only one impacted by the bug.

We think this has also fixed the error message when manually updating ID and are waiting on confirmation.