Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Problem

We are encountering an exception with the retry mechanism of message delivery using Domibus 4.0.1/2 and ActiveMQ 5.15.2.

 

This exception is similar to what we saw in the case of the failover configuration in the case when the ActiveMQ node which was pointed to in activeMQ.broker.host went down and the queue names were not accessible for Domibus to read. Having said that the exception below happens on my local development machine where I don't have the failover configuration.

 

 

Error in the logs: eu.domibus.jms.spi.InternalJMSException: Error getting destinations.

Exception with the retry mechanism of message delivery using Domibus.

 

Steps to reproduce the problem: 

- Send a message from C1 to C4 but let it will fail, in my specific case this started happening when a running C3 was turned off

- After the initial failure the following retries all fail with the exception listed below

 

 


The Domibus configuration section regarding ActiveMQ settings:

 # ---------------------------------- ActiveMQ ----------------------------------

 #The host of the JMS broker

activeMQ.broker.host=localhost

 #The name of the JMS broker

activeMQ.brokerName=localhost

 #The configuration file of the embedded ActiveMQ broker. IN case an external broker is used this property is not needed and it should be removed

#activeMQ.embedded.configurationFile=file:///${domibus.config.location}/internal/activemq.xml

 #The service URL of the MBeanServer

activeMQ.JMXURL=service:jmx:rmi://${activeMQ.broker.host}:${activeMQ.rmiServerPort}/jndi/rmi://${activeMQ.broker.host}:${activeMQ.connectorPort}/jmxrmi

 #The port that the JMX connector will use for connecting to ActiveMQ

activeMQ.connectorPort=1299 

#The RMI server port

activeMQ.rmiServerPort=1300

 #The connection URI that the clients can use to connect to an ActiveMQ broker using a TCP socket

activeMQ.transportConnector.uri=tcp://${activeMQ.broker.host}:61616

 #The username that is allowed to connect to the ActiveMQ broker

activeMQ.username=domibus

 #THe password of the username defined in the activeMQ.username property

activeMQ.password=changeit

  

The stacktrace: 

2019-02-05 13:29:30,936 [] [default] [] ERROR e.d.e.s.SendRetryWorker:41 - Error while enqueing messages.

eu.domibus.jms.spi.InternalJMSException: Error getting destinations

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ.findDestinationsGroupedByFQName(InternalJMSManagerActiveMQ.java:82)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ.browseMessages(InternalJMSManagerActiveMQ.java:168)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ.browseClusterMessages(InternalJMSManagerActiveMQ.java:160)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ$$FastClassBySpringCGLIB$$58ac8377.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)

        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ$$EnhancerBySpringCGLIB$$4037aeda.browseClusterMessages(<generated>)

        at eu.domibus.messaging.jms.JMSManagerImpl.browseClusterMessages(JMSManagerImpl.java:140)

        at eu.domibus.messaging.jms.JMSManagerImpl.browseClusterMessages(JMSManagerImpl.java:134)

        at eu.domibus.messaging.jms.JMSManagerImpl$$FastClassBySpringCGLIB$$f27411b8.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)

        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)

        at eu.domibus.messaging.jms.JMSManagerImpl$$EnhancerBySpringCGLIB$$68d539c0.browseClusterMessages(<generated>)

        at eu.domibus.ebms3.sender.RetryService.getQueuedMessages(RetryService.java:108)

        at eu.domibus.ebms3.sender.RetryService.getMessagesNotAlreadyQueued(RetryService.java:100)

        at eu.domibus.ebms3.sender.RetryService.enqueueMessages(RetryService.java:86)

        at eu.domibus.ebms3.sender.RetryService$$FastClassBySpringCGLIB$$728e9f15.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)

        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)

        at eu.domibus.ebms3.sender.RetryService$$EnhancerBySpringCGLIB$$9325a3f7.enqueueMessages(<generated>)

        at eu.domibus.ebms3.sender.SendRetryWorker.executeJob(SendRetryWorker.java:39)

        at eu.domibus.quartz.DomibusQuartzJobBean.executeInternal(DomibusQuartzJobBean.java:35)

        at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:75)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

Caused by: org.springframework.jmx.access.InvalidInvocationException: org.apache.activemq:type=Broker,brokerName=localhost

        at org.springframework.jmx.access.MBeanClientInterceptor.doInvoke(MBeanClientInterceptor.java:442)

        at org.springframework.jmx.access.MBeanClientInterceptor.invoke(MBeanClientInterceptor.java:350)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)

        at com.sun.proxy.$Proxy66.getQueues(Unknown Source)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ.getQueueMap(InternalJMSManagerActiveMQ.java:113)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ.findDestinationsGroupedByFQName(InternalJMSManagerActiveMQ.java:75)

        ... 44 common frames omitted

2019-02-05 13:29:35,022 [] [default] [] ERROR e.d.e.s.SendRetryWorker:41 - Error while enqueing messages.

eu.domibus.jms.spi.InternalJMSException: Could not find destination for [domibus.internal.dispatch.queue]

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ.browseMessages(InternalJMSManagerActiveMQ.java:170)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ.browseClusterMessages(InternalJMSManagerActiveMQ.java:160)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ$$FastClassBySpringCGLIB$$58ac8377.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)

        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)

        at eu.domibus.jms.activemq.InternalJMSManagerActiveMQ$$EnhancerBySpringCGLIB$$4037aeda.browseClusterMessages(<generated>)

        at eu.domibus.messaging.jms.JMSManagerImpl.browseClusterMessages(JMSManagerImpl.java:140)

        at eu.domibus.messaging.jms.JMSManagerImpl.browseClusterMessages(JMSManagerImpl.java:134)

        at eu.domibus.messaging.jms.JMSManagerImpl$$FastClassBySpringCGLIB$$f27411b8.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)

        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)

        at eu.domibus.messaging.jms.JMSManagerImpl$$EnhancerBySpringCGLIB$$68d539c0.browseClusterMessages(<generated>)

        at eu.domibus.ebms3.sender.RetryService.getQueuedMessages(RetryService.java:108)

        at eu.domibus.ebms3.sender.RetryService.getMessagesNotAlreadyQueued(RetryService.java:100)

        at eu.domibus.ebms3.sender.RetryService.enqueueMessages(RetryService.java:86)

        at eu.domibus.ebms3.sender.RetryService$$FastClassBySpringCGLIB$$728e9f15.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)

        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)

        at eu.domibus.ebms3.sender.RetryService$$EnhancerBySpringCGLIB$$9325a3f7.enqueueMessages(<generated>)

        at eu.domibus.ebms3.sender.SendRetryWorker.executeJob(SendRetryWorker.java:39)

        at eu.domibus.quartz.DomibusQuartzJobBean.executeInternal(DomibusQuartzJobBean.java:35)

        at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:75)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

Solution


Panel
bgColor#fff

Would it be possible to perform Please apply the following changes and perform another test? The scenario is the same with an external ActiveMQ broker: send a message from C2 to C3 while C3 is down and check if the retry is successful?

 

1. Change in domibus.properties

activeMQ.brokerName=domibus

 

2. In the external activemq.xml file change the following:

<broker useJmx="true" brokerName="domibus"


Content by Label
showLabelsfalse
max5
spacesCEKB
showSpacefalse
sortmodified
reversetrue
typepage
cqllabel in ("4_0_1","activemq","domibus","4_0_2") and type = "page" and space = "CEKB"
labelsDomibus ActiveMQ 4_0_1 4_0_2

...