Consuming from a queue, calling external web service -- how to not lose any messages - Forum - Technology Partner - Progress Community

Consuming from a queue, calling external web service -- how to not lose any messages

 Forum

Consuming from a queue, calling external web service -- how to not lose any messages

  • Hi,

    I have a use case where I need to consume some messages from a queue and call an external web service. The messages have some XML data, and the data should be sent to the external web service.

    The messages need to be processed in order, and it is important not to lose any. Even if the back end service is down for a while, I need to deliver them all in order when the service comes back up.

    What is the proper configuration for this?

    I have a process with an entry endpoint set to the queue, and it contains 1 sonic connect client service step. It seems when the back end service is dead, a connect timeout error is logged to the container log, but my message is consumed from the queue and I can't get it back.

    How can I configure this so that I get retries and the message is not consumed from the queue until I confirm that I got a non-fault soap message back from the service?

    Thanks

  • Eric,

    It looks like you have a couple of different challenges. First, message order. If you do not have clustered brokers, look into the 'exclusive' setting in the MQ Configuration and Management Guide. This should ensure that only one reader is active. If you do have clustered brokers, the sender should be configured in a way so that he can only send messages to one broker. If messages can go to more than one broker, message ordering becomes a programming task where you will have to reorder the messages.

    As for retrying messages, I haven't had to be concerned with preserving message order along with providing a retry service, but here's my two cents. I would use a scratch table in a database and set up a service that read from the queue and added the record to the database. Another service would pull messages in order from that table and delete the row if the service call succeeded. I'm sure there are ways to do that with a queue mechanism, but I can't think of a simple way to do that.

    Also, my experience goes up to 7.6. 8.x may have other ways to accomplish this.