10MB message size limit - Forum - Technology Partner - Progress Community
 Forum

10MB message size limit

  • We are using Sonic v7.6.2. We come to know from Sonic MQ documentation that maximum message size limit is 10MB.
    Does it apply to Sonic ESB Process exposed as a WebService?
    Does it apply to Sonic ESB Process which access external WebService?
  • This is a soft-limit and a default value only. It is imposed by the messaging broker. You can change this value or setting it to unlimited by configuring the broker's advanced property CONNECTION_TUNING_PARAMETERS.MAX_MSG_SIZE

    Quoting from the "Configuration and Management Guide", section "Configuring SonicMQ Brokers"

    CONNECTION_TUNING_PARAMETERS.MAX_MSG_SIZE
    — Specifies the maximum acceptable message size by setting its value as
    a non-negative integer value that specifies the allowed per-message
    size limit in megabytes. The default value is 10 MB. Setting the value to zero (0) disables the feature.

    And yes, it applies to any of the mentioned use cases since it will typically touch the broker.

    Thomas

  • Thanks Thomas!

  • Is 10MB message size a recommended size?

    How can we arrived at ideal max message size for our application?

    Does Sonic has any capacity/performance model that tracks message size/frequency/volumes with message latency and computing resources like memory or CPU requirements for the ESB?

  • 10 MB is what we test with but we have used larger message sizes before. Personally, I have used up to 100 MB for a single message. The larger you go though the more careful you need to be with what you are trying to do with the message. E.g. if you want to execute an XSL transformation then we have to parse to DOM, execute the transform, get a new DOM and serialize this again. Since a DOM is 5 time the original message and we have 2 of them we allready look at 1 + 5 + 5 + 1 = 12 times the orinignal message size that you need in heap in the ESB container doing the transform.

    In summary, while you can go larger the risk is you run out of memory.

    Thomas

  • As a comment, the issue with passing 10M messages in WebService calls is that you often find some clients will time out waiting for the response.  If you have a choice of size, and are using HTTP, you might have better luck optimizing for smaller sizes.

  • Thanks Thomas! 12x is a very scary number.
    Do you have any idea as how much memory it takes to validate and transform CSV based message (source and destination both formats are CSV)? Third party CSV library like OpenCVS, JavaCSV will be used for data validation and transformation.