
Anthony Hashemi (Customer) asked a question.
On a couple of occasions where our Marklogic cluster has been under heavy load, it has resulted in a couple of large requests being stuck in a "cancelling" status for around 20 hours or so. Is this normal, and how can we handle it within a day?
On a couple of occasions where our Marklogic cluster has been under heavy load, it has resulted in a couple of large requests being stuck in a "cancelling" status for around 20 hours or so.
During that period, the CPU of the EC2 instance one of the nodes stays at a higher baseline.
Is this normal, and how can we handle it within a day?
No, it's not normal. Sounds like something has gone wrong. Normally, when queries are in "cancelling" state, there is some rollback or cleanup happening, but it should not take hours.
What sort of appserver is this happening on? HTTP, XDBC, ODBC?
Hi Mads,
Thanks for the swift response. That was my understanding but the 2 requests I have seen, I believe were just querying the database so I assumed no cleanup would be necessary and indeed would be much quicker than 20 hours if so. I believe this is on HTTP.
James Kenwood reached out and suggested I create a support case which I have done, so hopefully will get to the bottom of it there.
I will update here when we get to the bottom of it.
Thanks again,
Anthony
On the host that is showing the "cancelling" request, run `sudo perf top` to see if it shows what it's doing. You could capture perf report and upload to the Support case. https://community.progress.com/s/article/using-perf-to-understand-performance
If the output of perf looks like it's "stuck" performing gzip compression (i.e. top items showing time spent in slide_hash, crc32_z, longest_match), then you might try disabling gzip compression for the HTTP appservers by applying the "HTTPServer Disable Compression" trace event in the Diagnostics page (Groups > group_name > Diagnostics ) of the Admin UI.
Unfortunately we have not been able to reproduce the issue yet. We have observed 2 distinct occurrences, generally when the instances have been under sever,e unexpected load. If we have capacity, we will try to reproduce it in a lower environment, and follow those steps you outlined.
Thanks,
Anthony