We are trying to install OE 11.5 to Windows 2012 Server R2 and get the following error "The following file(s) failed to self-register: C:\Program Files (x86)\Common Files\Progress Software\11\prox.dll;". This a second similar case to us. When we first met this we contacted Progress Support and produced a massive amount of log files they asked for (Windows Procmon etc). We took down virus protection, UAC and tried safe mode, but that didn't help.
What did help was that we took a clean server installation, installed OE 11.5 and just after that connected the server to the customer's domain and policies. I'm now asking has anyone had a similar problem and managed to solved it with an easier solution.