I've tried creating instances, restarted pc several times, but i still get error mentioned in the subject line.
The only reference I could find about this error is in this link, but it's not yet answered.
How to resolve?
OE 11.7 Classroom Edition.
What is likely happening is that the msagent’s (-minport and -maxport range) are defined in the TCP services file (C:\Windows\System32\drivers\etc\services).
The negative numbers you see is the port range in 16-bit signed integer: 65536 – 3534 = 62002 (65536 = max tcp port).
By default the port range is set to 62002-62202, try adjusting the range to i.e. 32002-32202 in the openedge.properties file in the conf folder of your PASOE instance and see if that resolves the problem.