SSL routines error with ABL Web Services or HTTP Client and SNI-enabled site.
SSL routines error with ABL Web Services or HTTP Client and ServerNameIndication enabled site.
SSL routines error occurs when trying to connect to an SNI-enabled site with the ABL Web Services or HTTP Client.
SSL error while connecting to WebService using the ABL Web Services or HTTP Client.

 

SSL debug logging shows:

[Day Mmm DD HH:MM:SS YYYY] ID-0x5b33cfd0 CTX-0x00000000 BIO-0x00000000 INFO  --- Initialized OpenSSL client logging
[Day Mmm DD HH:MM:SS YYYY] ID-0x5b33cfd0 CTX-0x00000000 BIO-0x00000000 INFO  --- Enabling SSL Client session reuse
[Day Mmm DD HH:MM:SS YYYY] ID-0x5b33cfd0 CTX-0x00000000 BIO-0x00000000 INFO  --- Set default SSL Client CipherSuites: AES128-SHA
[Day Mmm DD HH:MM:SS YYYY] ID-0x5b33cfd0 CTX-0x00000000 BIO-0x00000000 INFO  --- Configured with default SSL Client Protocol : TLSv1
[Day Mmm DD HH:MM:SS YYYY] ID-0x5b33cfd0 CTX-0x5b3eb710 BIO-0x00000000 INFO  --- SSL Client starting handshake with host (www.sni-enabled-host.com)
[Day Mmm DD HH:MM:SS YYYY] INTERNAL STATE OPERATION --- SSL State: 16 before/connect initialization
[Day Mmm DD HH:MM:SS YYYY] INTERNAL STATE OPERATION --- SSL_connect:before/connect initialization
>>> SSL 2.0 [length 0074], CLIENT-HELLO
    01 03 01 00 4b 00 00 00 20 00 00 39 00 00 38 00
    00 35 00 00 16 00 00 13 00 00 0a 07 00 c0 00 00
   33 00 00 32 00 00 2f 03 00 80 00 00 05 00 00 04
    01 00 80 00 00 15 00 00 12 00 00 09 06 00 40 00
    00 14 00 00 11 00 00 08 00 00 06 04 00 80 00 00
    03 02 00 80 68 5b b1 08 57 b9 7b 77 07 38 c9 49
    c6 c9 a0 7e 6f 95 e4 13 66 47 58 dc 2e 87 9b 39
    9b c1 96 44
[Day Mmm DD HH:MM:SS YYYY] INTERNAL STATE OPERATION --- SSL_connect:SSLv2/v3 write client hello A
[Day Mmm DD HH:MM:SS YYYY] INTERNAL STATE OPERATION ---  read:fatal:handshake failure
[Day Mmm DD HH:MM:SS YYYY] INTERNAL STATE OPERATION --- SSL_connect:error in SSLv2/v3 read server hello A
[Day Mmm DD HH:MM:SS YYYY] ID-0x5b33cfd0 CTX-0x5b3eb710 BIO-0x5b3d6c10 ERROR --- SSL Client handshake failure (336032784) SSL routines
[Day Mmm DD HH:MM:SS YYYY] ID-0x5b33cfd0 CTX-0x5b3eb710 BIO-0x5b3d6c10 INFO  --- Terminated SSL Client session

Performing the equivalent connection attempt using SSLC fails with the same errors:

sslc s_client -connect www.sni-enabled-host.com:443 -tls1 -cipher AES128-SHA -CApath %DLC%\certs

CONNECTED(00000124)
7356:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl\record\rec_layer_s3.c:1385:SSL alert number 40

...

Performing the equivalent connection attempt using SSLC and specifying the -servername parameter goes further:

sslc s_client -connect www.sni-enabled-host.com:443 -tls1 -cipher AES128-SHA -CApath %DLC%\certs -servername www.sni-enabled-host.com

CONNECTED(00000130)
depth=3 C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
verify error:num=20:unable to get local issuer certificate
Server did acknowledge servername extension.
...
 

This is expected behavior. SNI support does not cover ABL sockets at this time. As a result, the HTTP Client does not support SNI either.

SNI stands for Server Name Indication and is an extension of the TLS protocol. It indicates which hostname is being contacted by the browser at the beginning of the handshake process. This technology allows a server to connect multiple SSL Certificates to one IP address and gate.

If the certificate provided by the site has a subject alternative name in the certificate properties/details and this name differs from the action hostname of the machine SNI may be used to allow connectivity to the site.

Upgrade to OpenEdge 11.7.3 or later, where the SNI support has been added for the ABL Web Services and HTTP Client.

ABL Web Services Client
The following parameters are now available within the CONNECT() method:

• -servername : Sets the hostname for both the WSDL URL and the SOAP Endpoint.
• -wsdlServername : Sets the hostname for the WSDL URL.
• -soapServername : Sets the hostname for the SOAP Endpoint.

For example:

happsrv1:CONNECT("-WSDL https://www.example.com/wsdl -servername www.example.com").

ABL HTTP Client
A ServerNameIndicator property has been added to the ClientSocketConnectionParameters object. This sets the ABL socket's -servername property if set and when the connection is via HTTPS.
Support for the same setting has been added to the ClientLibraryBuilder and related types.

For example:

USING OpenEdge.Net.HTTP.IHttpRequest.
USING OpenEdge.Net.HTTP.IHttpResponse.
USING OpenEdge.Net.HTTP.ClientBuilder.
USING OpenEdge.Net.HTTP.RequestBuilder.
USING OpenEdge.Net.HTTP.IHttpClientLibrary.
USING OpenEdge.Net.HTTP.Lib.ClientLibraryBuilder.

DEFINE VARIABLE oLib AS IHttpClientLibrary NO-UNDO.
DEFINE VARIABLE oRequest  AS IHttpRequest NO-UNDO.
DEFINE VARIABLE oResponse AS IHttpResponse NO-UNDO.

ASSIGN
    oLib        = ClientLibraryBuilder:Build()
                                      :sslVerifyHost(NO)
                                      :ServerNameIndicator('some.servername.com')
                                      :library.                                                      
oRequest = RequestBuilder:Get('https://some.servername.com/'):Request.
oResponse = ClientBuilder:Build()
                         :UsingLibrary(oLib)
                           :Client
                           :Execute(oRequest).
                           
MESSAGE oResponse:StatusCode SKIP oResponse:StatusReason SKIP VIEW-AS ALERT-BOX.

References to other documentation:

What is SNI? 
https://www.ssls.com/knowledgebase/what-is-sni-technology/

Progress article(s):
How to enable SSL debugging in OpenEdge?
The ABL client does not support an SNI enabled Web Service