We have switched the frontend TLS certificate from a public CA (DigiCert) TLS/SSL certificate to an internal certificate issued by AWS Private CA.

After updating the TLS certificate, Automate UI seems to work correctly but we are getting an SSL error on chef-client runs and while bootstrapping new chef nodes.

Read on in the Resolution and other sections for more. 
 

Error Message:
 

Chef Infra Client encountered an error attempting to load the node data for node.  OpenSSL::SSL::SSLError: SSL Error connecting to (node url).

ERROR: SSL Validation failure connecting to host: (hostname) 

SSL_connect returned=1 errno=0 state=error: certificate verify failed (self-signed certificate in certificate chain) 

WARN: Error while reporting run start to Data Collector URL

 

The configured certificate was not set up properly and is missing certificates from the full certificate chain in the Automate configuration. In addition, the trusted_certs directory from the client machines should have this certificate chain in a .pem formatted file.

We suggest you update the Automate configuration with a full chain certificate. It needs to be more than one certificate first host certificate and another be the root cert for the Private CA. It would look like the below:
 

[global.v1.frontend_tls]]
cert = """-----BEGIN CERTIFICATE-----
<your host certificate>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<your private CA root certificate>
-----END CERTIFICATE-----
"""

Once you update the configuration file, patch it and restart the Automate service.

Other helpful checks to make and further details on this type of issue:

When you bootstrap a node, the Chef Infra Client copies the SSL certificates for the Chef Infra Server onto the node, into its Trusted_Certs directory. The trusted_certs directory on the node is located at:
- Windows Systems: C:\chef\trusted_certs
- All other systems: /etc/chef/trusted_certs

The trusted_certs directory on Chef Infra Client is a trusted certificate store for all communication in the Chef Infra system. Chef Infra trusts all SSL certificates stored in this directory.

Place private certificates in the trusted_certs directory to use them within Chef Infra Client.

Use the chef_client_trusted_certificate (https://docs.chef.io/resources/chef_client_trusted_certificate/) Chef Infra Client resource to manage these certificates continuously.

The default path of the SSL certificate authority (CA) bundle for Chef Infra Client is as follows:
- Windows System: C:\opscode\chef\embedded\ssl\cacert.pem.

- All other systems: /opt/chef/embedded/ssl/certs/cacert.pem 

You can use the SSL_CERT_FILE environment variable to specify the location for the SSL certificate authority (CA) bundle for Chef Infra Client. Please take a look at this link for more information: (https://docs.chef.io/chef_client_security/#ssl_cert_file).

The SSL certs should be updated in config.toml only. There are no other locations.

The certificate details should be in the below order.
 

-----BEGIN CERTIFICATE-----
(Your primary SSL certificate: your_domain_name.crt)
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
(Your intermediate certificate)
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
(Your root certificate)
-----END CERTIFICATE-----

That all needs to be in multi-line quotes, triple quotes like

"""
multi
line
stuff
here
"""


Please find the reference documentation links.

https://docs.chef.io/chef_client_security/

https://docs.chef.io/resources/chef_client_trusted_certificate/