After the upgrade from automate 3.x to 4.x we usually find the automate-minio service being in a CRITICAL state. 

The error messages that can be found in the journalctl logs are :

/hab/svc/automate-minio/data: Drive /hab/svc/automate-minio/data: found backend type fs, expected xl or xl-single

 

The above error says:

Indicates a mismatch between the expected storage backend configuration of MinIO and the actual storage backend that's being used.

Understanding MinIO Backends:

• fs (Filesystem):
  • This backend uses the local filesystem directly to store objects.
• xl (Erasure Coding):
  • This backend distributes data across multiple drives using erasure coding, providing data redundancy and fault tolerance.
  • It's designed for production environments where data durability and availability are critical.
• xl-single:
  • This is a variant of the XL backend designed for single-drive use. Although it uses the XL backend code, it does not provide any data redundancy.

 

Why the Mismatch Occurs During an Upgrade (3.x to 4.x):

Configuration Changes:

• Chef Automate 4.x introduced changes to its default MinIO configuration, specifically expecting an xl or xl-single backend for improved reliability.
• If your 3.x installation was using the fs backend (perhaps for a simple setup), the upgrade process might not have correctly migrated or converted the storage backend.

The error indicates that MinIO is finding existing data in the directory /hab/svc/automate-minio/data that was created using the simple fs storage backend. However, the MinIO service is now configured (or expected) to use the more robust xl or xl-single erasure coding backend. This mismatch results in the service failing to start.

 

Solution1: 

• Replace the fs to xl or xl-single in the file /hab/svc/automate-minio/data/.minio.sys/format.json



Solution2:

• Else you can follow: 
You can fix this by patching to remove builder, remove the pkgs and svc directories for automate-minio then repatch to add builder back and let it rebuild the directories. You need to make sure, you had no data or pkgs uploaded to the internatl builder, so you didn't loose anything configuration should all still there.

If you need any further assistance please reach out to Chef Support.