Information

Title	checkwidth validation is not calculated correctly with multibyte characters

URL Name	checkwidth-response-wrong-with-multi-byte-characters

Article Number	000185886

Environment	Product: OpenEdge Version: 11.4, 11.5, 11.6, 11.7 to 11.7.10, 12.0, 12.1, 12.2 to 12.2.5, 12.3 OS: All supported platforms

Question/Problem Description

When a field update inserts multi-byte characters in a UTF-8 session -checkwidth 1 or -checkwidth 2 validation against SQL Width raises erroneous warning 10840
Data insertion fails with 10841 when -checkwidth validation against SQL Width is enabled

The checkwidth response is wrong when multi-byte characters update a field:

When an ABL client has checkwidth 2 enabled, inserting 10 multi-byte characters into a field with format "x(10)" and _width 20 fails with error 10841
When an ABL client has checkwidth 1 enabled, inserting 10 multi-byte characters into a field with format "x(10)" and _width 20 Warning 10840 is raised depending

Same data (inserted without -checkwidth) can be queried successfully via SQL, showing that the MAX-WIDTH setting (_field._width) has not been breached.

Storing 10 x UTF-8 Euro characters (each Euro character being 3 bytes in size) into a field with format "x(10)" and _width 20:

LENGTH(myField, "CHARACTER") returns 10.
LENGTH(myField, "RAW") returns 30.

Steps to Reproduce	The following test uses the Euro symbol which is 3 bytes long. Integer value for the Euro symbol in UTF-8 is 14844588. 1. In a UTF-8 database, create a new table (test) with a single CHARACTER field (chkwdth) with format "x(10)"; default MAX-WIDTH setting for the field will be 20. ADD FIELD "chkwdth" OF "test" AS character DESCRIPTION "what width" FORMAT "X(10)" INITIAL "" LABEL "test field width" LABEL-SA "L28" POSITION 4 MAX-WIDTH 20 VIEW-AS "VIEW-AS FILL-IN NATIVE SIZE 12 BY .9" COLUMN-LABEL "From!UTF8!code" COLUMN-LABEL-SA "L16" HELP "_field._width" ORDER 20 2. Connect to the database with Procedure Editor using parameters: -cpinternal UTF-8 -cpstream UTF-8 -checkwidth 2 3. Run the following code: CREATE test. ASSIGN chkwdth = FILL(CHR(14844588), 10). Error 10841 will occur. Insertion succeeds if the FILL amount is reduced to 6 (3 x 6 = 18) since the SQL width of the field is 20.

Steps to Reproduce

The following test uses the Euro symbol which is 3 bytes long.
Integer value for the Euro symbol in UTF-8 is 14844588.

1. In a UTF-8 database, create a new table (test) with a single CHARACTER field (chkwdth) with format "x(10)"; default MAX-WIDTH setting for the field will be 20.

ADD FIELD "chkwdth" OF "test" AS character
DESCRIPTION "what width"
FORMAT "X(10)"
INITIAL ""
LABEL "test field width"
LABEL-SA "L28"
POSITION 4
MAX-WIDTH 20
VIEW-AS "VIEW-AS FILL-IN NATIVE SIZE 12 BY .9"
COLUMN-LABEL "From!UTF8!code"
COLUMN-LABEL-SA "L16"
HELP "_field._width"
ORDER 20

2. Connect to the database with Procedure Editor using parameters:
-cpinternal UTF-8 -cpstream UTF-8 -checkwidth 2

3. Run the following code:

CREATE test.
ASSIGN chkwdth = FILL(CHR(14844588), 10).

Error 10841 will occur.

Insertion succeeds if the FILL amount is reduced to 6 (3 x 6 = 18) since the SQL width of the field is 20.

Clarifying Information

The -checkwidth parameter is used to prevent data that exceeds the _width of a character, decimal, or raw field being inserted into the database by an ABL client to avoid problems when querying the same data with SQL clients, e.g. error -21002:

Column columnName# in table PUB.tableName has value exceeding its max length or precision. [Database Vendor Code: -210012]

Error Message	WARNING: Width of data is greater than <table.field> (rowid <rowid>) _width. (10840) ERROR: Width of data is greater than <table.field> (rowid <rowid>) _width. (10841)

Defect Number	Defect PSC00348404 / OCTA-3391

Enhancement Number

Cause

-checkwidth uses the RAW length of the data in its calculation.
The issue occurs because the checkwidth validation tries to get the current sql width from the RAW length (byte count).

Resolution

Upgrade to OpenEdge 12.4, 12.2.6, 11.7.11 when available where OCTA-3391 has been fixed.

When a check width on a character field the character count is used to validate the sql width.

Workaround

Notes

Progress Articles:

What is the -checkwidth parameter for?
Default calculations for SQL width
DBTOOL Max Width, Max ArrayWidth and Max ArrayWidth for extent fields

Keyword Phrase

Last Modified Date	7/8/2021 12:29 PM

checkwidth validation is not calculated correctly with multibyte characters

Information