Learn how to use the iconv() API.

For the last two months, in the articles "In Search of System Values" and "Accessing System Values," we've been using the subprocedure ConvertBuffer() to convert the UTF-8 (CCSID 1208) encoded XML output of the Retrieve Command Definition (QCDRCMDD) API to the CCSID of the running job. In those articles, any discussion of the ConvertBuffer() subprocedure was deferred to a later article. This is that "later" article. The source for the subprocedure we'll be discussing is shown below.

pConvertBuffer   b                                            

dConvertBuffer   pi                                            

dIconvOpen       pr           52   extproc('QtqIconvOpen')  

d ToCode                       32   const                    

d FromCode                     32   const                    

dIconv           pr           10i 0 extproc('iconv')          

d iconv_t                       52   value                    

d InputPtr                       *                            

d BytesToCvt                   10i 0                          

d OutputPtr                       *                              

d BytesAvlForCvt               10i 0                          

dIConvClose       pr           10i 0 extproc('iconv_close')    

d iconv_t                       52   value                    

dInputPtr         s               *   inz(%addr(CmdD_UTF8))

dOutputPtr       s               *   inz(%addr(CmdD_Job))  

dBytAvl_CmdD     s             10i 0 inz(%size(CmdD_Job))  

dRtnVal           s             10i 0                      

dFromCode         ds                 qualified            

d CCSID                         10i 0 inz(1208)              

d ConvAlt                       10i 0 inz(0)                

d SubstAlt                     10i 0 inz(0)                

d SSAlt                         10i 0 inz(0)                

d InpLenOpt                     10i 0 inz(0)                

d ErrOpt                       10i 0 inz(0)                

d                               8   inz(*ALLx'00')        

dToCode           ds                 qualified            

d CCSID                         10i 0 inz(0)                

d ConvAlt                       10i 0 inz(0)                

d SubstAlt                     10i 0 inz(0)                        

d SSAlt                         10i 0 inz(0)                        

d InpLenOpt                     10i 0 inz(0)                        

d ErrOpt                       10i 0 inz(0)                        

d                               8   inz(*ALLx'00')                

diconv_t         ds                                                

d                               10i 0 dim(13)                      

/free                                                              

iconv_t = IconvOpen(ToCode :FromCode);                            

RtnVal = Iconv(iconv_t :InputPtr :BytRtn :OutputPtr :BytAvl_CmdD);

RtnVal = IconvClose(iconv_t);                                    

/end-free                                                          

pConvertBuffer   e                        

While the subprocedure contains quite a bit in terms of prototype and data definition, there are only three executable lines, which represent the use of three APIs in performing the CCSID conversion from UTF8 to the job CCSID. These APIs are Code Conversion Allocation (QtqIconvOpen), Code Conversion (iconv), and Code Conversion Deallocation (iconv_close).

The QtqIconvOpen API is used first and identifies the type of CCSID conversion to be performed. There are two input parameters, and the API returns a return value.

The first parameter identifies the CCSID to convert to, the second parameter the CCSID being converted from. In addition, the second parameter defines characteristics of the CCSID conversion, such as conversion alternatives and how the length of the data to be converted is determined. Both of the input parameters to the QtqIconvOpen API use a common structure which is formatted as…

• A 4-byte integer identifying a CCSID value. The special value 0 indicates that the job CCSID is to be used or, if the job CCSID is 65535, the job default CCSID.
• A 4-byte integer, used only with the second input parameter, identifying the conversion alternative to be used. A value of 0 represents a round-trip conversion, 57 an enforced subset match conversion, and 102 a best-fit conversion.
• A 4-byte integer, used only with the second input parameter, indicating whether the number of substitution characters encountered (when the conversion alternative value is 57) is to be returned by the iconv API when CCSID conversions are performed.
• A 4-byte integer, used only with the second input parameter, indicating how DBCS shift-states are to be set when calling the iconv API.
• A 4-byte integer, used only with the second input parameter, specifying how the iconv API is to determine the length of data to convert. A value of 0 indicates that iconv will be explicitly provided with the length of data to convert by way of a parameter passed to the API, a value of 1 that the iconv API is to determine the length based on the presence of a null byte in the data stream to be converted.
• A 4-byte integer, used only with the second parameter, indicating whether it is an error when DBCS data is encountered when converting from a mixed-byte CCSID to a SBCS CCSID. A value of 0 indicates that the iconv API is to use a substitution character when DBCS data is encountered, a value of 1 that the API should return an error.
• An 8-byte character field that is currently reserved and must be set in its entirety to x'00's.

The return value of QtqIconvOpen is defined as an array of thirteen 4-byte integer values and is used in subsequent calls to the iconv and iconv_close APIs in order to identify the open CCSID conversion that is to be used by these APIs. With one exception, how the system uses this array is not documented. As a user of the API, you simply need to make sure you have allocated an array of thirteen 4-byte integer elements for this return value. The one exception is that if the first element of this array is returned with a value of -1, then an error was encountered when opening the CCSID conversion session. In this situation, errno can be examined for further information on the failure and subsequent calls to APIs such as iconv should not be done (using this returned descriptor).

Data structure ToCode is used in subprocedure ConvertBuffer() as the first parameter when calling the QtqIconvOpen API and data structure FromCode as the second parameter. The initialization of ToCode indicates that the XML data is to be converted to the job CCSID as subfield ToCode.CCSID is set to 0. The initialization of FromCode indicates that the data being converted is currently encoded as UTF8 (FromCode.CCSID is set to 1208), that a round-trip conversion is to be done (FromCode.ConvAlt is set to 0), and that the program will explicitly tell the iconv API how many bytes of data to convert (FromCode.InpLenOpt is set to 0).

While subprocedure ConvertBuffer()calls the QtqIconvOpen() API only once, a program (or job) can have multiple CCSID conversion sessions defined concurrently. Though perhaps a bit of overkill, you can have in excess of 100,000 types of CCSID conversions open at the same time.

The second API used is iconv and is used to perform the actual data conversion from one CCSID to another. There are five parameters passed to the iconv API, and the API returns a return value.

The first parameter to iconv is input-only and is a CCSID conversion session descriptor previously returned by the QtqIconvOpen API. This parameter is passed by value, as indicated in the iconv API prototype Iconv found in subprocedure ConvertBuffer.

The second parameter is a pointer that is used as both an input and an output parameter. On input to the iconv API, this pointer addresses the first character to be converted, and upon return by the API, this pointer addresses the first byte following the last byte used in the current conversion. In subprocedure ConvertBuffer, this parameter is prototyped as a pointer passed by reference. An alternative approach would be to prototype a pointer passed by value where that pointer value is set to the address of the updateable pointer variable.

The third parameter is a 4-byte integer value that can be used as both an input and output parameter. If the input length determination option (FromCode.InpLenOpt in subprocedure ConvertBuffer), when previously calling the QtqIconvOpen API, was set to 0, then on input to the iconv API, this parameter provides the number of bytes to be converted. Upon return from the API, the value of this parameter reflects the number of bytes left to convert. If an input length determination option (FromCode.InpLenOpt) value of 1 was set when calling QtqIconvOpen, then this parameter, when calling iconv, must be set to 0.

The fourth parameter to iconv is, like the second parameter, a pointer that is used for both input and output. On input, this pointer addresses the first byte to be used when writing the CCSID converted data. Upon return from the API, this pointer is updated to address the first byte following the last byte of the converted data that was written.

The fifth parameter is, like the third parameter, a 4-byte integer value that is used as both input and output. On input to iconv, this parameter contains the number of bytes available in the output buffer for the writing of converted data. Upon return from the API, this parameter reflects the number of bytes remaining (still available for use) in the output buffer after the writing of any converted data.

The return value of the iconv API is a 4-byte integer value providing status information related to the API. A return value of 0 or greater indicates that the conversion was successful while a value of -1 indicates failure (with errno providing additional information concerning the failure). A positive non-zero return value is possible if, when calling the QtqIconvOpen API, a count of the substitution characters encountered is requested (that is, if a value of 1 was used for FromCode.SubstAlt in subprocedure ConvertBuffer).

In subprocedure ConvertBuffer, the call to iconv is done with the following statement.

RtnVal = Iconv(iconv_t :InputPtr :BytRtn :OutputPtr :BytAvl_CmdD);

The parameter iconv_t is the CCSID conversion descriptor returned by the previous call to QtqIconvOpen and requests a CCSID conversion from UTF8 to the job CCSID. InputPtr is a pointer initialized to the address of our input buffer (subfield CmdD_UTF8 of data structure CmdD_RcvVar, which is the UTF8 encoded XML data returned by the Retrieve Command Definition (QCDRCMDD) API) and addresses the first byte of the XML data to be converted. BytRtn is the length of the XML data returned by the QCDRCMDD API—that is, how much UTF8 data is to be converted. OutPutPtr is a pointer initialized to the first byte of our output buffer (CmdD_Job) and addresses where the first byte of CCSID converted XML data is to be written. BytAvl_CmdD is the size of the CmdD_Job output buffer and defines how many bytes of converted XML data can be written in the job CCSID. The call to iconv by ConvertBuffer is a bit simplistic in that it assumes success and assumes that all of the XML data can be converted into an output buffer of 65527 bytes. But knowing the characteristics of the DSPSYSVAL command, this seems reasonable. In a future article, we can look at more-complex CCSID conversion environments where we may need to do a wee bit more checking on the returned results of the iconv API.

The third API is iconv_close and is used to close a previously opened CCSID conversion session. There is one input parameter, and the API returns a return value. The input parameter identifies the open CCSID conversion that is to be closed and is the conversion descriptor previously returned by the QtqIconvOpen API. The API return value can be used to determine whether iconv_close was successful or not in closing the CCSID conversion session. A value of 0 indicates success, and a value of -1 that an error was encountered. In an error situation, errno can be used to determine the type of failure.

While use of the iconv_close API is optional (all open CCSID conversion sessions will be closed implicitly when the job ends), each open CCSID conversion session does consume some system storage. So closing a session when you know you're done with it is recommended.

CCSID conversions, under program control, are that simple—or at least they are in the case of the LSTSYSVAL program.

Next month, we'll look at some additional considerations that come into play when doing CCSID conversions—in particular, conversions involving UTF8 as use of this encoding is rather pervasive when working in a network of systems. Though not difficult to accomplish, we'll also look at LSTSYSVAL support for displaying the QLOCALE system value (which I had originally intended for this month but am now deferring due to space reasons).

As usual, if you have any API questions, send them to me at This email address is being protected from spambots. You need JavaScript enabled to view it.. I'll see what I can do about answering your burning questions in future columns.

Bruce Vining is president and co-founder of Bruce Vining Services, LLC, a firm providing contract programming and consulting services to the System i community. He began his career in 1979 as an IBM Systems Engineer in St. Louis, Missouri, and then transferred to Rochester, Minnesota, in 1985, where he continues to reside. From 1992 until leaving IBM in 2007, Bruce was a member of the System Design Control Group responsible for OS/400 and i5/OS areas such as System APIs, Globalization, and Software Serviceability. He is also the designer of Control Language for Files (CLF).A frequent speaker and writer, Bruce can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it.. 

---

MC Press books written by Bruce Vining available now on the MC Press Bookstore.

		IBM System i APIs at Work Leverage the power of APIs with this definitive resource. List Price $89.95 Now On Sale

Also Read

The API Corner: Moving Jobs Within and Across Job Queues

The API Corner: What Time Zone Do You Want That Date and Time In?

The API Corner: How Do You Want That Date and Time?