06
Fri, Dec
6 New Articles

Access Database Files Dynamically in RPG

RPG
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Learn how to access database files via the UFCB. This practical example repairs invalid data in OPEN fields due to DBCS character truncation in a random database file.

 

Programs written in IBM i high-level languages (HLLs) such as RPG are bound to files they access statically, or in other words, at compile time. However, in VRM610, RPG was enhanced to allow passing files as parameters to RPG programs and procedures (see File Parameters). The enhancement makes accessing files from RPG more flexible, but RPG still cannot access a file as dynamically as the record I/O routines of the C libraryfor example, it can't open a file by name and then get the record length of the opened file, which is needed to perform further file I/O.

 

This article introduces techniques to access database files dynamically via the User File Control Block (UFCB), on which database file operations in all IBM i HLLs are based. As an RPG programmer, you might be surprised to find out that there is no real limitation that prevents you from accessing a database file determined at run-time; it is the RPG programming language that does not provide such support for you.

 

Along with introducing UFCB-based file operations, I'll demonstrate the practical ILE RPG utility program RPROPNFLD (rpropnfld.rpgle), which repairs invalid OPEN fields due to double-byte character set (DBCS) character truncation in a database file. The OPEN type is one of the four DBCS-capable character data types defined in IBM i (at either the OS level or the MI level) and can contain a mixture of double-byte character and single-byte character data. If double-byte character data is present, it must be surrounded by a shift-out control character (SO = hex 0E) and a shift-in control character (SI = hex 0F). If an OPEN field is truncated at right with a piece of the DBCS data without the tailing SI, the OPEN field becomes invalid and is likely to cause unpredictable errors. RPROPNFLD is expected to achieve the following tasks:

  • Open a database file by name dynamically for read and update
  • Determine the record length of the opened database file member after the member is opened
  • Read each record and locate the OPEN field in the read record by the input field position parameter
  • Check for validity of the data stored in the OPEN field and repair it if the data is invalid

The User File Control Block (UFCB)

Leif Svalgaard wrote the following in chapter 12 of his great e-book AS/400 Machine-level Programming:

Files are opened, read/written, then closed. At the MI-level there is no concept of opening/closing of files. Files are defined at the OS/400 level (CPF) above the MI. A program (even an MI-program) accesses a file through a User File Control Block (the UFCB). The UFCB defines the filename, library name, possibly a member name, buffer areas and all necessary other control information needed to manage the file. It also provides pointers to the various feedback areas and access to various control structures, such as the Open Data Path (the ODP). The main purpose of the UFCB is to provide device independence for I/O. You use a UFCB for database files, display files, spool files, save files, tape files, etc. The fact, that these files often have very different internal structure (the actual objects making up the files) is transparent to the programmer.

 

The design of this intermediate layer allows a program to access different types of database files and device files in the same manner, and provides an extendable infrastructure for future extensions.

 

For documentation on the UFCB, refer to the following resources:

  • Leif Svalgaard's e-book: Chapter 12, Input/Output in MI Using the SEPT, Chapter 13, File Conversion and Hashing, and Chapter 16, User-Defined 5250 Datastream I/O. Chapter 12 is also available in a presentation handout (handout.doc or handout.pdf).
  • The Query (QQQQRY) API's documentation in the Information Center covers the fixed portion and the variable portion of the UFCB. Similar information can be found in source member QSYSINC/H.QQQQRY, which was shipped with the System Openness Includes option of i5/OS.
  • Among the posts in the MI400 mailing list dating back to the early 2000s, you can find a lot of MI source examples and discussions about doing file I/O via the UFCB. Just go to http://archive.midrange.com/mi400/index.htm and search with keyword UFCB.

 

An additional way to obtain information about UFCB-based file operations is the OPM HLL compilers. You can compile an OPM HLL program that operates files with parameter GENOPT(*LIST) and investigate the OPM MI instructions in the resulting compiler listing. Actually, this was the way by which most IBM i experts outside of IBM figured out how file operations work.

 

The UFCB contains control information such as the name of the target file to open, open flags, and numerous open parameters. The UFCB contains status completion information and works as a link from the user program to the Open Data Path (ODP) of the opened file. After the target file is successfully opened, the first field of the UFCB, a space pointer addressing the ODP of the opened file, is updated, and space pointers in the UFCB that address various feedback areas within the ODP are updated.

 

The UFCB consists of a 208-byte fixed portion and a variable UFCB parameter list. The fixed portion of UFCB is defined as data structure ufcb_base_t in ufcb.rpgleinc, which is shown as follows:

 

   /**

     * DS ufcb_base_t -- fixed portion of   UFCB

     */

     d ufcb_base_t     ds                 qualified

     d     @odp                         *

     d     @inbuf                       *

     d     @outbuf                       *

     d     @open_fbk                     *

     d     @io_fbk                       *

     d     @next_ufcb                   *

     d     @sep_ind                     *

     d                              16a

     d     file                       10a

     d     lib_id                       5i   0

     d     lib                         10a

     d     mbr_id                       5i   0

     d     mbr                         10a

     d     last_dev                    10a

     d     dev_inx                     5i   0

     * Flags/State

     d     flags                       2a

     d     sh_sec_flg                   1a     overlay(flags)

     d     opn_flg                     1a     overlay(flags:2)

     d     ver                          2a

     d     rel                         2a

     d     inv_mrk_cnt                 10i   0

     * Mark counter options and blocked   record flag

     d     markcntflg                   1a

     d                               1a

     d     invactflg                   1a

     d                               1a

     d     nativeflg                   1a

     d                               2a

     d     opnscope                     1a

     d*                               5a

     * Bit 0 - equivalent to C's   "rtncode=y" parameter

     d     bang                         1a

     d                               4a

     d     inv_mrk_cnt_hi...

     d                               10i 0

     d                               7a

 

UFCB information introduced in this article is based on the excellent work of a few respected IBM i experts and a large amount of experimentation. However, due to the lack of official IBM documentation, the information provided in this article cannot be guaranteed to be thorough and completely accurate. Usage information of each individual field is to be discussed in a later section of this article, Open Database Files Dynamically via the UFCB.

 

The variable UFCB parameter list consists of a list of variable-length UFCB parameters and is ended by a special UFCB parameter, END_PARM_LIST (a BIN(2) field set to the value of hex 7FFF). Each variable UFCB parameter consists of a BIN(2) parameter-ID field followed by parameter-specific data. You can specify a negative value of the UFCB parameter ID field to deactivate the current UFCB parameter. For example, a parameter ID of value 6 indicates that a level-checking parameter is specified, and the level-checking information is provided in the parameter data to specify how level-checking should be performed against the file to open; however, a parameter ID of value -6 means the following the level-checking parameter data does not take effect, although being specified. For detailed information about the variable UFCB parameter list, please refer to the Query (QQQQRY) API's documentation in the Information Center.

The Open Data Path (ODP)

The ODP control block is the central control block of an opened device file or database member. According to Leif:

Internally there are two types of ODPs, a permanent ODP (sometimes called the prototype ODP) and an active ODP. When a device file/database member is created, an associated ODP is also created. When the device/member is opened, a temporary duplicate of the associated is created. This duplicate is the active ODP. The duplicate is destroyed when the file is closed or the job ends.

 

For database members, the permanent ODP at the MI-level is a cursor object (type/subtype X‘0D50’). It is simply the database member object. Its name is the concatenation of filename and member name. The associated ODP is also a cursor (type/subtype X‘0DEF’) called the operational cursor. Its name is the concatenation of filename, library name, and member name.

 

The first part of the ODP (often called the ODP root) has a fixed format The information contained in the root section includes:

- Status information

- Various size information about the space in which the ODP control block resides

- Offset to the Device File/Database MCB (Member Control Block)

- Offsets to various sections within the ODP

- Information (such as names and “open” options) common to several components

 

The following data structure odp_t defined in ufcb.rpgleinc describes the beginning part of ODP.

 

     /**

     * Open Data Path (ODP)

     */

     d odp_t          ds                 qualified

     d  
 
status                       4a

     d     dev_len                     10i   0

     d     opn_size                   10i   0

     * Offest to open feedback area

     d     opn_fbk_offset...

     d                              10i 0

     * Offset to Device Control Block (DCB)

     d     dcb_offset                 10i   0

     * Offest to I/O feedback area

     d     io_fbk_offset...

     d                               10i 0

     * @todo more fields in ODP

 

After a database file is successfully opened, the space pointer addressing the associated space of the active ODP is set in the UFCB provided by the user program (ufcb_base_t.@odp>). And the space pointers of the feedback areas (open feedback area and I/O feedback area) in the UFCB are set to address the corresponding feedback areas in the ODP.

Open Database Files Dynamically via the UFCB

To open a database file, the following UFCB parameters and options in the fixed portion of the UFCB should be specified:

  • File nameSet CHAR(10) ufcb_base_t.file to the name of the database file to open.
  • Library nameSet CHAR(10) ufcb_base_t.lib to the name of the library in which the database file resides. Special values *LIBL and *CURLIB are accepted. Experiments in VRM540 reveal that the BIN(2) ufcb_base_t.lib_id field is not checked and can be set to any value.
  • Member nameSet the BIN(2) ufcb_base_t.mbr_id field to MBRID (hex 0049) and set CHAR(10) ufcb_base_t.mbr to the database file member name. Special values *FIRST, *LAST, and *ALL are accepted. If ufcb_base_t.mbr_id is set to a value other than MBRID, the first member of the database file is selected.
  • FlagsThe CHAR(2) ufcb_base_t.flags field consists of a CHAR(1) share and secure flags field and a CHAR(1) open flags field, at offset 174 and 175, respectively, from the beginning of the UFCB. See the Query (QQQQRY) API's documentation in the Information Center for details. For example, in the RPROPNFLD example, where a database file member would be opened for read and update, the ufcb_base_t.flags field should be set to SS_PERMANENT + %bitor(OPN_INPUT : OPN_UPDATE) (hex 8028).

 

In the RPROPNFLD example, for convenience of detecting the end-of-file condition, you can set ufcb_base_t.rtncode to RTNCODE_YES (hex 80). (The effect is equivalent to specifying the "rtncode=y" parameter when opening a file via the _Ropen() routine of the C library.)

 

Most database files (physical files or logical files) are created with level-checking parameter set to LVLCHK(*YES). The target file of RPROPNFLD is determined at run-time, and RPROPNFLD has no knowledge about the record format(s) of the target file; therefore, it's reasonable to set the level-checking UFCB parameter (in the variable UFCB parameter list) to NO_LVLCHK (hex 000600), which is defined in ufcb.rpgleinc.

 

The following is the code piece extracted from rpropnfld.rpgle that initializes the UFCB parameters:

 

     d ufcb           ds                 qualified

     d     base                               likeds(ufcb_base_t)

     d     lvlchk                             likeds(lvlchk_parm_t)

     d     no_more                     5i   0

     d @ufcb           s               *   inz(%addr(ufcb))

 

     /free

           ufcb = *allx'00';

           ufcb.no_more = END_PARM_LIST;

           ufcb.base.file   = fnam.obj_name;

           ufcb.base.lib   = fnam.lib_name;

           ufcb.base.flags =

             SS_PERMANENT + %bitor(OPN_INPUT   : OPN_UPDATE);

           // rtncode=y

           ufcb.base.rtncode = RTNCODE_YES;

           // Do NOT perform level-checking

           %subst(ufcb.lvlchk : 1 : 3) = NO_LVLCHK;

     /end-free

 

To open/close a file, you need to call the User Domain System State (UDSS) APIs QDMCOPEN and QDMCLOSE, respectively. These APIs can be located in the System Entry Point Table (SEPT). Entry numbers 11 (QDMCLOSE) and 12 (QDMCOPEN) are the entry points to the Data Management Close/Open functions. Either of these APIs accepts only one parameter, a space pointer addressing the UFCB provided by the user program. The following code piece extracted from rpropnfld.rpgle shows the steps of locating QDMCOPEN and QDMCLOSE in the SEPT and calling QDMCOPEN to open the target file:

 

     d @pco           s               *

     d @sept           s              *     based(@pco)

     d sept           s               *   based(@sept)

     d                                       dim(7000)

 

     /free

           // Locate @sept

           @pco = pcoptr2();

 

           // Open DBF

           callpgmv( sept(sept_qcmcopen)

                   : @ufcb

                   : 1 );

     /end-free

 

As mentioned above, after a database file is successfully opened, the space pointer addressing the associated space of the active ODP is set in the UFCB (ufcb_base_t.@odp>). And the space pointers of the feedback areas (open feedback area and I/O feedback area) in the UFCB are set to address the corresponding feedback areas in the ODP. The open feedback area and I/O feedback area are documented in the Information Center:

  • Open feedback area
  • I/O feedback area

 

The open feedback area and the common I/O feedback area are defined as data structure open_fbk_t and io_fbk_base_t, respectively, in ufcb.rpgleinc.

 

After the target database file member is opened, the final step in this section is to retrieve the record length (open_fbk_t.max_rcdlen) of the opened database file member. Like this:

 

     d ofa             ds                 likeds(open_fbk_t)

     d                                       based(@ofa)

 

     /free

           // Retrieve record length

           @ofa = ufcb.base.@open_fbk;

           rcdlen = ofa.max_rcdlen;

     /end-free

Achieve Database File I/O Operations

According to Leif:

The SEPT contains pointers to the routines that read and write records, but these routines are different for different devices, so how do we know which ones to use? Finding the correct pointer is done through the Data Management Entry Point Table (DMEPT) in the Open Data Path (the ODP). After a file has been opened, this table has the correct indices into the SEPT for each I/O routine as fitting for the device in question. This allows the Data Management component of OS/400 to direct the I/O to the correct device/file consistent with user requests including overrides (e.g., to a different device or file type). Only this method for calling I/O routines allows Data Management to handle overrides and error conditions correctly.

 

Each entry in the DMEPT is set to an index or subscript into the SEPT at open time. If an entry does not apply to a particular device/file type, these entry points within the SEPT will be to a program that will signal an appropriate exception saying that the requested operation is not supported or applicable for that device. For example, all the output operations for this open input file lead to the same error routine -- SEPT entry number 69, QDMIFERR.

 

The DMEPT is located in a feedback area in the ODP, the Device Control Block (DCB), the content of which (including the DMEPT) is updated after file open. The odp_t.dcb_offset field is the offset to the DCB. Thus, the space pointer to the DCB can be calculated by adding odp_t.dcb_offset to the ODP pointer. DCB consists of the maximum number of devices, the number of devices in the ODP, and a Device Name (Definition) List (DNL), which is an array of device description for devices in the ODP. For a database file, the number of devices in the ODP is always 1. The DCB and the DNL entry are defined in ufcb.rpgleinc as data structure dcb_t and dnl_t, respectively. The Device definition list in the Information Center contains detailed description about fields in a DNL entry, except the DMEPT entries.

 

     /**

     * Device Control Block (DCB)

     */

     d dcb_t           ds                  qualified

     d     max_devs                     5u   0

     d     num_dev                     5u   0

     * Array of Device Name List (DNL)   entries

 

     /**

     * Device Name List (DNL) entry

     *

     * @remark Refer to "Device   definition list" in the Info-Center for more details

     *   http://publib.boulder.ibm.com/infocenter/iseries/v5r4/topic/dm/rbal3pdttr.htm

     */

     d dnl_t           ds                 qualified

     d     dev_name                   10a

     * @todo Check the `FM' component

     d     fm_offset                   10i   0

     d     fm_length                   10i   0

     d     lud_ptr_inx                 5u   0

     *

     * Start of Data Management Entry Point   Table (DMEPT)

     *

     * Get next

     d     get_inx                      5u 0

     * Get by RRN

     d     getd_inx                     5u   0

     * Get by key

     d     getk_inx                     5u   0

     * Put by RRN

     d     putd_inx                     5u   0

     * Put

     d     put_inx                     5u   0

     *   Put and get

     d     ptgt_inx                     5u   0

     * Update

     d     upd_inx                     5u   0

     * Delete

     d     dlt_inx                     5u   0

     * Force EOD

     d     feod_inx                     5u   0

     * Force EOV

     d     feov_inx                     5u   0

     * Commit

     d     cmt_inx                     5u   0

     * Rollback

     d     rlbk_inx                     5u   0

     * Release (free record lock)

     d     rls_inx                     5u   0

 

For example, the dnl_t.get_inx in the only DNL entry for an opened database file is 16, and the dnl_t.upd_inx is 19, which are the SEPT entry numbers of the QDBGETSQ and QDBUDR APIs.

Format of the Parameter Lists of the File I/O Routines

According to Leif:

All OS/400 CPF-level I/O routines accept three arguments: an open UFCB, an option list, and a control list. An argument may be left out or coded as a NULL pointer, if it is not applicable.

 

In ufcb.rpgleinc, the parameter lists of the file I/O routines are defined as the following:

 

     /**

     * Parameter list of I/O routines

     */

     d io_plist_t     ds                 qualified

     d     ptrs                           *   dim(3)

     d     @ufcb                           *   overlay(io_plist_t)

     d     @opt_lst                       *   overlay(io_plist_t:17)

     d     @ctrl_lst                       *   overlay(io_plist_t:33)

 

For simple operations, no control list is needed, so the control list parameter (io_plist_t.@ctrl_lst>) can be left as a null pointer.

 

The Data Management Option List consists (io_plist_t.@opt_lst>) of four 1-byte field:

  • Operation option byte (byte 0)
  • Share option byte (byte 1)
  • Data option byte (byte 2)
  • Device Support byte (byte 3)

 

Detailed descriptions of these 1-byte fields are available in the chapter Input/Output in MI Using the SEPT in handout.doc, or handout.pdf. Constants defined for the option list fields in ufcb.rpgleinc are the following:

 

     /**

     * Constants for I/O routine's option   list

     */

     /* Operation option byte (byte 0) */

     d io_opt_lst_0_release_all...

     d                 c                   x'00'

     d io_opt_lst_0_release_first...

     d                 c                   x'01'

     d io_opt_lst_0_get_first...

     d                 c                   x'01'

     d io_opt_lst_0_get_last...

     d                 c                   x'02'

     d io_opt_lst_0_get_next_wait...

     d                c                   x'03'

     d io_opt_lst_0_get_previous...

     d                 c                   x'04'

     d io_opt_lst_0_getk_next_unique...

     d                 c                   x'05'

     d io_opt_lst_0_getk_prev_unique...

     d                 c                   x'06'

     d io_opt_lst_0_getd_relcur...

     d                 c                   x'07'

     d   io_opt_lst_0_getd_ordinal_relstr_wait...

     d                 c                   x'08'

     d io_opt_lst_0_get_next_no_wait...

     d                 c                   x'13'

     d   io_opt_lst_0_get_next_wait_via_acc_input...

     d                 c                   x'23'

     d   io_opt_lst_0_get_next_wait_via_evt_handler...

     d                 c                  x'83'

     d io_opt_lst_0_get_cancel...

     d                 c                   x'0C'

     d io_opt_lst_0_get_same...

     d                 c                   x'0E'

     d   io_opt_lst_0_getd_ordinal_relstr_no_wait...

     d                 c                  x'18'

     d io_opt_lst_0_getd_next_mod_wait...

     d                 c                   x'0A'

     d io_opt_lst_0_getd_next_mod_no_wait...

     d                 c                   x'1A'

     d io_opt_lst_0_getd_unblocked_wait...

     d                  c                   x'0B'

     d io_opt_lst_0_getd_unblocked_no_wait...

     d                 c                   x'1B'

     d io_opt_lst_0_getd_cancel...

     d                 c                   x'0C'

     d io_opt_lst_0_getk_key_equal...

    d                 c                   x'0B'

     d   io_opt_lst_0_getk_key_before_or_equal...

     d                 c                   x'0A'

     d io_opt_lst_0_getk_key_before...

     d                 c                   x'09'

     d io_opt_lst_0_getk_key_after_or_equal...

     d                 c                   x'0C'

     d io_opt_lst_0_getk_key_after...

     d                 c                   x'0D'

     d io_opt_lst_0_put_wait...

     d                 c                   x'00'

     d io_opt_lst_0_put_no_wait...

     d                 c                   x'10'

     d io_opt_lst_0_put_get_next_wait...

     d                 c                   x'03'

     d io_opt_lst_0_put_get_next_no_wait...

     d                 c                   x'13'

   d   io_opt_lst_0_put_get_cancel...

     d                 c                   x'0C'

     d io_opt_lst_0_put_send_error_msg...

     d                 c                   x'0F'

 

     /* Share option byte (byte 1) */

     d io_opt_lst_1_get_for_read...

     d                c                   x'00'

     d io_opt_lst_1_get_for_update...

     d                 c                   x'03'

     d   io_opt_lst_1_get_for_read_only_no_position...

     d                 c                   x'10'

     d io_opt_lst_1_get_for_update_no_position...

     d                 c                   x'13'

 

     /**

     * Date option byte (byte 2)

     */

     d io_opt_lst_2_acc_rcd...

     d                 c                   x'00'

     d io_opt_lst_2_no_acc_rcd...

     d                  c                   x'01'

     d io_opt_lst_2_acc_all_rcd...

     d                 c                   x'02'

     d io_opt_lst_2_no_acc_all_rcd...

     d                 c                   x'03'

     d io_opt_lst_2_get_prior_acc_rcd...

     d                  c                   x'10'

     d io_opt_lst_2_get_prior_no_acc_rcd...

     d                 c                   x'11'

     d io_opt_lst_2_get_prior_acc_all_rcd...

     d                 c                   x'12'

     d io_opt_lst_2_get_prior_no_acc_all_rcd...

     d                 c                   x'13'

 

     /**

     * Device Support byte (byte 3)

     */

     d io_opt_lst_3_get...

     d                 c                   x'01'

     d io_opt_lst_3_getd...

     d                 c                  x'02'

     d io_opt_lst_3_put...

     d                 c                   x'05'

     d io_opt_lst_3_putget...

     d                 c                   x'06'

     d io_opt_lst_3_update...

     d                 c                   x'07'

   d   io_opt_lst_3_delete...

     d                 c                   x'08'

     d io_opt_lst_3_release...

     d                 c                   x'0B'

 

For example, the option list for a read-only operation might be the following:

 

     /free

          read_optl = io_opt_lst_0_get_next_wait

                 +   io_opt_lst_1_get_for_read_only

                 + io_opt_lst_2_acc_rcd

                 + io_opt_lst_3_get;

     /end-free

 

And the option list for a read-for-update operation might be the following:

 

     /free

           read_optl =   io_opt_lst_0_get_next_wait

                 +   io_opt_lst_1_get_for_update

                 + io_opt_lst_2_acc_rcd

                 + io_opt_lst_3_get;

     /end-free

 

Finally, a read operation followed by a update operation performed on an opened database file might look like this:

 

     d ufcb           ds                 qualified

     d     base                               likeds(ufcb_base_t)

     d     lvlchk                             likeds(lvlchk_parm_t)

     d     no_more                     5i   0

     d @ufcb           s               *   inz(%addr(ufcb))

     d dcb             ds                 qualified

     d                                       based(@dcb)

     d     base                              likeds(dcb_t)

     d     dnl                                 likeds(dnl_t)

     d                                     dim(1)

     d read_pl         ds                 likeds(io_plist_t)

     d read_optl       s             4a

     d update_pl       ds                  likeds(io_plist_t)

     d update_optl     s             4a

     d iofbk           ds                 likeds(io_fbk_base_t)

     d                                       based(@iofbk)

 

     /free

           // Locate IO feedback area

          @iofbk = ufcb.base.@io_fbk;

 

           // Locate DMEPT

           @odp = ufcb.base.@odp;

           @dcb = @odp + odp.dcb_offset;

 

           // Set parameter list of i/o   routines

           read_pl.@ufcb = @ufcb;

           read_optl = io_opt_lst_0_get_next_wait

                 +   io_opt_lst_1_get_for_update

                 + io_opt_lst_2_acc_rcd

                 + io_opt_lst_3_get;

           read_pl.@opt_lst = %addr(read_optl);

           read_pl.@ctrl_lst = *NULL;

           update_pl.@ufcb = @ufcb;

           update_optl =   io_opt_lst_0_put_wait

                 + io_opt_lst_1_get_for_read

                 + io_opt_lst_2_acc_rcd

                 + io_opt_lst_3_update;

           update_pl.@opt_lst = %addr(update_optl);

           update_pl.@ctrl_lst = *NULL;

 

           // Read a record

           callpgmv( sept(dcb.dnl(1).get_inx)

                   : read_pl.ptrs

                   : 3 );

           // ... ...

 

           // Update the record

           callpgmv( sept(dcb.dnl(1).upd_inx)

                 : update_pl.ptrs

                   : 3 );

     /end-free

 

Complete the RPROPNFLD Utility

With the information shown above, you can now complete the RPROPNFLD utility, which checks an OPEN field in a database file for validity and repairs the truncated DBCS data in it. The complete source of rpropnfld.rpgle is the following:

 

     /**

     * @file rpropnfld.rpgle

     *

     * Repaire invalid OPEN fields due to   DBCS character truncation

     * in a database file.

     * - Open a database file by name   dynamically

     * - Determine the record length of the   opened database file

     *     member after the member is opened

     * - Read each record looply and locate   the OPEN field in the

     *     read record by the input field position parameter

     * - Check for validity of the data   stored in the OPEN field and

     *     repair it if the data is invalid

     *

     * @example Invoke RPROPNFLD to repair   an OPEN field in DBF

     * *LIBL/SOMEFILE at position 1, with   length 10:

     * CALL RPROPNFLD('SOMEFILE *LIBL' X'0001' X'000A')

     */

 

     h dftactgrp(*no)

 

     /copy ufcb

     /copy mih-prcthd

     /copy mih-pgmexec

     /copy mih-comp

 

     * DS -- Qualified object name

     d qual_obj_name_t...

     d                 ds                 qualified

     d     obj_name                   10a

     d     lib_name                   10a

 

     * Prototype of mine

     d i_main          pr                 extpgm('RPROPNFLD')

     * Qualified database file name

     d     fnam                               likeds(qual_obj_name_t)

     * Position of the target OPEN field

     d     fld_pos                     5u   0

     * Length of the target OPEN field

     d     fld_len                     5u   0

 

     * Procedure to check for (and repair)   invalid OPEN data

     d check_and_repair...

     d                 pr             n

 

     d rcdlen         s             10u 0

     d ufcb           ds                 qualified

     d     base                               likeds(ufcb_base_t)

     d     lvlchk                             likeds(lvlchk_parm_t)

     d     no_more                     5i   0

     d @ufcb           s               *   inz(%addr(ufcb))

     d ofa             ds                 likeds(open_fbk_t)

     d                                       based(@ofa)

     d odp             ds                 likeds(odp_t)

     d                                       based(@odp)

     d dcb            ds                 qualified

     d                                       based(@dcb)

     d     base                               likeds(dcb_t)

     d     dnl                                 likeds(dnl_t)

     d                                     dim(1)

     d read_pl         ds                 likeds(io_plist_t)

     d read_optl       s             4a

     d update_pl       ds                 likeds(io_plist_t)

     d update_optl     s             4a

     d iofbk           ds                 likeds(io_fbk_base_t)

     d                                       based(@iofbk)

     d dbfiofbk       ds                 likeds(io_fbk_dbf_t)

     d                                       based(@dbfiofbk)

     * SEPT

     d @pco           s               *

     d @sept           s               *   based(@pco)

     d sept           s               *   based(@sept)

     d                                       dim(7000)

 

     d @rec           s               *

 

     d i_main         pi

     d     fnam                              likeds(qual_obj_name_t)

     d     fld_pos                     5u   0

     d     fld_len                     5u   0

 

     /free

           // [1] Initialize UFCB control options

           ufcb = *allx'00';

           ufcb.no_more = END_PARM_LIST;

          ufcb.base.file   = fnam.obj_name;

           ufcb.base.lib   = fnam.lib_name;

           ufcb.base.flags =

             SS_PERMANENT + %bitor(OPN_INPUT   : OPN_UPDATE);

           // rtncode=y

           ufcb.base.rtncode = RTNCODE_YES;

           // Do NOT perform level-checking

           %subst(ufcb.lvlchk : 1 : 3) =   NO_LVLCHK;

 

           // [2] Locate SEPT

           @pco = pcoptr2();

 

           // [3] Call the DM open routine (QDMCOPEN) to open DBF

           callpgmv( sept(sept_qcmcopen)

                  : @ufcb

                   : 1 );

 

           // [3.1] Retrieve record length

           @ofa = ufcb.base.@open_fbk;

           rcdlen = ofa.max_rcdlen;

             // (fld_pos + fld_len - 1)   should <= rcdlen

 

           // [4] Prepare for I/O operations

           // [4.1] Locate I/O feedback areas

           @iofbk = ufcb.base.@io_fbk;

           @dbfiofbk = @iofbk +   iofbk.spec_iofdk_offset;

 

           // [4.2] Locate DMEPT

           @odp = ufcb.base.@odp;

           @dcb = @odp + odp.dcb_offset;

 

           // [4.3] Set parameter list of i/o routines

           read_pl.@ufcb = @ufcb;

           read_optl =   io_opt_lst_0_get_next_wait

                 +   io_opt_lst_1_get_for_update

                 + io_opt_lst_2_acc_rcd

                 + io_opt_lst_3_get;

           read_pl.@opt_lst = %addr(read_optl);

           read_pl.@ctrl_lst = *NULL;

           update_pl.@ufcb = @ufcb;

           update_optl =   io_opt_lst_0_put_wait

                 +   io_opt_lst_1_get_for_read_only

                + io_opt_lst_2_acc_rcd

                 + io_opt_lst_3_update;

           update_pl.@opt_lst = %addr(update_optl);

           update_pl.@ctrl_lst = *NULL;

 

           @rec = ufcb.base.@inbuf;

           dow '1';

               // [4.4] Read a record

               callpgmv(   sept(dcb.dnl(1).get_inx)

                       : read_pl.ptrs

                       : 3 );

               // [4.5] Check for EOF

               if iofbk.rcd_read = 0;

                   leave;

               endif;

 

              // [5]   Check/Repair the current record

               if check_and_repair();

                   // [4.6] Update the current record

                   callpgmv(   sept(dcb.dnl(1).upd_inx)

                           : update_pl.ptrs

                          : 3 );

               endif;

           enddo;

 

           // Close DBF

           callpgmv( sept(sept_qcmclose)

                   : @ufcb

                   : 1 );

 

           *inlr = *on;

     /end-free

 

     /**

     * Procedure to check for (and repair)   invalid OPEN data

     * @pre

     *   - SPP @rec

     *   - 5U0 fld_pos

     *   - 5U0 fld_len

     *

     * @return *ON=invalid OPEN data

     */

     p check_and_repair...

     p                 b

 

     d @where         s               *

    d   dist           s             5u 0

     d rem             s             5u 0

     d dbcs_flag       s               n   inz(*off)

     d SO             c                   x'0E'

     d SI             c                   x'0F'

     d WS             c                   x'40'

     d ctrl_ch         s             1a   inz(SO)

 

     d check_and_repair...

     d                 pi             n

 

     /free

           // [5.1] Check for truncated DBCS character data

           // Search for control character SO

           @where = memchr(@rec : ctrl_ch :   fld_len);

           dow @where <> *NULL and dist   < fld_len;

               dist = @where - @rec;

               dbcs_flag = not dbcs_flag;

 

               if dbcs_flag;

                   ctrl_ch = SI;

               else;

                   ctrl_ch = SO;

               endif;

               // Search for next control   character

               @where = memchr(@rec + dist :   ctrl_ch : fld_len);

           enddo;

 

           // If dbcs_flag is still ON, there   isn't truncated DBCS

           // character data.

           if not dbcs_flag;

               return *OFF;

           endif;

 

           // [5.2] Repair truncated DBCS data

           rem = fld_len - dist - 1;

             // Number of bytes after to last   SO

           if rem <= 2;

               propb(@rec + dist : WS :   fld_len - dist);

           else;

               if %bitand(rem : x'0001') >   0; // rem is ODD

                   // Set the last character   to SI

                   propb(@rec + fld_len - 1 :   SI : 1);

               else;                         // rem is EVEN

                   // Set the last 2   characters to SI + x'40'

                   propb(@rec + fld_len - 2 :   SI : 1);

                   propb(@rec + fld_len - 1 :   WS : 1);

              endif;

           endif;

 

           return *ON; // Truncated DBCS data   detected!

     /end-free

     p                 e

 

Notes

  • [1] Initialize UFCB control options. Set ufcb.base.file to the input file name. Set ufcb.base.lib to the input library name. Specify SS_PERMANENT + %bitor(OPN_INPUT : OPN_UPDATE) as the value of the CHAR(2) ufcb.base.flags field to indicate opening the target database file for read and update. Set ufcb.base.rtncode to RTNCODE_YES (hex 80) to suppress the CPF5001 exception when end of file is reached. Disable record format level-checking by setting ufcb.lvlchk to NO_LVLCHK.
  • [2] Set @pco to the space pointer to the Process Communication Object (PCO) returned by the system built-in _PCOPTR2 so that the SEPT defined basing @sept (addressed by @pco) becomes valid. Now, the system pointers of DM open/close routines (QDMCOPEN and QDMCLOSE) can be invoked to open/close a file.
  • [3] Call QDMCOPEN to open the target database file passing the prepared UFCB.
  • [3.1] After the target database file member is opened successfully, the record length can be determined by the max_rcdlen field in the open feedback area.
  • [4] Prepare for I/O operations.
  • [4.1] Locate I/O feedback areas. For example, after a successful I/O operation, the Relative Record Number (RRN) can be determined by the rrn field of the io_fbk_dbf_t data structure, which represents the I/O feedback area for database files. In this example, the common I/O feedback area is addressed by space pointer @iofbk, and the I/O feedback area for database files is addressed by space pointer @dbiofbk.
  • [4.2] Locate the DMEPT by locating DCB (calculating the DCB pointer by adding odp.dcb_offset to the ODP pointer in the UFCB).
  • [4.3] Set parameter list of I/O routines.
  • [4.4] Read a record by calling the GET I/O routine returned by DM in the DMEPT (the QDBGETSQ API for a database file).
  • [4.5] Check for EOF via the iofbk.rcd_read field in the common I/O feedback area.
  • [4.6] Update the record previously read by calling the UPDATE I/O routine returned by DM in the DMEPT (the QDBUDR API for a database file).
  • [5] Check/repair DBCS data in the target OPEN field of current record by invoking procedure check_and_repair.
  • [5.1] Check for truncated DBCS character data by testing whether every SO control character has its corresponding SI.
  • [5.2] Fix the truncated DBCS data by padding an SI control character at the end of the OPEN field. rem is the number of bytes after the unpaired SO. If rem is odd, replace the last byte of the OPEN field with SI; if rem is even, replace the byte before the last byte of the OPEN field with SI and replace the last byte with SBCS white-space (hex 40).

 

Imagine that you have a physical file (PF) called A350 that contains an 8-byte OPEN field FO and a 1-byte character field FEND. There are two records in PF A350. The DBCS data in FO of the first record is truncated at right (without the ending SI control character). The result of a SQL statement, SELECT hex(FO), FO, FEND FROM A350, issued on PF A350 in STRSQL is the following:

 

060513JunleiFig1    

Figure 1: This is the "before" image.

 

The value of the FEND field of the first record cannot be displayed properly due to the invalid DBCS data in field FO.

 

Now, let's call the RPROPNFLD program to check and repair the OPEN data in the FO field of A350 like the following:

CALL PGM(RPROPNFLD) PARM(

       'A350     *LIBL' /* Qualified name of PF A350 */

       X'0001'           /* Field position of FO     */

       X'0008')           /* Length of field FO       */

 

Issue the SQL statement SELECT hex(FO), FO, FEND FROM A350 again on PF A350. Note the hexadecimal content of the FO field of the first record.

060513JunleiFig2

Figure 1: This is the "after" image.

 

Junlei Li

Junlei Li is a programmer from Tianjin, China, with 10 years of experience in software design and programming. Junlei Li began programming under i5/OS (formerly known as AS/400, iSeries) in late 2005. He is familiar with most programming languages available on i5/OS—from special-purpose languages such as OPM/ILE RPG to CL to general-purpose languages such as C, C++, Java; from strong-typed languages to script languages such as QShell and REXX. One of his favorite programming languages on i5/OS is machine interface (MI) instructions, through which one can discover some of the internal behaviors of i5/OS and some of the highlights of i5/OS in terms of operating system design.

 

Junlei Li's Web site is http://i5toolkit.sourceforge.net/, where his open-source project i5/OS Programmer's Toolkit (https://sourceforge.net/projects/i5toolkit/) is documented.

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: