18
Thu, Apr
5 New Articles

The API Corner: Diagnosing Decimal Data Errors

APIs
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Consider the advantages of the Convert Hex to Character (cvthc) API.

 

In developing a general-purpose utility to detect decimal data errors, we have made use of a few APIs. In the article "In Search of Decimal Data Errors," we saw how the List Fields (QUSLFLD) API can be used to find all numeric fields, within a user-specified file record format, that are defined as either zoned decimal or packed decimal. In addition to discovering the data type of each field, we were also able to determine the name of the field, the number of decimal positions, the number of digits to the right of the decimal point, the number of bytes used to store the field value, and the starting location of the field within the record format.

In the article "Detecting Decimal Data Errors," we saw how the Copy Numeric Value (CPYNV) API can be used, in conjunction with the information obtained with the QUSLFLD API, to determine which field(s) in a record would cause a MCH1202 Decimal data error due to the encoding of the data found in a given zoned or packed numeric field.

This month, I will introduce another API, one that allows us to "see" the field value causing the MCH1202. While implementing this API, we'll also enhance the FNDDDE program in another way. This additional enhancement is to replace our previous use of the RPG dsply operation code with a printer file. When the file being examined for decimal data errors might contain millions of records, a report listing the errors could be a whole lot more useful than a series of dsplys.

The Find Decimal Data Errors Printer File

Here's the source for our new printer file, FNDDDEPRTF.

A         R HEADING                   SKIPB(2)                

A                                     5'FNDDDE'                

A                                   13'USER:'                

A           CURUSR       10A O   19                        

A                                   114                        

A                                     'PAGE:'                

A                                  120PAGNBR                  

A                                     EDTCDE(Z)              

*                                                            

A                                   13'FILE:'                

A                                    SPACEB(1)              

A           FILE_IN       10A O   19                        

A                                   60'FNDDDE ANALYSIS REPORT'

A                                   114'DATE:'                

A                                  120DATE(*JOB)          

A                                     EDTCDE(Y)              

*                                                            

A                                   114'TIME:'                

A                                    SPACEB(001)            

A                                   120TIME                    

***************************************************************

A         R RRN_VAL                   SPACEB(1)              

A                                    1'RRN:'                  

A           RELNO_C       10A       6SPACEA(1)              

***************************************************************

A         R FLD_VALS_C               SPACEB(1)              

A                                    5'FIELD:'                

A           FLDNAME       10       15                        

A           FLD_DATA     100A       30                        

***************************************************************

A         R FLD_VALS_H               SPACEB(1)              

A                                   28'X'''                  

A           FLD_DATA     100A       30                        

***************************************************************

A         R NBRSUM                                            

A                                     3'SUMMARY OF NUMERIC +    

A                                     DATA ANALYSIS'          

A                                     SPACEB(2)                

A                                     6'VALID NUMERIC VALUES:'  

A                                     SPACEB(1)                

A           GOODNBR       20 0     30EDTCDE(1)                

A                                    6'BLANK VALUES FOUND:'    

A                                     SPACEB(1)                

A           BLNKNBR       20 0     30EDTCDE(1)                

A                                     6'NON-BLANK VALUES FOUND:'

A                                      SPACEB(1)                

A           NONBLNKNBR   20 0     30EDTCDE(1)                

***************************************************************

A         R THEEND                                            

A           FLD_DATA     100A       1                        

A                                     SPACEB(2)                

A                                   53                        

A                                     '***** E N D O F R E +

A                                    P O R T *****'          

A                                     SPACEB(2)                

There are six record formats defined within FNDDDEPRTF:

  • HEADING provides a heading for each page of the report. This heading includes the name of the file being analyzed.
  • RRN_VAL provides the relative record number of a record within the file containing one or more decimal data problems.
  • FLD_VALS_C provides two pieces of information about each field found to have a decimal data problem: the name of the field and either the text 'BLANKS FOUND' or, if the field contains values other than all blanks, a character view of the data associated with the field.
  • FLD_VALS_H provides a hex view of the data causing the problem when the decimal data error is not due to all blanks being found in the field.
  • NBRSUM provides a summary of the data that was found in the file. This summary includes the number of numeric fields with valid encodings ("good" field values if you will), the number of numeric fields with invalid encodings due to all blanks being found in the field, and the number of numeric fields with invalid encodings due to at least one non-blank value being found in the field.
  • THEEND provides status information about the report in addition to identifying the end of the report.

Assuming that you store the preceding source in member FNDDDEPRTF of QDDSSRC, you can create the printer file using this command:

CRTPRTF FILE(FNDDDEPRTF) SRCFILE(QDDSSRC)

Using the printer file, as opposed to the previous RPG DSPLY operations, allows us to output a character view of the data in error using FNDDDEPRTF record format FLD_VALS_C. To try and display the character values using the DSPLY operation would be a bit dangerous as fully one fourth of all possible EBCDIC encodings are below x'40' and represent control codes (such as 5250 display attributes) which, if sent directly to a display, can cause all kinds of problems. As a significant percentage of encoded values could be below x'40', and not represent displayable characters, FNDDDE also uses the FLD_VALS_H record format to output the hexadecimal values associated with fields in error in a displayable form.

The Convert Hex to Character (cvthc) API

To output the hexadecimal values, FNDDDE uses the Convert Hex to Character (cvthc) API, which is documented in the ILE C/C++ MI Library Reference manual.

The cvthc API takes 4-bit values (nibbles) of a field and formats each nibble as a displayable character. If a given field, for instance, is defined as being two bytes in length and has a value of x'2611', then cvthc would return the 4-byte character string '2611'.

The cvthc API defines three parameters: the first parameter is the receiver variable where cvthc returns the character string representation of various nibbles, the second parameter is the source variable that is to be converted from nibbles to characters, and the third parameter, which is passed by value, is a 4-byte integer set to the number of nibbles in the source variable to be converted.

Note that there is no parameter where you tell the API how large the receiver variable is. It is the API caller's responsibility to ensure that the receiver variable is large enough to hold the resulting character stringbasically, that the allocated size of the receiver variable, in bytes, is at least twice the number of nibbles (the value passed in the third parameter) to be converted.

Changes to the FNDDDE Program

With that introduction to the cvthc API, here is the latest version of the FNDDDE program with changes from last month shown in bold. In the program, the cvthc API is prototyped using the name CvtFrmNibbles.

h DftActGrp(*No) BndDir('QC2LE')                                      

                                                                      

fFileX     if   f32766       disk   extfile(FileXVar) usropn        

f                                     infds(FileXDS)                  

fFndDDEPrtFo   e             printer infds(PrtFDS)                  

                                                                      

*********************************************************************

                                                                      

dFndDDE           pr                 extpgm('FNDDDE')                

d File_In                       10a   const                          

d Lib_In                       10a   const                          

                                                                      

dFndDDE           pi                                                  

d File_In                       10a   const                          

d Lib_In                       10a   const                          

                                                                      

*********************************************************************

                                                                    

d CpyN           pr                 extproc('_LBCPYNV')            

d Rcv                               like(Target)                  

d RcvAttr                       7a   const                          

d Src                               const like(Source)            

d SrcAttr                       7a   const                          

                                                                      

d CvtFrmNibbles   pr                 extproc('cvthc')              

d Rcv                         100a                                  

d Src                         50a   const                          

d Length                       10i 0 value                          

                                                                    

d CrtUsrSpc       pr                 extpgm('QUSCRTUS')            

d QualName                    20a   const                          

d XtndSpcAttr                 10a   const                          

d IntSize                     10i 0 const                          

d IntValue                     1a   const                    

d PubAut                       10a   const                    

d Text                         50a   const                    

d Replace                     10a   const options(*nopass)  

d ErrCde                             likeds(QUSEC)            

d                                    options(*nopass)        

d Domain                       10a   const options(*nopass)  

d TfrSiz                       10i 0 const options(*nopass)  

d OptSpcAlgn                   1a   const options(*nopass)  

                                                              

d GetFDefn       pr                                          

                                                              

d LogRecIDVals   pr                                            

                                                              

d LogNbrError     pr                                          

                                                              

d LstFld         pr                 extpgm('QUSLFLD')        

d QualUsrSpc                   20a   const                  

d Format                       8a   const                  

d QualFileName                 20a   const                  

d RcdFmt                       10a   const                    

d OvrPrc                       1a   const                  

d ErrCde                             likeds(QUSEC)          

d                                     options(*nopass)        

                                                              

d RmvPM           pr                 extpgm('QMHRMVPM')      

d CSE                         10a   const                  

d CSECtr                       10i 0 const                  

d MsgKey                       4a   const                    

d MsgToRmv                     10a   const                  

d ErrCde                             likeds(QUSEC)          

d LenCSE                       10i 0 const options(*nopass)  

d CSEQual                     20a   const options(*nopass)  

d RmvUnhExcp                   10a   const options(*nopass)  

d CSEType                     10a   const options(*nopass)  

d AlwDftRpyRej                 10a   const options(*nopass)  

                                                              

d RtvFD           pr                 extpgm('QDBRTVFD')      

d RcvVar                       1a   options(*varsize)      

d LenRcvVar                   10i 0 const                  

d QualNameRtn                 20a                            

d Format                       8a   const                  

d QualFileName                 20a   const                  

d RcdFmt                       10a   const                  

d OvrPrc                       1a   const                    

d System                       10a   const                  

d FmtType                     10a   const                  

d ErrCde                             likeds(QUSEC)          

                                                              

d RtvUsrSpcPtr   pr                 extpgm('QUSPTRUS')      

d QualUsrSpc                   20a   const                  

d UsrSpcPtr                     *                              

d ErrCde                             likeds(QUSEC)              

                                                                

d SndEscMsg       pr                                            

d MsgID_In                     7a   const                      

d MsgDta_In                   256a   const                      

                                                                

d SndPgmMsg       pr                 extpgm('QMHSNDPM')        

d MsgID                         7a   const                      

d MsgFile                      20a   const                      

d MsgDta                     256a   const options(*varsize)    

d LenMsgDta                   10i 0 const                      

d MsgType                     10a   const                      

d CSE                         10a   const options(*varsize )  

d CSECtr                       10i 0 const                      

d MsgKey                       4a                              

d ErrCde                             likeds(QUSEC)              

d                                     options(*varsize)              

d LenCSE                       10i 0 const options(*nopass)        

d QualCSE                     20a   const options(*nopass)        

d WaitTime                     10i 0 const options(*nopass)        

d CSEDtaType                   10a   const options(*nopass)        

d CCSID                       10i 0 const options(*nopass)        

                                                                    

d SndSysMsg       pr                                                

                                                                    

*********************************************************************

                                                                      

d CpyNFailure     c                   const(00202)                  

d MaxNbrNumFlds   c                   const(1000)                    

d Packed         c                   const(x'03')                  

d SpcSize         c                  const(1048576)                

d Zoned           c                   const(x'02')                  

                                                                    

*********************************************************************

                                                                    

* Structures related to API QDBRTVFD                                

                                                                    

d myQDBQ25_Ptr   s               *                                  

d myQDBQ25       ds                 likeds(QDBQ25)                

d                                     based(myQDBQ25_Ptr)            

                                                                    

d myQDBQ36_Ptr   s               *                                  

d myQDBQ36       ds                 likeds(QDBQ36)                

d                                     based(myQDBQ36_Ptr)            

                                                                      

* Structures related to API QUSLFLD                                

                                                                    

d LstFldHdr_Ptr   s               *                                  

d LstFldHdr       ds                  likeds(QUSH0100)              

d                                     based(LstFldHdr_Ptr)          

                                                                

d LstFld100_Ptr   s               *                              

d LstFld100       ds                 likeds(QUSL0100)          

d                                     based(LstFld100_Ptr)      

                                                                

d CurNbrNumFlds   s             5u 0                            

d NumFlds         ds                 qualified                  

d NumFldsArray                 21a   dim(MaxNbrNumFlds)        

d   FldName                     10a   overlay(NumFldsArray :1)  

d   Bytes                       5u 0 overlay(NumFldsArray :11)  

d   SrcAttr                     7a   overlay(NumFldsArray :13)  

d   Type                       1a   overlay(SrcAttr :1)        

d   Scale                       3u 0 overlay(SrcAttr :2)        

d   Digits                     3u 0 overlay(SrcAttr :3)        

d                               10i 0 overlay(SrcAttr :4)        

d                                     inz(0)                    

d   BfrStrPos                   5u 0 overlay(NumFldsArray :20)  

                                                                    

d FileXDS         ds                                                

d RelNo               397   400i 0                                

                                                                      

d PrtFDS         ds                                                

d OFLine               188   189i 0                                

d CurLine             367   368i 0                                

                                                                      

d PSDS           sds                                                

d ExcpID               40     46a                                  

d CurUsr               358   367a                                  

                                                                      

d ErrCde         ds                 qualified                      

d Hdr                               likeds(QUSEC)                  

d MsgDta                     256a                                  

                                                                    

*********************************************************************

                                                              

d BlnkNbr         s            20i 0                          

d FileXVar       s             21a                            

d GoodNbr         s             20i 0                          

d RecIDLogged     s               n                            

d MsgDta         s            256a                            

d MsgKey         s             4a                            

d NonBlnkNbr     s             20i 0                          

d NotMCH1202     s               n                            

d QualNameRtn     s            20a                            

d RcdFmt         s             10a                            

d Source         s           100a   based(Source_Ptr)        

d Source_Ptr     s               *                            

d Target        s           100a                            

                                                              

d X               s             10i 0                          

                                                              

*********************************************************************

                                                                    

/copy qsysinc/qrpglesrc,qdbrtvfd                                    

/copy qsysinc/qrpglesrc,qusec                                        

/copy qsysinc/qrpglesrc,qusgen                                      

/copy qsysinc/qrpglesrc,quslfld                                    

                                                                    

*********************************************************************

                                                                    

iFileX     ns                                                        

i                                 132766 InpBfr                    

                                                                    

*********************************************************************

                                                                    

/free                                                              

                                                                    

write Heading;                                                    

                                                                  

read FileX;                                                      

dow (not %eof(FileX));                                          

                                                                  

     for X = 1 to CurNbrNumFlds;                                  

         Source_Ptr = %addr(InpBfr) + NumFlds.BfrStrPos(X) - 1;  

                                                                  

         monitor;                                                

             CpyN(Target :NumFlds.SrcAttr(X)                        

                 :Source :NumFlds.SrcAttr(X));                    

             GoodNbr += 1;                                        

                                                                  

         on-error CpyNFailure;                                    

             if ExcpID = 'MCH1202';                                

               LogNbrError();                                    

             else;                                                

                NotMCH1202 = *on;                                  

             endif;                                            

                                                                

         on-error *all;                                        

             NotMCH1202 = *on;                                  

         endmon;                                              

     endfor;                                                  

                                                                

     read FileX;                                              

     RecIDLogged = *off;                                      

enddo;                                                        

                                                                

close FileX;                                                  

                                                                

if (CurLine + 5) >= OFLine;                                  

     write Heading;                                              

endif;                                                        

                                                                

write NbrSum;                                              

                                                              

if NotMCH1202;                                              

     Fld_Data = 'Other numeric errors also found';            

else;                                                      

     // Remove all messages UNLESS a non-MCH1202              

     // was received                                          

                                                              

     RmvPM('*' :0 :' ' :'*ALL' :ErrCde);                      

     Fld_Data = 'Data analysis was completed successfully.';  

endif;                                                      

                                                              

if (CurLine + 4) >= OFLine;                                

     write Heading;                                          

endif;                                                      

                                                              

write TheEnd;                                              

                                                                    

close FndDDEPrtF;                                                  

                                                                    

*inlr = *on;                                                        

return;                                                            

                                                                    

// *****************************************************************

                                                                      

begsr *inzsr;                                                      

                                                                    

   // Set API QUSEC parameter to send exceptions                    

                                                                      

   QUSBPrv = 0;                                                    

                                                                    

   // Set API ErrCde parameter to not send exceptions              

                                                                    

   ErrCde.Hdr.QUSBPrv = %size(ErrCde);                              

                                                                  

   GetFDefn();                                                  

                                                                  

   // Get list of File_In fields and field attributes            

                                                                  

   RtvUsrSpcPtr('QUSLFLD   QTEMP' :LstFldHdr_Ptr :ErrCde);      

                                                                  

   if ErrCde.Hdr.QUSBAvl > 0;                                    

       if ErrCde.Hdr.QUSEI = 'CPF9801';                          

         // Create user space if not found                      

                                                                  

         CrtUsrSpc('QUSLFLD   QTEMP' :' ' :SpcSize :x'00'        

                   :'*ALL' :'UsrSpc for QUSLFLD output'          

                   :'*YES' :QUSEC :'*USER' :0 :'1');            

                                                                  

         RtvUsrSpcPtr('QUSLFLD   QTEMP' :LstFldHdr_Ptr :QUSEC);  

      else;                                                      

         // Any other error is a hard failure                

                                                              

         SndSysMsg();                                        

       endif;                                                  

   endif;                                                    

                                                              

   LstFld('QUSLFLD   QTEMP' :'FLDL0100'                        

         :(File_In + Lib_In) :RcdFmt :'0' :QUSEC);            

                                                              

   for X = 1 to LstFldHdr.QUSNbrLE;                          

       if X = 1;                                              

           LstFld100_Ptr = LstFldHdr_Ptr + LstFldHdr.QUSOLD;  

       else;                                                  

           LstFld100_Ptr += LstFldHdr.QUSSEE;                  

       endif;                                                  

                                                              

       // Load up numeric fields into NumFldsArray            

                                                                    

       select;                                                      

           when LstFld100.QUSDT = 'S';                              

               CurNbrNumFlds += 1;                                

               NumFlds.FldName(CurNbrNumFlds) = LstFld100.QUSFN02;

              NumFlds.Bytes(CurNbrNumFlds) = LstFld100.QUSFLB;    

               NumFlds.Type(CurNbrNumFlds) = Zoned;                

               NumFlds.Scale(CurNbrNumFlds) = LstFld100.QUSDP;    

               NumFlds.Digits(CurNbrNumFlds) = LstFld100.QUSigits;

               NumFlds.BfrStrPos(CurNbrNumFlds) = LstFld100.QUSIBP;

                                                                    

           when LstFld100.QUSDT = 'P';                              

               CurNbrNumFlds += 1;                                

               NumFlds.FldName(CurNbrNumFlds) = LstFld100.QUSFN02;

               NumFlds.Bytes(CurNbrNumFlds) = LstFld100.QUSFLB;    

               NumFlds.Type(CurNbrNumFlds) = Packed;              

              NumFlds.Scale(CurNbrNumFlds) = LstFld100.QUSDP;    

               NumFlds.Digits(CurNbrNumFlds) = LstFld100.QUSigits;  

               NumFlds.BfrStrPos(CurNbrNumFlds) = LstFld100.QUSIBP;

                                                                    

           other;                                                    

               // Don't care about other types                      

       endsl;                                                      

   endfor;                                                          

                                                                    

   FileXVar = %trimr(Lib_In) + '/' + %trimr(File_In);              

   open FileX;                                                      

                                                                    

endsr;                                                            

                                                                    

/end-free                                                          

                                                                    

*********************************************************************

                                                                    

p LogNbrError     b                                              

d LogNbrError     pi                                              

                                                                  

/free                                                            

                                                                  

if (not RecIDLogged);                                          

     LogRecIDVals();                                              

endif;                                                          

                                                                  

FldName = NumFlds.FldName(X);                                  

                                                                  

if %subst(Source :1 :NumFlds.Bytes(X)) = *blanks;              

     if (CurLine + 1) >= OFLine;                                  

       write Heading;                                            

       LogRecIDVals();                                          

     endif;                                                      

                                                                  

     Fld_Data = 'BLANKS FOUND';                                    

     write Fld_Vals_C;                                              

                                                                    

     BlnkNbr += 1;                                                  

else;                                                            

     if (CurLine + 2) >= OFLine;                                    

       write Heading;                                              

       LogRecIDVals();                                            

     endif;                                                        

                                                                    

     Fld_Data = %subst(Source :1 :NumFlds.Bytes(X));                

     write Fld_Vals_C;                                              

                                                                    

     Fld_Data = *blanks;                                            

                                                                    

     if NumFlds.Bytes(X) < 50;                                      

       CvtFrmNibbles(Fld_Data :%subst(Source :1 :NumFlds.Bytes(X))

                     :(NumFlds.Bytes(X) * 2));                      

       %subst(Fld_Data :((NumFlds.Bytes(X) * 2) + 1) :1) = '''';  

     else;                                                          

      CvtFrmNibbles(Fld_Data :%subst(Source :1 :49) :98);        

       %subst(Fld_Data :99 :1) = '''';                          

     endif;                                                        

                                                                    

     write Fld_Vals_H;                                              

                                                                    

     NonBlnkNbr += 1;                                              

endif;                                                            

                                                                    

/end-free                                                          

                                                                    

p LogNbrError     e                                                

********************************************************************

                                                                    

p LogRecIDVals   b                                                

d LogRecIDVals   pi                                              

                                                                  

/free                                                            

                                                                    

if (CurLine + 5) >= OFLine;                                      

     write Heading;                                                

endif;                                                          

                                                                    

RelNo_C = %char(RelNo);                                          

write RRN_Val;                                                  

                                                                  

RecIDLogged = *on;                                              

                                                                  

/end-free                                                        

                                                                  

p LogRecIDVals   e                                                

*******************************************************************

                                                                  

p SndSysMsg      b                                                

d SndSysMsg       pi                                              

                                                                  

/free                                                              

                                                                  

if ErrCde.Hdr.QUSBAvl <= 16;                                    

     SndEscMsg(ErrCde.Hdr.QUSEI :' ');                            

else;                                                            

     SndEscMsg(ErrCde.Hdr.QUSEI                                    

               :%subst(ErrCde.MsgDta :1                            

                 :(ErrCde.Hdr.QUSBAvl - 16)));                    

endif;                                                            

                                                                  

/end-free                                                        

                                                                  

p SndSysMsg       e                                                

********************************************************************

                                                                    

p SndEscMsg       b                                                

d SndEscMsg       pi                                                

d MsgID_In                     7a   const                        

d MsgDta_In                   256a   const                        

                                                                    

/free                                                              

                                                                    

SndPgmMsg(MsgID_In :'QCPFMSG   *LIBL'                            

           :MsgDta_In :%len(%trimr(MsgDta_In))                    

           :'*ESCAPE' :'*PGMBDY' :1                                

           :MsgKey :QUSEC);                                        

                                                                    

/end-free                                                          

                                                                    

p SndEscMsg       e                                                

*******************************************************************

                                                                  

p GetFDefn       b                                                

d GetFDefn       pi                                              

                                                                  

/free                                                            

                                                                  

   // Get File_In definition and record format name              

                                                                  

   RtvUsrSpcPtr('QDBRTVFD QTEMP' :myQDBQ25_Ptr :ErrCde);        

                                                                  

   if ErrCde.Hdr.QUSBAvl > 0;                                    

       if ErrCde.Hdr.QUSEI = 'CPF9801';                            

         // Create user space if not found                        

                                                                  

          CrtUsrSpc('QDBRTVFD QTEMP' :' ' :SpcSize :x'00'        

                   :'*ALL' :'UsrSpc for QDBRTVFD output'      

                   :'*YES' :QUSEC :'*USER' :0 :'1');          

                                                                

         RtvUsrSpcPtr('QDBRTVFD QTEMP' :myQDBQ25_Ptr :QUSEC);

       else;                                                    

         // Any other error is a hard failure                  

                                                                

         SndSysMsg();                                          

       endif;                                                  

   endif;                                                      

                                                                

   RtvFD(myQDBQ25 :SpcSize :QualNameRtn :'FILD0100'            

         :(File_In + Lib_In) :'*FIRST' :'0' :'*LCL'            

         :'*EXT' :ErrCde);                                    

                                                                

   if ErrCde.Hdr.QUSBAvl > 0;                                  

       SndSysMsg();                                            

   endif;                                                        

                                                                  

   if %bitand(%subst(myQDBQ25.QDBBits27 :1 :1) :x'20') = x'20';  

       MsgDta = %trimr(Lib_In) + '/' + %trimr(File_In) +          

               ' is not a table/physical file. Command ended';  

                                                                  

       SndEscMsg('CPF9898' :MsgDta);                              

   endif;                                                        

                                                                  

   // Get record format name                                    

                                                                  

   myQDBQ36_Ptr = myQDBQ25_Ptr + myQDBQ25.QDBFOS;                

   RcdFmt = myQDBQ36.QDBFT01;                                    

                                                                  

/end-free                                                        

                                                                  

p GetFDefn       e                                              

Not counting the various changes that were made in support of using the printer file FndDDEPrtTF, the major change of interest can be found in the function LogNbrError.

The first change to LogNbrError is the use of indicator RecIDLogged to keep track of whether or not record information, such as the relative record number, has already been printed when a decimal data error is encountered within a record. RecIDLogged, which is turned "off" in the mainline of FndDDE when a new record from FileX is read, is used to avoid printing redundant relative record number information when a given FileX record contains more than one decimal data error. On entry to LogNbrError, if RecIDLogged is "off," then function LogRecIDVals is called. LogRecIDVals simply prints the current relative record number, sets RecIDLogged to "on," and returns.

LogNbrError then prints the name of the field in error and either the text 'BLANKS FOUND', if the field contains all blanks, or the contents of the field, if the field contains any non-blank values. When non-blank data values are encountered, LogNbrError first prints the character representation of the data found, followed by the character representation of the data nibbles found (using the cvthc API). As the printer file field Fld_Data is defined with a length of 100 bytes, there is a check to determine the length of the field in error. If the field length, in bytes, is less than 50, then all of the nibbles are converted to character form for printing; otherwise, only the nibbles associated with the first 49 bytes are converted to character form and printed. Since the allocated size of Fld_Data is 100 bytes, and printing the first 49 bytes of data requires 98 print positions (recall that there are two nibbles per byte), this allows room in Fld_Data for a closing apostrophe to follow the data that is in error.

A second API-related change to FNDDDE is the conditional use of the Remove Program Messages (QMHRMVPM) API, prototyped as RmvPM, when exiting the program and only MCH1202s were encountered when testing the zoned decimal and packed decimal numeric data found in FileX. While the use of the operation codes of MONITOR and ON-ERROR allow the FNDDDE program to continue running in the event an exception such as MCH1202 is received, the monitored message is also retained in the job log. Rather than cluttering up the job log with a potentially large number of MCH1202 messages, messages that FNDDDE both expects and handles, FNDDDE uses the QMHRMVPM API to remove all these messages. As an alternative implementation, FNDDDE could have removed the MCH1202s as they were received, rather than when exiting the program, but I elected to remove them en masse as I do not anticipate any errors other than MCH1202 will be encountered when testing the numeric fields. If you're interested in learning more about this rather handy API, refer to the earlier "API Corner" article "Removing Error Messages from the Job Log."

Assuming that you store the preceding source in member FNDDDE of QRPGLESRC, then you can create the utility program using this command:

CRTBNDRPG PGM(FNDDDE)

Diagnosing Why a Field Has a Data Decimal Error

Last month, we loaded the test file DDEDATA with both "good" and "questionable" data. Now, we'll run our updated version of FNDDDE using this command:

CALL PGM(FNDDDE) PARM(DDEDATA *LIBL)

This should result in a printed report similar to what's shown below (the headings are left out due to margin considerations).

RRN: 2                                          

   FIELD:   ZNDFLD1       BLANKS FOUND      

   FIELD:   PKDFLD1       BLANKS FOUND      

   FIELD:   PKDFLD2       001                

                           X'F0F0F1'            

   FIELD:   ZNDFLD3       BLANKS FOUND      

RRN: 4                                          

   FIELD:   PKDFLD1       at                

                           X'81A3'              

   FIELD:   PKDFLD2       oss                

                           X'96A2A2'            

   FIELD:   ZNDFLD3       ibleHere          

                           X'89829385C8859985'  

SUMMARY OF NUMERIC DATA ANALYSIS                          

     VALID NUMERIC VALUES:                           23      

     BLANK VALUES FOUND:                             3      

     NON-BLANK VALUES FOUND:                         4      

Data analysis was completed successfully.                    

For relative record number 2 of DDEDATA, FNDDDE is reporting four problems. Three of the fields (ZndFld1, PkdFld1, and ZndFld3) are found to contain all blanks, and one field (PkdFld2) is found to contain the character value '001' or, in hexadecimal form, x'F0F0F1'. The cases where all blanks are found is easy enough to understand (and one I'm sure many of you have run into before), but what's wrong with the PkdFld2 value of '001'?

If you review the DDS defining DDEDATA or use the Display File Field Description command specifying FILE(DDEDATA), you'll see that field PkdFld2 is defined as being packed decimal. The problem is that the value x'F0F0F1' does not conform to the encoding rules of packed decimal data, rules which are introduced here in the ILE RPG Language Reference manual and further expanded on in the Machine Interface Concepts discussion of Arithmetic Operations under Packed Decimal Computation.

The system requires that packed decimal data have all nibbles except the last in the range of '0' through '9', with the last nibble (representing the sign) being in the range of 'A' through 'F' (where 'B' or 'D' represents a negative sign and 'A', 'C', 'E', or 'F' a positive sign). PkdFld2's value of x'F0F0F1' fails to meet these requirements in four of the six nibbles; the first, third, and fifth nibbles are 'F' (not '0' through '9') and the last nibble is '1' (not 'A' through 'F').

The value of PkdFld2 does, however, meet the encoding requirements of zoned decimal data, documented here in the ILE RPG Language Reference, where the second nibble of each byte must be in the range of '0' through '9' (the '0', '0', and '1' of x'F0F0F1') and the first nibble of the last byte must be in the range of 'A' through 'F' (the third 'F' of x'F0F0F1'), representing the value 1+. In the case of DDEDATA, this encoding of PkdFld2 as zoned decimal is just a curiosity, but in the real world, where data might be coming from another system/company, it could also, if a pervasive cause of decimal data errors, indicate an incorrect data definition in either the originator of the data or the recipient of the data.

Let's now look at the data associated with relative record number 4. Three of the fields (PkdFld1, PkdFld2, and ZndFld3) are identified in the report as being in error. With our previous discussion of the encoding requirements for packed decimal and zoned decimal fields, and the hexadecimal values being shown for these fields, you should be able to identify what is wrong with these fields. In the case of PkdFld1, there is an invalid 'A' where a number is required and an invalid sign value of '3'; in the case of PkdFld2, two invalid 'A' values and a bad sign ('2'); and in the case of ZndFld3, an invalid '8' where the sign is expected. Note that while there are a variety of both numeric and alphabetic values ('8's, '9's, and even a 'C') in the first nibble of various bytes of ZndFld3, this is not an error as only the first nibble of the last byte (the sign nibble) is used when working with zoned decimal fields. Change the second 'e' of 'Here' in the LOADDDE program to a '5' (that is, 'ibleHer5') and you'll find that ZndFld3 now has the value of 92358595+. Change the 'e' to 'N' and ZndFld3 is now 92358595-. The reason for this is that the '5' provides a valid positive sign ('5' is x'F5') and the 'N' a valid negative sign ('N' is x'D5').

A minor item to point out, related to the byte of data providing the sign value, is that when a numeric field such as PkdFld1 of relative record number 2 is all blanks, it's really only the byte containing the sign that is causing the decimal data error. The blank character is x'40', and neither the '4' for zoned decimal fields or the '0' for packed decimal fields will yield a valid sign value (which is 'A' through 'F'). All of the other bytes, however, are quite valid in that the '0's represent a valid numeric digit for zoned decimal and both the '4's and '0's represent valid numeric digits for packed decimal fields.

Continuing to look at relative record number 4, you might notice that the fields ZndFld1 and ZndFld2 are not included in the FNDDDE report even though the loaded values are the character strings 'dD' and 'IsP', respectively. Given the previous paragraph, you can probably guess what's happening. If you write a program (using the cvthc API, of course) to display the hexadecimal values of the character strings 'dD' and 'IsP' in character form, you will find that these field values are x'84C4' and x'C9A2D7', respectively. These values do conform to the rules of zoned decimal encodings and result (as we now know) in the numeric values of 44+ and 927- (or more accurately 92.7- as ZndFld2 is defined with one digit to the right of the decimal point). Due to the context of how these values are setas part of the larger character string 'BadDataIsPossibleHere'I would consider these numeric values to be bogus (though "good" in that they do not generate decimal data errors) and not "right" for processing. The key item here is that there can be data problems above and beyond decimal data errors.

By now, you most likely know why the sixth record of DDEDATA, loaded with the rather strange character string of '()E5q?xyz1-/"s0AT0J0D', doesn't show up in the FNDDDE report at all. That odd character string happens to result in hexadecimal values that conform to the rules for zoned decimal and packed decimal encodings. Verifying this, with the help of the cvthc API, is left as an exercise for you.

Next month's article will be the last planned change to the FNDDDE program. In addition to providing the relative record number of any record containing decimal data errors (which is currently being done), FNDDDE will be enhanced to also print the key values (assuming the file is keyed, of course) associated with any records containing decimal data errors. Certainly a nice feature if you want to "fix" the data.

As usual, if you have any API questions, send them to me at This email address is being protected from spambots. You need JavaScript enabled to view it.. I'll see what I can do about answering your burning questions in future columns.

Bruce Vining

Bruce Vining is president and co-founder of Bruce Vining Services, LLC, a firm providing contract programming and consulting services to the System i community. He began his career in 1979 as an IBM Systems Engineer in St. Louis, Missouri, and then transferred to Rochester, Minnesota, in 1985, where he continues to reside. From 1992 until leaving IBM in 2007, Bruce was a member of the System Design Control Group responsible for OS/400 and i5/OS areas such as System APIs, Globalization, and Software Serviceability. He is also the designer of Control Language for Files (CLF).A frequent speaker and writer, Bruce can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it.. 


MC Press books written by Bruce Vining available now on the MC Press Bookstore.

IBM System i APIs at Work IBM System i APIs at Work
Leverage the power of APIs with this definitive resource.
List Price $89.95

Now On Sale

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$0.00 Raised:
$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: