|Use String Manipulation Built-In Functions to Process External Files|
|Programming - RPG|
|Written by Thomas Snyder|
|Wednesday, 15 July 2009 01:00|
With the right RPG tools, you can make the most challenging data manipulation project fun.
Dealing with interface files from external systems is always an involved exercise in data validation. It's also a prime opportunity for using the string manipulation built-in functions (BIFs) available in RPG ILE.
Suppose you receive from an external system a file that is not available in an XML format, so you are required to process an ASCII text file. You have to be able to handle it and process it, especially if it contains data that generates revenue. You don't want to reject or misinterpret a single record, so you need to process that data and convert it into a usable format.
Here are some string manipulation BIFs that we will be using in our data-processing example program:
Here is a simple program that emulates a worst-case file record read without the file input/output complexities to distract from the main objective of string manipulation. I've created a simple string to represent a read operation that acts as a single line of data from an external file.
D inputBytes S 42A
D subBytes S 10A
D numberBytes S 10A
D junkInBytes S 10A
D junkOutBytes S 10A
D displayBytes S 52A
D posi S 10I 0
D* XLATE STRUCTURE FOR LOWER TO UPPER CASE CONVERSION + X'3A' FOR E'*
D lc S 27A
D uc S 27A
// Lowercase and Uppercase Characters
lc = 'abcdefghijklmnopqrstuvwxyz';
uc = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
// ALSO include conversion of Acute Accent "E'" to "E" in Cafe' ***
lc = %TRIM(LC) + X'3A';
uc = %TRIM(UC) + 'E';
junkInBytes = ':"''-,';
junkOutBytes = ' ';
numberBytes = '1234567890';
inputBytes = ' 319 '
+ 'Caf' + X'3A'
+ ', "Double" ''Quotes'':'
+ ' 1208-0807 ';
displayBytes = '[' + inputBytes + ']';
In the string, you can see spaces at the beginning and end of the string, an acute accent e, double quotes, single quotes, a colon, and a hyphen between the last numbers just for good measure.
When %trim is executed, you will see that both the leading and trailing blanks were removed.
displayBytes = '%trim['
When %trimR is executed, only the trailing blanks are removed.
displayBytes = '%trimR['
When %trimL is executed, only the leading blanks are removed.
displayBytes = '%trimL['
Using a combination of %check and %trimL, we are able to retrieve the first numeric value, which could have possibly been passed as a number in a right-justified field.
// Using Check to get the Front Number.
posi = %check(numberBytes:%trimL(inputBytes));
subBytes = %subst(%trimL(inputBytes):1:posi);
displayBytes = 'Front # ['
Using a combination of %checkR and %trimR, we can retrieve the last numeric value, which could have possibly been passed as a number in a left-justified field.
// Using CheckR to get the Back Number.
posi = %checkR(numberBytes:%trimR(inputBytes));
subBytes = %subst(%trimR(inputBytes):posi+1);
displayBytes = 'Back # ['
Retrieving the numbers at both the front and back of the string, the %subst BIF was used to pull the specified characters from the main string.
The lc->UC line shows the %xlate BIF providing one means of converting all of the lowercase letters to uppercase, with an additional conversion for the acute accent e.
// XLATE to UPPER Case
displayBytes = 'lc->UC['
The junk line shows the %xlate BIF in action again, removing any characters that you may find undesirable in your database when bringing the information in. Of course, junk is in the eye of the developer, so your list of undesirable characters may differ.
// Strip the Junk
displayBytes = 'junk['
And the clean line has the kitchen sink thrown at it, where all of the modifications were applied to the data to provide the intended clean results.
// All Cleaned Up
displayBytes = 'clean['
*inlr = *ON;
After running the program, you will see this output :
> call mcpress
DSPLY [ 319 Caf , "Double" 'Quotes': 1208-0807 ]
DSPLY %trim[319 Caf , "Double" 'Quotes': 1208-0807]
DSPLY %trimR[ 319 Caf , "Double" 'Quotes': 1208-0807]
DSPLY %trimL[319 Caf , "Double" 'Quotes': 1208-0807 ]
DSPLY Front # 
DSPLY Back # 
DSPLY lc->UC[ 319 CAFE, "DOUBLE" 'QUOTES': 1208-0807 ]
DSPLY junk[ 319 Caf Double Quotes 1208 0807 ]
DSPLY clean[319 CAFE DOUBLE QUOTES 1208 0807]
When the strings are displayed, they are surrounded by brackets to identify any leading and trailing blanks. In the final line of the output, you will see the results of applying all of the string manipulations, ending up with only the shiny, happy characters that will be entered into the database.
You may laugh, but I have actually had files that contained every one of these instances in a single file! No lie! But I am not one to say that it can't be done, so I did what I had to do.
Download the Code
You can download the code used in this article--as well as the fixed-format version--by clicking here.
The Right Tools for the Job
With the right RPG tools, you can take the most challenging data and make a fun project out of it.
|Last Updated on Wednesday, 15 July 2009 01:00|