|RPG's Persistent User Spaces|
|Programming - RPG|
|Written by Paul Tuohy|
|Tuesday, 04 December 2007 18:00|
In order to make proper use of user spaces, you need to change your traditional approach to writing programs. Traditional RPG programmers are accustomed to minimizing the overhead of a program; we've spent a lot of time ensuring that a program utilizes only the memory it actually needs (e.g., how often have you wondered how many elements to assign to an array?).
But the "modernized" RPG programmer takes a different view of minimizing the overhead. Memory is cheap, and the system and programs are now so fast that there is little or nothing to be gained from the minimalist approach. Dare I say it, but it is a PC style of programming: load the data into memory, manipulate it, and save it. Think of how your favorite spreadsheet or word processor program works.
This approach is also reflected in how RPG has changed to accommodate larger field sizes: 256 to 64K for character fields and 999 to 32,767 for the number of elements in an array. And wait until you see what happens with these limits in V6R1!
As with most RPG programmers, I first came across user spaces when using the list APIs. I soon realized that user spaces also had uses in applications and could prove invaluable in a Web environment that was dependent on CGI programs.
Basically, using user spaces taught me that not everything needs to be database based and at times it is better to keep a lot of information in memory.
In this article, I offer a quick review of user spaces and explain how to create and manage them. I will also show how you can use user spaces to handle the problem of persistence in a CGI environment.
A user space is an object (object type *USRSPC) that consists of bytes, the format and content of which are up to you. More precisely (even though it might sound extremely vague), a user space is whatever you want it to be. A user space is "loaded" into memory and is accessed in a program by using pointers and based variables; it is simply a stream of bytes that you can access directly from within a program.
The nearest that traditional programming can come to this concept is a data area, but you have to input and output data areas to and from programs. In contrast, you can immediately change the content of a user space by merely changing the value of a field in your program.
Also, a user space will only be "loaded" in memory once, which means that multiple programs (whether in the same job or multiple jobs) will share the same instance of the user space. This means that multiple programs in different programs are sharing the same portion of memory. So, when program PGMA in job JOBA changes the value of a field, it is actually changing the value of a field in PGMB running in JOBB, without PGMB having to do any I/O operations. The fact that a user space may be common to programs in multiple jobs at the same time (without the programs having to perform any I/O) is a feature that proves very useful in CGI programming.
A user space has a maximum size of 16 megabytes. To create a user space or manipulate its contents, you must use one of the system-supplied APIs, and the only system-supplied command for user spaces is Delete User Space (DLTUSRSPC).
Although user spaces have been on the system for a long, long time, it was only with the advent of pointers in RPG IV that they became a lot easier to use and a lot more beneficial.
The Persistence Problem
One of the issues with CGI is that the connection between the browser and the called program is not persistent, i.e., a request is sent from a browser to a program in a CGI job, and the program returns an HTML document to the browser; the connection is now "broken," and the next request to the CGI job could come from a different browser.
On the same basis, when a request is sent from a browser to a program in a CGI job, the program returns an HTML document, and when the browser makes a subsequent request, there is no guarantee that the request will go to the same CGI job.
If there is any data that must be maintained for a browser, a program must have some way of storing it for each requesting browser session and ensuring that the data is available to all CGI jobs. Although this problem could be solved using database files, there would be the requirement for CGI programs to constantly read, write, update, and delete records. User spaces provide a more efficient means of managing "persistent" information without the need for any I/O.
Let's look at an example.
At System i Developer, there is a small application we use to maintain information for planning our RPG & DB2 Summit conferences. This application is Web-based and is written using CGIDEV2 (details available at easy400.net).
All of the maintenance programs use externalized database processing subprocedures for database maintenance. The basic concept is that the client programs (CGIDEV2 programs in this example) have no idea of the format of the actual database. All access to the data is through getters and setters. If you are not familiar with the concept of externalizing a database, you can get your hands on some sample code here. (These examples do not have the user space processing applied, but the following information should make it easy to implement the required changes if you wish).
Here's the basic format for external processing (speaker data in this example):
The important parameter in all of these subprocedures is the Id. The module containing the database subprocedures contains two arrays: an array of database records it is currently "handling" and an array of single-character fields used to identify the next available record element. The NewSpeaker subprocedure returns the index (Id) where the requested database record (Speaker is the key) is being stored. The Id is then used as a parameter to any Get or Set subprocedure to identify the record to be processed. The client program calls the Release subprocedure when it is finished processing the data.
In order to incorporate this design into a CGI environment, the two arrays (handles and database records) must be common (or available) to any CGI jobs that might end up processing a request from a browser. If the two arrays are placed in a user space, they are both "common" to programs in all CGI jobs that need to access them; all the programs need is the Id to identify which element to process. Therefore, all the browser page has to return is the Id, which is easily achieved using a hidden field in a form.
Creating and Managing User Spaces
Since a user space will be required for each table (physical file) in my application, I decided to write a subprocedure that creates a user space (if it does not already exist), loads the user space into memory, and returns a pointer with the location of the user space. This pointer may then be used as the basing pointer for a based variable. Figure 1 shows the prototype for a subprocedure named GetStorageSpace.
Figure 1: This is the prototype for the GetStorageSpace subprocedure.
The GetStorageSpace subprocedure has three parameters:
The information that will be placed in the user space must be based on pointers. Figure 2 shows the definition of the two arrays (mentioned earlier) that will be mapped to the user space used for the Speaker database (Speakers is the record format on the Speaker table).
Figure 2: These based arrays will be mapped to the user space.
Figure 3 shows the definition of the CheckStorage subprocedure. This subprocedure calls the GetStorageSpace subprocedure, providing the name of the user space (STSPEAKERS) and the initial size (8M). The returned pointer (PtrHandles) is the basing pointer for the Handles array. The length of the Handles array is added to PtrHandles to calculate the value of the basing pointer (PtrStoreRecord) for the record array. This means that the first 32,767 bytes of the user space will contain the Handles array followed by the record array.
CheckStorage is executed as the first line in the NewSpeaker subprocedure.
Figure 3: Set the basing pointers.
The User Space APIs
The GetStorageSpace subprocedure is going to make use of the Create User Space (QUSCRTUS), Change User Space Attributes (QUSCUSAT), and Retrieve Pointer to User Space (QUSPTRUS) APIs. Figure 4 shows the corresponding prototypes, along with the definition of the SpaceAttribute data structure, which is used as a parameter to the QUSCUSAT API:
Figure 4: Here's the prototype for the QUSCRTUS, QUSCUSAT, and QUSPTRUS APIs.
The QUSCRTUS API consists of one mandatory parameter group and two optional parameter groups (remember, if one parameter in an optional parameter group is defined, then all parameters in the optional parameter group must be defined). These are the parameters for the QUSPTRUS API:
These are the parameters for the QUSCUSAT API:
There are four user space attributes that may be changed: the size of the user space, the initial value, automatic extendibility (the one we are interested in), and the transfer size request. The attribute data structure (SpaceAttribute in Figure 4) contains the following information:
These are the parameters for the QUSPTRUS API:
The GetStorageSpace Subprocedure
Figure 5: This is the GetStorageSpace subprocedure.
The main points to note are these (refer to the corresponding numbers in Figure 5):
Some Other Thoughts
This example showed a user space being used to make database data persistent across multiple CGI jobs. But it could be any data. A user space could be used to store information from multiple pages as a user enters information (think of the process of booking a flight online). The information is stored in a data structure array, and each requesting browser is assigned a sequence number that corresponds to an element of the data structure array. The sequence number is written to the browser and is returned as a parameter, as with a hidden field on a display file format. When a program in a CGI job receives a request from a browser, it uses the returned sequence number to identify the element containing the required persistent data.
Be very careful about assigning sequence numbers. What happens if a user closes the browser and reopens it or returns to a starting page? You may want to consider separate subprocedures for assigning a sequence number and checking the validity of a sequence number when a request is received; assignment and validity would be based on how long it has been (and that length of time is up to you) since a request was processed for a specific sequence number, which would allow for re-use of sequence numbers.
Having the data structure array mapped to a user space can enhance the process in a number of ways. Firstly, you can exceed the 32,767 element maximum; assuming that the user space can physically store more than 32,767 elements, you can simply change the basing pointer of the data structure array to have it start in a different portion of the user space (or even use a single data structure and use the starting pointer, the size of the data structure, and the sequence number to calculate where in the user space the data structure should be mapped). Secondly, during development and testing, having the data in a user space provides a means of viewing the "persistent" data in the program without having to debug it. All you have to do is write a program to view the data in the user space.