22
Mon, Apr
1 New Articles

The API Corner: Take Advantage of Open List APIs

APIs
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Do you know how to open, use, and close an open list?

 

This article is the fourth in a series that discusses how to find all *PGMs and *SRVPGMs that have a specific *MODULE bound into them. Before reading this article, you may find it beneficial to review the three prior articles:

•·                 "Module, Module, Who's Got My Module?"

•·                 "Finding Modules in a *SRVPGM"

•·                 "Finding All *SRVPGMs on the System"

While the stated intent is to find all uses of a given module, the actual purpose of this series of articles is to introduce the concepts and proper use of two types of system APIs:  List and Open List.

 

In the last article, "Finding All *SRVPGMs on the System," we saw the new and changed code for using the Open List of Objects (QGYOLOBJ) API, the Get List Entries (QGYGTLE) API, and the Close List (QGYCLST) API, but we had not had the opportunity to examine these additions in detail. We'll do so now. The approach we'll take is to step through the processing flow of the program, explaining any new data definitions as we encounter them. For space reasons, the source for program MODUSAGE is not being repeated in this article, so you may want to refer to the previous article.

 

The first set of changes is in the Setup procedure. Previously, all API calls had used an error code structure that specified that APIs should return errors as escape messages. We're now adding an API call to MODUSAGE (specifically, the call to the Get List Entries API in procedure OpnLstDone, which we'll discuss later) so that it's possible for the API to perceive an error situation that we'll be handling within MODUSAGE. Due to the relative expense of sending escape messages and monitoring for the message versus having the API directly return the message ID and exception data within the error code structure, we have added a new error code data structure named ErrCde. ErrCde is defined likeds(QUSEC) with an additional 128 bytes allocated for message data. By setting ErrCde.Hdr.QUSBPRV to a non-zero value, we're telling the API to not send escape messages for error situations. Most of the API calls in MODUSAGE continue to use the QUSEC instance of the error code. For now, only one API call is using the ErrCde instance of the error code structure.

 

The second change within Setup is adding initialization logic related to calling the Open List of Objects (QGYOLOBJ) API. MODUSAGE is using /copy qsysinc/qrpglesrc,qgyolobj to copy in the various IBM-provided data structure definitions for QGYOLOBJ. Within this include, IBM provides data structure QGYOSI for specifying the Open list Sort Information (the fifth parameter of QGYOLOBJ). By setting the Number of sort Keys subfield QGYNBRK to 0 in Setup, we are indicating that no sorting of the API output is needed. Also provided by IBM is the Open list Authority Control data structure QGYOAC (the eighth parameter of QGYOLOBJ). By setting the Format Length subfield QGYFL04 to the provided size of QGYOAC, we are indicating that no special authority checking is needed (that the defaults are sufficient). The Open list Selection Control data structure (the ninth parameter of QGYOLOBJ) QGYOSC is also provided by IBM, but in this case we need to specify an include/omit status field as the API provides no default. MODUSAGE therefore creates a new data structure SelCtl, defining SelCtl as likeds(QGYOSC) with a one-byte Status field immediately following QGYOSC. In Setup, MODUSAGE initializes SelCtl so that all objects are included in the API output.

 

The key item in this initialization is how both QGYOAC and SelCtl.Ctl are first set to *loval prior to setting specific subfields of these data structures. Both of these structures are used for input to an API and contain reserved subfields that must be set to a specific value. These reserved fields, QGYERVED04 of QGYOAC and QGYERVED05 of QGYOSC, must be set to x'00' (*loval), but RPG by default will initialize them to blanks (x'40's) as they are subfields of a data structure. In this situation, the correct way to initialize a subfield is to initialize the entire data structure. Initializing the subfield directly by name will work but can create a compatibility problem. This compatibility concern can arise if IBM starts to use either of these reserved fields for some purpose in a future release and, at that time, if IBM changes the name of the subfield. If we were to recompile MODUSAGE for any reason, the compile would fail as the reserved field name would no longer be defined. This concern can be eliminated by simply not using the name of any reserved field.

 

In the main procedure of MODUSAGE, following the call to Setup, we've added the call to the QGYOLOBJ API. The API call itself is quite straightforward and, as we reviewed the parameters for QGYOLOBJ in the previous article, we won't repeat that discussion here. We are asking that up to 50 *SRVPGM list entries be returned in the Receiver variable parameter RcvVar, which is defined as 4096 bytes in size (this size by the way is totally arbitrary and actually larger than needed--but sufficient for 50 list entries). Upon return from the API, MODUSAGE checks if the Information Complete subfield QGYIC07 of the QGYLI data structure shows that either complete ('C') or partial ('P') information was returned in the Receiver variable RcvVar.

 

If either is true, we enter into a FOR loop, which will continue as long as the Open List APIs continue to return either a 'C' or 'P' Information Complete status. Within the FOR loop, we set the Receiver Variable Entry Pointer pointer variable RcvVarEPtr to the address of the RcvVar variable. RcvVarEPtr is used as a base pointer for the Receiver Variable Entry data structure RcvVarE. RcvVarE, in turn, is defined as likeds(QGYORV01), the IBM-supplied definition for the Open list Receiver Variable. By setting RcvVarEPtr to the start of the RcvVar variable, RcvVarE now address the first list entry that was returned by QGYOLOBJ (or, in subsequent iterations of the FOR loop, returned by the Get List Entries API, QGYGTLE). MODUSAGE then enters into a second FOR loop that is exited when the number of Records Returned (QGYRRTN02) has been processed. This processing is done in the Check Service Program Module procedure, ChkSrvPgmMod, as was true in the original MODUSAGE program. After processing each list entry, the value of the RcvVarEPtr pointer is incremented by the Record Length of a returned list entry (QGYRL07) so that the next iteration of the inner FOR loop processes the next list entry. When this FOR loop has completed, MODUSAGE calls the Open List Done procedure, OpnLstDone, to determine if all list entries for the entire list (as opposed to the subset of the list returned in the current RcvVar) have been processed. If so, we leave the outer FOR loop; otherwise, we continue processing more list entries.

 

Upon exiting the outer FOR loop, we check to see if the Information Complete field QGYIC07 has the value of 'I'. This would indicate that a failure was encountered in building the QGYOLOBJ list and that not all *SRVPGMs were necessarily returned. In this case, an error message is displayed. If not, the summary message indicating how many module occurrences were found is displayed.

 

Within the ChkSrvPgmMod procedure, two changes were made. First, when calling QBNLSPGM, we are now only asking for module information related to the *SRVPGM identified by the current QGYOLOBJ list entry. This is done by using the QGYOLOBJ returned fields Object Name, RcvVarE.QGYON00, and Object Library, RcvVarE.QGYOL03. The second change is the addition of a LEAVE operation within the FOR loop of ChkSrvPgmMod. In the original MODUSAGE program, all *SRVPGMs and *MODULES were returned in one list, so we had to process all modules of a given *SRVPGM in order to access the modules of the next *SRVPGM. In this version of MODUSAGE, only the modules for a specific *SRVPGM are returned in the list, so once we have a match on the module name, we no longer need to process the remaining modules of that *SRVPGM. We now leave the loop and move on to the next *SRVPGM list entry returned by QGYOLOBJ.

 

We also added the new procedure Open List Done (OpnLstDone). This procedure is used to determine whether all list entries from the Open List API output have been processed. In case you're wondering, the reason this check is done in a separate procedure from the main procedure (which calls OpnLstDone) is so that we can reuse this logic when later adding *PGM support to MODUSAGE (using the List ILE Program Information API, QBNLPGMI). OpnLstDone first determines whether less than the total number of list entries in the open list have been processed. This is done by comparing the number of list entries processed (determined by adding the current first buffer record, QGYFBR01, to the number of records in the current buffer, QGYRRTN02) to the total number of list entries in the open list, QGYTR07. If all list entries have not been processed, OpnLstDone calls the Get List Entries (QGYGTLE) API, which is documented here. For your convenience, the parameter list for QGYGTLE is shown below, but because the parameters are very straightforward we won't review them individually.

Get List Entries (QGYGTLE) API

 

Required Parameter Group:

1

Receiver variable

Output

Char(*)

2

Length of receiver variable

Input

Binary(4)

3

Request handle

Input

Char(4)

4

List information

Output

Char(80)

5

Number of records to return

Input

Binary(4)

6

Starting record

Input

Binary(4)

7

Error code

I/O

Char(*)

 

  Default Public Authority: *USE

  Threadsafe: No

 

If all list entries have been processed (QGYFBR01 + QGYRRTN02 > QGYTR07), MODUSAGE determines if the building of the open list by the background server job has completed. This is done by checking the List Status Indicator field QGYLSI01. If QGYLSI01 is '2', indicating that the complete list has been built, then OpnLstDone calls the Close List (QGYCLST) API, documented here, and returns an *on logical variable to indicate that the list is done. If not, OpnLstDone calls the QGYGTLE API, requesting 50 more list entries, assuming that the total number of records may have increased since the previous call to QGYOLOBJ or QGYGTLE due to additional processing by the background server job.

 

It is this call to QGYGTLE that uses the ErrCde data structure we initialized earlier in anticipation of possibly getting an error returned by an API. When we call QGYGTLE, we are asking that the next 50 list entries from the QGYOLOBJ *SRVPGM open list be returned to the variable RcvVar. We identify the list we want the 50 entries from with the Request Handle variable QGYRH07 that was returned by QGYOLOBJ in the QGYLI data structure. We identify what relative record number within the open list we want the 50 returned entries to start from with the sixth parameter of QGYGTLE. It is this starting relative record number that may or may not cause QGYGTLE to believe an error has occurred.

 

Let's say we have exactly 50 *SRVPGMs (or some multiple of the requested number of list entries) in library A and some huge number of additional libraries that contain no *SRVPGMs. When we call QGYOLOBJ, it finds the 50 *SRVPGMs rather quickly and returns them to us, along with a total record count (QGYTR07) of 50 and a list status indicator (QGYLSI01) value of '1' - list is in the process of being built (as the background server job is still looking for *SRVPGMs in all of the other libraries). OpnLstDone, when calling QGYGTLE, is going to ask for 50 more records, starting at relative record number 51 (QGYFBR01 + QGYRRTN02). Once the background server completes building the list, QGYGTLE will determine that there are only 50 list entries in total and, as MODUSAGE asked for list entry number 51, inform us of the "error" with message GUI0006 - &1 is not valid for starting record number. Conceptually, the GUI0006 message is an indication of End of File. This "error" could be handled in a variety of ways. In the case of an error message actually being sent from the API, we could use a monitor group, an 'E' extender on a CALLP, a *PSSR, etc. In our current design, we elected to simply tell the API to not send an error message and to instead return the message ID and message data in the ErrCde data structure.

 

Due to this approach of not receiving exceptions from the API, we have a SELECT group following the call to QGYGTLE. If there is no error (ErrCde.Hdr.QUSBAVL = 0), then OpnLstDone returns an *off logical variable to indicate that more list entries have been returned for processing (that is, the list is not done). If there is an error returned by the API (ErrCde.Hdr.QUSBAVL <> 0), we check to see if the error was GUI0006 by examining the Exception ID field ErrCde.Hdr.QUSEI. If it's GUI0006, we close the open list and return an *on logical variable to indicate that all list entries have been processed. If the error message isn't GUI0006, we call the SndErrMsg procedure to send the error message as an escape message.

 

An important function of the OpnLstDone procedure, and one that is easy to forget, is in the closing of the open list. The system has no way of knowing when MODUSAGE is done processing the list entries associated with the open list. The open list will continue to exist until we either explicitly close the list with the QGYCLST API or the job ends. As open lists can be quite large, repetitively calling an application such as MODUSAGE, which opens a list on each call, and neglecting to close the list when we're done with it can cause the temporary storage associated with the job to grow and grow. You may not initially notice this consumption of storage when working with open lists of only a few KBs or MBs in size, but if you call MODUSAGE 100 times, and each time you open two open lists of even moderate size, you may start seeing the impact. Having a few GB-size open lists certainly makes it more obvious that we have a storage consumption (sometimes referred to as a memory leak) problem if the open lists are not closed when the application is done using the list.

 

I will point out that, with another design, it is also possible to avoid this "error" from QGYGTLE entirely. As with QGYOLOBJ, QGYGTLE supports a special value of 0 for the number of records to return and, related to this, a special value of 0 for the starting record number. When 0 is specified for these parameters, QGYGTLE returns only the list information data structure QGYLI and no list entries. Prior to calling QGYGTLE asking for 50 list entries, MODUSAGE could call QGYGTLE requesting that only QGYLI be returned. MODUSAGE could then determine if the total number of records QGYTR07 is now larger and/or if the open list is now complete (QGYLSI01) and know whether or not it is "safe" to call QGYGTLE for actual list entries. Either approach will work, I simply prefer in this case to assume that there are more list entries and check for a failure rather than to poll the open list to determine when to ask for more list entries and then call the API a second time to actually get the list entries. In a different environment--say one where we could do additional work within the application in lieu of processing additional list entries--a design approach of polling, rather than waiting for QGYGTLE to respond either with more list entries or message GUI0006 might be superior.

 

The SndErrMsg procedure, called by OpnLstDone, uses the Send Program Message (QMHSNDPM) API documented here. I won't go into the details of what is being done (that would be a whole different set of articles) other than to say that whatever error message QGYGTLE would have sent to MODUSAGE as an escape, we're now sending the same message to the caller of MODUSAGE.

 

That pretty much describes what changes have been made to MODUSAGE that enable us to now use the QGYOLOBJ API to determine what *SRVPGMs exist on the system and to then provide those *SRVPGM names to QBNLSPGM so that we can perform analysis of the modules bound into the *SRVPGM. In the next column, we'll add support for *PGM analysis in MODUSAGE by using the List ILE Program Information (QBNLPGMI) API.

 

Meanwhile, if you have other API questions, send them to me at This email address is being protected from spambots. You need JavaScript enabled to view it.. I'll see what I can do about answering your burning questions in future columns.

Bruce Vining

Bruce Vining is president and co-founder of Bruce Vining Services, LLC, a firm providing contract programming and consulting services to the System i community. He began his career in 1979 as an IBM Systems Engineer in St. Louis, Missouri, and then transferred to Rochester, Minnesota, in 1985, where he continues to reside. From 1992 until leaving IBM in 2007, Bruce was a member of the System Design Control Group responsible for OS/400 and i5/OS areas such as System APIs, Globalization, and Software Serviceability. He is also the designer of Control Language for Files (CLF).A frequent speaker and writer, Bruce can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it.. 


MC Press books written by Bruce Vining available now on the MC Press Bookstore.

IBM System i APIs at Work IBM System i APIs at Work
Leverage the power of APIs with this definitive resource.
List Price $89.95

Now On Sale

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$0.00 Raised:
$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: