18
Thu, Apr
5 New Articles

The API Corner: Monitoring a Job Queue

APIs
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

 

Learn the value of using the Job Notification Exit Point.

 

Back in April, the following question was posed on the MC Press Online CL forum:

"I am looking for advice on how to create a CL that will actively monitor a specific job queue and release jobs. All jobs that are submitted automatically go to hold status because the JOBD for end users is set to this. The business does not want to change the JOBD, so I need to come up with a way to actively monitor jobs for a specific queue and release jobs that are submitted. Any ideas?"


Initial responses to the question suggested using a tool that would convert data similar to what is found with the WRKJOBQ command to a database file and then reading the resulting database to determine if there are jobs currently on hold. Such a tool certainly represents a valid solution, but conversion to a database file also suggests (at least) two considerations. First that the CL program will most likely need to poll the job queue periodically, using the conversion tool to convert WRKJOBQ data to a database file, in order to determine if there are new jobs found in the resulting database fileso, based on the polling interval, there may be a delay between a job being submitted and the job actually running. Second that sometimes the conversion tool may return the fact that there are no jobs currently on the job queue in a held status, meaning that work was performed on the system (the conversion) only to find that there was nothing to do.

Better, to my thinking anyway, would be to have the system tell me when a job has been submitted and allow me to then release the job (if appropriate). Needless to say, there is such a capability and that's what we'll be looking at today. One item I want to point out, though. I'm quite aware that the "API Corner" is published in MC RPG Developer, so you might reasonably expect this article to contain an RPG program. But as the original question asked for a CL-based solution, this month's column will use CL and next month we'll look at an RPG-based solution (and one with a bit more function). I rationalize this as the "API Corner" is intended primarily to introduce you to system functions you may not be aware of and not, directly anyway, ways to code something in RPG. And who knows, you may even learn something about CL as CL in the 21st century is certainly not your parent's CL!

The exit point we will be using to solve today's problem is the Job Notification Exit Point, which was first introduced with V3R7. This exit point can be used to have the system send notification messages to a user-specified data queue when jobs are:

  1. 1.Placed on a job queue
  2. 2.Started
  3. 3.Ended

The notification message, in the case of a job being submitted to a job queue, provides information such as the qualified job name and the qualified job queue name. The data queue used should be created with an entry length of 144 bytes or greater, using a keyed sequence length of 4 bytes. The key values provided by the exit point are '0001' for job start notifications, '0002' for job end notifications, and '0004' for job queue submission notifications. By reading notification messages by key, you can have different programs process the various types of notifications. For today's purposes, we'll be looking for key values of '0004'. The following command will create a data queue, named MyDtaQ in library BVINING, that matches the above requirements.

CRTDTAQ DTAQ(BVINING/MYDTAQ) MAXLEN(144) SEQ(*KEYED) KEYLEN(4)

The definition of the job submitted notification message (as of V5R4; there's more in later releases) is:

Offset

Type

Field

Dec

Hex

0

0

CHAR(10)

Message identifier

10

A

CHAR(2)

Message format

12

C

CHAR(16)

Internal job  identifier

28

1C

CHAR(26)

Qualified job name

54

36

CHAR(20)

Qualified job queue name

74

4A

CHAR(8)

Time-stamp job entered system

82

52

CHAR(16)

Reserved

98

62

CHAR(1)

Job type

99

63

CHAR(1)

Job subtype

100

64

CHAR(44)

Reserved

The exit point will set the Message identifier field to the constant '*JOBNOTIFY' and the Message format field to the either the constant '01' for job start and end messages or '02' for job queue messages. The Qualified job name and Qualified job queue name fields are pretty much what you would expect (though fully documented in the Information Center API description).

With this introduction to the message layout out of the way, the following CL program provides two functions:

  1. Processes each notification message sent to BVINING/MYDTAQ from the Job Notification Exit Point that reflects a job being placed on the job queue BVINING/MYJOBQ while the subsystem MYSBS is active. This processing is basically to release any job submitted to the job queue. Note that there is no "real" error checking when releasing the job; the program simply tolerates any CPF message that may be sent in response to the Release Job command.
  2. Stops processing when receiving a data queue message containing the value 'STOP' in the first 10 bytes of the message.

Pgm                                                        

                                                          

Dcl       Var(&DtaQMsg) Type(*Char) Len(144)            

Dcl       Var(&MsgHdr)   Type(*Char) Len(10) +          

               Stg(*Defined) DefVar(&DtaQMsg 1)            

Dcl       Var(&MsgFmt)   Type(*Char) Len(2) +          

               Stg(*Defined) DefVar(&DtaQMsg 11)          

Dcl       Var(&QualJobN) Type(*Char) Len(26) +          

               Stg(*Defined) DefVar(&DtaQMsg 29)          

   Dcl       Var(&JobName) Type(*Char) Len(10) +        

                 Stg(*Defined) DefVar(&QualJobN 1)        

   Dcl       Var(&JobUser) Type(*Char) Len(10) +        

                 Stg(*Defined) DefVar(&QualJobN 11)        

   Dcl       Var(&JobNbr)   Type(*Char) Len(6) +        

                 Stg(*Defined) DefVar(&QualJobN 21)        

Dcl       Var(&QualJobQ) Type(*Char) Len(20) +          

               Stg(*Defined) DefVar(&DtaQMsg 55)          

   Dcl       Var(&JobQName) Type(*Char) Len(10) +      

                 Stg(*Defined) DefVar(&QualJobQ 1)        

   Dcl       Var(&JobQLib)   Type(*Char) Len(10) +      

                 Stg(*Defined) DefVar(&QualJobQ 11)      

                                                          

Dcl       Var(&LenMsg)     Type(*Dec) Len(5 0)          

Dcl       Var(&WaitTime)   Type(*Dec) Len(5 0) Value(-1)

Dcl       Var(&LenKey)     Type(*Dec) Len(3 0) Value(4)  

Dcl       Var(&Key)       Type(*Char) Len(4)   Value('0004')  

Dcl       Var(&LenSndrInf) Type(*Dec) Len(3 0) Value(0)  

                                                          

Call       Pgm(QRCVDTAQ) Parm(MYDTAQ BVINING +            

             &LenMsg &DtaQMsg &WaitTime +                  

             'EQ' &LenKey &Key &LenSndrInf ' ')            

                                                          

DoWhile   Cond(&MsgHdr *NE 'STOP')                        

           If Cond((&MsgHdr = '*JOBNOTIFY') *And +        

                   (&MsgFmt = '02') *And +                

                   (&JobQName = 'MYJOBQ') *And +          

                   (&JobQLib = 'BVINING')) Then(Do)        

             RlsJob Job(&JobNbr/&JobUser/&JobName)      

             MonMsg MsgID(CPF0000)

             EndDo

                                                            

           Call Pgm(QRCVDTAQ) Parm(MYDTAQ BVINING +        

                 &LenMsg &DtaQMsg &WaitTime +            

                 'EQ' &LenKey &Key &lenSndrInf ' ')        

EndDo                                                      

                                                          

EndPgm                                                    

The first set of DCL statements, &DtaQMsg through &JobQLib, defines a data structure mapping to the subfields of the notification message we are interested in. If you are not familiar with CL data structure definitions, you can think of the DCL DEFVAR keyword as providing support similar to the OVERLAY keyword of RPG D-specifications. &DtaQMsg represents the full message received (the name of the data structure if you will), &MsgHdr the Message identifier subfield of the message, &MsgFmt the Message format subfield, &QualJobN the Qualified job name (which is further defined with subfields &JobName, &JobUser, and &JobNbr, representing the three components of a qualified job name), and &QualJobQ the Qualified job queue name (which is further defined with subfields &JobQName and &JobQLib, representing the two components of a qualified object name).

The second set of DCL statements, &LenMsg through &LenSndrInf, corresponds to various parameters that are passed to the Receive Data Queue (QRCVDTAQ) API. &LenMsg is an output of the API providing the length of the message received, &WaitTime an input to the API indicating the number of seconds to wait for a data queue message if no message is currently available with the special value of a negative number indicating to wait forever, &LenKey an input to the API indicating the length of the key data parameter being passed, &Key a combined input and output parameter allowing the caller to identify a base key value to be returned and subsequently used by the API to return the actual key value returned, and &LenSndrInf an input to the API indicating if any information related to the sender of the data queue message should be returned.

Following the DCL statements, the program calls the Receive Data Queue API requesting a message from data queue BVINING/MYDTAQ. Due to the parameters passed on the API call, the program will wait forever for the next message, the message to wait for must have a key value of '0004', and the message is to be automatically removed from the data queue (after receiving it). No sender information is requested, and it is assumed by the API that the size of parameter &DtaQMsg is sufficient to hold the largest message that might be received (which is 144 bytes due to the MAXLEN parameter value used when creating the BVINING/MYDTAQ data queue). This assumption concerning the size of &DtaQMsg, and the behavior of removing the message, can be overridden by use of the second optional parameter group of the API, but the sample program simply goes with the default behavior.

When a message is received, the program enters a DOWHILE that is controlled by the value of message subfield &MsgHdr. If &MsgHdr is 'STOP', the DOWHILE is exited and the program ends; otherwise, an IF statement checks whether the received message reflects a newly submitted job of interest. The check is for &MsgHdr to be '*JOBNOTIFY', &MsgFmt to be '02', &JobQName to be 'MYJOBQ', and &JobQLib to be 'BVINING'. If all four tests are true, then the identified job is released using the Release Job (RLSJOB) command; otherwise, nothing is done in regard to processing the received message. Whether or not a job was released, the program then calls the QRCVDTAQ API, using the same parameters described previously, and re-enters the DOWHILE loop upon receiving another message.

 

One point should be made related to the IF check within the DOWHILE: As you will see shortly, you actually register subsystems to the Job Notification Exit Point, not job queues. As subsystems can have more than one job queue associated with them, the program needs to verify that the submitted job was indeed placed on the job queue that we want to release jobs from.

Assuming that the previous CL source is in member SBMDJOBS (Submitted Jobs) of source file QCLSRC, you can create the program into library BVINING using this command:

CRTBNDCL PGM(BVINING/SBMDJOBS)

To test SBMDJOBS, we'll create a subsystem MySbs along with related objects and entries. The following commands will create a subsystem named MySbs and a job queue named MyJobQ, add a job queue entry to subsystem MySbs for job queue MyJobQ, and add a routing entry to subsystem MySbs using program QSYS/QCMD and class QSYS/QBATCH. Library BVINING is used to hold all of the created objects.

CRTSBSD SBSD(MYSBS) POOLS((1 *BASE))

CRTJOBQ JOBQ(BVINING/MYJOBQ)

ADDJOBQE SBSD(BVINING/MYSBS) JOBQ(BVINING/MYJOBQ)

ADDRTGE SBSD(BVINING/MYSBS) SEQNBR(9999) CMPVAL(*ANY) +

PGM(QSYS/QCMD) CLS(QSYS/QBATCH)   

Once the subsystem and job queue are created, the following command registers the data queue BVINING/MYDTAQ with the Job Notification Exit Point:

ADDEXITPGM EXITPNT(QIBM_QWT_JOBNOTIFY) FORMAT(NTFY0100) +

PGMNBR(*LOW) PGM(BVINING/MYDTAQ) +

PGMDTA(*JOB 24 '0004MYSBS     BVINING   ')

While the Add Exit Program (ADDEXITPGM) command is generally used to register exit programs, in this case we're actually registering the data queue with the PGM parameter. The PGMDTA parameter specifies that we want notification messages related to jobs submitted (the '0004' portion of the program data) to a job queue in the active subsystem BVINING/MYSBS (the 'MYSBS     BVINING' portion of the program data) to be sent to the data queue BVINING/MYDTAQ (the PGM parameter).

To test everything, let's create two additional programs. The first program will be one that we submit, in a held status, to the job queue MYJOBQ in BVINING. The source is shown below:

Pgm                                              

SndMsg Msg('I''m FREE!!!') ToUsr(BVINING)      

EndPgm                                          

The program simply sends the message "I'm FREE!!!" to the message queue associated with user profile BVINING, so you may want to adjust the ToUsr parameter value to your *USRPRF. Assuming that the previous CL source is in member FREE of source file QCLSRC, you can create the program into library BVINING using this command:

CRTBNDCL PGM(BVINING/FREE)

The second program will be the one that sends a 'STOP' message to program SBMDJOBS. The source is shown below:

Pgm                                                        

Dcl       Var(&LenMsg) Type(*Dec) Len(5 0) Value(10)      

Dcl       Var(&LenKey) Type(*Dec) Len(3 0) Value(4)        

                                                            

Call       Pgm(QSNDDTAQ) Parm(MYDTAQ BVINING &LenMsg +      

             'STOP' &LenKey '0004')                        

EndPgm                                                      

The program simply send the message 'STOP', using the Send Data Queue (QSNDDTAQ) API, with a key value of '0004' to the data queue MYDTAQ in BVINING. Assuming that the previous CL source is in member STOP of source file QCLSRC, you can create the program into library BVINING using this command:

CRTBNDCL PGM(BVINING/STOP)

To test SBMDJOBS, and the releasing of held jobs from job queue BVINING/MYJOBQ, we need to start the program. There are many ways to do this, but one that seems appropriate to me is to define SBMDJOBS as an auto-start job entry to subsystem BVINING/MYSBS. In this way, we'll know that SBMDJOBS is active whenever the subsystem MYSBS is started. We can do this with the following commands (where you will again want to change the USER parameter value to an appropriate *USRPRF such as your own):

CRTJOBD JOBD(BVINING/SBMDJOBS) USER(BVINING) +

RQSDTA('CALL BVINING/SBMDJOBS')

ADDAJE SBSD(BVINING/MYSBS) JOB(SBMDJOBS) JOBD(BVINING/SBMDJOBS)

Now let's test what we've done. Start the subsystem using this command:

STRSBS SBSD(BVINING/MYSBS)

Now submit our FREE program, in a held status, to the job queue BVINING/MYJOBQ using this command:

SBMJOB CMD(CALL PGM(BVINING/FREE)) JOBQ(BVINING/MYJOBQ) HOLD(*YES)

Using the following command (and adjusting the MSG parameter value to reflect the user profile used in the FREE program)

DSPMSG MSGQ(BVINING)

you should see a display similar to the following:

From . . . : BVINING       06/10/13   11:21:52

I'm FREE!!!                                      

Submit the CALL to FREE a second time. You should now find two "I'm FREE!!!" messages in your message queue. Now use this command:

CALL PGM(BVINING/STOP)

Submit the CALL to FREE a third time. You should still only find two "I'm FREE!!!" messages in your message queue as SBMDJOBS has ended (due to the call to STOP) and will no longer be releasing jobs from job queue BVINING/MYJOBQ. Now use this command:

WRKJOBQ JOBQ(BVINING/MYJOBQ)

You should see that our most recent job submission is still on HLD status on the job queue. To restart processing, we can use another way (that is, not the auto-start job entry approach) to start SBMDJOBS such as this:

SBMJOB CMD(CALL PGM(BVINING/SBMDJOBS)) JOBQ(QSYSNOMAX)

You should now find the third "I'm FREE!!!" message.

Before ending this article, there are two items I would like to point out.

First is that if you've fully followed the preceding testing scenario, make sure to CALL STOP to end the instance of SBMDJOBS that was started using job queue QSYSNOMAX. If you neglect to do this and later end and restart the subsystem BVINING/MYSBS, you will encounter one of those wonderful "learning opportunities" as you will have both the QSYSNOMAX and the auto-start job entry instances of SBMDJOBS processing data queue BVINING/MYDTAQ. Having duplicate instances of SBMDJOBS isn't much of a problem when it comes to releasing held jobs but can get much more interesting in terms of handling messages such as 'STOP' (and any additional messages that you might add).

Second is that the Job Notification Exit Point support is very much driven by active subsystems. It was mentioned earlier that a given subsystem can have multiple job queues associated with it, but you can also have multiple subsystems associated with the same job queue. While a given job queue will be allocated to only one subsystem at a time, the subsystem that does allocate the job queue will control which Job Notification Exit Point definition entry is used, which in turn determines the data queue used by the system to send notification messages to. And most important to our discussion of the exit point: if no subsystem has allocated the job queue, then the Job Notification Exit Point will send notification messages to the default data queue QSYSDTAQ in QSYS (if you have created the data queue), otherwise to no data queue. That is, if you do not start subsystem BVINING/MYSBS and a job is submitted to job queue BVINING/MYJOBQ, while the subsystem is inactive, then the notification message will, if QSYS/QSYSDTAQ exists, be sent there. And if QSYSDTAQ does not exist, then no notification message will be sent at all. The submitted job will be on the job queue, but our SBMDJOBS program won't "see" it and the job will simply remain on the system in held status. If you never submit jobs to job queues that are associated with inactive subsystems, this is not a concern. If you do, however, submit jobs to unallocated job queues, then you may want to watch for next month's article as the use of QSYS/QSYSDTAQ will be one of the "RPG enhancements" mentioned at the start of this article.

The current SBMDJOBS program, where it simply releases jobs from a held status, is not overly exciting. But the ability to have the system notify you when a job is submitted (or started and/or ended), along with the qualified name of the job being processed by the system, can open up some real opportunities for system automation. Hopefully, this article will serve as a basis for some of the automation tools you've long been dreaming of. If nothing else, you now know how to automate the holding (as opposed to the releasing) of any jobs submitted by that associate of yours down the hall.

As usual, if you have any API questions, send them to me at This email address is being protected from spambots. You need JavaScript enabled to view it.. I'll see what I can do about answering your burning questions in future columns.

 

Bruce Vining

Bruce Vining is president and co-founder of Bruce Vining Services, LLC, a firm providing contract programming and consulting services to the System i community. He began his career in 1979 as an IBM Systems Engineer in St. Louis, Missouri, and then transferred to Rochester, Minnesota, in 1985, where he continues to reside. From 1992 until leaving IBM in 2007, Bruce was a member of the System Design Control Group responsible for OS/400 and i5/OS areas such as System APIs, Globalization, and Software Serviceability. He is also the designer of Control Language for Files (CLF).A frequent speaker and writer, Bruce can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it.. 


MC Press books written by Bruce Vining available now on the MC Press Bookstore.

IBM System i APIs at Work IBM System i APIs at Work
Leverage the power of APIs with this definitive resource.
List Price $89.95

Now On Sale

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$0.00 Raised:
$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: