TechTip: Running RPG Programs in Multi-Threaded Jobs

[H.Boldt]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
ShanePoad asked: why bother creating threads if you can't actually de-serialize the work load? Good question. Let's back up for a moment. Threads are typically used when handling requests in a TCP/IP server application. If a request can be handled quickly, then an answer can be returned immediately. But if there might be some latency involved in putting together an answer (such as accessing disk storage), the preferred approach is to spawn off a thread. (You could start a new process too, but that has additional overhead that's often not necessary. Basically, you don't want to introduce any unnecessary delay in processing the request.) In the past, it was usually only the systems programmer who needed to worry about this. However, these days you have HTTP servers, which may well call out to application code. Now back to your question, which I'll rephrase slightly: Why use RPG code in a multi-threaded environment if it's going to block requests at the RPG procedure level? That is, allow only one invocation of any procedure to be active at one time, thus totally eliminating any advantage that multi-threading might give you? Hmmm... Hmmm... Hmmm... Here are two possible answers: 1) You don't want all your nice existing RPG code to go to waste. 2) You don't want to learn Java or C. (Or any of dozens of other languages that handle threading better.) Sorry, I can't think of any other reasons why you would use RPG in a threaded environment. But I'm sure others with a better knowledge of the language can help. Cheers! Hans

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
I have a few thoughts / questions... 1. I think one of the points (#1) should be made clearer: Code Thread(*Serialize) in every module that could be in the call stack in a multi-threaded job. IE: If the call stack is 10 deep, all 10 need this H-spec. 2. Why wouldn't you just code Thread(*Serialize) in every single module then? 3. Are OPM CL programs already thread safe? CLLE programs? 4. Does coding Thread(*Serialize) still create just object? I swear I read a while back that this creates 2 objects, 1 hidden. Chris

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Chris, I learned about this a few years back so my memory may be spotty. A company I was working at had me write some procedures that were going to be called by web applications. I think the web apps were written in Java but I'm not so sure. I haven't worked with threads since. My understanding is that: 1. If you know the modules will run in multithreaded jobs, all modules called in the job must use Thread(*Serialize). 2. Yes. Code all modules that will run in multithreaded jobs that way. 3. No OPM program is threadsafe. I don't know about CLLE. 4. I remember hearing or reading the same thing. the second object somehow enforces the serialization. I also understand that RPG programs, both OPM and ILE do not spawn multithreaded jobs. As a general rule, I'd stay away from running any RPG procedure in multithreaded jobs. This is a quote from the ILE RPG manual, Ch 13: Control Specifcations: "Normally, running an application in multiple threads can improve the performance of the application. In the case of ILE RPG, this is not true in general. In fact, the performance of a multithreaded application could be worse than that of a single-thread version when the thread-safety is achieved by serialization of the procedures at the module level. Running ILE RPG procedures in a multithreaded environment is only recommended when required by other aspects of the application (for example, when writing a Domino exit program or when calling a short-running RPG procedure from Java). For long-running RPG programs called from Java, we recommend using a separate process for the RPG program." Tom.

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
1. If you know the modules will run in multithreaded jobs, all modules called in the job must use Thread(*Serialize). Tom, someone can't predict if a module written today might get called from a Java program coded a year from now (or anywhere in the call stack of that call). And if the Java programmer doesn't know RPG, they sure won't think twice about just calling the RPG program that doesn't use Thread(*Serialize). And if someone does realize that existing RPG modules might need to now use need to Thread(*Serialize), how do they determine which ones? Through lots of research? (Hawkeye/Abstract/PDM scans/etc) That's why I say why not just code it in EVERY RPG module? And you're correct, RPG is not a multi-threaded language. That's why having RPG and Java communicate via data queues is a nice clean hand-off. Thanks for the info. Chris

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
You're welcome Chris. About an RPG program someday being used in a multithreaded job... I think you have to pretty much trust that the developers at the time will do enough research to realize that calling RPG from Java may have ramifications. The java folks won't know what programs to call anyway without research as you suggested using tools, unless they've also done RPG. You can't predict anything. I'm in a situation now where I'm changing code I wrote in 1987. It's still being run today. It needs to output data that will go to the web instead of a standard report. At the time, I could have said, "Gee, that Al Gore discovery called the internet may take off some day. I should code my report program so that it can work with it." None of us are that forward-looking. Heck, we even coded 2-digit years back then. ;}. To code every program in a certain way in the event that it might be used in an unintended way later doesn't make sense. At least to me. Tom.

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
To code every program in a certain way in the event that it might be used in an unintended way later doesn't make sense. Tom, I just want to know how much overhead Thread(*Serialize) adds. If very little, then I'll just start adding to every program as I maintain/create them. I guess Barbara will have to chime in here, but it looks like this article was written a few years ago. Chris.

[H.Boldt]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Ralph wrote: Just guessing here, but wouldn't *Serialize invoke overhead to make it thread safe (synchronization at some level), the same synchronize that must be used prudently in Java programs for performance reasons? It's important to understand why serialization is necessary in RPG programs. The RPG run-time environment uses static global variables. This is pretty much mandated by various language features, such as files, LR-off return, and RPG's exception model. If you had a program active in two threads without some sort of serialization, you couldn't avoid the possibility that one thread might read the value of some static global while another is half-way through modifying it. THREAD(*SERIALIZE) was added to RPG in recognition of the fact that it would be too difficult to synchronize on every read and update of every global variable used by the run-time environment. On the other hand, if your procedures were totally re-entrant and used no static globals, then thread safety isn't much of an issue, and you wouldn't have to worry about the overhead of synchronizing access to shared resources. If, for example, you were to write a TCP/IP server app in Java or C or Python, you would spawn off a thread to process the request. Each thread can run pretty much independently so long as they do not access any shared resources. Threads may block on files operations, but the synchronization will be handled by the operating system. If your procedures are re-entrant except for a few shared static globals, you can put access to these shared resources in critical sections to synchronize access to them. But that's not the same as putting your whole application into one big critical section. The bottom line is THREAD(*SERIALIZE) was added to RPG allow RPG procedures to be called from Java server apps. It is not by any stretch of the imagination a preferred approach to multi-threading. Multi-threading is simply best left to languages that can support re-entrancy. Cheers! Hans

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Tom, Here's an example of a Java programmer wanting to call an existing RPG program and doesn't want to bother the RPG programmers. This is why I am saying just go ahead and code Thread(*Serialize). An ounce of prevention is worth a pound of cure. http://www.mcpressonline.com/mc?128@...SearchMark=2#2 Chris

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Hi Chris, I read that thread. Not knowing the Java progammer's relationship with the shop such as being a permanent employee, a temp, etc., what I'm about to say may not apply. Any programmer should be hashing this out within the shop to come up with a standard instead of programming without bothering the RPG programmers. You can do whatever you want in your shop. Like you, I'd be interested in knowing the potential problems, if any, and make sure that you're not using a pound of prevention for an ounce of cure. Let me know if you run into any problems with putting Thread(*Serialize) in each program. It may be worth looking at. Thanks Chris, Tom.

[Guest.Visitor]

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
It seems to me that people here are choosing performance over data integrity. Which option do you think the CEO, CIO, CFO, stock holders, BOD, etc would prefer? I ran some tests. I encourge others to do the same. I ran several tests over a program with and without Thread(*Serialize), 5,000,000 iterations over several tests. The performance is nearly the same, sometimes the Thread(*Serialize) actually wins. I imagine that database I/O will the bottleneck in most RPG modules, not Thread(*Serialize). Sure wish Barbara would add some input... Chris