TechTip: Running RPG Programs in Multi-Threaded Jobs

[Guest.Visitor]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
1. If you know the modules will run in multithreaded jobs, all modules called in the job must use Thread(*Serialize). Tom, someone can't predict if a module written today might get called from a Java program coded a year from now (or anywhere in the call stack of that call). And if the Java programmer doesn't know RPG, they sure won't think twice about just calling the RPG program that doesn't use Thread(*Serialize). And if someone does realize that existing RPG modules might need to now use need to Thread(*Serialize), how do they determine which ones? Through lots of research? (Hawkeye/Abstract/PDM scans/etc) That's why I say why not just code it in EVERY RPG module? And you're correct, RPG is not a multi-threaded language. That's why having RPG and Java communicate via data queues is a nice clean hand-off. Thanks for the info. Chris

[R.Daugherty]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Just guessing here, but wouldn't *Serialize invoke overhead to make it thread safe (synchronization at some level), the same synchronize that must be used prudently in Java programs for performance reasons? Or is the assumption that there would no overhead unless it really was in a multi-threaded job? I have to think it affects the compiled object code but maybe does very, very little unless there are other threads to cause blocking. To me it's the same thing as saying why not put synchronize on every Java method just in case. I am writing my first Java thread and sockets code this weekend for sure. I've been saying that for several weekends but this weekend for sure. rd

[R.Daugherty]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Another thought, in defense of RPG. Someone said that other languages like C and Java handle threading better, which is true, but some would walk away from the entirety of this thread thinking that Java and C handle threads and RPG doesn't, which is not the same thing at all. A sort of knee jerk C and Java know something that RPG doesn't would preclude why I am writing thread aware Java code this weekend. If Java is so much more advanced, why am I doing the equivalent of putting *Serialize in it? It's because there are performance costs to synchronize code, so we don't put synchronise around every block of Java code and *Serialize in front of every RPG program. But if you have designed a multi-threaded application, then you do. As I have asked elsewhere, just what would one do with threads in an ERP program except handling your own sockets communications anyway, and is it even desirable from an enterprise perspective? Remember, there is never anything new, just different tradeoffs. There is a reason we did what we did, and do what we do, and the tradeoffs that we chose in RPG. rd

[Guest.Visitor]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
You're welcome Chris. About an RPG program someday being used in a multithreaded job... I think you have to pretty much trust that the developers at the time will do enough research to realize that calling RPG from Java may have ramifications. The java folks won't know what programs to call anyway without research as you suggested using tools, unless they've also done RPG. You can't predict anything. I'm in a situation now where I'm changing code I wrote in 1987. It's still being run today. It needs to output data that will go to the web instead of a standard report. At the time, I could have said, "Gee, that Al Gore discovery called the internet may take off some day. I should code my report program so that it can work with it." None of us are that forward-looking. Heck, we even coded 2-digit years back then. ;}. To code every program in a certain way in the event that it might be used in an unintended way later doesn't make sense. At least to me. Tom.

[Guest.Visitor]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
To code every program in a certain way in the event that it might be used in an unintended way later doesn't make sense. Tom, I just want to know how much overhead Thread(*Serialize) adds. If very little, then I'll just start adding to every program as I maintain/create them. I guess Barbara will have to chime in here, but it looks like this article was written a few years ago. Chris.

[H.Boldt]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Ralph wrote: Just guessing here, but wouldn't *Serialize invoke overhead to make it thread safe (synchronization at some level), the same synchronize that must be used prudently in Java programs for performance reasons? It's important to understand why serialization is necessary in RPG programs. The RPG run-time environment uses static global variables. This is pretty much mandated by various language features, such as files, LR-off return, and RPG's exception model. If you had a program active in two threads without some sort of serialization, you couldn't avoid the possibility that one thread might read the value of some static global while another is half-way through modifying it. THREAD(*SERIALIZE) was added to RPG in recognition of the fact that it would be too difficult to synchronize on every read and update of every global variable used by the run-time environment. On the other hand, if your procedures were totally re-entrant and used no static globals, then thread safety isn't much of an issue, and you wouldn't have to worry about the overhead of synchronizing access to shared resources. If, for example, you were to write a TCP/IP server app in Java or C or Python, you would spawn off a thread to process the request. Each thread can run pretty much independently so long as they do not access any shared resources. Threads may block on files operations, but the synchronization will be handled by the operating system. If your procedures are re-entrant except for a few shared static globals, you can put access to these shared resources in critical sections to synchronize access to them. But that's not the same as putting your whole application into one big critical section. The bottom line is THREAD(*SERIALIZE) was added to RPG allow RPG procedures to be called from Java server apps. It is not by any stretch of the imagination a preferred approach to multi-threading. Multi-threading is simply best left to languages that can support re-entrancy. Cheers! Hans

[R.Daugherty]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Thanks for the additional info, Hans. It's not clear to me how Java would be accessing RPG globals to require synchronization protection of all things RPG from Java to allow RPG procedures to be called from Java server apps. rd

[R.Daugherty]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Chris, I read the article and the advice seems to preclude onerous overhead: Here are some rules of thumb: 1. Code THREAD(*SERIALIZE) in every module that might be run in a multi-threaded job. end quote rd

[R.Daugherty]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Ok, I read the article. You would have to have more than one Java thread calling subprocedures from the same ILE RPG module, and it's to protect system RPG processing code variables from being overwritten by two RPG code executions simultaneously in the same ILE module. Might not even deal with your own variables, which you would have to deal with as in Java with two threads accessing same variable, but in that case you would write the Java code with the variable designed to be accessed simultaneously. In RPG a global variable accessed by two procedures may be assumed (as one would normally expect in RPG) that one procedure will complete before the other is called. On the other hand, the RPG global variable could be designed to be accessed by multiple procedures in the module simultaneously, called from multiple Java threads, just as Java variables are. It seems totally comparable to the same situation in Java if you run multiple threads and don't synchronize mutually accessed data. The same thing will happen. It's a good call out from Barbara, I don't see anything that different going on between the languages. People doing stuff at this level need to design for it. Plus while Java seems superior because a JVM is running multiple threads, my understanding is that IBM and others are desperately trying to create a JVM that can share much better as ILE does because of huge scaling problems with JVM's being launched for every job that don't scale the way ILE does. So sure, if you a big enough hog you might seem superior. As usual, depends on what tradeoffs you're willing to make. rd

[H.Boldt]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Ralph wrote: ... I don't see anything that different going on between the languages. ... The difference is that in a language that supports thread-safety, you can put access to global resources within small critical sections. Thus, two or more threads can run at the same time, blocking only when a critical section is reached. In RPG, if THREAD(*SERIALIZE) is coded, the entire module (and anything it calls too) is in a critical section, and only one invocation of any procedure within the module can be active within a process at one time. Which rather defeats any advantage multi-threading can offer. See the difference yet? Cheers! Hans

[Guest.Visitor]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Tom, Here's an example of a Java programmer wanting to call an existing RPG program and doesn't want to bother the RPG programmers. This is why I am saying just go ahead and code Thread(*Serialize). An ounce of prevention is worth a pound of cure. http://www.mcpressonline.com/mc?128@...SearchMark=2#2 Chris

[H.Boldt]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Chris: I think I have to side with Tom here. Even if the additional overhead on THREAD(*SERIALIZE) isn't much, it would still be an unnecessary burden on modules that are not currently used in a threaded environment. As I mentioned before, since THREAD(*SERIALIZE) causes the entire module to block, it should never be considered a preferred programming technique anyways. Any potential use of RPG modules in a threaded environment should be examined very carefully. Cheers! Hans

[R.Daugherty]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
The point is is that there's nothing magical about Java, you have to go in and put synchronization protection around anything that multiple threads will access, same as we're doing with RPG here. Yes, it may be just a little more practical to run one procedure at a time like reading a data queue and blocking to finish processing before reading another, instead of having a system that reads off the data queue as fast as presented and spawning a thread for each read, where we could go into the code and figure out what all may be used simultaneously and put synchronization opcodes around everything necessary. Wouldn't that be good ERP programming? I think there's a reason OS/400 architecture is a little more practical than that. Speaking of which, *Serialize appears to me to be the equivalent of a virtual data queue for calls. I can't imagine any RPG programmer that would expect that multiple threads calling procedures in a service program in the job would be allowed to come in and start executing before the first call finished under any condition. I think when Java invokes ILE, there ought to be something in ILE that implements a *Serialize wrapper unless a job directive overrides it for those that would really want to do that. rd

[Guest.Visitor]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Hi Chris, I read that thread. Not knowing the Java progammer's relationship with the shop such as being a permanent employee, a temp, etc., what I'm about to say may not apply. Any programmer should be hashing this out within the shop to come up with a standard instead of programming without bothering the RPG programmers. You can do whatever you want in your shop. Like you, I'd be interested in knowing the potential problems, if any, and make sure that you're not using a pound of prevention for an ounce of cure. Let me know if you run into any problems with putting Thread(*Serialize) in each program. It may be worth looking at. Thanks Chris, Tom.

[H.Boldt]

TechTip: Running RPG Programs in Multi-Threaded Jobs

** This thread discusses the article: TechTip: Running RPG Programs in Multi-Threaded Jobs **
Ralph wrote: The point is is that there's nothing magical about Java, you have to go in and put synchronization protection around anything that multiple threads will access, same as we're doing with RPG here. ... You're still not getting it, Ralph. Let's look at it another way. Let's say your threaded code does not access ANY shared static storage. Your code written in Java or C would need NO synchronization. The code is already fully re-entrant. On the other hand, even if you do not code any shared static variables in your RPG module, you would still need THREAD(*SERIALIZE) because of RPG's run-time environment. OK, now add one shared static variable to your Java or C code. Only accesses to that shared variable need to be put into a critical section. The code can run safely in multiple simultaneous threads, blocking ONLY when the critical section is reached. Meanwhile, your RPG code blocks on every entry into the module. The difference is the possibility of greater throughput in the multi-threaded environment. With several threads running at once, one thread can still be running while others are blocked on I/O requests. But with RPG code, while one thread is blocked on an I/O request, others are blocked before the code is even started! Thus, the threads implemented in Java or C (or any other thread-safe language) can often complete faster. Ralph also wrote: I think when Java invokes ILE, there ought to be something in ILE that implements a *Serialize wrapper unless a job directive overrides it for those that would really want to do that. One problem with this suggestion is that you also have to consider an RPG module invoked from RPG modules called from Java. There's a reason Barbara suggests putting THREAD(*SERIALIZE) on ALL modules that can be invoked from Java. That includes modules called by other RPG modules called from Java. Consider three modules A, B, and C. A and B are called directly from Java, and these call C. C needs THREAD(*SERIALIZE) as much as A and B since A and B may well be active at the same time and may try to call C at the same time. The bottom line is this: Multi-threading is not an easy concept to grasp fully. THREAD(*SERIALIZE) is an easy way for RPG programmers to stay out of trouble, but is far from optimal. Cheers! Hans