Debugging Java from RPG JNI

[Guest.Visitor]

Try using Start Service Job (STRSRVJOB) command to service the job and then use STRDBG, similar to debuging a batch job. Why doesn't STRDBG work directly? I believe there is limitation with the setup you have outlined. If/when I find where that is documented then I'll post it. Tosh.

[Guest.Visitor]

Thanks Tosh, I just tried a simpler execution example and the program works! I think the error is somewhere in this complex SQL statement that I'm passing to the program. I'll try the STRSRVJOB and see what that does. Thanks a heap and Happy New Year! Best, Cliff PS. DOes anyone know how to get the STDOU stream from a Java program called through RPG JNI? I see some output flash very quickly to the screen when an error occurrs but I don't know how to trap that info.

[J.Pluta]

Never tried with JNI, but when you're using QSH you just override STDOUT to a database file: CRTPF QTEMP/STDOUT RCDLEN(256) OVRDBF STDOUT STDOUT Maybe this will work for JNI, too? Joe

[Guest.Visitor]

I just learned STRSRVJOB is not an option here because the Java does not run as a BCI boj. It runs within the RPG job natively hence the "N" in JNI. The issue here is kind of a mixed one because I'm trying to debug a program created with Eclipse (the open source version of the WSDc) that runs iSeries specific logic. In particular there is a JDBC snippet that I'm interested in analizing. It is using the native JDBC driver so I cannot run this piece of the code from my PC. I saw an option in eclipse for remote debugging but I'm not sure how to set my iSeries up so the remote debugger can attach to it. (I believe it connects through a port the default being 8000.) I looked on the IBM info center for documentation on remote JNI debugging and found nothing. They only include info regarding debugging command-line Java programs. I couldn't find an option in Eclipse to include debug info on the compile. This would have allowed me to do a command line debug just to sort things out. As you can see I having a hard time here. Help, Cliff

[Guest.Visitor]

I tried the OVRDBF command earlier without thinking to create the STDOUT PF first. It didn't work but maybe because STDOUT wasn't created. I'll try your suggestion if I can ever get the bug to re-occur. Thanks a heap! Cliff

[rdean400@yahoo.com]

If you have the WebSphere Development Tools, then you have access to the IBM Distributed Debugger. It will step into Java code from RPG. I've done it, and I seem to remember there might be a trick to doing it (maybe .java file has to be present with the .class file?). Anyway, it works. Only caveat is that I wouldn't call it a speed demon, by any stretch of the imagination. --Robert

[Guest.Visitor]

I'm not sure what you mean by WebSphere Development Tools. I have the... (looking in desk drawer...) wait a minute! I DO have the WebSphere Development Tools disks. Are you referring to the disks that ship w/ V5R1 upgrade? I was treating this as outdated technology and have been reluctant to install it. I don't know alot about VAJ and I've grown pretty comfortable w/ Eclipse. Truth is I didn't want to install all this stuff if I didn't have to. However, if what your saying is true I might just bite the bullet and install it anyway. I'm assuming that I'll have to debug the RPG from Code/400 right? Thanks a bunch everybody! Keep the replies comming, Cliff

[Guest.Visitor]

I just installed WDSc 4.0 on my PC and downloaded Eclipse 2.0.2 but when I started the eclipse.exe I cant find the WDSC development tools and wont able to start WSDc. I cant even add to plug-ins. Is there an easy way to do this. Thanks a lot. Sky

[Guest.Visitor]

Skywalker, I'm assuming you installed Eclipse 2.0.2 after WDSc. This is probably why you don't see the WDSc dev tools. WDSc already includes Eclipse and installing Eclipse overtop of a WDSc install is probably wiping out the WDSc stuff. Un-install everything and start over from the beginning. Just install WDSc and forget about downloading eclipse because it should already install with WDSc. Cliff PS. Could you burn and mail me a copy of your WDSc CDs?

[rdean400@yahoo.com]

5 CDs and a DVD. Should contain CODE/400, Toolbox for Java, VARPG, VAJ, and Webfacing tools along with the IBM Distributed Debugger. You don't have to use VAJ to use the debugger. If you've grown comfortable with Eclipse, stick with Eclipse (I think it's a better IDE anyway). --Robert

[Guest.Visitor]

I just tried the override again and specified it at the job scope. I then ran my program seeing the error info flash before me. I checked the STDERR and STDOUT files I created in QTEMP and both were empty. Also it doesn't look like RPG is catching and reporting Java exceptions. Cliff

[Guest.Visitor]

Robert, I'll try the WDT400 stuff. Thanks for the tip. Cliff

[Guest.Visitor]

I'm using the new RPG JNI support and executing the Java app from RPG for optimization purposes. I suppose I could strip out the raw raw Java invocation code for debugging, I was just hoping I wouldn't have to do that. Cliff

[Guest.Visitor]

Try this to override to database file: pgm dcl &java *char 256 chkobj obj(qtemp/stdout) objtype(*file) monmsg cpf9801 exec(do) crtpf qtemp/stdout rcdlen(96) enddo clrpfm qtemp/stdout ovrdbf file(stdout) tofile(qtemp/stdout) + mbr(stdout) ovrscope(*calllvl) chgvar &java ('java YourClass') qsh cmd(&JAVA) dltovr file(*ALL) lvl(*) endpgm And this to override to a print file: pgm dcl &java *char 256 ovrdbf file(stdout) tofile(qsysprt) + ovrscope(*calllvl) chgvar &java ('java YourClass') qsh cmd(&JAVA) dltovr file(*ALL) lvl(*) endpgm This works only when executing your java class via QSH not the JAVA CL command.

[Guest.Visitor]

Creating a debug class like for example (very rough draft); <hr>My disclaimer: This was done very quickly so I do not claim excellence in this example. It is just an example. <hr> <pre> <hr>Debug class<hr> import java.util.Vector; public class Debug{ static Vector currentMsgs = new Vector(); static int currentIndex = -1; public static void debug(String msg) { currentMsgs.add(msg); } public static String getNextMessage() { if(currentIndex==currentMsgs.size()-1) currentIndex = -1; else currentIndex++; return (String)currentMsgs.get(currentIndex); } public static int getCount() { return currentMsgs.size(); } public static void clear() { currentMsgs.clear(); currentIndex = -1; } } </pre> You could output statements like: <hr>PrintJVM2 class<hr> <pre> import java.io.*; class PrintJVM2{ static public void listJVM() { File[] f = File.listRoots(); Debug.debug("
Listing File System Roots..."); for(int i=0; i<f.length; i++) Debug.debug(f[i].getPath()); Debug.debug("
Listing File Separators..."); Debug.debug("File separator as String: " + File.separator); Debug.debug("File separator as Char: " + File.separatorChar); Debug.debug("Path separator as String: " + File.pathSeparator); Debug.debug("Path separator as Char: " + File.pathSeparatorChar); Debug.debug("
Listing JVM properties..."); Debug.debug("Java version: " + System.getProperty("java.version")); Debug.debug("Java vendor: " + System.getProperty("java.vendor")); Debug.debug("Java vendor URL: " + System.getProperty("java.vendor.url")); Debug.debug("Java installation directory: " + System.getProperty("java.home")); Debug.debug("Java Virtual Machine specification version: " + System.getProperty("java.vm.specification.version")); Debug.debug("Java Virtual Machine specification vendor: " + System.getProperty("java.vm.specification.vendor")); Debug.debug("Java Virtual Machine specification name: " + System.getProperty("java.vm.specification.name")); Debug.debug("Java Virtual Machine implementation version: " + System.getProperty("java.vm.version")); Debug.debug("Java Virtual Machine implementation vendor: " + System.getProperty("java.vm.name")); Debug.debug("Java Runtime Environment specification version: " + System.getProperty("java.specification.version")); Debug.debug("Java Runtime Environment specification name: " + System.getProperty("java.specification.vendor")); Debug.debug("Java class format version number: " + System.getProperty("java.class.version")); Debug.debug("Java class path: " + System.getProperty("java.class.path")); Debug.debug("List of paths to search when loading libraries: " + System.getProperty("java.library.path")); Debug.debug("Name of JIT compiler to use: " + System.getProperty("java.compiler")); Debug.debug("Path of extension directory or directories: " + System.getProperty("java.ext.dirs")); Debug.debug("Default temp file path: " + System.getProperty("java.io.tmpdir")); Debug.debug("Operating system name: " + System.getProperty("os.name")); Debug.debug("Operating system architecture: " + System.getProperty("os.arch")); Debug.debug("Operating system version: " + System.getProperty("os.version")); Debug.debug("File separator ("/" on UNIX): " + System.getProperty("file.separator")); Debug.debug("Path separator (":" on UNIX): " + System.getProperty("path.separator")); try{ Debug.debug("Line separator ("\n" on UNIX): " + Integer.decode(System.getProperty("line.separator"))); } catch(NumberFormatException nfe){}; Debug.debug("User's account name: " + System.getProperty("user.name")); Debug.debug("User's home directory: " + System.getProperty("user.home")); Debug.debug("User's current working directory: " + System.getProperty("user.dir")); } } </pre> <hr>TestJNI pgm<hr> And then report then in the RPG program like: <pre> h Debug DftActGrp(*No) ActGrp(*new) Bnddir('QC2LE') d printf pr extproc('printf') d * value options(*string) d * value options(*string:*nopass) d * value options(*string:*nopass) d * value options(*string:*nopass) d listJVM pr extproc(*java d :'PrintJVM2' d :'listJVM') d static d getCount pr 10i 0 extproc(*java d :'Debug' d :'getCount') d static d getNextMessage... d pr o extproc(*java d :'Debug' d :'getNextMessage') d class(*java:'java.lang.String') d static d getBytes pr 256a extproc(*java d :'java.lang.String' d :'getBytes') varying d newline c x'25' d msg s o class(*java:'java.lang.String') d txt s 256a d max s 10i 0 d count s 10i 0 /free listJVM(); max = getCount(); for count = 1 to max; msg = getNextMessage(); txt = getBytes(msg); printf(newline+'%s':%trim(txt)); endfor; *inlr = *on; /end-free </pre> This is very rough but you could trigger the message retrieval by converting the java exception to an ILE exception - one possibility. You could also code the Debug class to output to System.out or to the Vector based on -D (setting a JVM property for Debug).

[Guest.Visitor]

I like that idea! I could wrapper that concept in a generic ILE module for retrieving JVM diagnostics and such. I'm just baffeled that there isn't anything included in the new ILE support to facilitate this type of stuff. Thanks Jim. Cliff

[Guest.Visitor]

Clifton, I havent tried this yet, but maybe the following would work: When you use JNI to call Java from RPG, the as400 will start up a jvm if one is not already running. Normally the RPG JNI api's handle the jvm startup automatically for you. In this case you need to control the startup of the jvm so you can add on the extra remote debugging parms (so the jvm is listening for a remote debugger connection...). You can use the JNI_CreateJavaVM command to startup the jvm with the parms you need. Some sort of helpful sample code exists in chapter 11 of the "WebSphere Development Studio: ILE RPG Programmer's Guide" SC09-2507-03. The trick would be to see if you can duplicate the effect of something like (java -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,s erver=y,suspend=n,address=29000) to lauch the jvm. I dont care for the eclipse remote debugging, so I use bugseeker. That is just a personal preference, though. Alex

[Guest.Visitor]

Alex, I appreciate your feedback. I've been struggling with the JNI stuff trying to figure out how to set the JVM options using the invocation API. I think i'm getting close. So far I've found out how to set the version for the invoke API. I'm still not quite sure if I should use the JavaVM or the JDK1_1 argument structures. Also I'm not quite sure if I can set different options upon attaching to a JVM. That's a non-issue though since I plan on setting them on initialization. What I found out is that on V5R1 the JVM ends if you start it in debug, end your debug, and then recompile. I'm not sure the exact sequence of steps that causes it to end. I thought it would always hang around until the job ended. I'm thinking this could cause problems on V5R2 where you can only start one JVM per process. I'm not sure if that's just one total or just a concurrency restriction. Thanks a heap, Cliff

[Guest.Visitor]

Clifton, Do you have access to a V5R2 system to try starting a jvm, ending it, and starting another? Surely it is just a limit to one JVM at a time per process. Right now I only have access to a v5r1 box, so I cant test that here. I guess I could just sign out and back in again... Please post a sample of the invocation code once you figure out how to set the initargs. I am about a month away from needing to do the same thing myself. Another concern on my end is the THREAD(*SERIALIZE) requirement for any ILE RPG code that might be run in any multithreaded environment. I am not too worried about the case of RPG calling Java since in my case that wont be in a multithreaded environment. However, I had thought to use JNI to call RPG from Java --- which definitely would be in a multithreaded environment. Since THREAD(*SERIALIZE) forces serialized access to the RPG module, it will be a huge potential bottleneck. I am running a bunch of tests on the other options. If you have any suggestions I would love to hear them. Alex

[B.Morris]

Alex Garrison wrote: > > Clifton, > > Do you have access to a V5R2 system to try starting a jvm, ending it, and starting another? Surely it is just a limit to one JVM at a time per process. Right now I only have access to a v5r1 box, so I cant test that here. I guess I could just sign out and back in again... > Alex, in V5R2 you can't start another JVM after you end one. This is documented in the V5R2 What's New section for Java. Apparently (and I don't know or understand the details of this) destroying the JVM doesn't actually destroy it, so starting another one would cause two to exist.

[J.Pluta]

"Prior to V5R2, it would be possible for the JNI_GetCreatedJavaVMs function to return more than one Java virtual machine in its list of Java virtual machines. In V5R2, the JNI_GetCreatedJavaVMs returns at most one Java virtual machine. Prior to V5R2, the JNI_CreateJavaVM function could be called repeatedly within a single process, and with each successful invocation a separate and distinct Java virtual machine was created. In V5R2, the JNI_CreateJavaVM function returns an error code (JNI_EEXIST -5 VM Already Created as defined in jni.h) whenever a Java virtual machine has already been created in the current process. Prior to V5R2, the DestroyJavaVM function destroyed the Java virtual machine and returned its resources to the process. In V5R2, the DestroyJavaVM function returns an error code (JNI_ERR -1 error defined in jni.h)." From my reading of this, Barbara, it used to be that multiple calls to JNI_CreateJavaVM actually created multiple VMs, and that you could then select between them. Now, you can only have one. If you create one, it is "assigned" to the job. Trying to create one when one exists returns an "already created" error, and when you ask for a list of active VMs, you will only get one entry in the list. Trying to end the VM returns an error. So basically as long as a JVM is up, the create API won't let you create a new one. That makes sense. When a JVM crashes or ends for any reason, though, there should be some way around this. There should be some way to notify the job that the JVM associated with this job is gone kerplunkt, and that you can start a new one. Or am I reading this wrong? Because otherwise, what if a utility used, say, QShell to fire up a JVM and then, as is normal, did a System.exit(0) to end the utility? Would the JVM then end, and the job no longer be able to execute that command? Or would the end be a "pseudo-end", and the JVM actually be suspended and just wait around for another "pseudo-start"? That's a possibility. There are issues with static initializers that could come into play, but that's an application design issue. Joe

[Guest.Visitor]

Joe, If a utility uses QSH to fire up a JVM then that JVM would be associated with the BCI job that QSH is running under. After System.exit(0) was invoked from the method the entire job would end unless the Java was invoked from a shell script that had further processing to do (more script statements). The calling job would always be able to start a new QSH job to run more Java programs. This would be the ugly workaround to the situation I described. I'm still not sure of the details of the situation though. I'm not sure if on V5R2 you can't have another JVM started ever or if the restriction is just no concurrent JVMs running. I agree that there should be some workaround for this. Maybe the JVM will be modified to stay around forever even after debug sessions. Though I don't think that would be practical. Cliff

[Guest.Visitor]

Alex, I'll make sure I post my solution. I'm 75% there. Cliff

[B.Morris]

Joe Pluta wrote: > ... > So basically as long as a JVM is up, the create API won't let you create a new one. That makes sense. When a JVM crashes or ends for any reason, though, there should be some way around this. There should be some way to notify the job that the JVM associated with this job is gone kerplunkt, and that you can start a new one. Or am I reading this wrong? > ... Joe, my understanding ("understanding" is too strong a word) is that when the JVM ends (crashes, is destroyed, whatever), it doesn't really end, it just becomes unavailable for anything new. I don't know why it's not possible to completely destroy it.

[Guest.Visitor]

I'm confused. On V5R2, if the JVM crashes does this mean you would have to signoff or start a new job before you could run any more Java code? If so, why has this decision been made? It sounds like a short coming. Also according to your reply the JVM never unloads or stops. Instead it becomes unavailable for anything new. I'm assuming that means that the JVM stays resident in memory while some internal flag is set indicating it is unavailable. I also assume all JVM related threads are stopped in the event of failure so in essence nothing is actually "running". Are you saying that there is no way to completely unload the JVM from memory?

[R.Daugherty]

This is a really intelligent memo from a Sun engineer on the problem with Java. It applies to Solaris, not OS/400, and I understand that this post may be construed as trolling. If so, please delete, no problem. I found the memo to be an excellent summary. rd from FooCompany.com: Sun Microsystems The Java Problem The Java Problem Author: Julian S. Taylor Reviewed by: Steve Talley, Mark Carlson, Henry Knapp, Willy (Waikwan) Hui, Eugene Krivopaltsev, Peter Madany, Michael Boucher Executive Summary While the Java language provides many advantages over C and C++, its implementation on Solaris presents barriers to the delivery of reliable applications. These barriers prevent general acceptance of Java for production software within Sun. A review of the problem indicates that these issues are not inherent to Java but instead represent implementation oversights and inconsistencies common to projects which do not communicate effectively with partners and users. Within Sun, the institutional mechanism for promoting this sort of communication between partners is the System Architecture Council codified in the Software Development Framework (SDF). We propose that the process of releasing our Java implementation will benefit from conformance with the SDF. Introduction This document details the difficulties that keep our Solaris Java implementation from being practical for the development of common software applications. It represents a consensus of several senior engineers within Sun Microsystems. We believe that our Java implementation is inappropriate for a large number of categories of software application. We do not believe these flaws are inherent in the Java platform but that they relate to difficulties in our Solaris implementation. We all agree that the Java language offers many advantages over the alternatives. We would generally prefer to deploy our applications in Java but the implementation provided for Solaris is inadequate to the task of producing supportable and reliable products. Our experience in filing bugs against Java has been to see them rapidly closed as "will not fix". 22% of accepted non-duplicate bugs against base Java are closed in this way as opposed to 7% for C++. Key examples include: 4246106 Large virtual memory consumption of JVM 4374713 Anonymous inner classes have incompatible serialization 4380663 Multiple bottlenecks in the JVM 4407856 RMI secure transport provider doesn't timeout SSL sessions 4460368 For jdk1.4, JTable.setCellSelectionEnabled() does not work 4460382 For Jdk1.4, the table editors for JTable do not work. 4433962 JDK1.3 HotSpot JVM crashes Sun Management Center Console 4463644 Calculation of JTable's height is different for jdk1.2 and jdk1.4 4475676 [under jdk1.3.1, new JFrame launch causes jumping] In personal conversations with Java engineers and managers, it appears that Solaris is not a priority and the resource issues are not viewed as serious. Attempts to discuss this have not been productive and the message we hear routinely from Java engineering is that new features are key and improvements to the foundation are secondary. This is mentioned only to make it clear that other avenues for change have been explored but without success. Here we seek to briefly present the problem and recommend a solution. Defining the Java Problem These are the problems we have observed which we believe indicate the need for an improved implementation and a modified approach. 1. The support model seems flawed Since Java is not a self-contained binary, every Java program depends fundamentally upon the installed Java Runtime Environment (JRE). If that JRE is broken, correction and relief is required. This sort of relief needs to happen in a timely manner and needs to fix only the problem without the likelihood of introducing additional bugs. Java Software does not provide such relief. Java packages are released (re-released) every four or five months, introducing bug fixes and new features and new bugs with each release. These releases are upgrading packages which remove all trace of the prior installed packages and cannot be down-graded in the event of an error. The standard release taxonomy used by the Architecture Review Committees (ARCs) was developed for use by Solaris and our other mission-critical software products to help solve these and many other problems. It is impractical for a project based on Java to correct bugs in the Java implementation. Java Software corrects bugs only by releasing an entire new version. For that reason, projects seek to deliver their own copy of Java so they can maintain it without fear of a future upgrade. Outside vendors, such as TogetherJ, specify a particular release of Java for their product. The customer must locate that release and install it. If a future product seeks to use a different version, that version has to be installed side-by-side with the prior version or TogetherJ may no longer function. The ARCs commonly see project submittals requesting permission to ship their own version of Java. The ARCs have been routinely forbidding projects to do this even though they are aware of specific cases wherein interfaces or their underlying behaviors have changed incompatibly across minor releases. The threat of losing the ability to directly support such a substantial part of their product has inhibited projects from choosing Java as their implementation language and caused widely-discussed problems for customers of projects that have used Java. Consider that the Java language supports rapid development, simple testing and access to a wide variety of platforms. Why are the shelves at CompUSA (a Linux friendly store) not crammed with W32/Linux/etc offerings written in Java? As it stands client-side Java remains primarily a web language partly because the Netscape platform runs Java 1.1.5 and has not changed for years. It is buggy but very stable. This indicates that Java must strictly enforce backward compatibility across minor releases and must adhere to Sun release taxonomy for the identification of releases. Further, existing releases must support some sort of remedy akin to a patch so that existing installations can be corrected through existing methods. 2. The JRE is very large. The JRE is significantly larger than comparable runtime environments when considering resident set size (memory dedicated to this specific program). It has been seen to grow to as much as 900M. This has a drastic effect on both performance and resource usage. It also means that multiple JREs present critical resource constraints on the servers for such thin-client systems as SunRays. Typical resident set requirements for Java2 programs include: Hello World 9M SMC Server 38M SLVM GUI 60M Component Manager 160M TogetherJ 300 - 900M The largest program in that list is TogetherJ. From the standpoint of resource requirements, TogetherJ does much of what Rational Rose does but Rational Rose appears to function in less than 250M. Startup time is effected as well. For example, on an Ultra10 TogetherJ requires 5 minutes to load and start. SMC, Sun's flagship system admin console, takes between one and two minutes to reach the point that it can be used. Some of this problem appears to relate to the JRE. We do not have the time or money to conduct a serious side-by-side study of Java vs other languages and are therefore calling upon our personal experiences with Java development. The fact that these experiences are hard to quantify forces us to try to support the validity of this concern through existing research. A study performed by an outside team appears to indicate a rough parity in performance between Java and a common implementation of another OO language called Python (see IEEE Computing, October 2000, "An Empirical Comparison of Seven Programming Languages" by Lutz Prechelt of the University of Karlsruhe). Both platforms are Object Oriented, support web applications, serialization, internet connections and native interfaces. The key difference is that Python is a scripting language. This means there is no compilation to byte code so the Python runtime environment has to do two things in addition to what the Java runtime environment does. It has to perform syntax checks and it must parse the ascii text provided by the programmer. Both of those tasks are performed at compile time by Java and so that capability does not have to be in the JRE. Given this data, it appears that the JRE can actually be simpler than the Python RE since Java does at least some of this work at compile time. The example above of "Hello World" is a good method for getting an idea of the minimum support code required at runtime. This support code includes garbage collector, byte code interpreter, exception processor and the like. Hello World written in Java2 requires 9M for this most basic support infrastructure. By comparison, this is slightly larger than automountd on Solaris8. The Python runtime required to execute Hello World is roughly 1.6M. Further examples of what is possible include the compiling OO languages Eiffel and Sather which fit their garbage collector, exception processor and other infrastructure into roughly 400K of resident set. While the Java VM (as demonstrated above) grows rapidly as more complex code is executed, the Python VM grows quite slowly. Indeed, an inventory control program written entirely in Python having a SQL database, a curses UI, and network connectivity requires only 1.7M of resident set. This seems to indicate that the resident set requirements of the JRE could be reduced by at least 80%. Imagine what happens if our current implementation of Java were ubiquitous and all 150 users on a SunRay server were running one and only one Java program equivalent to Component Manager above. The twenty-four gigabytes of RAM the server would have to supply exclusively to these users is well beyond the typical configuration. RAM is cheap but performance is what we sell, all customers on that SunRay server would see significant performance degradation even with the maximum amount of RAM installed as all other processes were forced to reside on swap. The resident set size required by the JRE makes it impractical to run Java in an initial Solaris install environment. It is impractical to run it as a non-terminating daemon. A Java daemon could be started from inetd run long enough to do its job and then quit but the rpc protocol required to pass the socket port to the daemon is very complex and not Java-friendly. Java applications cannot be executed at boot time since the loading of the VM introduces an unacceptable performance degradation. If the Java runtime were as small as that of Python, it is likely that the Java daemon would become popular and could provide basic services to applications written in any number of languages. 3. Extensions do not support modularity. As new extensions are introduced, they are released separately under their own names and distributed generally. Each one may go through several revisions as separate modules. At some point, they are then folded into base Java, tying base Java's version to the versions of dozens of smaller yet distinct functionalities. These functionalities are then restricted to a draconian backward-compatibility rule since once folded in, they are no longer selectable modules. Examples include modules that used to be called Swing, RTI, IDL, JSSE and JAAS. These are all good things that should be part of Java. Our concern is that these are not separable modules which can evolve as requirements change. The Java system for evolving the interface (deprecation) does not serve production software very well. Once the interface disappears, the product just breaks. If the Java base were simpler and the more advanced features (those most likely to be deprecated) were delivered as versioned modules, it would be possible for a commercial product to retain it's older modules on the system and survive a large number of Java upgrades. Production quality programs written in Java, like TogetherJ, indicate a specific Java version which must be installed before the program is run. If another program is installed, requiring a higher Java version, the user may be forced to decide which program stays and which goes away. Alternatively, the other Java version could be installed to a different base directory but this requires considerable sophistication on the part of the user, complicates administration and violates the ARC big rule that common software must be shared. 4. It is not backward-compatible across minor releases. Among the various incompatibilities across minor releases are: a) In JDK 1.1 Class.fields() returns only public variables. In 1.2, protected and private variables are returned. b) Swing table sizing calculation changed from Java 1.3 to 1.4. c) Swing JFrame launch behavior changed significantly from Java 1.2.2 to Java 1.3.1. Each of these examples is simple, but they demonstrate the general problem that people cannot program for a particular release of Java and expect that their programs will continue to run. This is a serious problem now, but has the potential to become a show-stopper as technology such as auto-update advances. What is perhaps more important is that the perception of Java as an unstable platform is widespread. This perception is restated with every Java-based project to come to ARC. Within Sun, Java is not viewed as a satisfactory language for the construction of commercial applications. This perception and the record require addressing. The Java Problem is Recognized Internally That our Java implementation is perceived as inappropriate for many uses is supported by internal documents and policies. For example: 1. SOESC AI - 092501.2 Java Dependencies for Deployment In this document provided to SOESC, John Perry describes the concerns regarding the Solaris "JVM dependencies for deployment". Following is an excerpt: ------- - Large footprint of applications when run on Solaris. A simple application ("hello world" type) has a total footprint of 35-40 megs on Solaris 9 (build 48, using Java 1.4 build 82) on both Intel and Sparc machines. Sparc machines, by far, have a much higher resident footprint then Intel machines (~30 megs, compared to ~11 megs). The same program run on a Windows machine has a footprint of ~5 megs, resident footprint being ~3.5 megs - Slow start up times prevents Java applications from being started while Solaris is booting up and during mini-root time. This requires applications which are written in Java to have some kind of mechanism to start-up after the OS has been fully started. - Instability of Native code (JNI) which can cause the entire VM to crash. ------- 2. Teams Are Looking for Options The CIMOM (supporting WBEM) is a Java daemon. It initially occupies around 40M of RSS but grows from there. In order to address this problem, at least one Sun Engineer, Peter Madany, has been doing research to determine Java daemon memory utilization when running on a currently unsupported J2ME VM on Solaris. In other words, we are looking into demonstrating that resource exhaustion on Solaris Servers could be avoided by using some of the techniques used in an edition of Java intended for very small systems. 3. New Projects Explain Why They Are Not Using Java Quoting from the recently submitted Nile case (SARC/2001/617) now under review: ------- These libraries should be commercial implementations and must be in native platform code (ie not Java or Perl). Native code is a requirement because one of the core requirements for the proxy is for minimum impact on the target host. Java has too large a footprint (both memory and disk image) and may not be installed on the customer's host. ------- 4. ARCs Include the Java Problem in Rejection Reasoning Quoting from the recently rejected SunMC PMA case (LSARC/2000/457): ------- The CLI interpreter is implemented in Java, and the overhead of starting a JVM for each command execution is prohibitive. At least one of the votes to reject was related to this inappropriate use of Java. The Solaris implementation of Java is slow and very large. While this project did not provide a measurement of resident set for their CLI, the minimum RSS for the JVM is known to be 9MB and the typical RSS for a similar Java program is 30 to 40MB, and takes up to 15 seconds to start. The project team admitted in the review that this CLI may be used on a daily basis. For such a CLI, the delays and resource requirements of the Solaris Java implementation are unacceptable. ------- 5. Customers and Field Engineers Are Noticing the Problem Following is an excerpt from Kevin Tay's e-mail to three Java aliases regarding a customer installation of a third-party product written in Java called Vitria. We see typical very large RSS numbers compared to a WinNT implementation combined with increased resource usage from Solaris7 to Solaris8: ------- Customer said they have something like 450+ container servers and 80+ automator server for the Vitria system. So the estimation for the hardware RAM is around 9GB for USII machine and 14-15GB for the USIII machine. Questions: 1. Why is Sun systems using so much more memory? 2. Why is the UltraSPARC III/Solaris 8 system using a lot more memory than a UltraSPARC II/Solaris 7 system (with every other thing being equal)? 3. How can I reduce the memory utilization of the UltraSPARC III system? ------- NOTE: The response to this e-mail was to suggest moving to a different build of Java 1.2.2 since the indicated build on Solaris 8 had a known bug; it should be noted, however, that the 9GB memory footprint for Solaris7 is still unusually large. 6. Close Call in Solaris9 Bug ID 4526853 describes a bug in Core Java which used to be an external module called JSSE. Among other products, PatchPro and PatchManager depend on the JSSE. As long as the module could be used, the JSSE interface could be trusted to remain stable despite extensive changes in core Java. Now the Java architecture makes it impossible to use the module. This bug in core Java completely disables PatchPro and PatchManager. It was introduced in build 83 of Java 1.4. It was detected and corrected before the final build of Solaris9. If it had not been detected before the final build, it would have shipped with Solaris9 FCS. For those products that depend upon JSSE and operate on multiple OSs, there would have been no recourse except to deliver with their product an entire new Java distribution. This distribution would have to upgrade the existing Java installation. The fact that various products depend upon specific versions would mean that such an upgrade would carry the risk of breaking other Java-based software on the target system. Correcting the Java Problem We strongly recommend that management require Java to conform to the Software Development Framework especially from the standpoint of ARCreview. We believe that the next release of the Sun Java implementation should be brought to ARC while still in the prototype phase. Both PSARC and LSARC have dealt with the Java issues peripherally, recognizing numerous problems but unable to effect change in the underlying source of the difficulties - namely Java. By bringing the Sun Java implementation through ARC, these issues can be resolved.

[Guest.Visitor]

How does this apply to the JNI issue I'm having? A brief scan of the article revealed only a slight mention of JNI instabilities that crash the JVM. I agree that this is nice to know info but I think I missed the part where it applies to the topic. I don't mean to insult I just want to make sure I'm not missing something relevant. With all due respect, I appreciate the post. I haven't read it entirely (it's rather long) I'll read it when I get home tonight. Cliff

[R.Daugherty]

Clifton, it had some stuff about the JNI instability. For sure was not some answer to your problem. Just disregard, no problem. rd

[Guest.Visitor]

Barbara, I also am confused on this. Can you point me to any IBM iseries docs that describe the jvm behavior in v5r2? Alex

[Guest.Visitor]

Ralph, Yeah seems like trolling to me ;). The comp.lang.java.advocacy newsgroups have been tearing into this memo for a couple of days now. Those interested can jump over there to check out the controversy. Really most of it is much_todo_about_nothing, in my opinion. Alex

[R.Daugherty]

oh, and here I thought it was news. Thanks for the info, Alex. It apparently failed the smell test... :) rd

[J.Pluta]

Nothing gets comp.lang'rs all tore up like an internal memo. <grin> And your take on it is pretty much valid, Alex. There are problems with Java development, but they're pretty much the same problems any living software entity has, from Linux to RPG to sendmail to DOOM. Bugs get prioritized poorly, software gets bloated. No! Really? <blockquote><tt> >snicker< </tt></blockquote>

[B.Morris]

Alex Garrison wrote: > > Barbara, > > I also am confused on this. Can you point me to any IBM iseries docs that > describe the jvm behavior in v5r2? > Alex, see the V5R2 Memo to Users, here: http://publib.boulder.ibm.com/iseries/v5r2/ic2924/info/rzaq9.pdf. Joe Pluta already posted the relevant bits in this thread - his subject was "One JVM at a time, not one forever?".

[Guest.Visitor]

The documentation in the reference quoted by Joe is just not clear or comprehensive enough to answer the questions. Of course I read the "memo to users", but the three paragraphs on the jvm really just bring up more questions than it answers. Consequently I was hoping IBM had better documentation that spelled out the jvm behavior more clearly. For example: "Prior to V5R2, it would be possible for the JNI_GetCreatedJavaVMs function to return more than one Java virtual machine in its list of Java virtual machines. In V5R2, the JNI_GetCreatedJavaVMs returns at most one Java virtual machine." Are we talking about one JVM for the whole system, or one per job? If two people signon and call JNI_GetCreatedJavaVMs, will they be running in the same JVM? If so, then I have only one JVM per system -- not good for many reasons (need to run multiple jdks, desire to have websphere run in its own jvm so it cant be affected by anything else going on in other jobs, multiple class loaders.....). If each job can get their own JVM (but only one per job), what happens when the jvm for one user blows up? Do I have to abort the whole job? IPL the box? Put another quarter in the slot? If we do get one jvm per job, what happens when that job ends? Does the given jvm end? If not, what happens to the heap? I can easily fire up Java jobs that take 60MB of heap. The jobs end, is the heap released to the memory pool? Does anyone out there know of some IBM docs detailing jvm behavior in v5r2? Anything that maybe got handed out at COMMON? Alex Garrison

[Guest.Visitor]

Hello all, I keep getting myself into these corners where I don't see a easyway out. I'm trying to debug a Java program I wrote. The problem is I wrappered this Java program with an RPG JNI program. (In other words I'm using the new Java support for RPG to call the Java from RPG.) The RPG program is complimented with a good ol' command definition object to disguise it as one of our ol' fashioned home-baked CL utilities. It was working for quite some time now until I decided to muck with it not remembering where I'd left off last time. Now it's time to debug and I'm stuck! I used the eclipse workbench to create the Java program. I threw the .class files in a .jar and shipped it to the IFS. I put it in the classpath and viola! Instant RPG access via the new support. Only thing is the ol' ILE debugger doesn't seem to be able to step into the source. I then tried running just the Java program from the command line with RUNJVA and a *Debug option. "Source file not available..." says the ol' green screen debugger. "Well why are you using that old green screen stuff for debugging?" Good question. The skinny is I don't have the full WSDc toolset from IBM even as we are on V5R1. (We never ordered the disks.) I would ask management to order them for me but you see, that's another story. I'm not even supposed to be coding Java here! (It's the forbidden language like swear words in church.) It'll probably take another six months to a year before I get the chance to play on the new WSDc stuff. So I got the next best thing. I downloaded eclipse from the net since it's free and went to work on that. My situation in summary (minus all the ramblings): I have the Eclipse open source IDE (...and incidentally the Websphere Development Tools disks offered prior to the Eclipse integration.) I have V5R1 and a pretty cool RPG program that uses the new stuff to call the Java program. I have a bug somewhere in my Java (probably due to the fact that I'm stuck on JDK1.2.2 and the APIs I'm using depend on JDK1.3.1+) I have JDK 1.4.1 installed on my Win2K desktop for development (...also the likely suspect of issues since our '400 is stuck on JDK1.2.2) I have limited patience for things which I have no control over and am reaching my boiling point. What are my options? Should I downgrade my development client to JDK1.3.1 or lower? I don't want to move backwards in technology. Should I install the older Websphere devlopment stuff w/ VAJ? I'm not familiar with VAJ and I'd hate to waste time with another outdated bloated IDE. (I've been searching Java IDEs for the longest and finally found my true love in Eclipse.) Should I port the danged source file to the IFS, compile it, and hack my system classpath once again to get things working? (All these steps just feel unnecessary but it would probably take less time than writing this discussion.) What to do? Regards, Cliff (Probably my last post for the year 2K2)

[Guest.Visitor]

Haven't tested it yet but I'm getting there. This was provided compliments of Barbara Morris on the other forum. <hr width=50 align=left>Code ('http://www.mcpressonline.com/mc/showcode@@.6ae55441/35')

[B.Morris]

Alex Garrison wrote: > > The documentation in the reference quoted by Joe is just not clear or > comprehensive enough to answer the questions. Of course I read the "memo to > users", but the three paragraphs on the jvm really just bring up more > questions than it answers. Consequently I was hoping IBM had better > documentation that spelled out the jvm behavior more clearly. > ... snip some good questions > Does anyone out there know of some IBM docs detailing jvm behavior in v5r2? > Anything that maybe got handed out at COMMON? > Alex, if the answers to your questions aren't readily available in the Info Center, they should be. You should do a Feedback from the Info Center - it's the stubby pencil in the top right. To answer your main question, it's one JVM per job. Using the RUNJVA command or doing a java command from QSH doesn't count - those start up separate jobs.