Converting Data

[MCWebsite.Staff]

** This thread discusses the article: Converting Data **
** This thread discusses the Content article: Converting Data **
0

[H.Boldt]

Converting Data

** This thread discusses the article: Converting Data **
First, my first experience with data conversions was back in school when I was asked to move some program I wrote from a Burroughs B6700 to a PDP 11/45 running Unix. The only choice available at the time was to write the program to tape. My first attempt failed since the Unix system didn't recognize the tape labels. My second attempt failed because the B6700, an ASCII machine, by default wrote the tape in EBCDIC format. My third attempt finally worked. This experience left me, not just with a taste of the problems of ASCII/EBCDIC co-existence, but also with a thorough dislike for tapes. Second, Bob, you mention that the method you describe is slower and more restrictive that the preferred approach. Why not mention the preferred approach here? You're probably referring to the widely used "iconv()". Information about iconv() is readily available, and can be found with a simple Google search. Here is one summary from the midrange.com archives of how to use iconv() in RPG. (Actually, unless Barbara has made a change in the past 18 months, the RPG runtime uses iconv() to implement EBCDIC/Unicode conversions.) Third, the previous point suggests a quick and dirty way to convert EBCDIC to ASCII in RPG without even calling any API's. Remember that the first 127 ASCII characters map one to one to the first 127 Unicode characters. If your EBCDIC string represents characters in this range, use the %UCS2 built-in to get the Unicode characters. Every other byte in the resulting string is the corresponding ASCII character. (I'm not sure I'd recommend this in practice. Better to use iconv() directly, and get proper conversions for the upper half of the ASCII character set too.) Fourth, in many cases, why bother with data conversion at all? For example, say you're writing a new application that uses HTTP (or some other internet protocol) as its primary user interface. You can avoid the trouble and expense of data conversion by just storing your data in ASCII form in the database. RPG doesn't really care, unless you're comparing data to literals in your program. But even there, you can always do the EBCDIC to ASCII conversion of the literals up-front. Cheers! Hans

[shane]

Converting Data

** This thread discusses the article: Converting Data **
Has anyone ever heard of a shop running the AS/400 in ASCII instead of its default EBCDIC? I am a little surprised we haven't seen IBM pushing this. Shane.

[Guest.Visitor]

Converting Data

** This thread discusses the article: Converting Data **
No, I haven't. I heard someone a while back tried to change the system value QCCSID to ASCII and got an error message saying it was not allowed. Not sure about today though. Chris

[shane]

Converting Data

** This thread discusses the article: Converting Data **
If it weren't for the big difference in collating sequence of numeric values I don't think we would notice much. Of course I would have to retire the logical NOT symbol in CL programs (shift on 6 on non-pc keyboards). LOL.

[R.Cozzi]

Converting Data

** This thread discusses the article: Converting Data **
Hans, Yes iconv() is the preferred method I was referring to. I use it in my xLateCCSID() subprocedure in RPG xTools. I've written both an RPG subprocedure wrapper and a C wrapper as well--both allow the iconv() handle to be saved so that the conversion stays open until you close it. It seems simplicity is best with the RPG crowd. Simply saying "Convert to ASCII" is preferred over, use these three APIs to convert between CCSIDs. (iconv, iconv_open() iconv_close()). If %USC is so nice, why not add %ascii()?

[Guest.Visitor]

Converting Data

** This thread discusses the article: Converting Data **
Thanks Bob, responding to your last paragraph in this article, for the free advice in 2005. I too hope it can continue to come for free. I support, use, and recommend MC RPG Developer! (yep, that must be the name.) Happy new year to everyone at MC.

[H.Boldt]

Converting Data

** This thread discusses the article: Converting Data **
First, I never said %UCS2 is nice. I simply offered that technique as a curiosity. (I'm sorry if I didn't make that more clear.) Fact is, UCS2 is rather outdated since the Unicode standard now includes more glyphs than can be represented in UCS2 format. Regarding simplicity, many RPG programmers also profess to being concerned about performance. As you know, that's quite understandable. After all, if your company is spending 6-7 figures on iSeries hardware, they want to make darn sure that the machine is constantly running above 90% utilization. Using a compiled language is one way to make sure you're getting the most out of your hardware. Unfortunately, as we've seen before, the simple solution and the best performing solution aren't necessarily the same thing. Just by adding an ease-of-use wrapper to an API adds a layer of performance cost. Depending on the interface definition, that cost may be quite significant, especially when dealing with long character strings. One more point, code page conversion is rarely "simple". As you know, there are multiple EBCDIC code pages and also multiple ASCII code pages. If you want to do things properly (and who doesn't?), you need to be very much aware of the specific code pages you're dealing with, and how data gets translated at each stage of processing. For example, you may have to be concerned with the code page of the incoming data in an HTTP POST request. I suppose my point here is that you may not be doing anyone any favors by offering "ease-of-use" wrappers for code page conversion. Better to provide good education on the proper use of the preferred techniques. But that's just my own humble opinion. Cheers! Hans

[David Abramowitz]

Converting Data

** This thread discusses the article: Converting Data **
ShanePoad wrote: If it weren't for the big difference in collating sequence of numeric values I don't think we would notice much. You may notice more than you think. As far as I know, there is more than just a collating sequence involved here. EBCDIC uses block transfer mode. This transmits data an entire page at a time, while ASCII transmits characters one at a time. The difference to the OS would be gargantuan. Dave

[H.Boldt]

Converting Data

** This thread discusses the article: Converting Data **
Huh?!? As far as I know, there's no requirement on any code page defining how data encoded in that code page is to be transferred. Perhaps you're confusing the issue with the old 3270/5250 terminals that transfer EBCDIC characters a screen at a time and other dumb terminals that transfer ASCII a byte at a time? These days, most data, regardless of code page, is transmitted via packets through some communications protocol (for example, ethernet or some internet protocol). Hans

[shane]

Converting Data

** This thread discusses the article: Converting Data **
David, the topic your refering to is related, but mostly from the stand point of convention. Syncronous is a block transfer communication method and asyncronous is the byte transfer method you mentioned. EBCDIC and ASCII are just well accepted standards for character representation. Syncronous is typically used in EBCDIC environments(mostly because of the mainframe heritage), while asyncronous is typically found in ASCII environments like PCs. From a pure technical perspective, there is nothing stopping you from creating a new character representation, call it FRED, and communicate with it. You just have to share with others its definition. Shane.