TechTip: Talk to the Internet with cURL PDF Print E-mail
Written by Alan Seiden   
Thursday, 18 May 2006

Use the i5's PASE environment to run cURL, the popular open-source communication utility.

The i5 can run a variety of open-source software, thanks to i5/OS PASE's support of AIX binaries. One fine specimen is cURL (client for URLs), a popular program for Internet communication.

From cURL's manual: "cURL is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT, TELNET, LDAP or FILE)...cURL offers a busload of useful tricks."

It's easy to use cURL to request pages from Web sites, or, with a little Control Language (CL) code, to automate Web-based tasks:

  • Check whether your site is running.
  • Get Web-based information according to the URL parameters you pass.
  • Execute an action on another server via a URL.
  • Upload a file to a Web site that supports HTTP file uploads.

This article will get you started by demonstrating a simple Web page download.

Get cURL

Some kind souls on the Web have compiled and packaged the AIX binary for us, so we don't have to compile our own. For this demonstration, we'll choose a non-SSL-enabled version of cURL (SSL encryption would require installing many prerequisites).

Download the cURL command-line binary. It is archived at UCLA's Public Domain Software Library for AIX. Choose the download from the Binary column where the row shows AIX=4.2 and Version=7.13.1. Although this package is somewhat old, it is more complete than that supplied for newer versions. The file can also be downloaded directly (this could change over time) from ftp://aixpdslib.seas.ucla.edu/pub/curl/RISC/4.2/exec/curl.7.13.1.tar.Z. The ".tar.z" extension indicates a UNIX-style compressed archive file. Save the file to your hard drive.

Extract and Install cURL

Although you could use PASE's uncompress/tar commands on your iSeries/i5 to extract the contents of the ".tar.z" file, I prefer to extract the files on my Windows PC with a program such as the free ZipGenius.

The extracted contents comprise a directory structure. Of that structure, we are interested in the /usr/local/bin folder. Look inside the bin folder for a file called curl. Copy curl to your i5's root file system. I chose to copy curl to a folder called /coolprograms using a Windows Explorer mapped drive. If I had not been able to map a drive for some reason, I could have used iSeries Navigator or FTP.

Get a Web Page

Let's try it!

Launch the PASE interactive (terminal) environment:
CALL QP2TERM

Once inside PASE:
cd /coolprograms [or whatever folder you copied curl into]
curl www.google.com

If the i5 is connected to the Internet, you will see the output on your screen (Figure 1).


http://www.mcpressonline.com/articles/images/2002/seiden_alan_techtip_cURLV4--05260600.png

Figure 1: The command curl www.google.com retrieves the HTML of Google's home page. (Click images to enlarge.)

Redirect the Output to a File

We may wish to redirect cURL's output to a file for later analysis or other use. In our example, we will create an output directory and then adapt our curl command to send the output there. A redirection symbol (>) sends the HTML to a file of our choice.

mkdir /htmloutput
curl www.google.com > /htmloutput/google.html

Verify the successful download by viewing the contents with UNIX's cat command:
cat /htmloutput/google.html

The results of these commands are shown in Figure 2.

http://www.mcpressonline.com/articles/images/2002/seiden_alan_techtip_cURLV4--05260601.png

Figure 2: cURL sends output to a file whose contents are then displayed by the UNIX cat command.

Automate cURL by Calling It from CL

For ultimate control, we can automate cURL by calling it via QP2SHELL or QP2SHELL2 from our CL programs:

/* The -o means we will indicate where to save our result. */

/* The -s means silent mode: Do not show status or errors. */
/* (Note that the -o and -s arguments must be lowercase) */

CALL PGM(QP2SHELL2) +
PARM('/coolprograms/curl' '-o' +
'/htmloutput/webpage.html' +
'http://www.thesite.com' '-s')

Use this CL command as a template for many possibilities of letting your i5 talk to the Internet.

Resources

Manual Page for cURL
Using cURL to Automate HTTP Jobs
Mailing Lists for cURL
History of cURL by its creator, Daniel Stenberg

Alan Seiden is Senior Developer and Technical Lead at Strategic Business Systems, Inc., in Ramsey, New Jersey, where he helps clients reach their business goals using IBM i5, Microsoft, and open-source technology, with an emphasis on usability. Alan is an advisory board member of the New York City Usability Professionals Association. Contact Alan at This e-mail address is being protected from spam bots, you need JavaScript enabled to view it .


Last Updated ( Thursday, 18 May 2006 )
 
Discuss (7 posts)
bruceanthony@yahoo.com
TechTip: Talk to the Internet with cURL
Nov 29 2006 11:56:00
Article was very helpful. Does anyone know where one can download a binary version of curl with SSL for the iSeries?
#120590

H.Boldt
TechTip: Talk to the Internet with cURL
May 31 2006 10:53:00
There are a number of tools that offer more or less the same functionality. One method I sometimes use is <a href="http://www.python.org/">Python</a>'s <a href="http://docs.python.org/lib/module-urllib.html">urllib</a>. For casual use, you can even invoke the URL retrieval methods from a Python command line, for example, <code>urllib.urlretrieve('http://www.google.com/','mylib/google.html')</code>. <p>(For those who didn't know, there is a <em>native</em> <a href="http://www.iseriespython.com/">Python port for the iSeries</a>.) <p>Cheers! Hans
#120589
Guest.Visitor
TechTip: Talk to the Internet with cURL
May 30 2006 16:40:00
Let's test your i5's internet connection. <p>From an i5/OS command line, run this command: <BR>
ping www.google.com <p>Let us know what appears in the status line (line 24). <p>--Alan Seiden
#120588
Guest.Visitor
TechTip: Talk to the Internet with cURL
May 30 2006 15:37:00
Just like the other poster here, this is my first use of PASE - thanks for the article. Unfortunately, though I am able to access intranet sites via curl, I get only "(6) Couldn't resolve host" when I try to access an external site. This is no doubt due to the company firewall. Any ideas on how to poke thru the firewall with this command? Thanks.
#120587
Guest.Visitor
TechTip: Talk to the Internet with cURL
May 19 2006 11:11:00
I am continuing to learn about open source an AIX on the AS400. I have installed the PASE environment, but have not done anything with it. Your article was very straigtforward and is the first useful thing I've done with PASE. Of course, it took me a while to figure out the difference between "/" and "". Once I got past that, I was able to duplicate the tasks in the article. This is the first time I've been able to do that. Thanks. <p>---Dale
#120586
A.Seiden
TechTip: Talk to the Internet with cURL
May 19 2006 01:35:00
If this article about cURL provoked any questions or ideas, please post them here. We'd like to hear them. <p>Alan Seiden
#120585
MC Press Web Site Staff
TechTip: Talk to the Internet with cURL
Nov 29 2006 12:58:00
This is a discussion about <B>TechTip: Talk to the Internet with cURL</b>.<p align='center'><a href=http://www.mcpressonline.com/mc?1@232.1KNKfHX1eQT.17@.6b3741a6>Click here for the article</a>.</p>
#120584


Discuss...
User Rating: / 0
PoorBest 
Related Articles
< Prev   Next >

The following White Papers can be found at the MC White Paper Center


The following trial software can be found at the MC Press Software Center.   



   MC-STORE.COM