08
Fri, Nov
10 New Articles

V4R4 APIs Expand AS/400 IFS File Sizes to 256 GB

APIs
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

As applications become more data-intensive, the need to store and manipulate large amounts of information grows far beyond the current 2 GB limit imposed by existing interfaces. With existing 32-bit interfaces, sizes, offsets, and lengths are represented as signed 32-bit integers and are limited to the maximum values that can be represented in a signed integer, e.g., no file can be larger than 2 GB for a variable defined as a signed integer.

In OS/400 V4R4, the AS/400 Integrated File System (AS/400 IFS) provides 64-bit versions of its UNIX-type APIs. These 64-bit APIs allow applications to store and manipulate files that are much larger than 2 GB. With the new 64-bit interfaces, sizes, offsets, and lengths are represented as 64-bit integers, which eliminates the 2 GB limit. AS/400 IFS users can now store and manipulate large files up to 256 GB in the Root (/), QOpenSys, and user-defined file systems by using the new 64-bit versions of the OS/400 UNIX-type APIs.

How to Use Large File Support

AS/400 large file support is easy to use. It enables applications that use existing 32- bit AS/400 IFS interfaces to continue to run without recompilation or code changes if they do not intend to use AS/400 large file support or the new 64-bit AS/400 IFS interfaces. Applications that use large file support can either have the existing 32-bit interfaces mapped to their new 64-bit version counterparts, or you can code directly to the new 64-bit interfaces. The following compile-time flags are provided for applications to use the AS/400 IFS 64-bit or large file support:

• _LARGE_FILES allows automatic mapping of 32-bit interfaces and data types to their 64-bit versions.

Applications compiled using the _LARGE_FILES flag will have their invocations of 32-bit interfaces mapped to the 64-bit versions and all the appropriate data types and data structures will also be mapped to 64-bit versions. For example, open() will be mapped to open64() and off_t will be defined as type long long (originally defined as type int).

• _LARGE_FILE_API allows applications to access 64-bit interfaces and data types directly. Applications compiled using the _LARGE_FILE_API flag can code directly


to 64-bit interfaces and use 64-bit data types and data structures. For example, applications can use open64() and use off64_t (defined as type long long).

Application Protection Mechanism

Under this structure, applications are prevented from inadvertently using large files without using large file support (i.e., 64-bit APIs with the appropriate compile flag). Applications will need to open a large file with the open64() API. An attempt to open a large file with the open() API will fail unless the application is compiled with the _LARGE_FILES flag. Alternatively, open() API with O_LARGEFILE in oflag can also be used to open a large file.

If the application intends to access large offsets or sizes on file descriptor-based operations, e.g., lseek(), ftruncate(), then the file must be opened for large file access and the 64-bit version of these APIs must be used.

New Data Types and Structures

With 64-bit enablement, a number of data types and data structures are redefined, as shown in Figure 1. The standard definition for those data types will be mapped to their 64- bit definitions when the applications are compiled with the LARGE_FILES flag. The 64-bit definitions are available when the applications are compiled with the LARGE_FILE_API flag.

Large File API Support

Similarly, a number of new 64-bit AS/400 IFS API interfaces are provided to allow applications to create and manipulate large files in the AS/400 IFS. The existing 32-bit AS/400 IFS interfaces are mapped to 64-bit interfaces when user applications are compiled with _LARGE_FILES defined. And, like the data types and structures, the 64-bit interfaces are only available to applications when they are compiled with the _LARGE_FILE_API flag. Figure 2 presents the standard 32-bit API interfaces and their 64-bit versions.

Application Programmer Hints, Examples, and Warnings

The following are strategies that your programmers can use when considering how to use large file support on the AS/400 IFS once you migrate to OS/400 V4R4:

• Ignore large file support in the AS/400 IFS. Your applications will run as they did before large file support was introduced. However, if an application attempts to open a file that is larger than 2 GB-1, the open() operation will fail. Any attempt to open a large file without using the open64() API will fail with an errno value of EOVERFLOW (3525). Similarly, an attempt to read or write beyond 2 GB-1 when the starting offset is greater than 2 GB-2 will fail unless the file is opened in a large file access mode. Read operations fail with an errno value of EOVERFLOW (3525) when the starting offset is greater or equal to 2 GB-1. The write() API will also fail with an errno value of EFBIG (3507) in this situation.

• If you do not want to use large file support, but you want to be able to open large files (without going beyond the limit of 2 GB-1), then define _LARGE_FILE_API in the source code and open the file using open64(). This solution gives you access to 64-bit data types and interfaces, but it does not map standard definitions to their 64-bit version. Be aware that, since you have opened the file as a large file, read() and write() will not fail when you go beyond 2 GB-1 with a starting offset greater than 2 GB-2. However, other APIs may fail when you go beyond the 2 GB-1 limit. For example, an lseek operation to seek beyond 2 GB-1 will fail with errno value of EOVERFLOW. Similarly, an fstat() operation will fail if the size of the file is greater than 2 GB-1 with errno value of EOVERFLOW.

This technique enables you to mix and match the use of standard and 64-bit definitions. However, applications programmers need to be very careful when mixing and matching. For example, you could open a large file using open64() and use the lseek() API to seek within the 2 GB-1 boundary. An attempt to seek beyond 2 GB-1 would fail unless lseek64() is used. However, read() and write() would be enabled to go beyond 2 GB-1 and an application may inadvertently read or write data that is beyond its reach.


• If you want to use large file support without changing your code, then define _LARGE_FILES in the source code and recompile. All the standard definitions will be mapped to their 64-bit version. However, you need to be careful because certain APIs can now return offsets and sizes that may not fit in integer data types. For example, after being mapped to stat64, using the stat() API on a large file will return a size that will not fit into an integer data type. So, if you are storing offsets and sizes in local variables, you may need to change those local variables.

• If you want to use large file support with code changes, define _LARGE_FILE_API in the source code, change all the standard definitions to their 64-bit version, and recompile. From a code clarity standpoint, this solution is preferred.

Large File Support without the Pain

Large file support in the AS/400 IFS is similar to the large file support provided on AIX. The large file support is provided in such a manner that existing applications working with regular files will not be impacted. The large file support on the AS/400 is very easy to use, and two compile time flags are provided for applications to choose the way they want to use the large file. In addition, the following OS/400 V4R4 commands are also enabled to work with or display large stream files stored in the AS/400 IFS:

• Work with Object Links (WRKLNK)
• Display Object Links (DSPLNK)
• COPY Object (CPY)
• Move Object (MOV)
• Copy to Stream File (CPYTOSTMF)
• Copy from Stream File (CPYFRMSTMF)
• Save Object (SAV)
• Restore Object (RST)
• Display Diskette (DSPDKT)
• Display Save File (DSPSAVF)
• Display Tape (DSPTAP) With OS/400 V4R4, IBM is anticipating the needs of a growing set of key applications that rely on the AS/400 IFS for their file storage. By using 64-bit interfaces, the AS/400 IFS allows applications to use and support very large files so that businesses can grow their applications beyond the 2 GB barrier.

Related Reading

AS/400 Unix-Type APIs V4R4 (SC41-5801-03)


Standard Definition 64-bit Definition

off_t off64_t struct flock struct flock64 off_t l_start; off64_t l_start; off_t l_len; off64_t l_len; F_SETLK, F_SETLKW, F_GETLK F_SETLK64, F_SETLKW64, F_GETLK64 struct stat struct stat64
off_t st_size; off64_t st_size; unsigned long st_allocsize; unsigned long long st_allocsize; struct statvfs struct stavfs64

_Bin8 f_blocks; unsigned long long f_blocks; _Bin8 f_bfree; unsigned long long f_bfree; _Bin8 f_bavail; unsigned long long f_bavail;

Figure 1: These are the redefined data types and data structures (along with their 32-bit counterparts) you’ll need for 64-bit enablement.

Existing Interfaces 64-bit Definition

open() open64() creat() creat64() fstat() fstat64() lstat() lstat64() stat() stat64() ftruncate() ftruncate64() lseek() lseek64() statvfs() statvfs64() fstatvfs() fstatvfs64()

note: Qp0lGetAttr() and DosSetFileLock() APIs are also enabled for large file access.

Figure 2: These are the 64-bit implementations of the standard 32-bit APIs.


BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: