When a file contains variable-length fields, the data of the fixed-length fields of its deleted records can still be retrieved.
A known technique to retrieve the records deleted from a file consists of saving the file to a save file and extracting the records from the save file. This technique is based on the fact that, in many cases, the records of the file are stored in the save file maintaining their format, apart from a status byte added by the system at the beginning of each record.
However, there are situations where the format of the records in the save file is different from the original format. This happens when one of the fields of the file is variable-length, nullable, or of type date-time (DDS types L, T, and Z). In these cases, we can still retrieve the data of the fixed-length fields of the deleted records and, what's more, the data of some variable-length fields for some of those records.
Some Terms
From now on, when we talk about the length of a field, we will be referring to the number of bytes (not characters) and, for variable-length fields, will not take into account the two initial bytes.
So the current length of a variable-length field will refer to the number of bytes taken up currently by the data of the field, its maximum length to the maximum number of bytes that its data can take up, and its allocated length to the number of bytes that the system allocates for its data.
For example, the maximum length of a single-byte variable-length field of 10 characters (10A VARLEN) is 10 (not 12), and its current length can be from 0 to 10. The maximum length of a graphic variable-length field of 10 characters (10G VARLEN) is 20 (not 22), and its current length can be 0, 2, 4...or 20.
We will say that a variable-length field is of type 1 if the number of bytes allocated by the system for the field is equal to its maximum length. If it is less than its maximum length, we will say that the field is of type 2.
The space allocated by the system for a variable-length field is calculated as shown in Figure 1:
Figure 1: This is the space allocated for variable-length fields.
Note that the system can allocate some space for a variable-length field without the user's request. On the other hand, when the user requests the system to allocate a number of characters for a variable-length field, the number of characters eventually allocated by the system can be greater than the one requested. We will see later how to obtain the length of the space allocated.
Data Records Saved to a Save File
When a physical file is saved to a save file, the records of the physical file are stored in the save file with a format different from the original one. First of all, every record of the physical file is stored in the save file in a logical record A and, in some cases, in a logical record B.
Records A contain the data of the fixed-length fields, the variable-length fields of type 1, and the variable-length fields of type 2 whose current length is less than or equal to their allocated length. Records B contain the data of the variable-length fields of type 2 whose current length is greater than their allocated length.
To simplify the following description, let's suppose that only one of the members of the physical file is saved to the save file.
Records A (corresponding to the records of the saved member) are stored consecutively in positions 1 to 512 of a group of records of the save file. These positions of these records of the save file make up what we will call the area A of the save file. Records B are stored (in general, consecutively) in positions 1 to 512 of another group of records of the save file. We will call these positions the area B of the save file.
Records A and B contained in a save file are shown in Figure 2:
Figure 2: Here's an example of a save file containing a physical file.
To find out where area A begins, we have to look for a record of the save file whose positions 15 to 24 contain the name of the saved member. This is the first record of the save file containing information about that member. Then, look for a record whose position 2 is equal to x03 and whose position 33 is equal to x80. This is the first record of area A and contains the first logical record A from position 33.
To identify the initial record of area B, we have to look for the last record of the save file belonging to area A (we will see how to in the section "Format of Records A") and then for a record whose position 2 is equal to x13. This is the first record of area B.
Areas A and B are divided into segments 16MB long (2^24 bytes). As each record of the save file contains 512 bytes, a segment is stored in 32K (32,768) records of the save file. The first 32 bytes of a segment are reserved for the segment itself.
A record A can be stored in two consecutive segments. In this case, the part of record A stored in the second segment will begin in position 33 of the segment (as its first 32 bytes are reserved).
A record B cannot be stored in two consecutive segments. If a record B does not fit into the remaining space of a segment of area B, the whole of it will be stored in the following segment from position 33.
Length of the Space Allocated for Variable-Length Fields
The length (in bytes) allocated by the system for each variable-length field of the physical file, as well as whether they are of type 1 or 2, is stored in the save file between the first record containing information about the saved member and the first record of area A (see section "Data Records Saved to a Save File").
We have to look for a record of the save file whose positions 47 to 56 contain the name of the member and then for a record whose positions 1 to 2 contain, in binary format, the number of fields of the file. An array with one item per field of the file is stored in this record from positions 33 to 512 (and, if necessary, in the following records from positions 1 to 512). Each item of this array is 32 bytes long. Its format is shown in Figure 3:
Figure 3: This is the format of an item containing variable-length info for a field.
For an example, see section "Example of Records A and B for a File with Variable-Length Fields."
Format of Records A
RecordA = Status [VarLenInfo] [NullInfo] FieldA1 ... FieldAn [VarLenFiller]
where [ ] shows that an element can be present or not and n is the number of fields of the physical file.
Status
1 byte. Values x80 or xA0 show a valid record, xC0 or xE0 a deleted record, and x00 or x01 the end of the area A.
VarLenInfo = RecordBSegment RecordBOffset RecordBLen
Exists if any field of the physical file is variable-length. In this case, it is 8 bytes long.
RecordBSegment
2 bytes, binary format.
Number of segment of area B where record B lies. It is 0 when there is no record B associated.
RecordBOffset
4 bytes, binary format.
Offset of record B inside the segment. This offset ranges from x0000 0000 to x00FF FFFF (that is, byte 1 is always x00).
The absolute offset of record B in area B can be calculated as follows:
(RecordBSegment - 1) * 2^24 + RecordBOffset.
RecordBLen
2 bytes, binary format.
Length (in bytes) of the record B. Can be 0.
NullInfo
Exists if any field of the physical file allows the value null. In this case, 1 bit is used for every field of the file to store whether it is null (1) or not (0). Therefore, the number of bytes needed is n / 8 rounded up, where n is the number of fields of the file.
FieldAj = FixedField or VarField for j = 1 to n
VarLenFiller
Exists if any field of the physical file is variable-length and the length (in bytes) of record A is not a multiple of 16.
It contains as many bytes x00 as necessary for the length of the record A to be a multiple of 16.
FixedField
Contains the data of a fixed-length field. This data has the same format as the physical file except for the types shown in Figure 4:
Figure 4: These are the types with different formats in a physical file and a record A.
Figure 5 shows how some date-time values are stored in a record A:
Figure 5: These are date, time, and timestamp values in record A.
As a field of type date is stored in record A as a number of days, we can obtain its equivalent in *LONGJUL format by taking any of the dates of Figure 5 as a reference and taking into account the leap years.
VarField = [VarFieldFiller] VarFieldLen [VarFieldAllocSpace]
VarFieldFiller
1 byte x00. Exists if the offset of VarField in record A is odd. This way, the offset of VarFieldLen will always be even.
VarFieldLen
2 bytes, binary format.
Contains the current length (in bytes) of this variable-length field.
Possible values: from 0 to 32740 (x7FE4)—that is, up to 32740 single-byte characters or 16370 graphic characters—and -32768 (x8000). The value x8000 shows that the data of this variable-length field is not stored in record A but in its associated record B.
VarFieldAllocSpace
Space allocated (if any) in record A for the data of this variable-length field. Its first VarFieldLen bytes contain the data of the variable-length field when VarFieldLen is different from x8000.
Can exist even when VarFieldLen is equal to x8000.
See section "Length of the Space Allocated for Variable-Length Fields" to learn how to obtain the length of VarFieldAllocSpace.
Properties of Records A
All of the records A have the same length.
If the physical file contains a variable-length field, the length (in bytes) of record A is a multiple of 16.
The offset in record A of the length of a variable-length field—that is, the offset of VarFieldLen—is always even.
The record number 1 of the physical file member is stored in record A number 2, the record number 2 in record A number 3, and so on. The record A number 1 does not correspond to any record of the physical file member.
A record A can be stored in two consecutive segments.
Format of Records B
RecordB = RRN1 AccumLenB1 ... AccumLenBm [DataB1] ... [DataBm] +
RRN2 EndOfRecord
where [ ] shows that an element can be present or not and m is the number of variable-length fields of type 2 of the physical file. See section "Length of the Space Allocated for Variable-Length Fields" to learn how to obtain m.
RRN1
4 bytes, binary format.
Relative record number of the record of the physical file member whose data is stored in this record B. Can be 0.
AccumLenBk for k = 1 to m
2 bytes, binary format.
Length (in bytes) occupied by the data of the k first fields of the record B—that is, the sum of the bytes occupied by DataB1, DataB2...and DataBk.
DataBk for k = 1 to m
Contains the data (if any) for the kth variable-length field of type 2 of the physical file. This data occupies AccumLenBk - AccumLenB(k - 1) bytes, being AccumLenB0 equal to 0.
RRN2
4 bytes, binary format.
Relative record number of the record of the physical file member whose data is or was stored in this record B.
EndOfRecord
1 byte with value x73.
Properties of Records B
Records B can have different lengths.
The offset and length of record B are stored in its corresponding record A (in VarLenInfo).
When the offset of record B is different from 0, its length can still be 0 (meaning that record B contains no data right now).
A record B cannot be stored in two consecutive segments.
Deleted Records
For a record deleted from a physical file, all the data in its corresponding record A remains unchanged except Status—which is set to xC0 or xE0—and RecordBSegment, RecordBOffset, and RecordBLen—which are set to 0—so the data stored in NullInfo, FieldA1...and FieldAn can be retrieved. On the other hand, and unfortunately, we have no direct reference to the location of its record B.
Example of Records A and B for a File with Variable-Length Fields
Physical file FILE1:
A R RFILE1
A FLD1 3A
A FLD2 30A VARLEN
A FLD3 30A VARLEN(10)
A FLD4 20G VARLEN(5) CCSID(13488)
A* UCS-2: Space=x0020 1=x0031 2=x0032 3=x0033
A FLD5 3A ALWNULL
CRTPF FILE(QTEMP/FILE1) MBR(MEMBER1)
insert into QTEMP/FILE1 values
('AAA', '' , 'FOURTEEN BYTES',
vargraphic('12345' , 5, 13488), 'END'),
('BBB', 'VARLEN', 'FOURTEEN BYTES',
vargraphic('123456', 6, 13488), 'END'),
('CCC', '' , 'OVER FOURTEEN BYTES',
vargraphic('12345', 5, 13488), null )
Save file:
SAVOBJ OBJ(FILE1) LIB(QTEMP) OBJTYPE(*FILE)
DEV(*SAVF) SAVF(QTEMP/SAVF1)
FILEMBR((FILE1 (MEMBER1))) UPDHST(*NO)
The format of records A and B are shown in Figures 6 and 7, respectively:
Figure 6: This is the format of records A for FILE1.
Figure 7: This is the format of records B for FILE1.
Records of the save file:
First record containing information on MEMBER1:
RRN From To
*...+....1....+....2....+....3....+....4....+....5 33 1 50
FILE1 MEMBER1 ° ø
FFFFCCDCF44444DCDCCDF44444444444440900070000000000
FFFF693510000045425910000000000000B000000000000000
Record with "MEMBER1" in position 47:
RRN From To
*...+....1....+....2....+....3....+....4....+....5 41 1 50
m o-¯L Ø a °FILE1 MEMB
00000903296BD000000000008200F0008009CCDCF44444DCDC
01083404D60C3000010000000000F00010B069351000004542
....+....6 51 60
ER1
CDF4444444
5910000000
Record with the number of fields in positions 1 and 2:
< beginning of item > end of item
V Variable-length indicator ww allocated length
nn number of variable-length field of type 2
RRN From To
*...+....1....+....2....+....3....+....4....+....5 43 1 50
Ë §
000000000000000700B0000000000000000000000000000000
05030A100009000303500000000000000403000A0000000000
< V nnww
....+....6....+....7....+....8....+....9....+....0 51 100
Ø
00000000000000000100000800000000000000000000000001
00000000000000040E000E000001000000000000000000040E
>< V nnww ><
....+....1....+....2....+....3....+....4....+....5 101 150
Ø µ
0001080000000000000000000000000200020A000000000000
00000000020E0000000000000000060800000000030A000000
V nnww >< V nnww
....+....6....+....7....+....8....+....9....+....0 151 200
Ø
00020000000000000200000000000000000000000080000000
00000000000403000C20000000000000000000000000000000
>< V nnww >
First (and only) record of the area A:
Aj___ FieldAj S Status v VarLenInfo n NullInfo
f filler ww length of variable-length field
RRN From To
*...+....1....+....2....+....3....+....4....+....5 49 1 50
© u o-¯L Ø
00000B0A296BD0000000000000030000800000000044400000
03083404D60C30000100000000000100000000000800000000
A1_A2_A3
Svvvvvvvvn fwwww
....+....6....+....7....+....8....+....9....+....0 51 100
Ø AAA FO
0000000000000000000000000044408000000000CCC00000CD
00000000000000000000000000000000000000001110000E66
______________A4__________A5_ A1_A2_A3__
ww fSvvvvvvvvn fwwww
....+....1....+....2....+....3....+....4....+....5 101 150
URTEEN BYTES ? ? ? ? ?END Ø BBB Ø FOUR
EDECCD4CEECE000303030303CDC08000002020CCC08000CDED
4935550283520A0102030405554000100000102220000E6649
____________A4__________A5_ A1_A2_A3____
ww fSvvvvvvvvn fwwww
....+....6....+....7....+....8....+....9....+....0 151 200
TEEN BYTESØ END Ø CCC Ø
ECCD4CEECE800000000000CDC08000004020CCC00080000000
35550283520000000000005540001000102833300000000000
__________A4__________A5_ A1_A2_A3______
ww fSvvvvvvvvn fwwww
....+....1....+....2....+....3....+....4....+....5 201 250
? ? ? ? ?
00000000000303030303444000000000000000000000000000
000000000A0102030405000000000000000000000000000000
________A4__________A5_
ww fS
First (and only) record of the area B:
Bk___ DataBk nnnn RRN wk AccumLenk
E EndOfRecord
RRN From To
*...+....1....+....2....+....3....+....4....+....5 57 1 50
© u o-¯L VARLEN ?
01000B0A296BD00000000000000000000000000001ECDDCD03
03083404D60C30000100000001000000000206060251935501
B1____B3
nnnnw1w2w3
....+....6....+....7....+....8....+....9....+....0 51 100
? ? ? ? ? Ë OVER FOURTEEN BYTES Ë
0303030303000070000000101DECD4CDEDECCD4CEECE000070
02030405060002300030003036559066493555028352000330
__________ B2_________________
nnnnEnnnnw1w2w3 nnnnE
Appendix
Figure 8 contains the maximum values for database files that we have been implicitly dealing with:
Figure 8: These are the maximum values for database files (from V5R3 to V7R1).
More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.
TRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now! Request Now.
Forms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.
Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:
Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.
IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn:
LATEST COMMENTS
MC Press Online