Debian Codegraph @

vms/NOTES.TXT

raw
1 VMS Notes for Info-ZIP Zip 3.0 and UnZip 6.0
2 ============================================
3
4 This document describes some VMS-specific behavior and implementation
5 details of the Info-ZIP Zip and UnZip programs.
6
7 Last modified: 2008-04-10.
8
9
10 Command-line Case
11 -----------------
12
13 Zip and UnZip now include code which can preserve the case of
14 command-line parameters and options, which obviates quoting upper-case
15 options like "-V" or "-Z". This works on non-VAX systems with a
16 sufficiently recent C RTL, and SET PROCESS /PARSE_STYLE = EXTENDED.
17 (Sufficiently recent here means __CRTL_VER >= 70301000, which includes
18 VMS V7.3-1 with a C Run Time Library ECO, or V7.3-2 or newer.) This
19 code uses the decc$feature_set_value() function to enable the
20 DECC$ARGV_PARSE_STYLE feature. There is a small range of C RTL versions
21 where this function is unavailable, but where manually setting the
22 logical name DECC$ARGV_PARSE_STYLE to "ENABLE" will work. HELP CRTL
23 leads to some additional information on these features.
24
25
26 File Name Case (ODS5)
27 ---------------------
28
29 In general, Zip 3.0 and UnZip 6.0 should handle file name case (and
30 extended file names) in reasonable ways on ODS5 disks.
31
32 Zip offers a variety of "-C" (/PRESERVE_CASE) options to control how
33 case is handled when adding files to an archive. The default settings
34 ("-C2-", /PRESERVE_CASE = NOODS2, down-case ODS2 file names; "-C5",
35 /PRESERVE_CASE = ODS5, preserve case of ODS5 file names) should be
36 consistent with previous Zip versions for files on ODS2 disks, and
37 reasonable for files on ODS5 disks.
38
39 UnZip should preserve case when it extracts to an ODS5 destination
40 disk (unless "-2" (/ODS2) is specified). (Note that previous UnZip
41 versions, including version 5.52, did not properly preserve case for
42 directories, which were always up-cased.)
43
44 The Zip and UnZip builders should work properly on ODS2 and ODS5
45 disks, with old (pre-ODS5) and new (case-conscious) versions of MMS (or
46 MMK). All testing was done with SET PROCESS /CASE_LOOKUP = BLIND.
47 Various problems may be expected with /CASE_LOOKUP = SENSITIVE.
48
49 For consistency, the builders should always create product files
50 (.OBJ, .EXE, .HLB, and so on) with upper-case names, whether the build
51 is done on an ODS2 or ODS5 disk. Note, however, that in a world with
52 both ODS2 and ODS5 disks, and old and new Zip and UnZip versions, it's
53 possible to encounter lower-case product file names. For example, a VMS
54 binary kit could be created on an ODS2 disk, and a Zip archive created
55 from that (using Zip 2.x, or Zip 3.x with default settings). Such a Zip
56 archive would contain down-cased names for those product files, and
57 those lower-case names would then normally be preserved when UnZip was
58 used to extract that archive onto an ODS5 destination. Normally, things
59 will work regardless of such case changes, but there may be some
60 untested combinations of unexpected name cases and quirky MMS (or MMK)
61 behavior, where something goes wrong. Complaints are always welcome,
62 but it may not be possible to get everything to work as expected with
63 every version of VMS, MMS (or MMK), Zip, and UnZip, on every file
64 system.
65
66 It might help matters if _all_ VMS binary kits were produced on ODS5
67 disks, and packaged using (case-preserving) Zip version 3.x, but this
68 would certainly be different from the way things have been done before,
69 and maintaining control over this process is essentially impossible.
70
71
72 Symbolic Links (ODS5)
73 ---------------------
74
75 VMS V8.3 offers support for symbolic links (symlinks) on ODS5 disks.
76 In previous Zip and UnZip versions, the generic code for symlinks was
77 disabled, and there was no VMS-specific code for symlinks. Now, by
78 default, Zip and UnZip attempt to support symlinks wherever the C
79 headers and C run-time library include the functions needed for symlink
80 support. This means non-VAX systems with __CRTL_VER >= 70301000, so
81 this includes VMS V7.3-1 and up, and thus symlink-capable Zip and UnZip
82 programs may be built on systems which do not themselves offer symlink
83 support. (Various run-time failures may be expected if symlinks are
84 encountered on pre-V8.3 systems, either in a file system or in a Zip
85 archive.)
86
87 Symlink support can be disabled at build-time, if desired, by
88 defining the C macro NO_SYMLINKS. (See comments in the builder
89 regarding LOCAL_UNZIP or LOCAL_ZIP, as appropriate.) For example, using
90 MMS to build UnZip:
91
92 MMS /DESCRIP = [.VMS] /MACRO = ("LOCAL_UNZIP=NO_SYMLINKS=1")
93
94 or, using the command procedure to build Zip:
95
96 LOCAL_ZIP == "NO_SYMLINKS=1"
97 @ [.VMS]BUILD_ZIP.COM
98 DELETE /SYMBOL /GLOBAL LOCAL_ZIP
99
100 The Zip "-v" (/VERBOSE) report should include SYMLINK_SUPPORT in its
101 list of "Zip special compilation options" if Zip was built with symlink
102 support. Currently, UnZip is less convenient, but searching the UnZip
103 executable (or EXTRACT.OBJ) for "symlink" should fail if symlink support
104 is missing (or succeed if it's present):
105
106 $ SEARCH /NOOUTPUT UNZIP.EXE SYMLINK ! No symlink support.
107 %SEARCH-I-NOMATCHES, no strings matched
108 or:
109 $ SEARCH /NOOUTPUT EXTRACT.OBJ SYMLINK ! Symlink support.
110 $ SHOW SYMBOL $STATUS
111 $STATUS == "%X00000001"
112
113
114 File I/O Performance
115 --------------------
116
117 When compiled using DEC/Compaq/HP C (not GNU C or VAX C), the Zip and
118 UnZip file I/O code now includes access callback functions which are
119 used to try to set some RMS parameters to non-default values, with the
120 intention of improving file I/O speed. This affects reading an archive
121 file in UnZip and writing one in Zip. (Reading and writing the
122 individual data files are handled in more exotic ways, making these
123 parameters less important for them.)
124
125 Currently, the built-in default parameters enable read-ahead and
126 write-behind, using a multi-buffer count of 2, and a multi-block count
127 of 127 (the maximum). For writing the archive, the default extend
128 quantity is 16384 blocks (8MB), with truncation enabled. This
129 combination is believed to be, at worst, fairly harmless for most
130 situations, and, in most cases, to provide a substantial speed
131 improvement, especially with large archives.
132
133 This code allows SET RMS_DEFAULT parameters to override the built-in
134 default values. On some old VMS versions, sys$getjpi() can not provide
135 the SET RMS_DEFAULT values, and in this situation, the callback function
136 will not try to use its improved parameter values. Users on such old
137 VMS versions who seek improved I/O speed may wish to bypass this check,
138 which requires changing the code in the get_rms_defaults() function in
139 [.VMS]VMS.C. The "-vv" (/VERBOSE = MORE) option on both programs
140 enables diagnostic messages which show the operation of the callback
141 function. A message showing a failure status from sys$getjpi()
142 indicates this problem.
143
144 Sample results (UnZip shown, Zip similar):
145
146 VMS VAX V5.4, VAX C. Callback code disabled, no messages:
147 WIMP $ unzip -tvv TESTMAKE.ZIP
148 Archive: SYS$SYSDEVICE:[UTILITY.SOURCE.ZIP.UNZIP60C]TESTMAKE.ZIP;1
149 [...]
150
151 VMS VAX V5.5-2, DEC C. SYS$GETJPI() fails (%SYSTEM-F-BADPARAM):
152 WEAK $ unzip -tvv TESTMAKE.ZIP
153 Get RMS defaults. getjpi sts = %x00000014.
154 Archive: DUA1:[UTILITY.SOURCE.ZIP.UNZIP60C]TESTMAKE.ZIP;1
155 [...]
156
157 VMS VAX V7.3, DEC/Compaq C. Callback code works:
158 WUSS $ unzip -tvv TESTMAKE.ZIP
159 Get RMS defaults. getjpi sts = %x00000001.
160 Default: deq = 0, mbc = 0, mbf = 0.
161 Open callback. ID = 1, deq = 16384, mbc = 127, mbf = 2.
162 Archive: ALP$DKA0:[UTILITY.SOURCE.ZIP.UNZIP60C]TESTMAKE.ZIP;1
163 [...]
164
165 VMSV5.5-2 is too old. V7.3 is new enough. Anyone with more precise
166 information is invited to contribute it.
167
168 Users who find other parameter sets more beneficial, or who find
169 particular problems with this set are welcome to comment.
170
171 In this version, as in previous versions, when UnZip expands a -V
172 archive, it allocates the entire extent of a data file before writing
173 any of its data. In some previous versions, this could cause the
174 destination disk to be locked for a considerable time (minutes), if
175 highwater marking was enabled on that disk. Now, the FAB SQO
176 ("sequential access only") flag (or equivalent) is set, which prevents
177 this troublesome disk locking.
178
179 In some previous versions, when UnZip expanded a non-V archive, it
180 did no pre-allocation, and used the default extension quantity. This
181 could slow file creation significantly for large files. Now, space for
182 extracted files is pre-allocated, and the same SQO ("sequential access
183 only") flag is set, as with a -V archive.
184
185
186 Changes to the "-V" (/VMS) Option
187 ---------------------------------
188
189 The intent of the "-V" (/VMS) option was to store VMS file attributes
190 in a Zip archive, allowing UnZip to extract an exact copy of a file on a
191 VMS system, including all its VMS attributes.
192
193 In Zip before version 2.31, using the "-V" (/VMS) option created an
194 archive which usually contained data from beyond the EOF (End-of-File)
195 marker in a data file, but generally not all the disk blocks allocated
196 for the file. When extracted on a VMS system, the result was usually
197 acceptable (because the data from beyond the EOF marker were usually
198 ignored). However, when extracted on a non-VMS system, the resulting
199 file was usually corrupted by being NUL-padded to the next larger 16KB
200 multiple in size.
201
202 Now (Zip 2.31 and later), with "-V" (/VMS), Zip truncates a data file
203 at EOF, and portable-format files (Stream_LF, fixed-512) should be
204 extracted properly on a non-VMS system. On a VMS system, well-formed
205 files (that is, those with no valid data beyond EOF) should also be
206 restored correctly.
207
208 With the new "-VV" (/VMS = ALL) option, the archive includes all
209 allocated blocks for the file (including those beyond EOF). When
210 extracted on a VMS system, the original file should be reproduced with
211 as much fidelity as possible, but on a non-VMS system, most files will
212 be seen as corrupt because of the data from beyond EOF.
213
214
215 Changes to Program Exit Status Values
216 -------------------------------------
217
218 Zip and UnZip exit with 32-bit VMS status values which are formed
219 from their internal OS-independent status values. In previous program
220 versions, this was done by converting the internal success code (0) into
221 %x00000001 (SS$_NORMAL), and converting the other internal warning and
222 error codes using an artificial control/facility code, 0x7FFF (which
223 includes some reserved bits), and a severity value which was determined
224 according to rules specified in the VMS-specific exit function.
225 Curiously, the internal status codes were left-shifted by 4 bits instead
226 of 3, so all the resulting VMS message codes (bits 13:3) were even.
227
228 Zip and UnZip now have facility names and codes assigned by HP
229 (UnZip: IZ_UNZIP, 1954; Zip: IZ_ZIP, 1955). Now, by default, the
230 programs exit with standard 32-bit VMS status values which differ from
231 the old ones in several ways: The official facility code is used, and
232 the facility-specific bit is set. (For compatibility with older
233 versions, the internal status codes are still left-shifted by 4 bits.
234 This also makes it easier to extract the internal status code from a
235 hexadecimal representation of the VMS status code.) The builders also
236 create non-executable message files (UNZIP_MSG.EXE and ZIP_MSG.EXE) so
237 that, after a suitable SET MESSAGE command, the program messages will be
238 available from DCL. For example:
239
240 $ SET MESSAGE dev:[dir]ZIP_MSG.EXE
241 $ ZIP FRED.ZIP no_such_file
242 zip warning: name not matched: no_such_file
243
244 zip error: Nothing to do!
245 (dev:[dir]FRED.ZIP;)
246
247 ALP $ WRITE SYS$OUTPUT F$MESSAGE( $STATUS)
248 %IZ_ZIP-W-NONE, Nothing to do
249
250 The message files may be copied into SYS$MESSAGE to make them generally
251 available, although this could cause some confusion if multiple versions
252 of the programs are used on the system, and their error message source
253 files differ. Each different destination directory will get its own
254 UNZIP_MSG.EXE or ZIP_MSG.EXE ([.ALPHA], [.ALPHAL], [.VAX], and so on),
255 but all of the same-architecture files are equivalent to each other.
256 That is, on an Alpha system, any of the [.ALPHA*]ZIP_MSG.EXE files could
257 be used; on an IA64 system, any of the [.IA64*]ZIP_MSG.EXE files could
258 be used; and on a VAX system, any of the [.VAX*]ZIP_MSG.EXE files could
259 be used. (Similar for UNZIP_MSG.EXE, of course.)
260
261 If desired, the programs may be built to use the old exit status values
262 by defining a C macro with the old facility value:
263 "CTL_FAC_IZ_UNZIP=0x7FFF" (UnZip) or "CTL_FAC_IZ_ZIP=0x7FFF" (Zip).
264 (See comments in the builder regarding LOCAL_UNZIP or LOCAL_ZIP, as
265 appropriate.) This will maintain compatibility with older program
266 versions, but will make the programs incompatible with the new error
267 message files.
268
269
270 VMS File Attribute Schemes
271 --------------------------
272
273 Zip's "-V" (/VMS) option causes VMS file attributes to be stored in
274 an archive. Since Zip version 2.2 (released in 1996), Zip has, by
275 default, stored VMS file attributes using a scheme ("PK") which is
276 compatible with the one used by PKWARE in their PKZIP product. Before
277 that, a different scheme ("IM") was used. UnZip versions before 5.2
278 support only the older IM scheme, but since UnZip version 5.2, both
279 schemes have been supported by UnZip.
280
281 The IM scheme has not been well tested recently, but it is still
282 available. Some problems were seen when the IM scheme was used with
283 symbolic links on VMS V8.3. Details on how build Zip to use the IM
284 scheme instead of the PK scheme are included in comments in the main
285 builder files. Look for VMS_IM_EXTRA in [.VMS]BUILD_ZIP.COM or IM in
286 [.VMS]DESCRIP.MMS.
287
288 The "special compilation options" section of a "zip -v" ("zip
289 /verbose") report should show either VMS_PK_EXTRA or VMS_IM_EXTRA,
290 according to how Zip was built.
291
292
293 UTC Date-Times
294 --------------
295
296 Zip archives traditionally include local (MS-DOS compatible)
297 date-time information for files. Since Zip version 2.1, it has also
298 been possible to store UTC date-time information in the archive, and
299 since UnZip version 5.2, UnZip has been able to use this UTC date-time
300 information when extracting files.
301
302 On VMS, support in the C run-time environment for UTC became
303 available with VMS V7.0. UTC support in Zip and UnZip is automatically
304 enabled at compile time, if it is available on the system where the code
305 is compiled (__CRTL_VER >= 70000000). It may be disabled at compile
306 time by defining the C macro NO_EF_UT_TIME. Details on how build Zip
307 and UnZip with additional C macros defined are included in comments in
308 the main builder files. Look for LOCAL_[UN]ZIP in
309 [.VMS]BUILD_[UN]ZIP.COM or in [.VMS]DESCRIP.MMS. For example, using MMS
310 to build UnZip:
311
312 MMS /DESCRIP = [.VMS] /MACRO = ("LOCAL_UNZIP=NO_EF_UT_TIME=1")
313
314 or, using the command procedure to build Zip:
315
316 LOCAL_ZIP == "NO_EF_UT_TIME=1"
317 @ [.VMS]BUILD_ZIP.COM
318 DELETE /SYMBOL /GLOBAL LOCAL_ZIP
319
320 The "special compilation options" section of a "zip -v" ("zip
321 /verbose") or "unzip -v" ("unzip /verbose") report should show
322 USE_EF_UT_TIME if the program was built with UTC support.
323
324
325 Building with the LIST option using MMK or MMS
326 ----------------------------------------------
327
328 Currently, building with MMK or MMS using the LIST option (as in
329 "/MACRO = LIST=1") may cause a failure for some old versions of the DEC
330 C compiler. The LIST option currently adds "/show = (all, nomessages)"
331 to the CC command line, and some old DEC C compilers do not support the
332 "nomessages" keyword. When VAX C is used, this keyword is omitted, but
333 the builder does not distinguish between the various DEC/Compaq/HP C
334 versions. The work-arounds are to use BUILD_[UN]ZIP.COM, or edit
335 [.VMS]DESCRIP_SRC.MMS to remove the troublesome keyword.
336
337
338 GNU C
339 -----
340
341 Zip and UnZip have been built using GNU C (VAX) version 2.3, mostly
342 for fun, but serious users are encouraged to report any interest in
343 continuing this activity. The GNU C 2.3 header files were missing some
344 things, including definitions of SEEK_CUR, SEEK_END, and SEEK_SET. The
345 VMS-specific code now expects to find unixio.h and unixlib.h, which were
346 absent from the GNU C 2.3 distribution.
347
348 To work around these difficulties, the Zip and UnZip kits include
349 some emergency replacement unixio.h and unixlib.h files which appear to
350 work for these programs, at least. To install them, use commands like
351 the following:
352
353 COPY [.VMS]UNIXIO_GCC.H GNU_CC_INCLUDE:[000000]UNIXIO.H
354 COPY [.VMS]UNIXLIB_GCC.H GNU_CC_INCLUDE:[000000]UNIXLIB.H
355 SET PROTECTION W:RE GNU_CC_INCLUDE:[000000]UNIXIO.H, UNIXLIB.H
356
357 There may be an error in the GNU C header file ATRDEF.H which can
358 cause Zip to fail, when making a "-V" archive, with a spurious "could
359 not open for reading" error message, followed by more bad behavior. It
360 probably also causes trouble of some kind in UnZip. To check the
361 questionable macro definition, use a command like the following:
362
363 SEARCH GNU_CC_INCLUDE:[000000]ATRDEF.H ATR$S_JOURNAL
364
365 This should show something equivalent to this:
366
367 #define ATR$S_JOURNAL 0x001
368
369 If you see "0x002" (or equivalent) instead of "0x001" (or equivalent),
370 then this value must be corrected in the file before building Zip or
371 UnZip.
372
373 You may also see several warnings from the compiler caused by other
374 defects in the GNU C header files, such as:
375
376 <various>: warning: passing arg 4 of `qsort' from incompatible pointer type
377
378 [...]rab.h:134: warning: unnamed struct/union that defines no instances
379 [...]rab.h:143: warning: unnamed struct/union that defines no instances
380
381 These warnings appear to be harmless.
382