Debian Codegraph @

doc/background

raw
1 BACKGROUND
2 Frederik Eaton ([email protected])
3 2003/3/26
4
5 ORIGINS
6
7 AVFS originated as a shared library which was intended to intercept
8 shared library calls from an executable to glibc, overriding a basic
9 set of the glibc filesystem-related functions to give transparent
10 access to a number of "virtual" filesystem modules. Most of the
11 modules are intended to provide access to files stored in various
12 types of archives, but there are several which are used to access
13 files remotely, via common protocols such as ssh or ftp, and several
14 others which don't fall into either category.
15
16 MC-VFS
17
18 The design of avfs is based on that of mc-vfs, a library which was
19 created to enable Midnight Commander to use virtual filesystem modules
20 like the ones described above. MC-VFS consists of replacements for the
21 basic file operations (namely open, close, read, write, opendir,
22 readdir, closedir, telldir, seekdir, stat, lstat, fstat, chmod, chown,
23 utime, readlink, symlink, link, unlink, rename, chdir, lseek,
24 mknod). The functions provided by mc-vfs are designed to work as
25 drop-in replacements for the corresponding glibc functions, making it
26 easier to modify an existing application to use them.
27
28 This design goes much farther than facilitating better graphical file
29 browsers or more convenient utilities - by linking a file server such
30 as NFS to mc-vfs, one can export an mc-vfs virtual filesystem through
31 the kernel as a standard kernel filesystem, making it available to
32 every user on the system.
33
34 USERVFS
35
36 Pavel Machek's uservfs (originally podfuk) was to my knowledge the
37 first project to take advantage of this idea. Early versions of his
38 program consisted of patches to NFS, but because of problems with
39 kernel deadlock and the way NFS allocates filehandles (you'll have to
40 ask him, I don't understand), he switched to Coda. Neither NFS nor
41 Coda is ideal as the kernel side of a userland virtual file system -
42 NFS because rpc.nfsd is single threaded and has a deadlock problem
43 when used with virtual filesystems, Coda because, among other things,
44 its protocol makes random access to files impossible. (Another
45 project, Jeremy Fitzhardinge's userfs, solved these problems by
46 providing its own kernel filesystem, but the kernel code for that was
47 never brought up to date with newer kernels, and instead of using an
48 API similar to mc-vfs, it had an interface closer to the kernel VFS
49 interface which made it difficult to write new modules for it.)
50
51 AVFS-PRELOAD
52
53 The original version of AVFS, avfs-preload, took a different approach
54 to serving userland filesystems. It was modeled after a program
55 called zlibc, which is supposed to transparently compress and
56 uncompress your files for you. Referred to as "the LD_PRELOAD hack",
57 the method used by zlibc, and later avfs-preload, circumvented the
58 kernel completely, avoiding the shortcomings of existing kernel
59 filesystems and allowing users without root access to install and use
60 the programs themselves.
61
62 To use the original version of avfs, the user would simply set the
63 LD_PRELOAD environment variable to point to the avfs shared
64 library. Then, when he started an executable from that shell, the
65 linker would load libavfs.so before glibc. Libavfs.so overrode a few
66 key glibc functions with avfs' own virtual replacements, modifying the
67 executable's view of the root filesystem to include avfs' virtual
68 modules.
69
70 Since nearly all programs which make system calls do so through glibc,
71 the preload method was compatible with almost everything and worked
72 quite well. Although the preload method is no longer preferred for a
73 number of reasons, it is still possible to compile avfs to use it.
74
75 PROBLEMS WITH THE PRELOAD METHOD
76
77 Why did the preload method fall out of favor? There are two (??) main
78 problems with it. First, for some reason newer versions of glibc
79 (>2.1??) made certain key functions private so they couldn't be
80 overridden by programs like avfs. Since these functions were used
81 internally by many other public glibc functions, fixing avfs to work
82 with the newer glibc would have meant overriding an impractical number
83 of additional glibc calls. (Note: Solaris libc still works with
84 avfs-preload, it is only Linux/glibc which has this problem) The
85 second reason is that, for all the cleverness behind the original
86 LD_PRELOAD hack, there are still some things that are impossible
87 without a real kernel filesystem - the obvious one is making your
88 filesystem visible from within the kernel, but there are other issues,
89 such as keeping virtual file descriptors open over exec(). (because
90 avfs-preload can't do this, it was never able to handle shell
91 redirections such as "gzip < virtual_file"; you had to do "cat
92 virtual_file | gzip" instead) XXX this paragraph is out of date and
93 needs to be updated.
94
95 AVFSCODA
96
97 The current replacement for the preload method is similar to uservfs -
98 called avfscoda, it exports a real kernel filesystem using Coda,
99 reading commands from the kernel Coda module through the /dev/cfs0
100 character device. To use avfscoda you must start avfscoda, mount
101 /dev/cfs0 on /overlay, and install a special kernel module redir.o
102 which redirects failed lookups in your root filesystem to /overlay (so
103 you can pretend that the virtual files you see are actually on your
104 root filesystem together with the other files). Redir has been tested
105 on 2.2.x and 2.4.x kernels, and is unlikely to become incompatible
106 with future versions since it works by patching the system call table,
107 an interface which tends to be fairly static.
108
109 OTHER VFS LIBRARIES
110
111 Because the code for libavfs - the modules and vfs infrastructure -
112 comprise the bulk of the project, we can say that avfs is first and
113 foremost a virtual filesystem library. So we should take the
114 opportunity to mention some of its cousin vfs libraries here:
115
116 mc-vfs
117
118 Also see above. mc-vfs is a predecessor to avfs, and is currently used
119 in mc and gmc. (?? is it being actively developed?) (?? which avfs
120 modules other than extfs are inherited from mc-vfs?)
121
122 gnome-vfs
123
124 - foundation for nautilus (Gnome's new "second generation" file
125 manager), but intended to be used by other gnome apps as well
126 - extensible via shared library modules (like avfs)
127 - unlike avfs, treats 'stackable' modules like 'tar' separately from
128 root modules like 'ftp'; protocols such as ftp, http, smb are
129 specified in URI's, while stackable tar, gzip are invoked with the
130 familiar '#'.
131
132 Example: "http://server/path/to/file.tar#tar/path/to/file.gz#gzip" in
133 gnome-vfs would be
134 "/#http:server|path|to|file.tar#utar/path/to/file.gz#ugz" in avfs
135
136 - unlike avfs, gnome-vfs' API is not intended to be interchangeable
137 with glibc's:
138 - gnome-vfs refers to files with a pointer to GnomeVFSHandle, rather
139 than using integer file descriptors;
140 - gnome-vfs has extra functions implementing asynchronous I/O (by
141 spawning helper threads or processes)
142
143 - last I checked, gnome-vfs had very little documentation
144
145 kio
146
147 Something like a KDE version of gnome-vfs?? Was unable to find any
148 documentation of it.