AVFS API Overview Frederik Eaton (frederik@ugcs.caltech.edu) 2003/3/26 Writing a module There are several different ways to write a new filesystem module for avfs. If you only need basic functionality and you don't care about efficiency, you might want to look at extfs. extfs allows you to implement simple filesystem modules as short, standalone scripts. It is documented in extfs/README.extfs. If extfs isn't good enough, then you can use the low-level C API, or one of the helper libraries built on top of it. There are currently four such libraries: 'filter', for modules which act on single files (see ubzip2.c); 'archive', for archive-style modules (see uar.c); 'remote' for things like ssh, rsh, and ftp (see rsh.c); and 'state' which exports a procfs-like "get/set" filesystem (see ftp_init_ctl in ftp.c). An example of a module built directly on the low-level API is 'volatile', in modules/volatile.c. The final source of information about each of the different interfaces is always the avfs source code. This document tries to give a high-level overview which should help make the code more understandable. The low-level interface Data structures Let's start with ventry and vfile. These are somewhat analogous to the Linux kernel's 'inode' and 'file' structures, respectively. struct ventry { void *data; struct avmount *mnt; }; struct vfile { void *data; struct avmount *mnt; int flags; int ptr; avmutex lock; }; A ventry just represents a path in the file system. You can convert from a path to a ventry using av_get_ventry (internal.h). In order to read or write to the corresponding file, one must obtain a vfile by "opening" the ventry with av_open (oper.h). This calls the 'open' method of the appropriate filesystem module. This method, along with many others, is stored as a function pointer in an object of type 'struct avfs'. 'struct avfs' is the data structure we use to define a module, something like the Linux kernel's super_operations, inode_operations, and file_operations put together: /* from src/avfs.h */ struct avfs { /* private */ struct vmodule *module; avmutex lock; avino_t inoctr; /* read-only: */ char *name; struct ext_info *exts; void *data; int version; int flags; avdev_t dev; void (*destroy) (struct avfs *avfs); int (*lookup) (ventry *ve, const char *name, void **retp); void (*putent) (ventry *ve); int (*copyent) (ventry *ve, void **retp); int (*getpath) (ventry *ve, char **retp); int (*access) (ventry *ve, int amode); int (*readlink)(ventry *ve, char **bufp); int (*symlink) (const char *path, ventry *newve); int (*unlink) (ventry *ve); int (*rmdir) (ventry *ve); int (*mknod) (ventry *ve, avmode_t mode, avdev_t dev); int (*mkdir) (ventry *ve, avmode_t mode); int (*rename) (ventry *ve, ventry *newve); int (*link) (ventry *ve, ventry *newve); int (*open) (ventry *ve, int flags, avmode_t mode, void **retp); int (*close) (vfile *vf); avssize_t (*read) (vfile *vf, char *buf, avsize_t nbyte); avssize_t (*write) (vfile *vf, const char *buf, avsize_t nbyte); int (*readdir) (vfile *vf, struct avdirent *buf); int (*getattr) (vfile *vf, struct avstat *buf, int attrmask); int (*setattr) (vfile *vf, struct avstat *buf, int attrmask); int (*truncate)(vfile *vf, avoff_t length); avoff_t (*lseek) (vfile *vf, avoff_t offset, int whence); }; Notice that each of the three structures has a "void *data" member, which is used to store filesystem-specific information. Functions in 'struct avfs' with a 'void **retp' parameter use 'retp' to return one of these pointers - for lookup(), copyent(), and getpath(), *retp will be used as the 'data' member of a new ventry; for open() it is put into a vfile.data. Most operations listed above have a corresponding wrapper function prefixed with "av_" in 'oper.h' (e.g. av_open for open, av_lseek for lseek); when accessing files from other modules, the module writer should always use these functions rather than trying to look up the method and call it directly. Exceptions are the entry management operations putent, copyent, getpath which can be accessed through av_copy_ventry, av_free_ventry, and av_generate_path respectively (in avfs.h), and the lookup operation which is called recursively by av_get_ventry (in internal.h). The Object System Avfs employs a simple system for garbage collection through reference counting. void *av_new_obj(avsize_t nbyte, void (*destr)(void *)); void av_ref_obj(void *obj); void av_unref_obj(void *obj); #define AV_NEW_OBJ(ptr, destr) \ ptr = av_new_obj(sizeof(*(ptr)), (void (*)(void *)) destr) av_new_obj creates a new object with the specified size and destructor, and with a reference count of 1. The returned pointer addresses a free region at least nbytes long. The reference count and destructor are stored immediately before this region in memory. av_ref_obj increments the reference counter associated with the given block; av_unref_obj decrements the reference counter and calls the destructor function when the count reaches 0. Examples of how these functions are used can be found throughout the code. The common task of allocating memory for a fixed-size data structure can be abbreviated by calling the AV_NEW_OBJ macro with the pointer to be assigned to and a destructor function; because of this, the function av_new_obj is almost never called directly. Of the data structures which have been introduced so far, only struct vfile and struct avfs are allocated as reference-counted objects; ventry and avmount are allocated and freed using av_* versions of malloc and free (see avfs.h). Because the avfs core is used in long-running processes such as avfscoda and avfs_server, care must be taken to avoid leaking memory in a module by forgetting to decrement a reference or free a pointer. Utility libraries Avfs uses a few internal utility libraries throughout the code. One example is a facility for organizing information into a named hierarchy, called Namespace Namespace is pretty straightforward. I'll skip over the details since they belong in a separate document (which doesn't exist as of this moment, but reading the code is easy). The two important data types are 'struct entry' and 'struct namespace', both defined opaquely in "namespace.h". An empty namespace is created with struct namespace *av_namespace_new(); 'struct entry' has a 'void *data' member which can be accessed with void av_namespace_set(struct entry *ent, void *data); void *av_namespace_get(struct entry *ent); To look up an entry, use struct entry *av_namespace_resolve(struct namespace *ns, const char *path); or struct entry *av_namespace_lookup(struct namespace *ns, struct entry *parent, const char *name); The first function looks up an absolute path, starting at the root of the namespace. The path components are separated with '/'. The second function looks up an child entry in the 'parent' entry. If 'name' is null, then it returns the parent of 'parent'. When a path or name doesn't exist in the namespace, both functions create a new entry and return that. As with most functions returning objects, these return an entry with an incremented reference count - you are responsible for decrementing the reference count when you are done with the returned entry. Nothing else inside namespace holds references to entries except an entry's own children, so when you unreference a childless entry it disappears. Cache This is another semi-important general-use facility in avfs which requires some explanation. It manages a cache of files in /tmp. Saying "manages" is a bit of a stretch, though, because all the responsibility for creating and deleting the files is placed on the user. The only thing that 'cache' does is maintain a list of objects. These are of the kind described above, typically created with AV_NEW_OBJ and carrying a virtual destructor located just before the object memory area. With each object is associated a disk usage number. Two things must hold for objects used in 'cache': they must be unreferenced with av_unref_obj after insertion into the cache (so that 'cache' holds the only reference to them), and the virtual destructor must delete some file on the same filesystem as /tmp, which is the same size as reported by the cache object. So even though the 'cache' module itself never worries about creating or deleting files, each object in its list represents a disk file, and the module can regulate disk usage indirectly by unreferencing (and causing the destruction of) these objects. The 'cache' module keeps track of the disk usage of its own objects in a global variable, and checks the amount of free space on the filesystem containing /tmp whenever the size of a cache object is changed or when other avfs modules call av_cache_diskfull(). Whenever the amount of free space drops below a certain level, or the space used by the cache objects grows above a certain level, 'cache' starts deleting objects from its list in least-recently-used order. (These levels are configurable through "#avfsstat/cache") Here is the API (from cache.h): struct cacheobj *av_cacheobj_new(void *obj, const char *name); void *av_cacheobj_get(struct cacheobj *cobj); void av_cacheobj_setsize(struct cacheobj *cobj, avoff_t diskusage); void av_cache_checkspace(); void av_cache_diskfull(); Note that the 'cacheobj' object returned by av_cacheobj_new is a standard avfs object; as per the convention, it is returned with a 1 reference count and should only be unreferenced when it is no longer needed. This will cause the stored "void* obj" to be deleted. See remote.c for an example usage - remnode holds a reference to a cacheobj (remnode.file); in rem_get_file, av_cacheobj_get is called to retrieve a remfile object. At this point the returned object will be null if it has been purged, in which case it must be recreated and reinserted into the cache. Higher-level filesystem libraries Here I'll touch briefly on four of the higher-level libraries, which can simplify implementation of some of the more common types of filesystems. State This is the simplest library. It is similar to procfs in linux. To use it, you create a namespace with av_state_new and populate that namespace with 'statefile' objects, one for each virtual file you want to export. Each 'statefile' object has a pair of 'get' and 'set' functions which are called when data is read and written, respectively, from the corresponding virtual file. int av_state_new(struct vmodule *module, const char *name, struct namespace **resp, struct avfs **avfsp); struct statefile { void *data; int (*get) (struct entry *ent, const char *param, char **resp); int (*set) (struct entry *ent, const char *param, const char *val); }; The 'ent' argument is the entry for the file in your original namespace. XXX what is param? 'val' is the value to set; 'resp' holds the value retrieved by get(). There is also a shortcut for creating files in /#avfsstat (from internal.h): void av_avfsstat_register(const char *path, struct statefile *func); Filter 'filter' lets you create modules which provide access to some translation of the data in a file. It is currently only used for decompression and compression in the ugzip, gz, ubzip2, bz2, and uz modules, but it could be used for simple decryption/encryption and other things. Create a module with the following function: int av_init_filt(struct vmodule *module, int version, const char *name, const char *prog[], const char *revprog[], struct ext_info *exts, struct avfs **resp); 'name' is the name of the module. The translation is carried out by an external command, which is specified in the 'prog' and 'revprog' arguments. 'revprog' is needed to allow writing to the virtual file, and should perform the reverse operation of 'prog'. See gz.c for an example invocation. (XXX: say more about filter implementation. filtprog, serialfile) Archive Used in extfs, uar, urar, utar, and uzip. Provides access to 'archive' formats; the result is a virtual directory hierarchy. I'll just give an overview here because I don't understand much of the structure of the code. For details, see e.g. uzip.c. To instantiate an archive module, use av_archive_init. This fills in most of a 'struct avfs', including all the methods. The avfs.data member points to a 'struct archparams', which you are supposed to initialize yourself. The methods of this structure define your module. They are called by the archive library's own 'struct avfs' methods. struct archparams { void *data; int flags; int (*parse) (void *data, ventry *ent, struct archive *arch); int (*open) (ventry *ve, struct archfile *fil); int (*close) (struct archfile *fil); avssize_t (*read) (vfile *vf, char *buf, avsize_t nbyte); void (*release) (struct archive *arch, struct archnode *nod); }; The open, close, read, and release methods are optional. Only the parse method is required. 'parse' is called when an archive is first used and creates a list of all of the file entries in the archive. These are stored internally in a 'namespace' object in 'struct archive'. New entries are added by obtaining a 'struct entry' with av_arch_create() or av_arch_resolve() (archive.h, archutil.c) and then associating with it a node of type 'struct archnode' with av_arch_new_node(). Among other things, the 'archnode' data structure holds basic stat flags and permissions for the file, the byte offset of the file data within the archive, and a 'void* data' member which can be used to point to a module-specific data structure. This should be filled in by 'parse'. 'void *data' is the struct archparams.data member (for your own use); 'ventry *ent' is the location of the archive to be parsed; and 'arch' is the (opaque) main archive data structure, which is needed by av_arch_create et al. 'read' reads data from a file. The vfile.data member is an archfile: struct archfile { vfile *basefile; struct archive *arch; struct archnode *nod; struct entry *ent; /* Only for readdir */ struct entry *curr; int currn; void *data; }; If the file is stored contiguously starting at its offset in the archive, then av_arch_read(), the default value for archparams.read, will suffice. The archive library can figure out what to do since the file offset has already been stored in the archnode by 'parse'. But if the file is compressed (see uzip.c) or sparse (see utar.c), then it will need special handling in your module, and you must define a custom 'read' method. 'open' and 'close' can be defined if you need to allocate resources or special data structures in preparation for reading a file. 'open' is called with an archfile after all its fields but 'data' have been filled in, at the very end of the open sequence; 'close' is called at the beginning of the close sequence before anything has been deallocated. Examples can be found in uzip.c. 'release' is called when an archive is no longer being used. It should be defined if the 'parse' method allocated resources which need to be freed. Remote The remote library is used by the ssh (rsh.c), rsh, ftp, dav, and floppy modules. It differs from 'archive' in that the directory tree is read one directory at a time (rather than all at once), and files are read completely if at all (rather than allowing random access). A remote module is defined with the following structure: struct remote { void *data; char *name; int flags; int (*list) (struct remote *rem, struct remdirlist *dl); int (*get) (struct remote *rem, struct remgetparam *gp); int (*wait) (struct remote *rem, void *data, avoff_t end); void (*destroy) (struct remote *rem); }; This should be passed to av_remote_init, which will give you a 'struct avfs*': int av_remote_init(struct vmodule *module, struct remote *rem, struct avfs **resp); As always, remote.data is for your own use. 'name' is the name of your module. XXX define 'flags'. The rest of the fields are methods: list: obtain a directory listing. It should call av_remote_add to add all the entries of dl->hostpath.path on dl->hostpath.host to the remdirlist object. get: start transferring a file. Your implementation should start copying a remote file to a temporary file (possibly with av_get_tmpfile) and return the name in gp->localname. You should point gp->data to an object which will let you find out how much has been transferred (so you can implement remote.wait). In addition, this object will be unreferenced when the file is no longer needed, so its destructor should delete the temporary file. wait: block until a certain amount of data has been read. The 'void *data' parameter is from gp->data in remote.get. This function should return 1 if the data is ready, 0 if EOF was reached, and -error if there was an error. destroy: do cleanup. Should call av_free(rem), possibly after freeing module-specific data pointed to by rem.data. THE END