vfs(n) Tcl-only Virtual File Systems vfs(n)NAME
::vfs - Commands and Procedures to create virtual filesystems
SYNOPSIS
package require Tcl 8.4
package require vfs ?1.2.1?
vfs::filesystem info
vfs::filesystem mount
vfs::filesystem unmount
vfs::accessMode mode
vfs::matchDirectories types
vfs::matchFiles types
vfs::matchCorrectTypes types filelist ?inDir?
DESCRIPTION
The ::vfs package provides commands to query, mount and unmount virtual
filesystems, and provides as Tcl libraries some facilities for helping
the writing of new virtual filesystems in Tcl. Once a virtual filesys‐
tem is in place, the standard Tcl file, glob, cd, pwd, open commands,
including all their C APIs in the Tcl library (e.g. Tcl_FSOpenFileChan‐
nel, Tcl_FSMatchInDirectory,...), can be used within the filesystem
(and indeed, properly written extensions such as Tk which may open or
read files will also transparently access the virtual filesystem).
Because all of Tcl's FS activity passes through a single layer, it can
all be intercepted. This package does just that. Notice that this is
quite different to overloading the file command in Tcl. We are actu‐
ally providing vfs replacements for C commands like access, stat. By
implementing just a handful of commands at this low level, we ensure
that all commands at higher levels function irrespective of what is
going on inside the FS layer.
Tcl's filesystem hooks operate on a per-process basis. This means
every Tcl interpreter in the same process/application sees the same
filesystem, including any virtual filesystems.
The package require vfs command should be used to access this library.
It automatically registers the vfs hooks into Tcl's filesystem, and
these will not be removed until Tcl exits (if desired, control over
this could be exposed to Tcl in the future). However, the vfs package
will at that stage not have any new filesystems mounted, so it will
have little effect. Note that package require vfs has two effects.
First of all, when it is issued in any Tcl interpreter it will ensure
the vfs hooks have been registered with Tcl's core just once (and if
any of those interpreters are later deleted, the vfs hooks will still
remain registered - they remain until Tcl exits). The second effect is
to provide the command vfs::filesystem which allows the interpreter to
intercept filesystem commands and handle them with Tcl code in that
interpreter.
There are three somewhat unsupported subcommands of vfs::filesystem,
fullynormalize path, posixerror int, internalerror ?script?, which are
used to normalize a path (including any final symlink), to register a
posix error code with a Tcl error, and to trap/report internal errors
in tclvfs implementations respectively.
vfs::filesystem mount ?-volume? path command
To use a virtual filesystem, it must be 'mounted'. Mounting
involves declaring to the vfs package that any subdirectories of
a given path in the filesystem should be handled by the given
command which should be a Tcl command or procedure in the inter‐
preter in which the vfs::filesystem is executed. If the ?-vol‐
ume? flag is given, the given mount point is also registered
with Tcl as a new volume (like a new drive which will appear in
file volumes). This is useful (and required for reasonable
operation) for mounts like ftp://. For paths mounted inside the
native filesystem, it should of course not be given. The new
filesystem mounts will be observed immediately in all inter‐
preters in the current process. If the interpreter is later
deleted, all mounts which are intercepted by it will be automat‐
ically removed (and will therefore affect the view of the
filesystem seen by all interpreters).
vfs::filesystem unmount path
This unmounts the virtual filesystem which was mounted at path
(hence removing it from Tcl's filesystem), or throws an error if
no filesystem was mounted there.
vfs::filesystem info ?path?
If no arguments are given, this returns a list of all filesys‐
tems mounted (in all interpreters). If a path argument is
given, then the command to be used for that path is returned, or
an error is thrown if no vfs is mounted for that path. There is
currently no facility for examining in which interpreter each
command will be evaluated.
vfs::filesystem fullynormalize path
Performs a full expansion of path, (as per 'file normalize'),
but including following any links in the last element of path.
IMPLEMENTING A TCL ONLY VFS
The vfs package will intercept every filesystem operation which falls
within a given mount point, and pass the operation on to the mount
point's command in the interpreter which registered it. In general this
occurs by the C equivalent of an evaluation like this: eval $command
[list $subcmd $root $relative $actualpath] $args.
Here subcmd may be any of the following: access, createdirectory,
deletefile, fileattributes, matchindirectory, open, removedirectory,
stat, utime. If command takes appropriate action for each of these
cases, a complete, perfect virtual filesystem will be achieved, indis‐
tinguishable to Tcl from the native filesystem. (CAVEATS: right now I
don't expose to Tcl all the permission-related flags of 'glob').
The remaining arguments specify a file path on which to operate (all
commands operate on one of these), and any additional arguments which
may be required to carry out the action. The file path is specified by
three arguments: root is the part of the path which lies outside this
filesystem's mount point, relative is the part of the path which lies
inside this filesytem, and actualpath is the original (unnormalized)
name of the path which was used in the current command wherever it
originated (in Tcl or C). For example, if C:/foo/bar/mount.zip/xxx/yyy
is a path in your filesystem, where mount.zip is a zip archive which
has been mounted (on top of itself) and contains xxx/yyy, and the cur‐
rent working directory is inside xxx, and we evaluate a command like
file exists yyy, then rootwill be C:/foo/bar/mount.zip, relative will
be xxx/yyy, and actualpath will be yyy. The file separator between the
root and relative is omitted.
Note that most filesystem operations will only require the relative
argument to work correctly, but the other arguments are actually
required for correct operation of some subcommands.
Almost all of these commands should either return correctly (i.e. with
a TCL_OK result at the C level) or they should use vfs::filesystem
posixerror to signal the appropriate posix error code. If a Tcl error
is thrown, that should be considered a bug, but it will be interpreted
as an unknown posix error in the filesystem call. The exceptions to
these rules are those filesystem commands which are able to specify a
Tcl error message directly: open (when an interpreter is given),
matchindirectory and fileattributes (for a set or get operation only).
These three commands are allowed to throw any Tcl error message which
will be passed along to the caller, or they may throw a posix error
which will be handled appropriately.
The actual commands are as follows (where r-r-a represents the standard
argument triplet of root, relative and actualpath):
command access r-r-a mode
Return TCL_OK or throw a posix error depending on whether the
given access mode (which is an integer) is compatible with the
file.
command createdirectory r-r-a
Create a directory with the given name. The command can assume
that all sub-directories in the path exist and are valid, and
that the actual desired path does not yet exist (Tcl takes care
of all of that for us).
command deletefile r-r-a
Delete the given file.
command fileattributes r-r-a ?index? ?value?
If neither index nor value is given, then return a list of all
acceptable attribute names. If index is given, but no value,
then retrieve the value of the index'th attribute (counting in
order over the list returned when no argument is given) for the
given file. If a value is also given then set the index'th
attribute of the given file to that value.
command matchindirectory r-r-a pattern types
Return the list of files or directories in the given path (which
is always the name of an existing directory), which match the
pattern and are compatible with the types given. It is very
important that the command correctly handle types requests for
directories only (and files only), because to handle any kind of
recursive globbing, Tcl will actually generate requests for
directory-only matches from the filesystem. See vfs::matchDi‐
rectories below for help.
command open r-r-a mode permissions
For this command, mode is any of "r", "w", "a", "w+", "a+". If
the open involves creating a file, then permissions dictates
what modes to create it with. If the open operation was not
successful, an error should be thrown. If the open operation is
successful, the command should return a list of either one or
two items. The first item (which is obligatory) is the name of
the channel which has been created. The second item, if given,
is a Tcl-callback to be used when the channel is closed, so that
the vfs can clean up as appropriate. This callback will be
evaluated by Tcl just before the channel is closed. The channel
will still exist, and all available data will have been flushed
into it. The callback can, for example, seek to the beginning
of the channel, read its contents and store that contents else‐
where (e.g. compressed or on a remote ftp site, etc). The
return code or any errors returned by the callback are ignored
(if the callback wishes to signal an error, it must do so asyc‐
nhronously, with bgerror, for example), unless the 'internaler‐
ror' script has been specified, when they are passed to that
script for further action.
command removedirectory r-r-a recursive
Delete the given directory. recursive is either 0 or 1. If it
is 1 then even if the directory is non-empty, an attempt should
be made to recursively delete it and its contents. If it is 0
and the directory is non-empty, a posix error (EEXIST) should be
thrown.
command stat r-r-a
Return a list of even length containing field-name and value
pairs for the contents of a stat structure. The order is not
important. The option names are dev (long), ino (long), mode
(int), nlink (long), uid (long), gid (long), size (long), atime
(long), mtime (long), ctime (long), type (string which is either
"directory" or "file"), where the type of each argument is given
in brackets. The procedure should therefore return with some‐
thing like return [list dev 0 type file mtime 1234 ...].
command utime r-r-a actime mtime
Set the access and modification times of the given file (these
are read with 'stat').
VFS HELPERS
The vfslib provides a number of Tcl procedures which can help with
writing command procedures to handle the above possibilities. These
are:
vfs::accessMode mode
converts an integer access mode to a somewhat more preferable
string, any of F X W XW R RX RW.
vfs::matchDirectories types
Does types want directories included?
vfs::matchFiles types
Does types want files included?
vfs::matchCorrectTypes types filelist ?inDir?
Returns that subset of the filelist (which are either absolute
paths or names of files in inDir) which are compatible with the
types given.
VFS DEBUGGING
Use something like this to debug problems in your implementation:
vfs::filesystem internalerror report ; proc report {} { puts stderr
$::errorInfo }
LIMITATIONS
There are very few limitations to the vfs code. One subtlety that you
may encounter is if you mount a case-sensitive virtual filesystem into
a case-insensitive system (e.g. the standard Windows or MacOS fs) and
your code relies on case-insensitivity, then it will not run properly
in the virtual filesystem. Of course if your code relies on case-
insensitivity, it wouldn't run under Tcl on Unix either, so the best
solution is to fix your code!
We may add link and lstat commands in the future to allow virtual
filesystems to support reading and writing links - this is supported by
the C API, but has simply not been exposed to Tcl in this extension,
yet.
The Tcl 'Tcl_FSMatchInDirectory' function takes a variety of type
information in a Tcl_GlobTypeData structure. We currently only expose
the 'type' field from that structure (so the 'permissions' and MacOS
type/creator fields are ignored).
KEYWORDS
vfs, filesystem, file
Vfs 1.2.1 vfs(n)
