[OpenAFS-devel] Re: Good UI/design practice for showing file transfer status?

[Andrew Deason]

On Sun, 8 Dec 2013 19:23:18 -0600
Troy Benjegerdes <hozer@hozed.org> wrote:

> So one thing I've run into quite frequently is that I write a file on
> some machine on a slow DSL link into a sudirectory of my home
> directory, and then I'll do something like 'ls -lha' in that same
> directory, sorta hoping to get a useful file size report maybe telling
> me the progress of how much data actually got to the server.

'pv' is a handy tool for seeing progress of copying a file around (among
other things), but of course that will proceed in huge unpredictable
jumps.

> What happens is the 'ls' hangs and I can't even ctrl-C it or put it
> in the background.

That should only happen possibly if we're hanging on creating the
directory entry before writing data. Can you '\ls'? That is, ls without
doing something like 'ls -l' or anything else that requires stat()ing?

Is this... Linux? Or something else?

> I imagine something like the dropbox or google drive interfaces that
> show an 'upload progress' indicator would be nice, but what's the
> right way to handle this, and how would I actually get this
> information? 

Well, are you talking about making such an indicator from existing
available information, or altering the client to make more information
available? For the former, you can read this from 'rxdebug', as you've
said to get a rough idea of how quickly you're writing data to the
server.

However, via altering the client, you could have a tool give the kernel
a fid or a path, and the kernel responds with how many bytes have been
written in the current transfer for that fid, how much we will write,
and what server it's going to. If you want to see this for every active
transfer, you could have something else to just list all of the
currently active transfers. From that information, a gui or something
else could display a progress bar and time estimate, etc.

I don't think this would even be that difficult. The transfer is started
in rxfs_storeInit (src/afs/afs_fetchstore.c), and that's where you get
the total length of the transfer. Each time we've transferred some data,
we hit the code right after the ops->write call in afs_GenericStoreProc
(afs_fetchstore.c), or at the end of afs_linux_splice_actor or
afs_linux_read_actor (src/afs/LINUX/osi_fetchstore.c). If you set up and
update a structure for transfer progress in those places, it should be
easy to set up a pioctl to read that info.

Anything like that, of course, would only give the progress for an
individual transfer, which is not necessarily the entire amount of data
an application wants to write to the file. But if you're just 'cp'ing a
single file that is much smaller than the cache, it should be
effectively a single transfer.

It seems like the mariner log maybe would be appropriate for this? But I
don't really know much about how that functions; not sure if that's
useful.

-- 
Andrew Deason
adeason@sinenomine.net