[OpenAFS] intermediate file updates

[Neulinger, Nathan]

One option - if the programs are truly having output "redirected to files",
you might consider providing a simple wrapper that does:

my_grid_job | job_logger filename

job_logger:

	open specified output file
	while (!eof)
		read some lines
		write some lines
		fflush

(now, I didn't think fflush was sufficient, I thought files only got written
on close.) If that's the case, you could also:

		every so often
			close and reopen file in append mode


It's not idea, but it does get you what you need without modifying programs.


-- Nathan

------------------------------------------------------------
Nathan Neulinger                       EMail:  nneul@umr.edu
University of Missouri - Rolla         Phone: (573) 341-4841
Computing Services                       Fax: (573) 341-4216


> -----Original Message-----
> From: Derek Atkins [mailto:warlord@MIT.EDU] 
> Sent: Thursday, January 10, 2002 10:10 AM
> To: Wim.Glassee
> Cc: openafs-info@openafs.org
> Subject: Re: [OpenAFS] intermediate file updates
> 
> 
> Then syncing files is not an option, I'm afraid.  AFS is currently
> write-on-close().  I think it would take a lot of work to change
> that behavior.  If nothing else, your applications would have
> to be changed to cause "the right thing" to happen.
> 
> The reason it works with NFS is that NFS has no client-side caching.
> With AFS, clients write to their local cache and only flush out to
> the server at close().
> 
> -derek
> 
> "Wim.Glassee" <glassee@uia.ua.ac.be> writes:
> 
> > The users only 'use' the programs on the cluster, they 
> didn't write them,
> > so calling fflush() is not an option I'm afraid.
> > 
> > > If they fflush() the output file it should get written back to the
> > > server.
> > >
> > > -derek
> > >
> > > "Wim.Glassee" <glassee@uia.ua.ac.be> writes:
> > >
> > > > Hi,
> > > >
> > > > I'm running Sun Grid Engine on a linux cluster with AFS 
> as distributed
> > > > file system. When a job is started on a cluster node, 
> all output and error
> > > > messages are redirected to files. These files are in 
> the cluster users'
> > > > home directory on AFS. I noticed that these files 
> remain 0-length until
> > > > the job is finished. When I use NFS for this, they are updated
> > > > automatically. Is there any way to get the same 
> behaviour in AFS, 'cause
> > > > the cluster users would like to find out what is 
> happening in their job
> > > > while it's still running.
> > > >
> > > > Thanks,
> > > >
> > > > wim
> > > >
> > > > _______________________________________________
> > > > OpenAFS-info mailing list
> > > > OpenAFS-info@openafs.org
> > > > https://lists.openafs.org/mailman/listinfo/openafs-info
> > >
> > > --
> > >        Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
> > >        Member, MIT Student Information Processing Board  (SIPB)
> > >        URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
> > >        warlord@MIT.EDU                        PGP key available
> > >
> > 
> 
> -- 
>        Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
>        Member, MIT Student Information Processing Board  (SIPB)
>        URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
>        warlord@MIT.EDU                        PGP key available
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>