[OpenAFS] sanity check please.

[Lars Schimmer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Pucky Loucks wrote:
> Thanks so much for you response Lars, I've commented below.
> 
> On 4-Sep-05, at 4:01 AM, Lars Schimmer wrote:
> 
>>
>> | 1) Is this going to become a huge management issue?
>>
>> Depends.
>> If you use some nice scripts, it could be managed easy.
>> At all its just kinda "vos create volume" "fs mkmount path volume" "fs
>> setacl path rights" and "backup volume". And after all, the big  overview
>> ;-) As long as you manage your cell well enough, itīs easy. Please  donīt
>> create one dir with all volumes in it.
>>
> am I correct in that a volume isn't replicated until a "vol release" 
> "fs checkvolumes"? i.e. as I write new images I'll need to decide  when
> I should run those commands?  I might just end up replicating it  at the
> end of the day. for 1/2 day.

Yes, a volume is only replicated with a "vos release" command.
So if you write files to a rw volume, they won't appear in the RO volume
until you replicate the RW to a RO or more RO volumes.

>> | 2) If I end up getting 5 thousand images a day would I want it to 
>> be  in
>> | it's own volume so I could replicate each "day"?
>>
>> What do you mean "images"? CD images? DVD images?
> 
> jpeg images. small ones.  i.e. between 50kb and 300kb. also we'll be 
> storing 15 second video clips and audio clips approx 350kb

Oh, that images ;-)
It's gonna be a rough time with that small files if there are too much
in a volume. But at all 5000*500kb is max. 2.5 GB, seems to be OK.


>> You can create one volume for each day, for each 5000 images. But  think
>> about: 5000 images in one directory is a real mess to search through.
>> And if that are "big" images, the size of this daily volume grows  fast,
>> so a replicate volume takes far more time. Replication is near line
>> speed (as long as there are no large amounts of small files >1kb; but
>> you talk about images, so that files should be larger), but e.g.  100 gig
>> in a volume takes its time to replicate at 100Mbit.
>> I suggest 50 volumes a 100 images/day, e.g. numbered "day.001.01" or
>> else, as you can find the volume easy and you can easy replicate them
>> with a script. And if you distribute these volumes over 5-10 file
>> servers, the replication process span over the network and is faster
>> ended at all. Speed is a question of design.
>>
> this sounds like a good idea.  all of the logic of what file is  located
> where is in my applications database so I won't really need  to search
> from the command line for a file.

Hehe, ok :-)

>> | 3) what's the recommend max size for a volume?
>>
>> I once worked with 250 GB volumes. But to replicate these big  volumes
>> suxx.
>>
> good to know that you used such a large volume, even if it was really 
> slow for replication

Right. But I suggest you to do 1 or 2 or more volumes for each day.

>> And there is limit in files in a volume: max. 64k files with <16  letters
>> allowed in one volume.
>> ~From a mailinglist-entry:
>> The directory structure contains 64K slots.
>> filenames under 16 chars occupy 1 slot.
>> filenames between 16 and 32 chars occupy 2 slots
>> filenames between 33 and 48 chars occupy 3 slots, and on
>>
> this part confuses me, do you have the link to the original topic?

https://lists.openafs.org/pipermail/openafs-info/2002-September/005812.html


>> And please be aware, big file support (>2 GB/File) was introduced in
>> 1.3.7x of OpenAFS.
>>
>> | 4) These files will be served via apache is that an issue? (my
>> | understanding is it's not)
>> As far as openafs.org runs from openafs in a apache, I assume not.
>>
> that's what I thought
> 
> again thanks so much for you help.  I ordered the book Managing AFS 
> from Prentice Hall yesterday. Hope it's a good read. :)

I don't know, I used the net, much time and some hot-testings ;-)

> Pucky
> 

Cya
Lars
- --
- -------------------------------------------------------------
TU Graz, Institut für ComputerGraphik & WissensVisualisierung
Tel.: +43 316 873-5405       E-Mail: l.schimmer@cgv.tugraz.at
PGP-Key-ID: 0xB87A0E03
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDHHKqVguzrLh6DgMRAkz3AKC1Szi9qHj0sg88cj4QRfT9igdr6gCfcHXo
uUnE+inYI5pDla8ioHHHAzI=
=Oo15
-----END PGP SIGNATURE-----