[OpenAFS] Scripts To Move Volumes
Russ Allbery
rra@stanford.edu
Fri, 12 Sep 2003 14:49:47 -0700
Lan Zhang <chz@mercury.cs.wayne.edu> writes:
> Does any one have a script to move volumes from one afs file server to
> the other? The script knows how to move the volumes based on the status
> of the volumes to be moved. For example, the script will run vos remove,
> vos addsite and vos release when the volume to be moved is RO; The
> script will run vos move when the volumes to be moved is RW; the script
> will run vos remove, vos move, vos addsite, and vos release when the
> volume to be moved is RW and RO.
Here's the script that we use. I keep meaning to put it on the web
somewhere....
#!/usr/bin/perl -w
$ID = q$Id: mvto,v 1.3 2003/06/11 22:14:03 eagle Exp $;
#
# mvto -- Move an AFS volume from anywhere.
#
# Written by Russ Allbery <rra@stanford.edu>
# Based on code by Neil Crellin <neilc@stanford.edu>
# Copyright 1998, 1999, 2001, 2003
# Board of Trustees, Leland Stanford Jr. University
#
# A smart vos move, or a vos move that assumes you know what you're doing,
# depending on your point of view. mvto parses the output of vos examine
# for a volume to figure out where it is and then puts it where you want it.
# It also supports replicated volumes, and is able to figure out how the
# replication pattern of a volume differs from what you want it to be and
# correct it.
##############################################################################
# Modules and declarations
##############################################################################
use strict;
use vars qw($ID $VOS);
use Date::Parse qw(str2time);
use Getopt::Long qw(GetOptions);
# Find a usable version of vos.
($VOS) = grep { -x $_ } qw(/usr/bin/vos /usr/pubsw/bin/vos);
##############################################################################
# Overrides
##############################################################################
# Override system with something that checks return status.
use subs qw(system);
sub system {
CORE::system (@_) == 0
or die "$0: @_ failed (status " . ($? >> 8) . ")\n";
}
##############################################################################
# AFS information
##############################################################################
# Given a server name and a partition, fully qualify both and return them as a
# list of ($server, $partition). Accepts - as the partition to pick the least
# loaded partition on that server, or a list of letters or letter ranges to
# pick the least loaded of the partitions on the server from that range.
sub findpartition {
my ($server, $part) = @_;
$server = 'afssvr' . $server if ($server =~ /^\d+$/);
if ($part eq '.' || (length ($part) > 1 && $part =~ /^[a-z-]+$/)) {
$part = 'a-z' if $part eq '.';
open (INFO, "$VOS partinfo $server |") or die "$0: can't fork: $!\n";
my @free;
local $_;
while (<INFO>) {
if (m%^Free space on partition (/vicep[$part]): (\d+) K %) {
push (@free, [ $1, $2 ]);
} elsif (m%^Free space on partition (/vicep.)%) {
next;
} else {
die "$0: vos partinfo said $_\n";
}
}
@free = sort { $$b[1] <=> $$a[1] } @free;
$part = $free[0][0];
} else {
$part =~ s%^(?:/?vicep)?%/vicep%;
}
die "$0: invalid partition $part\n" if ($part !~ m%^/vicep[a-z]$%);
return ($server, $part);
}
# Given a volume name, determines various characteristics of the volume and
# returns them in a hash. 'size' gets the volume size in KB, 'rwserver' and
# 'rwpart' get the server and partition for the read-write volume, 'ro' gets a
# hash of server and partition values for the replicas, 'sites' gets a count
# of the number of sites the volume is replicated on, and 'unreleased' gets a
# boolean value saying whether there are unreleased changes.
sub volinfo {
my ($volume, $checkro) = @_;
my (%results, $rotime, $rwtime);
open (VEX, "$VOS examine $volume |")
or die "$0: can't fork $VOS examine: $!\n";
local $_;
while (<VEX>) {
if (/^\Q$volume\E\s+\d+ (RW|RO|BK)\s+(\d+) K\s+On-line\s*$/) {
die "$0: $volume is $1, not RW\n" unless $1 eq 'RW';
$results{size} = $2;
} elsif (/^\s+server ([^.\s]+)\.\S+ partition (\S+) RW Site\s*/) {
die "$0: saw two RW sites for $volume\n" if $results{rwserver};
$results{rwserver} = $1;
$results{rwpart} = $2;
} elsif (/^\s+server ([^.\s]+)\.\S+ partition (\S+) RO Site\s*/) {
$results{ro}{$1} = $2;
$results{sites}++;
} elsif (/^\s+Last Update (.*)/) {
my $tmp = $1;
$rwtime = str2time($tmp);
}
}
close VEX;
die "$0: unable to parse vos examine $volume\n"
unless ($results{rwserver} && $results{size});
if ($results{sites}) {
open (VEX, "$VOS examine $volume.readonly |")
or die "$0: can't fork $VOS examine for readonly: $!\n";
while (<VEX>) {
if (/^\s+Last Update (.*)/) {
my $tmp = $1;
$rotime = str2time($tmp);
}
}
close VEX;
if ($rwtime > $rotime) { $results{unreleased} = 1 }
}
return %results;
}
##############################################################################
# Implementation
##############################################################################
# Usage message, in case the command line syntax is wrong.
sub usage { die "Usage: $0 <vol> <server> <part> [<server> <part> ...]\n" }
# Parse our options.
my $fullpath = $0;
$0 =~ s%.*/%%;
my ($help, $force, $justprint, $version);
Getopt::Long::config ('bundling', 'no_ignore_case');
GetOptions ('help|h' => \$help,
'dry-run|just-print|n' => \$justprint,
'force|f' => \$force,
'version|v' => \$version) or exit 1;
if ($help) {
print "Feeding myself to perldoc, please wait....\n";
exec ('perldoc', '-t', $fullpath);
} elsif ($version) {
my $version = join (' ', (split (' ', $ID))[1..3]);
$version =~ s/,v\b//;
$version =~ s/(\S+)$/($1)/;
die $version, "\n";
}
# Volume name is always the first argument. Pull it off and figure out where
# this volume is.
usage if (@ARGV < 3);
my $volume = shift;
usage if (@ARGV % 2 != 0);
my %volume = volinfo $volume;
print "\n$volume on $volume{rwserver} $volume{rwpart} ($volume{size} KB)";
print " with unreleased changes" if $volume{unreleased};
print "\n";
for (keys %{ $volume{ro} }) { print " replica on $_ $volume{ro}{$_}\n" }
print "\n";
die "$0: replica sites given and $volume is unreplicated\n"
if (!$volume{sites} && @ARGV > 2);
# Now parse the first argument, which says what server to put the RW on.
my (@commands, $needrelease, $needspace);
my ($toserver, $topart) = findpartition (shift, shift);
my %ro = %{ $volume{ro} };
if ($volume{rwserver} eq $toserver) {
print "$volume is already on $volume{rwserver}\n";
$needspace++;
if ($ro{$volume{rwserver}}) {
delete $ro{$volume{rwserver}};
} elsif ($volume{sites}) {
push (@commands, [ qw(vos addsite), $volume{rwserver},
$volume{rwpart}, $volume ]);
$needrelease++;
}
} else {
push (@commands, [ qw(vos move -v), $volume, $volume{rwserver},
$volume{rwpart}, $toserver, $topart ]);
push (@commands, [ qw(vos backup), $volume ]);
if ($volume{sites} && (!$ro{$toserver} || $ro{$toserver} ne $topart)) {
push (@commands, [ qw(vos remove), $toserver, $ro{$toserver},
"$volume.readonly" ]) if $ro{$toserver};
push (@commands, [ qw(vos addsite), $toserver, $topart, $volume ]);
$needrelease++;
}
}
# Grab the remaining arguments, which give the desired replication. Build a
# hash table of the servers the volume is already replicated on, and check the
# desired state against the current state. Consider having a replica on the
# right destination server to be good enough and don't move replicas on the
# server.
my $replicas = 1;
while (@ARGV) {
my ($server, $part) = findpartition (splice (@ARGV, 0, 2));
if ($ro{$server}) {
print "$volume is already on $server\n";
$needspace++;
delete $ro{$server};
} else {
push (@commands, [ qw(vos addsite), $server, $part, $volume ]);
$needrelease++;
}
$replicas++;
}
if ($volume{sites}) {
if ($replicas < $volume{sites}) {
die "$0: would reduce replication from $volume{sites} sites to"
. " $replicas sites\n";
} elsif ($replicas > $volume{sites}) {
die "$0: would increase replication from $volume{sites} sites to"
. " $replicas sites\n";
}
}
# Refuse to release a volume with unreleased changes unless --force was given
# on the command line.
die "$0: volume has unreleased changes, use --force to force a release\n"
if ($volume{unreleased} && $volume{sites} && $needrelease && !$force);
# Do the volume release, if necessary.
if ($volume{sites} && $needrelease) {
push (@commands, [ qw(vos release -v -f), $volume ]);
}
# Now clean up any remaining unwanted replicas.
for (keys %ro) {
push (@commands, [ qw(vos remove), $_, $ro{$_}, "$volume.readonly" ]);
}
# Okay, run our commands.
print "\n" if $needspace;
for (@commands) {
print "@$_\n";
unless ($justprint) { system (@$_) }
}
__END__
##############################################################################
# Documentation
##############################################################################
=head1 NAME
mvto - Move an AFS volume from anywhere
=head1 SYNOPSIS
mvto [B<-nf>] I<volume> I<server> I<partition> [I<server> I<partition> ...]
=head1 DESCRIPTION
B<mvto> is a smart B<vos move> that uses B<vos examine> to determine where
the volume is currently located and how it is currently replicated. It
essentially allows the user to say "make the volume distribution look like
this" and it will make the changes necessary to do that. For replicated
volumes, the first server/partition pair is taken as the location of the
read/write and every additional server/partition pair is taken as a site to
put a replica. (One replica is automatically put on the same partition as
the read/write, if any replication sites are specified, so the result will
be a replication site on every server/partition pair given.)
If the volume is already located on the same server as the destination, even
if it's on a different partition, this is considered by B<mvto> to be "good
enough" and the volume will not be moved. Similarly with replication sites,
if there is already a replication site on that server (even on a different
partition), that replication site won't be moved or removed and will be
counted as one of the replication sites for the volume. To move volumes
between partitions on the same server requires more finesse and special
cases since one cannot have two replicas on the same server, so it should be
done by hand.
If any details about the replication of the volume had to be changed (and
the volume is replicated), the volume will be released. In practice, this
means that unless the volume is already located on all of the same servers
given on the command line, already has a replication site on the same
partition as the read/write, and already has the right number of replication
sites, the volume will be released if replicated.
If the volume needs to be released, B<mvto> will check to see if it has any
unreleased changes. If so, it will refuse to perform any operations unless
the B<--force> (or B<-f>) command-line option is given to avoid accidentally
releasing volumes with unreleased changes.
If the read/write volume has to be moved, B<mvto> will run B<vos backup> on
the volume after the move (since volume moves have a side effect of deleting
the backup volume). Don't use this program on volumes that shouldn't have a
backup volume.
B<mvto> will neither increase nor decrease the replication of a volume. If
the number of replication sites should be changed, or if the volume is
currently unreplicated and should be replicated, this should be done by hand
before running mvto.
AFS servers may be specified as just a number; all numeric server names will
have C<afssvr> prepended to them.
Partitions may be specified as a simple letter, as C<vicepX>, or as
C</vicepX>. More than 26 partitions on one server is not supported.
Partitions may also be specified as C<.>, in which case the parition on that
server with the most free space according to B<vos partinfo> is chosen, or
as a string of letters and letter ranges such as C<ace-gm>, in which case
the partition of the set specified with the most free space is chosen. (In
this example, the set is /vicepa, /vicepc, /vicepe through /vicepg, or
/vicepm on the given sever.)
B<mvto> passes the verbose flag to most B<vos> commands it runs, and passes
the B<-f> flag to B<vos release>.
=head1 OPTIONS
=over 4
=item B<-h>, B<--help>
Print out this documentation (which is done simply by feeding the script to
C<perldoc -t>).
=item B<-v>, B<--version>
Print out the version of B<mvto> and exit.
=item B<-f>, B<--force>
Release a volume if a release is required, even if that volume has
unreleased changes. Without this flag, B<mvto> will refuse to release a
volume that has unreleased changes.
=item B<-n>, B<--dry-run>, B<--just-print>
Print out volume status information and the commands that B<mvto> would run,
but don't execute any of them.
=back
=head1 EXAMPLES
Move the volume ls.trip.windlord, wherever it is, to afssvr3 /vicepd:
mvto ls.trip.windlord afssvr3 /vicepd
Move the volume ls to afssvr5 /vicepa, with replication sites on that same
partition, on afssvr6 /vicepk, and on afssvr10 /vicepb:
mvto ls 5 a 6 k 10 b
Move the volume pubsw to the partition on afssvr10 with the most free
space, with one replication site on afssvr11 on whichever partition of the
first three has the most free space. This volume will be released even if
it has unreleased changes.
mvto pubsw 10 . 11 a-c
=head1 SEE ALSO
vos(1), vos_addsite(1), vos_backup(1), vos_examine(1), vos_move(1),
vos_release(1), vos_remove(1)
=head1 AUTHOR
Russ Allbery <rra@stanford.edu>, based on a much simpler script by Neil
Crellin <neilc@stanford.edu> that only handled unreplicated volumes.
=cut
--
Russ Allbery (rra@stanford.edu) <http://www.eyrie.org/~eagle/>