[OpenAFS] not enough space in target directory

Sun, 27 Oct 2013 03:41:57 -0400

This is a cryptographically signed message in MIME format.

--------------ms020308000003020008010802
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 10/22/2013 12:18 PM, Dyer, Rodney wrote:
> This problem is affecting us as well.
>=20
> I'm not entirely convinced this problem couldn't be fixed with some AFS=
 client trickery.

As you so often like to tell me the application should attempt the
operation to the file system and let it fail if it is going to fail.
The problem with the Explorer Shell is that it tries to be smarter than
the file system.  The Explorer Shell maintains a cache of all of path
components including which volume that path is in, what the volume
attributes are and how much free space is present there.  When that
cache becomes corrupted the explorer shell makes incorrect decisions
about which volume a path refers to and how much free space there is.

To the best of my knowledge there is no ability to reset the explorer
cache other than killing the explorer process.  When the explorer shell
guesses that there is no free space at a particular path it will never
even try the operation.  It assumes it is smarter.

> Our issue is that we have a drive mapped to the cell root.
>=20
> Since our drive is mapped to the root, the application 'thinks' the ent=
ire file system on that 'drive' is one 'volume'.

I'm not sure how you have come to this conclusion.  Lets say that your
cell layout looks like:

 \\afs\college.edu\user\username\windows\profile.V2\

where

  college.exe mp-> root.cell.readonly

  user        mp-> root.user.readonly

  username    mp-> user.username

  windows     =3D directory

  profile.V2  =3D directory

and you have mapped R: to \\afs\college.edu

If the application thinks that the entire tree under R: is one volume
that is not the fault of AFS.  AFS tells the application this is not the
case by setting the Reparse Point File Attribute on each mount point and
by setting the Surrogate Bit in the Reparse Point Tag value.  The RP
Surrogate bit tells the application that the reparse point is a place
holder for some other object which might not be in the same volume.
The application doesn't need to know anything about AFS.  It just needs
to follow the rules for parsing the output of Win32 Directory Queries
and File Information Queries.

If you have an application that is incapable of processing reparse
points you should raise that issue with the application vendor because
that application is going to break on every Windows File System that
supports reparse points.

> Some of our applications try to be 'smart' and check the 'disk free' be=
fore they perform their writes.

The applications do this because the Windows Cache (not the AFS cache)
will permit an application to write into the cache as much data as it
wants until the file system tells the Windows Cache that there is no
more room.  Unfortunately some network file system clients have a very
poor idea of the free space on the server (think SMB/CIFS).  As a result
the application can write hundreds of MBs or GBs of data into the
Windows cache before the Windows cache begins to lazy write the data to
the network.  If it turns out there is no room, the application is told
a network error occurred and the data is lost.  In XP a balloon would
appear with an error message indicating that data was lost.  In Vista
and above the error is written to the Windows Application Event Log but
the end user isn't even told.

So applications have a very good reason to check the free space or use
non-cached writes if they actually care about their data getting to the
file server when the disk is "remote" because aren't all "remote" disks
"SMB/CIFS"?

> (Note: In my history as a programmer, we would just open the file for '=
write' and keep writing until we run out of space with a 'out of space' e=
rror occurs from the 'C' calls.  I think the error was ENOSPC, 'No space =
left on device'. )

I have just explained the problem with this approach.

> I understand that -if- the AFS client were to return an arbitrarily lar=
ge size number for the 'disk free' on them mapped drive, and the applicat=
ion writes into a sub-folder, then that sub-folder may be an AFS volume t=
hat doesn't have that 'disk space' available.  In that case the applicati=
on would lose data when going over quota for the volume.  So some decisio=
n was made to return 'zero bytes available' on all drives mapped to the A=
FS root.

The AFS redirector reports the actual amount of free space for each
volume when asked.  If the AFS redirector is never asked, it never
tells.  Readonly volumes have 0 free space and therefore the reported
free space is 0 bytes.

No one is forcing applications to query the amount of free space on the
volume.  If an application is doing so its because the author felt that
it was necessary to protect the user.

Lying about the size of the partition, the size of the volume and the
amount of free space accessible to the user puts the application at
risk.  By lying the application developer that does the right thing is
being punished in place of the application developer that does the wrong
thing.

> You get around the issue by mapping a drive directly to the AFS volume =
in which you need to write to.

You could also contact the application author and file a bug report to
get the application fixed.

> If you don't care about losing data, one solution is to 'symlink' the c=
ell into the C: drive, as follows...
>=20
>      C:\>mklink /d c:\afs \\afs\root.afs
>=20
>      A DIR of c:\ will then return what is available on your C: drive.

Which is great until the free space on the C: drive drops to nothing
because of shadow volume snapshots, pagefile increases, or other
reasons.   Current free space on my C: disk.

         618,112 bytes free
              99.9 % in use

It was 5GB free about 12 hours ago.  During disk backups the volume
checkpoint and NTFS journal uses up all of the free space.

All you are doing is trading one problem for another.

> We ran the AFS client for years on Windows XP and it always returns 2 T=
B as free on a mapped drive.

This statement is incorrect.   As an SMB server it is not possible for
the individual AFS volumes to be reported as independent devices.
Therefore the size of the one volume reported to the SMB client is that
of the entire AFS name space.  That is effectively infinite but such a
volume cannot be reported.  So the volume size was always reported as
2TB and the free space as 1TB.

Then you would run Microsoft Office and find that sometimes your
powerpoint or word or excel documents would get corrupted because the
application believed the data was written to disk and it was only during
the cache flush that occurs when the last handle is closed that the data
would be pushed to the file server where there would be no space.

There was also the problem that applications couldn't write a file
larger than 1TB and some applications would get very confused when the
sum of all files in a directory tree were larger than the partition size.=

Finally because there was no mechanism of exposing volume boundaries,
when a user tried to delete a tree in the explorer shell it would walk
the entire tree to the leafs and start deleting stuff.   What it should
have done is deleted the mount point if the user had permission and done
nothing outside the current volume if the user didn't have permission.

> This is a VERY annoying problem.

Yes it is and the correct way to get it fixed is to file bug reports
with the application vendors.  Asking the AFS redirector to break the
rules only makes matters worse.

> I wish AFS would prevent you from writing more into a volume than the q=
uota size that you were allocated.  In that case returning ENOSPC, 'No sp=
ace left on device' should be applicable.

The AFS file server does prevent you from writing more into a volume
than the quota size and the AFS redirector queries the file server after
each 1MB of data is written on a given file to update its estimates.
But the AFS redirector cannot return an error on a WriteFile() request
it has not yet been asked to perform.

Whether or not the AFS redirector returns out of space errors is
irrelevant.  The applications that have issues are having issues because
they are attempting to work around issues with other file systems by
querying the free space or the volume attributes and are doing so
incorrectly.

The AFS redirector returns two errors:  Out of space when the partition
is full and out of quota when the volume quota is reached.

Jeffrey Altman

--------------ms020308000003020008010802
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIINITCC
BkIwggUqoAMCAQICEDirAC//rpa3Vv85Wvtd5xswDQYJKoZIhvcNAQEFBQAwgcoxCzAJBgNV
BAYTAlVTMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1
c3QgTmV0d29yazE6MDgGA1UECxMxKGMpIDE5OTkgVmVyaVNpZ24sIEluYy4gLSBGb3IgYXV0
aG9yaXplZCB1c2Ugb25seTFFMEMGA1UEAxM8VmVyaVNpZ24gQ2xhc3MgMSBQdWJsaWMgUHJp
bWFyeSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eSAtIEczMB4XDTExMDkwMTAwMDAwMFoXDTIx
MDgzMTIzNTk1OVowgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3Jh
dGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29u
YSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwg
U3Vic2NyaWJlciBDQSAtIEc0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxuwn
/R1j9DsdisHTHMjIgoa2uEqGkqqBXHLKMA0vnkEiVzAhJZCao/SsKsaIF4ZhchN2LuwDyyeb
jyCAN+DkitpVplAP/LlcI2mJQqG6H6/vDvmkyQrx+DeyxtmSSq5937hEH5u6P4wG/tgjT0hR
I2pghKjuJy9g35byGiqMPI8AzE/L+iCOvDX24fCatgXz/B0/xhR7DtryBeTTgwKmxWlwtKnk
VunbHVz0pjbia7UeKi3cvrvuOgSwMAitX2hsxr0GloiE5+apZC28ODC7iCbDZ2ZmtLR3+cCh
xw5y72bi5bnK4POFdzWY3tQcsP5mceI4y258T0BV65fZqBge7QIDAQABo4ICRDCCAkAwOAYI
KwYBBQUHAQEELDAqMCgGCCsGAQUFBzABhhxodHRwOi8vcGtpLW9jc3AudmVyaXNpZ24uY29t
MBIGA1UdEwEB/wQIMAYBAf8CAQAwbAYDVR0gBGUwYzBhBgtghkgBhvhFAQcXATBSMCYGCCsG
AQUFBwIBFhpodHRwOi8vd3d3LnN5bWF1dGguY29tL2NwczAoBggrBgEFBQcCAjAcGhpodHRw
Oi8vd3d3LnN5bWF1dGguY29tL3JwYTA0BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnZl
cmlzaWduLmNvbS9wY2ExLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwx
GjAYBgNVBAMTEVZlcmlTaWduTVBLSS0yLTk3MB0GA1UdDgQWBBSt+cOTci21uShh5KTXYNXE
Cl4aATCB8QYDVR0jBIHpMIHmoYHQpIHNMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVy
aVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsT
MShjKSAxOTk5IFZlcmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBD
BgNVBAMTPFZlcmlTaWduIENsYXNzIDEgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBB
dXRob3JpdHkgLSBHM4IRAItbdVaEVIULAM+vOEjOsaQwDQYJKoZIhvcNAQEFBQADggEBANaP
wdqbiPKzbE0fWC+6AVFddMFG6MO4e5/WQPHv/zK6iWvADjRDn6SZ5qTwXUgzYoWFYf4jiCKM
YJsrnGVJlMSiOCRIpVylUEto6WIip5PomSJuPVu7EEIOH0x1RzRWCY/4vYw881y70pZwVHBi
Te/REL6dSCxe7IZrB4LwPeElJygs4BZ2HrP95WKW0oo9Xyuu+1zCE7dlY8s0dkOf1oeZq26t
lcEAP0Yngf813iMOQ9wUXzL5yinvwlIw9ZnduYH4OiUgjYJo8rkhhXRmBOGGORYy8i3WKqjJ
3tkAAk/jGCDFpYFWtpXe04Kt+HslvmR8LqC6cCz4+XXidE0HbYQwggbXMIIFv6ADAgECAhAl
sq27FC4B63Z8q1Jzxx+CMA0GCSqGSIb3DQEBBQUAMIGmMQswCQYDVQQGEwJVUzEdMBsGA1UE
ChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5bWFudGVjIFRydXN0IE5ldHdv
cmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuU3ltYW50ZWMg
Q2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHNDAeFw0xMzAxMTUwMDAwMDBa
Fw0xNDAxMTYyMzU5NTlaMIHOMS4wLAYDVQQDDCVQZXJzb25hIE5vdCBWYWxpZGF0ZWQgLSAx
MzU4Mjc2MTA4NjMxMSswKQYJKoZIhvcNAQkBFhxqYWx0bWFuQHlvdXItZmlsZS1zeXN0ZW0u
Y29tMQ8wDQYDVQQLDAZTL01JTUUxHjAcBgNVBAsMFVBlcnNvbmEgTm90IFZhbGlkYXRlZDEf
MB0GA1UECwwWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEdMBsGA1UECgwUU3ltYW50ZWMgQ29y
cG9yYXRpb24wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQChjpVdVk6XeKFff4To
xGglVcP3FVY95LfwfBUKbQKppRYgfKBFW1RIEG+KXpc3IGOOTUX0Lg1xaukRwuoqv/pqRMNK
JabXcr1RFuCFg9xfcmP5Z+65qQ+IRxYCKdcNfu3GdS7rMOY57+VLU7aZnnMgnt0oE42awze8
gYQSJsdjZKKZS9DBnzxJYfzqhIw7txoMdV7rwPAZm5yujsqI/eWuZPZ+qZ+8GTsnHJzZNvc6
KrDGU6aVfknM+qf92hXA2VXvmtf1B17BBbGX6Hgat71Ufw5oXFly6/Vt+e5mKIozXytw8qPX
NllsG89ITVzn0OovcpcSRFBrXanzVn7JVj33AgMBAAGjggLVMIIC0TAMBgNVHRMBAf8EAjAA
MA4GA1UdDwEB/wQEAwIFoDAgBgNVHSUBAf8EFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwHQYD
VR0OBBYEFMC1SMuRefXyNAwkZM+Lgu7iJ6nFMCcGA1UdEQQgMB6BHGphbHRtYW5AeW91ci1m
aWxlLXN5c3RlbS5jb20wHwYDVR0jBBgwFoAUrfnDk3IttbkoYeSk12DVxApeGgEwggErBggr
BgEFBQcBAQSCAR0wggEZMIIBFQYIKwYBBQUHMAKGggEHbGRhcDovL2RpcmVjdG9yeS52ZXJp
c2lnbi5jb20vQ04lMjAlM0QlMjBTeW1hbnRlYyUyMENsYXNzJTIwMSUyMEluZGl2aWR1YWwl
MjBTdWJzY3JpYmVyJTIwQ0ElMjAtJTIwRzQlMkMlMjBPVSUyMCUzRCUyMFBlcnNvbmElMjBO
b3QlMjBWYWxpZGF0ZWQlMkMlMjBPVSUyMCUzRCUyMFN5bWFudGVjJTIwVHJ1c3QlMjBOZXR3
b3JrJTJDJTIwTyUyMCUzRCUyMFN5bWFudGVjJTIwQ29ycG9yYXRpb24lMkMlMjBDJTIwJTNE
JTIwVVM/Y0FDZXJ0aWZpY2F0ZTtiaW5hcnkwXQYDVR0fBFYwVDBSoFCgToZMaHR0cDovL3Br
aS1jcmwuc3ltYXV0aC5jb20vY2FfNTYxYzEwMzY5MGM5N2E2OTI0N2EwZWYwNzFhYzgxYWYv
TGF0ZXN0Q1JMLmNybDBsBgNVHSAEZTBjMGEGC2CGSAGG+EUBBxcBMFIwJgYIKwYBBQUHAgEW
Gmh0dHA6Ly93d3cuc3ltYXV0aC5jb20vY3BzMCgGCCsGAQUFBwICMBwaGmh0dHA6Ly93d3cu
c3ltYXV0aC5jb20vcnBhMCoGCmCGSAGG+EUBEAMEHDAaBhFghkgBhvhFARABAgIEAYazFxYF
MTA5MjIwDQYJKoZIhvcNAQEFBQADggEBAHVBsY+l21fY0twxzNnO0QbadbRT4n7k3rYfOMbP
noBDkYQx5lcrYn19f7e+ADMPdW/MY2ZjFM76aiRgiTo2IjMB7z4vyX6hfGJKTIHX9loDW0H2
z2o0jYqXDQtXx9A/gfMtVh9J+8O5IuCPsVtXgI4p6kRQHN+Er2Rzu1I4BILxE9HsDb/ruX6p
NXZSUQ2AcMP87ZVfz+reumMpJgWWAoiQWJCgp+qZ2c2AG+yV+FstiMpxIj/qB9+BrFRuam8d
IGbH0tIS2tyc7pgit8Pid2Zv7HGT2NFu69INsKIyqGImhMVuOUaCq/kNi3Z+6R5Hm3ljift8
ZF52Kiq4whe8KlcxggRSMIIETgIBATCBuzCBpjELMAkGA1UEBhMCVVMxHTAbBgNVBAoTFFN5
bWFudGVjIENvcnBvcmF0aW9uMR8wHQYDVQQLExZTeW1hbnRlYyBUcnVzdCBOZXR3b3JrMR4w
HAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlN5bWFudGVjIENsYXNz
IDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzQCECWyrbsULgHrdnyrUnPHH4IwCQYF
Kw4DAhoFAKCCAmswGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcN
MTMxMDI3MDc0MTU3WjAjBgkqhkiG9w0BCQQxFgQURbeiFvQJJX3RXmaQ2KBApsAV0zswbAYJ
KoZIhvcNAQkPMV8wXTALBglghkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4G
CCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCB
zAYJKwYBBAGCNxAEMYG+MIG7MIGmMQswCQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMg
Q29ycG9yYXRpb24xHzAdBgNVBAsTFlN5bWFudGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsT
FVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRp
dmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHNAIQJbKtuxQuAet2fKtSc8cfgjCBzgYLKoZIhvcN
AQkQAgsxgb6ggbswgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3Jh
dGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29u
YSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwg
U3Vic2NyaWJlciBDQSAtIEc0AhAlsq27FC4B63Z8q1Jzxx+CMA0GCSqGSIb3DQEBAQUABIIB
AC4cdGKPq8dcDBP5oTEmk0quBRtFDi5RP1Hle1O63Zqkq5c2OQm4q+r1oZH+k5plnm6Ox3yS
a7WGyaOsdFiEkYH/dGpjDPfEwskUjWuhs4BeHgSyX2Ca6TBxtY5LxGtiGfo3OrErkTJQPVYG
CMohND+EePeK1Pro/NOOJb5klLQFS7e9imZjGjmqoZTRzLuSZPOglt2s5P6fve0luiTp6nR2
kVUIqmVAArD5jpF09Yy3g4A09YN8iQ5yq84Mbh1ovwTA2hsMbROoMhIgLU1SR7rrF75b6Icb
d731KBVuSN2mEr7p5S4CW056Gs4HmvfLXoN/3kzariRbvUodPLLkBNMAAAAAAAA=
--------------ms020308000003020008010802--