[OpenAFS-devel] Windows Memory Leak - Revisited!

[James Peterson]

Below is a patch that fixes the biggest of the memory leaks associated with
Window's current version of OpenAFS.

Basically the memory leak is due to the differences between Microsoft's
implementation of the SMB protocol.  Later versions (after ??? 2001) did not
send SMB FindClose during a directory scan.  Back in Feb 2001 I wrote the
following:

"W2K and XP does not send FindClose when doing a directory scan.  When
smb_ReceiveTran2SearchDir() processes FindFirst or FindNext the last SMB
call is known by an empty directory (no file names).  Unfortunately Bit 1 of
the Flags is not set (close search if end of search) and the routine
and smb_DeleteDirSearch() is never called.  This patch looks for 0 file
names left to determine that we are done with the directory scan.   This
problem was reproducible while trying to scan a directory with symbolic
links."

The current versions of the SMB protocol implimentation ends up setting the
EOS but not setting returnedNames to zero.  The following patch I've
included tests for both of these conditions.  The 'diff' is based on source
"openafs-snap-2003-02-25".

diff -NurEB --exclude-from=exclude bas/WINNT/afsd/smb3.c
upd/WINNT/afsd/smb3.c
--- bas/WINNT/afsd/smb3.c	2002-11-26 19:30:16.000000000 -0800
+++ upd/WINNT/afsd/smb3.c	2003-01-21 06:29:32.000000000 -0800
@@ -2365,7 +2364,7 @@
          * or if something went wrong, close the search.
          */
         /* ((searchFlags & 1) || ((searchFlags & 2) && eos) */
-	if ((searchFlags & 1) || (returnedNames == 0)
+	if ((searchFlags & 1) || (returnedNames == 0) || ((searchFlags & 2) &&
eos)
         	|| code != 0) smb_DeleteDirSearch(dsp);
 	if (code)
         	smb_SendTran2Error(vcp, p, opx, code);

This patches does halt the memory leakage so that the program will grow upto
the size of the memory cache and stop.  Enjoy!

James Peterson
"Integrity is the Base of Excellence"