[OpenAFS-devel] Windows Memory Leak - Revisited

[Lantzer, Ryan]

The patch submitted by James Peterson appears to have resolved the
performance problem I was seeing when accessing 250000 files in 12000
directories on Windows XP with OpenAFS. Without this patch, the number
of used status cache entries (status cache entries with a refCount
member greater than 0) steadily increases. Because the script that I was
testing with took so long to run, I never let it run through all 12000
of the directories, but one time I waited long enough to see more than
700 status cache entries in use. After this patch, the number of used
status cache entries sits at about 3, even after 10000 status cache
entries have been allocated.

I have a patch to src\WINNT\afsd\cm_callback.c that looks like it should
fix another place where status cache entries might be "held" without
being "released". Nathan Neulinger found this place in the Windows cache
manager code that looks like it leaks a status cache entry. I have not
performed any tests yet to see if this patch causes any problems, but
I'm pretty sure that this change should be made.

Ryan Lantzer

--- cm_callback.c.orig	2002-07-10 14:38:54.000000000 -0500
+++ cm_callback.c	2003-02-27 17:05:15.000000000 -0600
@@ -112,6 +112,7 @@
 			smb_NotifyChange(0,
 			 FILE_NOTIFY_GENERIC_FILE_FILTER,
 			 dscp, NULL, NULL, TRUE);
+		if (dscp) cm_ReleaseSCache(dscp);
 	}
 }



James Peterson james@abrakus.com  wrote:
Wed, 26 Feb 2003 21:36:51 -0800=20

Below is a patch that fixes the biggest of the memory leaks associated
with
Window's current version of OpenAFS.

Basically the memory leak is due to the differences between Microsoft's
implementation of the SMB protocol.  Later versions (after ??? 2001) did
not
send SMB FindClose during a directory scan.  Back in Feb 2001 I wrote
the
following:

"W2K and XP does not send FindClose when doing a directory scan.  When
smb_ReceiveTran2SearchDir() processes FindFirst or FindNext the last SMB
call is known by an empty directory (no file names).  Unfortunately Bit
1 of
the Flags is not set (close search if end of search) and the routine
and smb_DeleteDirSearch() is never called.  This patch looks for 0 file
names left to determine that we are done with the directory scan.   This
problem was reproducible while trying to scan a directory with symbolic
links."

The current versions of the SMB protocol implimentation ends up setting
the
EOS but not setting returnedNames to zero.  The following patch I've
included tests for both of these conditions.  The 'diff' is based on
source
"openafs-snap-2003-02-25".

diff -NurEB --exclude-from=3Dexclude bas/WINNT/afsd/smb3.c
upd/WINNT/afsd/smb3.c
--- bas/WINNT/afsd/smb3.c	2002-11-26 19:30:16.000000000 -0800
+++ upd/WINNT/afsd/smb3.c	2003-01-21 06:29:32.000000000 -0800
@@ -2365,7 +2364,7 @@
          * or if something went wrong, close the search.
          */
         /* ((searchFlags & 1) || ((searchFlags & 2) && eos) */
-	if ((searchFlags & 1) || (returnedNames =3D=3D 0)
+	if ((searchFlags & 1) || (returnedNames =3D=3D 0) || ((searchFlags &
2) &&
eos)
         	|| code !=3D 0) smb_DeleteDirSearch(dsp);
 	if (code)
         	smb_SendTran2Error(vcp, p, opx, code);

This patches does halt the memory leakage so that the program will grow
upto
the size of the memory cache and stop.  Enjoy!

James Peterson
"Integrity is the Base of Excellence"