[OpenAFS-devel] dealing with rxevent queue stalls

[Mark Vitale]

--Apple-Mail=_53394073-057E-4EDB-9ADD-71FE7A8D0669
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


On Sep 24, 2013, at 11:27 AM, Andrew Deason <adeason@sinenomine.net> =
wrote:
> Logging it is definitely helpful and OK, though logging from rx is not
> great. Are you currently using existing mechanisms by just printing to
> stderr, or some new mechanism for logging from rx?
I'm just using dpf() in the prototype; of course I'd need something =
better for any upstream version.

>> 2) Volume releases suffer from poor performance and occasionally fail
>> with timeouts.
>>  - root cause was heavier-than-normal vlserver load (perhaps caused
>>  by disk performance slowdowns); this starved LWP IOMGR, which in
>>  turn prevented LWP rx_Listener from being dispatched (priority
>>  inversion), leading to a grossly delayed rxevent queue.
>=20
> I'm not sure if I'm mistaken as to what this is about, or if I just =
find
> this phrasing really confusing. I thought the issue here was just that
> ubik proceses (such as vlserver) use plain read() and write() calls to
> read and write from disk; so if they take a while, all LWPs will =
freeze
> because we cannot preempt the LWP waiting on i/o. Is that correct?
Yes, that's correct; both cores for the LWP case showed we were waiting =
for an (LWP-synchronous) disk read to complete.  I think we are saying =
the same thing, but it sounds clearer the way you say it!  But it's not =
_just_ that the disk IO is synchronous; the site reported that disk IO =
was indeed suffering from poor performance at the time.

--
Mark Vitale
mvitale@sinenomine.net



--Apple-Mail=_53394073-057E-4EDB-9ADD-71FE7A8D0669
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iQIcBAEBAgAGBQJSRHm7AAoJECQB9O5MipHIKOUQAKvZxQ+vIsnp3hEcZPV+qmff
ZqGABzjJj7ix7mtKwGp/duuhp1AR7EdgarKIQiTxa/aCViJojI7pgP71SL9KBmnv
ttwT+fDwIqQCfC3GQHgj6WX+B5qur7T7mK5mUKPGAA5y0QZsNngr70F6cGqbIVuA
X9yoyDWAJv2ufgt5xACGuHJyNVH4oGdh5rlOMoLjOWi/d8XjzMhXMBZ+iIAKGRCf
uhA7eOO0PaR1rJjdcDrsbUIVTQUMBV3z2WDU+HeNeGWynIYuas5i974odHmuwh3O
3bLKT4zkjrqwA/S4Sua9p849ZVMKd23defnfdWy4Q0PIla5V8bHDiX6qn/qwmXCM
HNqOMGHI3kE/gafxnvteMPP78ThUcE7Uq2e81wNXKcxqh8/zK63cn0TBX83A8kmV
csWlYY7vxmS70bSdZZwf+LRFaaT91Yrtfn59kxL2FzKFMbjkAT/Mz+UCHxcWeDKG
3z1QbOqxDOkSVFHmNmuJHVp6qhx528yPvUwQXiYIHmpFGq/ZjUhKchWljfg8w5Hk
P9lqRuu2nXmrfPJLIuiF2pXMvsEwP+UEYRsJIpQ8LpADG6+STSWaTrIdbrfrr1oj
cVvybtLtsMo25Zw4gNv5Ui0TXFWwEnaR8lcHoZT/9S7JMOTqhLQClxL4wV14XU81
wXJPYkshTvK1va8LunOT
=gFOW
-----END PGP SIGNATURE-----

--Apple-Mail=_53394073-057E-4EDB-9ADD-71FE7A8D0669--