find_entity_views sometimes gets error '500 read failed'

I've got a perl program that uses the vSphere api talking to a vcenter 5.1.0 947673.  Every minute the program issues a call to find_entity_views to get the list of VirtualMachines on the VCenter.  We have one deployment where the call fails sometimes with a perl exception like this:

exception = $VAR1 = bless( {

                 '-file' => '/usr/lib/perl5/site_perl/5.8.8/Error.pm',

                 '-text' => 'SOAP request error - possibly a protocol issue: 500 read failed:',

                 '-line' => 38,

                 '-package' => 'Error'

               }, 'Error::Simple' );

In looking at the logs on the VCenter I see this:

[viclient:SoapTran:W:15] 2013-07-12 15:27:06.150  Invoke 7787 Finish WaitForUpdates on PropertyCollector:propertyCollector [vcenter] - Serial:0.000, Server:300.013

Suppressed: VirtualInfrastructure.Soap.MethodInvocationSoapImpl: The request failed because the remote server 'vcenter' took too long to respond. (The command has timed out as the remote server is taking too long to respond.)

[viclient:Error   :W:15] 2013-07-12 15:27:06.150  ConnectionError occurred while executing WFU

VirtualInfrastructure.Exceptions.RequestTimedOut: The request failed because the remote server 'vcenter' took too long to respond. (The command has timed out as the remote server is taking too long to respond.)

   at VirtualInfrastructure.Soap.SoapServiceWrapper.DoInvokeSync(ManagedObject mo, MethodName methodName, Object[] parameters, Int32 timeoutSecs)

   at VirtualInfrastructure.Soap.SoapTransport.VirtualInfrastructure.Transport.InvokeMethod(ManagedObject mo, MethodName methodName, Object[] pars)

   at VirtualInfrastructure.ManagedObject.InvokeMethod(MethodName methodName, Object[] pars)

   at Vmomi.Core.PropertyCollector.WaitForUpdates(String version)

   at VirtualInfrastructure.Updates.PollerDispatcherImpl.ExecuteSingleWFU()

   at VirtualInfrastructure.Updates.PollerDispatcherImpl.PollWaitForUpdates()

System.Net.WebException: The command has timed out as the remote server is taking too long to respond.

   --- End of inner exception stack trace ---

The call doesn't always fail -- it'll go for a period where it actually works then slips back into not working with the above error.  It's very strange.

Does anyone have any ideas as to what it could be or at least where I could start looking?  Any and all information would be greatly appreciated!

Jonny L.