ESX 4.0 Update 1 has been re-released with changes to avoid this issue. The installation process checks for running agents and stops them before proceeding.
The re-released ESX 4.0 Update1 is referred to as ESX 4.0 Update 1a and is available via vSphere Update Manager (VUM) and the VMware Downloads site.
http://downloads.vmware.com/d/info/datacenter_downloads/vmware_vsphere_4/4
