I finally found a solution.
It seems it fails if you give some parameter marked as mandatory (scsiCtlrUnitNumber, hotAddRemove)... Theses parameters seem to be automatically defined, a dump of the device will show that.
Here is the code used:
my $controller_spec =
VirtualDeviceConfigSpec->new( device => VirtualLsiLogicController->new( key => 0,
device => [0],
busNumber => $scsi_controller,
sharedBus => VirtualSCSISharing->new('noSharing')),
operation => VirtualDeviceConfigSpecOperation->new( 'add' ) );
my $vmspec = VirtualMachineConfigSpec->new(deviceChange => [$controller_spec] );
eval {
$vm_view->ReconfigVM( spec => $vmspec );
};