While creating a reservation in a DHCP scope (Windows 2008 R2) I received the error mentioned.
After some investigation I discovered that Windows 2008 R2 doesn’t allow reservations that are not in the scope, although the subnetmask (for example 255.255.0.0) is big enough. Changing the subnetmask isn’t an option, so you have to delete and re-create the scope.
You can export the settings with the following command :
netsh dhcp server ip-adres server scope X.X.X.X dump > c:\dump,txt
Then edit the subnet mask
and import the scope with the following command :
netsh exec c:\dump.txt
Using Windows 2003 and Windows 2008 (not R2) it is possible to add reservations within the subnetmask but out of the DHCP scope!