Saturday, September 28, 2013

Work around ipsec setkey error "The result of line 2: (null).”

In some Ipsec old howto, AH and ESP are specified through two settings when configuring ipsec SA manually:

add 10.0.0.216 10.0.0.11 ah 24500 -A hmac-md5 "1234567890123456";          
add 10.0.0.216 10.0.0.11 esp 24501 -E 3des-cbc "123456789012123456789012";

spdadd 10.0.0.216 10.0.0.11 any -P out ipsec
   esp/transport//require
   ah/transport//require;

It results error message like “The result of line 2: (null).” when running with “setkey” with 3.7.10 linux kernel. The solution is to combine merge hmac setting into ESP, like this:

add 10.0.0.216 10.0.0.11 esp 24501 -E 3des-cbc "123456789012123456789012"
  -A hmac-md5 "1234567890123456";          

spdadd 10.0.0.216 10.0.0.11 any -P out ipsec
   esp/transport//require
   ah/transport//require;