David Cameron says it is important to "speak frankly" after criticism of his comments about Pakistan's record on tackling terrorism.