Actually you don't need to do so - at least in Vimperator you can enter 'pass through mode' by pressing ctrl-z. This disables vimperator keys and allows you to interact with sites in 'normal way'.