If you borrow offshore and want to deduct the interest cost for tax purposes, you will either:
(i) need to make sure that the loan is treated as an onshore loan by the lender with interest being paid in NZ; or
(ii) deal with the non-resident withholding tax issue which means either (a) forgetting about tax deductibility in NZ or (b) paying non-resident withholding tax on the interest (bank will require a gross up) and filing some returns with the IRD in NZ.