Gorse was introduced as a hedge plant and is common around pastoral land as the native trees establish the gorse will fail to thrive and eventually be shaded out. However, in the mean time:

Control: Broadcast spray is practical in areas where there is no native vegetation. However where occasional bushes persist amongst naturally occurring natives, cut the stems and  stump swab: glyphosate (roundup)(250ml/L) or metsulfuron-methyl 600g/kg Escort or Associate) (2g/L) or triclopyr 600 EC (grazon) (250ml/L) or Tordon Brushkiller (100ml/L) or Vigilant gel.