Effective weed management in pastures is critical for maintaining the productivity of grazing land. Autonomous ground vehicles (AGVs) are increasingly being considered for weed localization and treatment in agricultural land. Weeds, however, can be difficult to distinguish from background plants, due to similarities in colour, shape and texture. While deep learning approaches can be used to solve the localization issue, they are computationally expensive, and require a large volume of training images in order to combat overfitting. In this paper we present a novel Extreme Learning Machine based network for segmenting weeds from the background pasture. The proposed method utilizes a combination of LBP, HOG and colour features, and is tested on four small datasets, achieving a high mean Intersection over Union of 87.1, 79.5, 81.6 and 87.6 for Bathurst burr, horehound, thistle and serrated tussock respectively.