A key component in any target or anomaly detection algorithm is the characterization of the background. We investigate several approaches for estimating the background level at a given pixel, based on both the local neighborhood around that pixel and on the global context of the full image. By framing this as a regression problem, we can compare a variety of background estimation schemes, from standard signal processing approaches long used in the hyperspectral image analysis community to more sophisticated nonlinear approaches that have recently been developed in the image processing community. These comparisons are performed on a range of images including single band, standard red-green-blue, eight-band WorldView-2, and 126-band hyperspectral HyMap imagery.