Common aims of habitat studies are to differentiate between (i) suitable and unsuitable sites for a given species, and (ii) sites used by different communities of species. To quantify differences between sites, field data of site use must be precise enough that true underlying between-site variability is not masked by within-site measurement error. We designed a pilot study to guide the development of a survey protocol for a habitat study on bats in an agricultural landscape in southeastern Australia. Three woodland sites and two scattered tree sites of 2 ha each were surveyed for nine consecutive nights. At three locations within each site (spaced > 50 m apart) one or two Anabat detectors were mounted 1 m above ground or in a tree (2 m above ground). We used mixed regression models to quantify multiple sources of variability in bat calling activity, and graphical data analysis to visualise how increases in survey effort were likely to affect inference. For the five most active species, we found that typically over 40% of variability in nightly detections occurred at the between-site level; approximately 10% occurred between locations within sites; approximately 20% was explained by night-to-night differences; and approximately 30% of variability was not attributable to systematic variation within experimental units. Differences in community composition between sites were clearly evident when two or more detectors per site were used for four or more nights. We conclude with six general considerations for the design of effective habitat studies. These are to (i) consider key contrasts of interest; (ii) use data from mild, calm, dry nights only; (iii) calibrate detectors; (iv) use multiple detectors where possible, or move a single detector within a site; (v) survey for multiple nights; and (vi) where vertical differentiation in habitat use is likely, mount detectors at different heights. These considerations need to be balanced within the context of financial and logistical constraints.