William H. Brown, Downham Market