We can free this ref_map as soon as the fetch is complete. It is not
used for the automatic tag following, nor is it used to disconnect the
transport. This avoids some confusion about why we are holding onto
these refs while following tags.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>