Upstream repositories
An upstream is a repository that your repository proxies: when your repository doesn't hold a requested file, it requests the file from its upstreams. With upstreams, one Artifex repository can serve your own artifacts and everything your builds depend on, so a build needs one Maven repository URL instead of a list.
Only someone with the repository's Admin role can manage upstreams.
Add an upstream
An upstream is a preset repository, a remote Maven repository named by its URL, or another repository in Artifex, named by its owner and repository name.
To add one, follow these steps:
Open the repository and go to Upstreams.
Click New upstream.
Under Kind, select A preset repository, A repository on the internet, or Another repository here.
For a preset, in the Preset list, select the repository.
Otherwise, in the Name field, enter a display name. For a remote repository, in the URL field, enter the repository URL, starting with
https://. For an Artifex repository, in the Repository field, enterOWNER/REPOSITORY_NAME.Optional: expand Authentication to set credentials, or Advanced for the options described in this document.
Click Add upstream.
Presets
A preset is a well-known repository that Artifex configures for every account: Maven Central, the Gradle Plugin Portal, Google Maven, Minecraft Libraries, NeoForged, Minecraft Forge, FabricMC, JitPack, and Sonatype Snapshots. A preset upstream takes its name, its URL, its content rules, and every other setting from the preset, and changes when the preset changes. You can only turn it off, move it, or remove it.
Add Maven Central as the preset rather than by its URL: Maven Central blocks requests from Artifex, and the preset uses a Maven Central mirror instead.
For an Artifex upstream, you must be able to read the repository that you name, and it can't be the repository that you're configuring.
Order
When the repository doesn't hold a file, Artifex requests it concurrently from every upstream that can hold it and uses the response from the earliest upstream in the list that has the file. Put the upstream whose copy you prefer first. When the repository's Pin authoritative upstreams setting is on, a version's authoritative upstream supplies its later files.
When an earlier upstream serves the file, Artifex cancels the pending requests to later upstreams and creates no negative cache entries for them.
To change the order, in the upstream's row, click Move up or Move down.
Artifex skips an upstream that's turned off. To turn off an upstream without removing it, turn off the Request from NAME switch on the upstream, where NAME is the upstream's name.
To prevent requests to an upstream for artifacts that it can't serve, use a content rule instead. For more information, see Content rules.
Upstream authentication
For a remote repository that needs authentication, expand Authentication and fill in the following:
- Username: the username to send.
- Password: the password or token to send.
The password is never shown again. When you edit the upstream, leaving the field empty retains the stored password.
Advanced options
| Option | Default | What it does |
|---|---|---|
| Timeout | 2000 ms | The timeout for the upstream's response headers. Between 100 and 60000 milliseconds. |
| Negative cache TTL | 300 seconds | How long a negative cache entry for a not-found response lasts. Between 0 seconds and 7 days. |
| User agent | CodeSoh-Artifex/1.0 (+https://artifex.soh.gg) | The HTTP User-Agent header value. Set it when an upstream expects a particular value. |
| Accept HTML responses | Off | When off, an HTML response is treated as not found, because it's usually a login page or an error page. Turn it on only for an upstream that serves artifacts with an HTML content type. |
| Fetch Gradle variant files eagerly | On | When Gradle Module Metadata (.module) arrives from this upstream, fetch every file that its variants list. See Gradle Module Metadata in Artifex. Because Gradle requests all of a module's files from the repository that supplied its .module file, this prevents a build from failing on a variant whose files the repository holds only some of; the repository setting When requesting a missing file without write access is the other option. |
What happens on a request
When a build requests a file that the repository doesn't hold, Artifex does the following:
- Takes every upstream in order, skipping any that's turned off, any whose content rules say it can't hold this artifact, and any that has a negative cache entry for this file.
- Requests the file from all of them concurrently and uses the response from the earliest upstream in the list that has it. An Artifex upstream serves only its own files and doesn't request from its own upstreams.
- Caches the file and serves it. Concurrent requests for the file wait for this fetch rather than causing a second request to the upstream.
Artifex proxies a file from an Artifex upstream without caching it, so the file doesn't add to the repository's file count or storage.
The repository that receives the request decides access, not the upstream. Anyone who can read a repository can read what its Artifex upstreams supply through it, so don't make a repository public while it has an upstream that holds something private. Artifex checks that you can read an Artifex upstream only when you add it.
Files from an upstream are marked as such on the file page, together with the upstream URL and the checksums that were verified. Artifex doesn't store a file whose content doesn't match the checksum that the upstream published.
Missing versus unavailable
If every upstream that can serve the file responds with not found, your build reports the dependency as not found.
If an upstream is unavailable because it timed out, returned an error, or sent a file whose checksum didn't match, your build reports the repository as unavailable rather than the dependency as not found, and a build that retries after the upstream recovers resolves normally.
Negative caching
A not-found response creates a negative cache entry for each upstream and file. The entry lasts for the upstream's Negative cache TTL. Builds often request files that don't exist, such as a sources JAR file that was never published, and the negative cache prevents repeated requests for them to the upstream.
Set the duration to 0 to turn off negative caching, which is useful while you're setting up an upstream.
Negative cache entries are removed in the following cases:
- You change the upstream's settings.
- You add or remove one of its content rules.
- You remove the upstream.
- You click Clear negative cache under Advanced in the upstream's Edit dialog, or in its Details for a preset.
Use Clear negative cache when a file has been published upstream and you don't want to wait for its negative cache entry to expire.
Maven metadata from an upstream is refreshed every 10 minutes and never negatively cached, so a version published upstream appears within 10 minutes.
Authoritative upstreams
The first time a version is requested, the upstream that served it is stored as that version's authoritative upstream.
The authoritative upstream is recorded whether Pin authoritative upstreams is on or off. It changes with the following actions:
- Deploying a file into the version clears its authoritative upstream. The version is stored in your repository.
- Deleting the repository's files for the version lets the next request choose an authoritative upstream again.
- Removing the authoritative upstream lets the next request choose among the remaining upstreams. Files already fetched stay in the repository.
When Pin authoritative upstreams is on, requests for the version's missing files go only to its authoritative upstream. This prevents another upstream from supplying different bytes for part of the same version. If that upstream is unavailable, the build reports the repository as unavailable instead of using another upstream.
When the setting is off, each missing file resolves in upstream order. Serving a file from another upstream doesn't replace the version's authoritative upstream.
Maven metadata
Artifact-level Maven metadata combines your own versions with the versions reported by every upstream that can serve the artifact. When Pin authoritative upstreams is on, version-level metadata for a snapshot comes only from its authoritative upstream, so timestamped snapshots stay consistent with their files.
Remove an upstream
To remove an upstream, follow these steps:
- Open the repository and go to Upstreams.
- In the upstream's row, click Remove.
Its content rules are removed with it. Files that were already fetched from it remain in the repository and are still served.
Advanced
Credentials are sent as HTTP Basic authentication.
For a remote repository, Artifex reads the checksums that the upstream sends with the file, in headers such as x-checksum-sha1, requests the checksum files for the rest, and compares them with the bytes that arrived. If they don't match, the file isn't stored and the request gets a 503 Service Unavailable response with a message naming the algorithm.
The Timeout is measured from when Artifex sends the request to the upstream until the response headers arrive, not from when the build's request arrives at Artifex.
The following table shows how an upstream's response is treated:
| What the upstream did | What Artifex does |
|---|---|
Responded 404 or 410 | Treats the file as missing and creates a negative cache entry |
Responded 200 with an HTML content type, and Accept HTML responses is off | Treats the file as missing and creates a negative cache entry |
| Responded with another error status, timed out, or the connection failed | Reports the upstream as unavailable |
| Sent bytes whose published checksum doesn't match | Reports the upstream as unavailable |
| Was still responding when an earlier upstream served the file | Cancels the request and creates no negative cache entry |
If every upstream that can serve the file responds with not found, the request gets a 404 Not Found response. If no upstream has the file and at least one was unavailable, the request gets a 503 Service Unavailable response with Retry-After: 30 and a message naming the upstream and what went wrong.