Data methodology
How Alveos collects, stores, transforms, and labels software-distribution metrics.
Principles
Every metric must name its source, meaning, freshness, coverage, and transformation. Missing data stays missing: Alveos does not fill outages with estimates. “Official”, “verified”, or “independent” is not used unless the source actually supports that claim.
Source classes
- Public provider source: direct public APIs or listings such as VS Code Marketplace, Open VSX, GitHub Releases, PyPI, and Packagist.
- Public dataset / trusted third party: named datasets such as PSF BigQuery or PypiStats.
- First-party product telemetry: the connected product sends lifecycle events to Alveos (Beacon; Streaming Zebra coming soon). The numbers come from the product and the publisher’s counting method — event-driven, not fetched from a marketplace.
- Manual publisher import: for example VS Code Publisher Hub data supplied by the publisher. It remains separate from public Gallery counters.
What the metrics mean
Alveos stores what the source delivers. What the source does not deliver is not captured here.
| Source | Delivered and captured | Missing |
|---|---|---|
| VS Code Marketplace (Gallery) | Public counters (downloadCount, install, updateCount) — acquisition |
Currently active installations: the Gallery does not provide that, and Alveos does not capture it here |
| Open VSX | downloadCount (VSIX downloads) |
Active installations, unique users, uninstalls: neither delivered nor captured |
| GitHub Releases | download_count of release assets |
Clones, private repositories, source archives |
| PyPI / PypiStats | File downloads (daily series without mirrors) | Unique users and installations, because the source does not count them |
| Packagist | downloads.total / monthly |
Installations and unique users |
| Beacon / lifecycle | Install, update, uninstall, usage, and build events reported by the product | Marketplace counters; uninstalls are often incomplete |
| Publisher Hub (import) | Daily-stats days the publisher supplies | Not Gallery SSOT; website VSIX only if the file includes that column |
Counters with different meanings are not silently presented as the same metric.
Collection, freshness, and gaps
Alveos fetches public stores once a day at 05:20 Europe/Berlin. It stores today’s UTC day. A failed fetch skips the day, keeps history, and does not write zeros. Gallery, Open VSX, GitHub, PyPI JSON, PypiStats, and Packagist are not backfilled three days.
What the graph does: from the first stored day, the solid line holds the last known value (step-hold). If a fetch is missing in between, Alveos draws the same value dashed — last known stock, not a forecast and not a trend curve over the real points. Missing days in the window are marked. Days before the connection are not gaps.
If the last successful day is two or three days ago, the series is delayed; from the fourth day it is stale. That is a display label, not a backfill.
Two exceptions rewrite recent days when the next fetch succeeds: Streaming Zebra stores D−2 and D−1 from the summary; PyPI countries (BigQuery, paying plans) upsert the last seven UTC days. Publisher import and Beacon are not daily fetches and are not scored as missed gaps.
Transformations and comparability
Alveos retains raw provider integers and labels derived values such as cumulative-counter deltas, sums, ratios, imported values, and filtered partitions. Direct delivery and GitHub lanes that can describe the same delivery are kept separate. Streaming Zebra and Beacon updates are not added twice.
Reproducible exports
CSV and JSON exports include provider, external ID, source class, metric semantics, coverage start, cadence, transformation version, source URL, and a SHA-256 row hash. Historical snapshots currently have UTC day precision: fetched_on contains the day, time_precision is day, and fetched_at remains empty rather than containing an invented time.
Provider references
| Source | Fetch | Cadence |
|---|---|---|
| VS Code Gallery, Open VSX, GitHub Releases, PyPI JSON, PypiStats, Packagist | Cron stores today’s UTC day | Daily 05:20 Europe/Berlin |
| PyPI countries (BigQuery, paying plans) | Upsert of the last seven UTC days | Same cron |
| Streaming Zebra (coming soon) | Summary; heals D−2 and D−1 | Same cron |
| Beacon | Product sends | Event-driven, not a daily fetch |
| Publisher Hub | xlsx import | On import |
Auditable corrections
Signed-in publishers can report an already stored connection under Data corrections: UTC day, metric (downloads, installs, or updates), proposed value, and reason. The request records the raw observed value. A correction does not fill a missed fetch and does not write an estimate into a gap.
Operator review, acceptance, rejection, application, and supersession are append-only events in a tamper-evident hash chain. An applied correction is a versioned read-time overlay: the provider snapshot remains unchanged. Graphs, totals, and exports use the active overlay. CSV/JSON identify the request and original provider values with correction_request_ids and correction_original_json. A newer approved correction revokes the older overlay but preserves both audit histories.
Questions or suspected data errors: support@alveos.eu.