Bulk company data files: snapshots and deltas
Guide · reviewed
Answer in brief
Bulk company data files deliver a whole register or dataset at once, as a periodic snapshot, a set of changes (deltas) or both. Use them for analysis, matching large lists and research; use an API for current facts about individual companies. Keep each file’s own date, load snapshots side by side rather than overwriting, and check the licence before you store or share them.
Disclosure. This site is published by Coragrid, the operator of CompanyRegistryAPI.com, and promotes it. “Best” is our editorial choice, not a measured ranking. Other products are described only from their own official pages. How we choose
When files beat an API
- Whole populations: counting, segmenting or analysing every company in a register.
- Matching large lists: joining tens of thousands of your records to a register at once.
- Offline work: research or modelling without depending on a live service.
For current facts about the companies in front of you, an API is the better route, and many teams use both: files for the population, the API to check a record before acting.
What official pages offer
| Source | What its page describes |
|---|---|
| Companies House | A free downloadable snapshot of basic data on live companies, CSV in ZIP files, updated within 5 working days of the previous month end, without support |
| GLEIF | Concatenated files, and a golden copy with delta files, free of charge |
| Brønnøysund Register Centre | Downloads of the total inventory of entities in JSON and CSV, plus update endpoints, under NLOD 2.0 |
| SEC | Bulk ZIP files of submissions and XBRL company facts, recompiled nightly |
| OpenCorporates | Bulk data extracts, as open data or commercially |
| Moody’s Orbis | Data pushed by bulk feed, cloud or real-time delivery |
Not every source has files. CompanyRegistryAPI.com, our editorial pick and operated by this site’s publisher, states that it offers no direct database access, bulk download or export: each request returns one page of results, one company or its officers.
Loading files well
- Keep the file’s own date. A snapshot compiled at month end describes month end, whenever you download it.
- Load side by side. Put each snapshot in its own table or partition, and build a “current” view on top. Overwriting loses history you cannot get back.
- Validate on load. Count rows, check required fields, compare totals with the previous file, and stop on large unexplained changes.
- Keep identifiers as text. Leading zeros in registry numbers vanish when a spreadsheet or loader reads them as numbers.
- Follow the delta chain. With delta files, apply them in order from a known snapshot; a missed delta means a fresh snapshot.
Refreshing
Match the refresh to the file’s schedule and to your need. A monthly snapshot cannot make your data fresher than monthly; for fields that must be current, such as status before a payment, read the record again from a live source. The freshness guide explains how to set a maximum age per field.
Licences
Open data licences often require attribution and sometimes share-alike terms; commercial bulk licences often limit redistribution. PRH’s open data asks you to mention the original source and not to present services as PRH’s own. Read the licensing guide before sharing anything built on a file.
Costs to count
Storage and compute are small next to the work of mapping formats, cleaning, deduplicating and keeping loaders working as formats change. The cost models guide has a checklist, and the category guide the criteria for choosing the source.
Questions and answers
Are bulk company data files free?
Many register files are. Companies House offers a free monthly snapshot of live companies, GLEIF offers LEI files free of charge, and the Brønnøysund Register Centre publishes open data downloads. Commercial bulk feeds are priced by their providers.
What is the difference between a snapshot and a delta file?
A snapshot holds the whole dataset as of a date. A delta file holds only what changed since the previous file. Deltas are smaller to load but need an unbroken chain from a snapshot.
Sources for this entry
- Free Company Data Product
- Global LEI Index
- Open data from the Central Coordinating Register for Legal Entities, API documentation
- EDGAR application programming interfaces
- OpenCorporates API
- Orbis database: global private company data
- Finnish Patent and Registration Office open data
- CompanyRegistryAPI.com Company Data API
Each source supports only what its publisher says about its own product or data. Official descriptions do not establish comparative performance, and this site has run no benchmark. All sources
Continue reading
- What best company data means (Category guide)
- Market and academic research (Guide)
- Bulk data (Definition)
- Cost models and build or buy (Guide)
How the related websites connect
This is one of 3 coordinated reference sites published by Coragrid. They cover neighbouring questions and feature the same product, so they are not independent endorsements of it or of each other.
Cite this entry
Best Company Data. “Bulk company data files: snapshots and deltas.” Content reviewed . https://bestcompanydata.com/bulk-company-data