Get API key

Export SEC companyfacts to CSV with Python

Use Python’s standard library to export bounded pages of reported SEC facts, preserving units, periods, accession numbers, and a separate source-provenance file.

WHAT YOU WILL BUILD

Request a specific concept and unit, follow the returned next offset within a deliberate page budget, and write both a CSV and a provenance sidecar. The script reports whether more results remain instead of disguising a partial export as complete.

Requirements: Python 3.10 or later; standard library only. An API key from your dashboard, stored in the SEC_API_KEY environment variable.

Run the workflow

Save this as workflow.py and run python3 workflow.py. The production base URL is used by default. Keep credentials out of source control.

import csv
import json
import os
from urllib.parse import urlencode
from urllib.request import Request, urlopen

key = os.environ.get("SEC_API_KEY")
if not key:
    raise RuntimeError("Set SEC_API_KEY first")
base = os.environ.get("SEC_API_BASE_URL", "https://sec-api.dev")
rows, metadata, seen_offsets = [], [], set()
offset, next_offset = 0, None
max_pages = 3

for _ in range(max_pages):
    if offset in seen_offsets:
        raise RuntimeError("Repeated pagination offset")
    seen_offsets.add(offset)
    params = urlencode({"taxonomy": "us-gaap", "concept": "Assets",
                        "unit": "USD", "limit": 100, "offset": offset})
    request = Request(base + "/api/v1/facts/320193?" + params,
                      headers={"Authorization": "Bearer " + key})
    with urlopen(request, timeout=30) as response:
        payload = json.load(response)
    rows.extend(payload["data"])
    metadata.append(payload["meta"])
    next_offset = payload["pagination"]["nextOffset"]
    if next_offset is None:
        break
    offset = next_offset

fields = ["taxonomy", "concept", "unit", "value", "start", "end",
          "filed", "accessionNumber", "form"]
with open("companyfacts.csv", "w", newline="", encoding="utf-8") as file:
    writer = csv.DictWriter(file, fieldnames=fields)
    writer.writeheader()
    writer.writerows({field: row.get(field) for field in fields} for row in rows)

source_changed = len({item["fetchedAt"] for item in metadata}) > 1
with open("companyfacts-provenance.json", "w", encoding="utf-8") as file:
    json.dump({"pages": metadata, "hasMore": next_offset is not None,
               "nextOffset": next_offset, "sourceChanged": source_changed},
              file, indent=2)
print(json.dumps({"rows": len(rows), "hasMore": next_offset is not None,
                  "nextOffset": next_offset, "sourceChanged": source_changed}))

Start with a narrow, interpretable selection

This export asks for the us-gaap Assets concept in USD for a single CIK. The exact filter makes the output easier to inspect than a mixed file containing many concepts and units. Each CSV row still carries the concept and unit: your downstream analysis should not depend on remembering the request from a terminal session. Keep end, start when supplied, filed, and accessionNumber because reporting period and filing provenance answer different questions. Missing start values are left blank rather than invented.

Use a page budget you can explain

The example has a three-page budget with up to 100 requested rows per page. That is a deliberate limit on work, not a claim that every company or concept fits inside 300 rows. It follows the next offset supplied by the response and checks for an accidental repeated offset. When the budget ends before the dataset does, hasMore is true and the next offset is preserved in the sidecar. Increase the budget only after checking your account allowance and the size of the intended export. Respect 429 retry guidance; do not wrap the script in an uncontrolled retry loop.

Save provenance beside the CSV

A CSV alone cannot express all of the response-level context. The second output file records the source metadata for every page, whether more observations remain, and whether fetchedAt changed during the export. A change does not itself prove a data problem; it tells you the pages were not all served from the same cache retrieval. If reproducibility matters, investigate that boundary and decide whether to restart from a stable source snapshot. This script detects the condition rather than promising snapshot isolation that the endpoint does not provide.

Keep observation export separate from accounting logic

The output is a collection of reported observations, not an automatically reconciled balance sheet. Several rows may need consideration for a particular reporting interval. Do not group solely by the concept name and silently keep the last row. Document any preferred-value selection separately, retaining the original observations and accession numbers. The standard-library HTTP client raises an error on non-success responses, so a failed request stops the export rather than producing a misleading success message. Run the script in a directory where creating the two named output files is appropriate; it overwrites those files on a subsequent run.

Sources and further reading

The underlying SEC dataset behavior is documented in the official SEC API reference and SEC technical specifications. Response-field names shown here belong to the sec-api.dev endpoint contract. The scripts print actual results and do not embed fabricated filing or financial data.

All tutorials ←