Download SEC filing metadata CSV and preserve its headers
Export a bounded annual-filing metadata page with Python, saving source and pagination headers next to the CSV.
Download the CSV bytes and save provenance headers in a sidecar. Explicitly retain the next offset so a single export page cannot be mistaken for a full archive.
Requirements: Python 3.10 or later; standard library only. An API key from your dashboard, stored in the SEC_API_KEY environment variable.
Run the workflow
Save this as workflow.py and run python3 workflow.py. The production base URL is used by default. Keep credentials out of source control.
import json
import os
from pathlib import Path
from urllib.parse import urlencode
from urllib.request import Request, urlopen
key = os.environ["SEC_API_KEY"]
base = os.environ.get("SEC_API_BASE_URL", "https://sec-api.dev")
params = urlencode({"cik": "320193", "form": "10-K", "history": "all",
"limit": 100, "format": "csv"})
request = Request(base + "/api/v1/datasets/filings?" + params,
headers={"Authorization": "Bearer " + key})
with urlopen(request, timeout=60) as response:
content = response.read()
headers = {k: v for k, v in response.headers.items()
if k.lower().startswith(("x-data-", "x-history-", "x-next-", "x-total-"))}
Path("filings.csv").write_bytes(content)
Path("filings-provenance.json").write_text(json.dumps(headers, indent=2))
print(json.dumps({"bytes": len(content), "provenance": headers}))Save metadata outside the CSV
CSV records do not carry the entire response envelope. The script saves data-source, retrieval, snapshot, coverage, total, and next-offset headers where supplied. Keep both files together. Authorization is intentionally excluded from the sidecar so the audit record does not become a leaked credential.
Interpret a page as a page
The endpoint returns up to100filing rows for this request. An empty next-offset header means the selected source result is exhausted; a nonempty value requires another request. Coverage refers to the requested source history, while pagination refers to how much of that result this download contains. These are different dimensions of completeness.
Know the export transformations
CSV fields that could be interpreted as spreadsheet formulas are escaped by the service. Missing values export as empty cells. If exact string preservation matters for a programmatic archive, use JSON and retain its envelope. The script overwrites its two named files, so run it in a suitable output directory. It does not download the original filing documents.
Sources and further reading
The underlying SEC dataset behavior is documented in the official SEC API reference and SEC technical specifications. Response-field names shown here belong to the sec-api.dev endpoint contract. The scripts print actual results and do not embed fabricated filing or financial data.
All tutorials ←