The 5-step data source vetting playbook
Run before a feed becomes the spine of 10,000 pages.
— Step 1 (Data): Confirm licensing permits public republishing. Gate: ambiguous terms → fail, don't ship on hope.
— Step 2 (Data): Profile completeness per field. Gate: any core field below 80% fill rate can't be a page-gating field.
— Step 3 (Data): Measure freshness. Record the source's update cadence and stamp dateModified from it. Gate: stale-beyond-X feeds get flagged, not silently served.
— Step 4 (Data): Test uniqueness. Gate: if 90% of rows share identical values in your display fields, the dataset can't differentiate pages — fail.
— Step 5 (Dev): Build a schema contract (field names, types, nullability). Gate: a feed that breaks the contract halts the pipeline, never publishes garbage.
Guardrail: one source of truth per field; no silent fallbacks that mix sources.
Ship gate: don't publish until all boxes are checked.
Scale Engine SOP
@ScaleEngineSOP
The 5-step data source vetting playbook
Этот пост опубликован в Telegram-канале Scale Engine SOP. Подписаться можно по ссылке: @ScaleEngineSOP.