Topic
Data and context
Which facts and definitions can this system rely on?
A correct calculation can answer the wrong question if people mean different things by revenue, a customer, or an active order. Identify the source, definition, and revision the decision needs.
Start with an approved definition, scope data repair to one decision, and check whether an extraction contains its required records. The notes keep operational facts, generated context, evaluation references, and telemetry uses separate.
Notes in this topic
Read-only can still expose the wrong data
Permission to fetch a record does not automatically include permission to send every field to a model.
Read the noteThe SQL ran. Did it answer the right question?
A query can execute without errors and return the wrong total. Check what each row represents before accepting the number.
Read the noteWhich cases belong in your evaluation set?
Five passing tests can hide a missing sixth. Look at the cases behind the score before deciding what the system can do.
Read the noteDoes the model need to think, or find the evidence?
Before buying a stronger model, check what information it received. The answer may depend on a document search never found.
Read the noteHow much data repair does this decision need?
Find the records needed for the job before starting a company-wide cleanup. A missing budget can block a finance draft; duplicate mailing addresses might not matter.
Read the noteWhich revenue definition should the agent use?
Two reports can both say revenue and mean different things. Settle the definition, scope, and unresolved facts before letting an agent answer.
Read the noteThe API returned 200. Did all the data arrive?
A successful request can leave out the order that would change your decision. Check which records should have arrived before using the extract.
Read the note