Document assistants can make project information easier to use. Their value depends on finding the right evidence, recognising what is missing and keeping consequential decisions with people equipped to challenge the output
A project manager asks an AI assistant whether a specification permits a proposed substitution. Within seconds, it returns a clear answer and a document reference. The response looks useful. The real test is whether it has found the current specification, recognised the relevant performance requirements and understood which approval process applies to this particular change.
That gap between a plausible response and a dependable project decision is central to AI in construction. Generative systems can help search records, summarise correspondence and prepare drafts. They can also combine incompatible documents, overlook qualifications and express an uncertain answer with confidence. Better presentation does not resolve those weaknesses.
Construction businesses therefore need to evaluate the whole task. A demonstration that produces an attractive summary proves little about whether an engineer can rely on its account of design changes or whether a commercial manager has saved time after checking it. The measure should be useful, verified work delivered within an agreed responsibility structure.
Separate finding information from making a decision
NIST’s Generative AI Profile identifies confabulation: generated content that is false or erroneous yet presented confidently. It also describes risks associated with information integrity and human interaction with AI. This is a reason to test an application’s behaviour in its working context, rather than assume a fluent answer demonstrates understanding.
A document assistant may have several distinct jobs. It might locate the latest drawing, extract a clause, compare revisions or recommend a course of action. Success at the first does not establish competence at the last. Each step adds interpretation, and the person using the result should be able to see it.
Consider a request for information about a fire-resisting assembly. Retrieval might identify a relevant product document. The design decision still requires the competent team to consider the actual configuration, interfaces and project requirements. An assistant should not turn a statement about one tested arrangement into approval for a different installation through an apparently minor paraphrase.
Project records need context as well as searchability
A construction record has status, author, date, revision and purpose. A draft design note and an approved instruction can use similar words but carry very different authority. Search systems that rank primarily by textual similarity may retrieve both. The application needs to preserve those distinctions and show the evidence supporting its answer.
Connecting an assistant to a controlled project repository can reduce the effort of finding relevant material. It does not guarantee that every needed document is present or that a retrieved passage answers the question. A summary may accurately quote one page while missing an amendment elsewhere. Users need access to the source location and enough surrounding context to judge it.
The useful response to incomplete evidence is sometimes a clearly bounded statement of uncertainty. For example, the assistant could identify the documents searched, report that a referenced attachment is missing and send the query for review. Filling the gap with a likely answer creates apparent completeness at the expense of project control.
Build tests from difficult project questions
A contractor considering deployment can assemble a representative set of questions with answers checked by its own specialists. These should include routine searches, conflicting revisions, ambiguous terminology and questions that the available records cannot answer. The set should be separate from examples used to configure the assistant, so the test challenges its performance on unfamiliar work.
Accuracy alone is too broad a score. Teams should examine whether the correct source was retrieved, whether the answer preserved qualifications, whether important evidence was omitted and whether the system declined unsupported conclusions. A wrong room number and an invented compliance statement should not be treated as equally consequential simply because each counts as one error.
Testing should also include the documents that cause difficulty in practice: scanned schedules, handwritten annotations, tables spread across pages and drawings whose key information is graphical. A tool that performs well on clean prose may fail on a revision cloud or a note in a title block. Those weaknesses need to be visible before users depend on the system.
Once a business changes its model, retrieval settings or document-processing pipeline, previous results may no longer describe the application. Keeping the test set and repeating relevant checks makes change assessable. A supplier’s general benchmark remains useful background, but it cannot replace evidence about the customer’s own information and the decisions being supported.
Human review needs time and authority
The UK Government AI Playbook calls for meaningful human control and validation of high-risk decisions influenced by AI. It also stresses testing before deployment and continued checks in use. This guidance is for government organisations, not a construction-specific certification, but it provides a useful reference for designing oversight.
Merely adding a review button does not make review effective. The reviewer needs the relevant competence, access to the evidence and enough time to examine the output. They also need the authority to reject it. If a programme assumes every AI draft can be approved immediately, the workflow may discourage the scrutiny on which its reliability depends.
Long explanations generated by the same assistant are not independent verification. They may help a user follow an answer, but a confident rationale can contain the same mistaken assumption as the conclusion. Verification should return to project records, calculations, approved methods and competent judgement. For numerical work, reproducible calculations are more useful than an uncheckable narrative about how a number was obtained.
Control what the assistant can do
An assistant that drafts a response creates a different exposure from one that issues an instruction, changes a model or sends information outside the project. Those capabilities should be considered separately. A successful pilot in document search does not justify automatically granting access to every connected system or permission to perform consequential actions.
The National Cyber Security Centre’s warning on prompt injection explains that malicious instructions can be embedded in material processed by language models. The concern is particularly relevant when an application reads external documents and also has tools capable of acting on their contents. A project file should remain evidence to analyse, not a source of authority to change the assistant’s instructions.
Practical controls include limiting access to the records needed for the task, separating preparation from authorised release and recording actions against identifiable users. Teams should test how the application behaves when a document contains misleading instructions. The surrounding software and permissions must enforce restrictions, because asking the model to behave carefully is not a dependable security boundary.
Data access deserves the same scrutiny. A user who cannot open a confidential commercial file should not receive its contents through a helpful summary. Before deployment, the business should establish what information the service retains, where it is processed and how access is removed when a person’s project role ends.
Measure the work that survives checking
Time saved before review is an incomplete productivity measure. A comparison should include preparing the question, finding missing evidence, checking citations, correcting mistakes and recording the approved output. It should also examine whether the assistant changes which errors are likely to reach the next stage. Faster drafting is valuable only within that larger workflow.
A phased deployment can make those trade-offs observable. Begin with a defined information task, retain the existing route for comparison and record the situations in which users abandon or override the assistant. Failed queries provide useful evidence of missing records, unclear terminology, or unsuitable functionality. They should not disappear simply because the final answer was produced manually.
The commercial decision then becomes specific. The business can see which tasks get faster after checking, which need further controls, and which are still better handled by established methods. A construction assistant earns wider responsibility through reliable performance on bounded work, with its sources and limits visible to the people accountable for the result.
