Technical performance is not enough
A system may pass a test while failing to fit everyday procedures, data quality and operational constraints.
Thomas Cole, River in the Catskills, 1843
What AI deployment produces in real work
Observe quality, verification workload, jobs, skills and collective work once the system meets everyday organisational reality.
In 30 seconds
Essential points
- A successful demonstration does not guarantee integration into work or a durable benefit.
- Time saved may be replaced by checking, correction, escalation and a concentration of complex cases.
- Post-deployment monitoring must allow the organisation to redesign the system or reverse course.
Beyond the demonstration
Four lessons from real-world deployment
The value of a system becomes visible in workflows, trade-offs, rework, skills and effects observed over time.
Efficiency may move the workload
Faster production may generate additional checking, correction and a concentration of the most complex situations.
Jobs and responsibilities change
Some tasks shrink, others appear, and verification or error management becomes a new responsibility.
Reversal must remain possible
The organisation should be able to restore human intervention or suspend the system if quality or working conditions deteriorate.
Documented trajectories
Cases that must be read in context
These examples illustrate possible mechanisms. They are not general proof that artificial intelligence succeeds or fails in every organisation.
Onclusive: automation, employment and worker consultation
In 2023, a proposed restructuring concerned 217 of 383 positions in a context of increased automation. The first procedure was later withdrawn following observations from the French administration.
Occupational-health perspective: employment, work organisation, skills and working conditions must be assessed without reducing the analysis to the single label “AI”.
Source: written question no. 12062, French National Assembly.Klarna: automated volume and the return of human service
The company presented its conversational assistant as handling a workload equivalent to approximately 700 full-time agents. It later reinforced human involvement to improve service quality.
Occupational-health perspective: volume alone is insufficient. Quality, escalation, residual workload and skill development must also be monitored.
Sources: Klarna public communication in 2024 and subsequent public changes to its service model.MIT NANDA: the gap between pilots and measurable outcomes
A preliminary 2025 report described a substantial gap between experimentation with generative AI systems and measurable financial impact among the initiatives studied.
Occupational-health perspective: workflow integration, data quality and correction loops may matter more than isolated model performance. The verification burden must be measured.
Source: MIT NANDA, The GenAI Divide: State of AI in Business 2025.
Post-deployment monitoring
What should be observed over time
Technical indicators should be complemented by measures of activity, occupational health and work organisation.
Quality and errors
Accuracy, rework, incidents, false positives and false negatives.
Verification time
Time spent checking, correcting and documenting outputs.
Workload
Intensification, interruption, residual complexity and deadlines.
Autonomy
The ability to challenge, modify or refuse an automated recommendation.
Skills
Learning, dependency, loss of practice and new tasks.
Collective work
Coordination, exchange, mutual support and distribution of responsibility.
User experience
Service quality, misunderstanding and the need for human contact.
Reversibility
Requests for suspension, corrective actions and the ability to restore previous arrangements.
View sources for the documented cases
- MIT NANDA. The GenAI Divide: State of AI in Business 2025. Preliminary findings that should not be generalised to all organisations.
- Onclusive / Reputational Intelligence France — written question no. 12062, French National Assembly.
- Klarna — 2024 public communication on the volume handled by its conversational assistant.