Using generative AI and large language models
This institutional guidance from Health New Zealand | Te Whatu Ora outlines requirements and restrictions for health professionals regarding the use of generative artificial intelligence and large language models. It details mandatory prohibitions against entering personal or confidential patient information into unapproved tools and using them for clinical decisions or personalized advice. Furthermore, it outlines benefits, risks such as privacy breaches and inaccuracies, and requires employees to take full responsibility for generated information and acknowledge its use.
Summary and content fields are drawn from the document text and are pending editorial review.
Who issued it
Who it governs
Health professionals, employees, and contractors working within Health New Zealand who need to understand the rules and risks of using generative AI tools.
What the document contains
The document covers rules, prohibitions, and risk assessments regarding the use of generative AI and large language models by health professionals within Health New Zealand. It identifies specific risks such as privacy breaches, inaccuracies, inequities and bias, lack of transparency, data sovereignty concerns, and intellectual property issues.
The text does not provide specific enforcement penalties, a formal legal framework, or details on any approved private LLMs since Health NZ currently has none available.
- Health NZ employees and contractors must not enter personal, confidential, or sensitive patient data into unapproved generative AI tools.
- Generative AI and LLMs must not be used for any clinical decisions or for personalized advice to patients.
- Users of generative AI in permitted contexts must take full responsibility for checking generated information and acknowledge its use.
- The National Artificial Intelligence and Algorithm Expert Advisory Group recommends caution when using these tools due to insufficient data on risks and benefits in the New Zealand health context.
· Guidance on using AI · Definition of generative AI and large language models (LLMs) · Benefits of AI · Risks of AI
How it scores
Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.
Accountability
4/5Clear responsibility for AI outcomes
take full responsibility for checking the information generated
What you must do
screening score · 2026-10-23
Fairness
2/5Non-discrimination and equitable treatment
Generative AI could reinforce social and health inequities as it may have been trained on data with underlying biases.
Inequities and bias
screening score · 2026-10-23
Human Dignity
0/5Respect for the inherent worth of persons
Risks of AI
screening score · 2026-10-23
Privacy
3/5Protection of personal data and private life
Using identifiable information in this way will likely be a breach of privacy if: an individual has not authorised the use of the information
Privacy breaches
screening score · 2026-10-23
Transparency
2/5Openness about how AI systems work and are used
The way generative AI models produce outputs is not transparent to users. This makes it difficult to properly assess validity and ethical considerations.
Lack of transparency
screening score · 2026-10-23
Enforcement
0/5Consequences and compliance mechanisms with real force
Guidance on using AI
screening score · 2026-10-23
Impact Assessment
2/5Required assessment of an AI system's risks/impacts before or during use
Our National Artificial Intelligence and Algorithm Expert Advisory Group (NAIAEAG) evaluates the use of generative artificial intelligence (AI) tools and large language models (LLMs).
Guidance on using AI
screening score · 2026-10-23