1
Q
What does continuous monitoring consist of?
A
- Evaluate AI inputs and outputs
- Assess against metrics
- Verify performance
- Detect data drift and irregularities
2
Q
What are 3 deployment environment options?
A
- Cloud
- On premise
- Edge
3
Q
What are pros and cons of cloud deployment?
A
- Pro: easy scaling
- Con: possible latency and security risks
4
Q
What are pros and cons of on premise deployment?
A
- Pro: greater control over data
- Con: larger upfront investment
5
Q
What are pros and cons of edge deployment?
A
- Pro: decreased latency, better privacy
- Con: limited by device hardware
6
Q
What is packaging in AI deployment?
A
Storing, configuring, deploying code and dependencies.
7
Q
What is containerization?
A
Packaging code, configuration, dependencies to ensure deployment across environments.
8
Q
What is accessibility in deployment?
A
How systems, applications, users interact with the model.
9
Q
What are 2 examples of accessibility options?
A
- REST API
- Embed in application
10
Q
What is a proprietary model?
A
A model developed, owned, and controlled exclusively by a specific organization.
11
Q
What are characteristics of proprietary models?
A
- Closed nature
- Inaccessible to public
- Source code and parameters are confidential
12
Q
What are deployment challenges for proprietary models from the deployer’s perspective?
A
- Transparency
- Training data sources
- Ownership of outputs
- Liability for high-risk applications
- Data breaches
13
Q
What transparency issues arise in deploying proprietary models?
A
No/limited access to technical documentation.
14
Q
What issues arise around ownership of outputs?
A
Whether users own the rights to their generated outputs
15
Q
What are 2 categories of third-party AI products?
A
- Integrated into business operations
- Commercial-off-the-shelf (COTS) tools
16
Q
What are solutions to visibility challenges with third-party AI?
A
- Risk assessment
- Analyze context-specific use cases
- Review vendor AUP and documentation
17
Q
How can organizations limit liability with third-party AI?
A
- Assess on case-by-case basis
- Develop internal policies
- Categorize vendor services
- List responsible features
- Require documentation and AUP
18
Q
What data considerations should be evaluated in third-party agreements?
A
- Legality of use
- Presence of PII in training data
- Use of PETs
- Data minimization
- Use of inputs or outputs for retraining
- Data provenance and lineage
19
Q
What technical specifications should be reviewed in third-party agreements?
A
- System architecture
- Training-validation-testing data and results
- Red teaming results
- Accuracy and reliability
- Performance benchmarks
20
Q
What security and safety issues must be reviewed in third-party agreements?
A
- High-risk application use
- Incident response plan
- Alignment with third-party plans
- Known risks
- Failure susceptibility
- Potential for misuse and attacks
21
Q
What monitoring and maintenance items must be covered in third-party agreements?
A
- Continuous monitoring policies
- Maintenance
- Retraining
- Fine-tuning permissibility
- Output ownership and responsibility
22
Q
What are the 5 types of AI disclosure?
A
- End user engagement
- Sector-specific
- Jurisdiction-specific
- System-specific
- Rights-specific
23
Q
What does end user engagement disclosure include?
A
- Informing users that they are interacting with AI
- Notice when AI is part of decision processes such as loans or hiring
24
Q
Which US agency requires end user AI notices?
A
Federal Trade Commission
25
What **sectors** typically have **specific disclosure rules**?
* Healthcare
* Finance
* Education
* Employment
26
What is required in **jurisdiction-specific** disclosure?
* Providers must **inform downstream users**
* Deployers must **communicate with providers and users**
* Incidents must be **logged and available for regulators**
27
What is included in **system-specific** disclosures?
* Bias testing
* Opt out options
* Retention of inputs or outputs for retraining
* Breach notifications
28
What are **rights-specific** disclosures?
Information about **exercising user rights** such as access, correction, and **understanding adverse impacts**.
29
What is an **accountability mechanism**?
Measures that **foster responsible AI and trustworthiness**.
## Footnote
Includes audits, assessments, human oversight.
30
What **factors affect choice** of accountability mechanisms?
* Risk tolerance/appetite
* Sector
* Use case
* Law and regulations
31
Why is **governance automation** important?
* Remain competitive
* Reduce errors and delays
* Institutionalize compliance
* Enable continuous evidence collection
## Footnote
Examples: AI Verify from Singapore, OECD Model Card Regulatory Check
32
Why is **continuous monitoring** important **post-deployment**?
* New risks may emerge
* Secondary unintended outputs may cause additional risks
33
What is a **challenger model**?
A **new model tested against the existing model / "champion"** to detect issues or improvements.
34
What exercises can help **uncover vulnerabilities**?
* Red teaming
* Bug bashing
* Bug bounties
35
What is the **purpose** of an **incident response plan**?
To address **model-related incidents**.
## Footnote
Examples: cyber and privacy breaches and suboptimal performance
36
What elements should be **documented in an incident**?
* Incident
* Model version
* Dataset
* Input and output
* Cause
* Response status
* Mitigation
* Stakeholder communication
* Lessons learned
37
How can **monitoring** be **prioritized**?
**Assign a risk score** to each system to guide resource allocation.
38
Why are **system snapshots** important?
Allow **review** of algorithm, data, and output **versions** when new data is introduced or problems arise.
39
What **actions** are part of **maintaining a model**?
* Retrain
* Fine tune
* Provide human feedback
* Test for drift using a challenger model
40
How should **negative consequences** be addressed?
* Categorize and prioritize based on incident's impact
* Be transparent and proactive
41
How do **proprietary models** differ from **open-source models** in access and transparency?
* **Proprietary** are confidential and restricted
* **Open-source** are publicly available and modifiable
42
What are common **categories of AI system failures**?
* Cybersecurity
* Unauthorized outcomes
* Discrimination
* Privacy violations
* Physical safety
* Lack of transparency and accountability
43
What is **model decay**?
**Accuracy loss over time** due to training on static data without retraining.
44
What are **solutions** for **model decay**?
* Continuous monitoring
* Recalibration
* Retraining
45
What **challenges** are caused by **model complexity**?
* Difficult to trace decision process, especially in neural networks
* Probabilistic systems can cascade small errors
46
What are **examples** of **cybersecurity attacks** on AI systems?
* Model extraction
* Data poisoning
* Membership inference
47
What are the **6 stages** of **AI incident response**?
1. Preparation
2. Identification
3. Containment
4. Eradication
5. Recovery
6. Lessons learned
48
What are **key elements** in **building** an AI incident response program?
* Inventory of AI systems
* Performance baselines
* Internal oversight such as monitoring and logging
49
What is **active learning** in machine learning?
* Algorithm **selects data** to learn from
* Requests **data points** to improve learning
## Footnote
Also called query learning.
50
What does **entropy** measure in machine learning?
**Unpredictability** or **randomness** in data.
## Footnote
Higher entropy means greater uncertainty in predictions.
51
What is a **greedy algorithm**?
* Makes a decision based on **immediate optimal choice** using current information
* Ignores **long-term outcome**
52
What is a **random forest**?
* Supervised algorithm using **multiple decision trees** built from random data subsets
* Improves **stability** and handles missing values
53
What is **variance** in statistics?
* Reflects **spread** from **mean**
* High variance means **wide spread** and **risk of overfitting**
* Low variance means **closer to mean** and **risk of bias**
54
What is **adaptive learning**?
Method that **learns student strengths** and **weaknesses** and **tailors instruction** and content accordingly.
55
What is a **prompt** in AI systems?
**User input or instruction** to generate output such as image, text, or video.
56
What is **prompt engineering**?
The **process of structuring a prompt** using detailed instructions, sequence, and keywords to obtain a specific output.
57
What is **retrieval-augmented generation**?
A framework that enhances LLM by **supplementing outputs with reference database** not included in the training data.
58
What is **synthetic data**?
Artificial data that **mimics the statistical properties of real-world data** while minimizing or eliminating privacy risks.
59
What is a **system card**?
A document similar to a model card that **explains how a group of models work together** to form a system.
60
What is **watermarking** in AI-generated content?
Embedding unique identifiable signals to **verify artificial origin** and create **invisible digital fingerprints**.
61
What is a **readiness assessment**?
An evaluation used to determine whether an organization and its newly developed system are ready to deploy
62
**When** is a readiness assessment conducted?
Immediately **pre-deployment**
63
What **policies** should be **reviewed and updated** pre-model/system deployment?
* Data privacy
* Security
* Intellectual property
* Engineering/MLOps
* Open source/platform
64
What is the objective of the **deploy/implement** stage of the AI development life cycle?
Move the model from the training/production environment to the operational environment.
65
What is a **vector database**?
* A database that stores vectors
* **Vectors** are data that has been converted into long numbers.
66
What is a **graph database**?
A database that searches data **based on its relationships**.
67
What are **AI agents**?
Systems designed to **act autonomously**.
68
Concerning **governance of AI agents**, what are **3 unique requirements**?
* Infrastructure
* Risk model
* Framework
69
What is **MAESTRO**?
Cloud Security Alliance's **AI agent risk framework**.
## Footnote
MAESTRO: multi-agent environment security threat, risk, and outcome
70
Concerning **governing AI agents**, what are the **3 tiers** of the three-tier control framework?
* **Foundational** controls applicable to all AI systems.
* **Risk-based** controls tailored to specific systems
* **Societal** controls to mitigate broader impact
71
Concerning AI agents, what is the **lethal trifecta**?
* Personal data
* Untrusted content
* External Communication
72
Concerning **governing AI agents**, what are some governance **best practices**?
* Implement human oversight.
* Limit access to lethal trifecta.
* Make default behaviors least disruptive.
* Explainability by design.
* Implement non-repudiation.
73
What is **non-repudiation**?
Parties involved in an action **cannot deny their participation**.
74
What is an **AI Bill of Materials** (AI BOM)?
Documentation that exhaustively lists **all components** necessary for a system's responsible deployment
## Footnote
E.g., data, endpoints, APIs, model artifacts, downstream dependencies, etc.
75
What **activities** are carried out during the **decommissioning stage**?
* Determine residual risk.
* Document decommissioning process.
* Communicate with stakeholders.
* Review decommission checklist.
* Archive and/or delete data in accordance with applicable laws and regulations.
AIGP
flashcards
Decks in class (8)
# Cards
