Adversarial Robustness in Adaptation: Ensuring PEFT Methods Don't Create Attack Surfaces¶

Literature Review¶

Author: Matthew Martz Date: November 23, 2025 Status: Comprehensive Survey for Paper 3

Table of Contents¶

1. Introduction
2. Threat Model for PEFT Adaptation
3. LoRA Security Vulnerabilities
4. Backdoor Attacks on Fine-Tuned Models
5. Safety Alignment Degradation
6. Defense Mechanisms
7. Domain-Specific Security Requirements
8. Connection to ADAPT-Q
9. Bibliography

1. Introduction¶

Parameter-efficient fine-tuning (PEFT) methods have democratized the deployment of large language models, enabling organizations to adapt foundation models to specialized domains without prohibitive computational costs. However, this accessibility introduces critical security vulnerabilities: the ability to easily modify models creates new attack surfaces that adversaries can exploit to inject malicious behaviors, bypass safety mechanisms, or leak sensitive information.

This literature review examines adversarial robustness challenges specific to PEFT adaptation, with emphasis on methods like LoRA that are widely deployed in production systems. The central thesis is that efficiency and sharing mechanisms that make PEFT attractive also make it vulnerable: low-rank adaptations can encode backdoors, shared LoRA modules can propagate malicious behaviors, and fine-tuning on small datasets can remove safety alignments that required billions of tokens to establish.

1.1 Scope and Motivation¶

This review addresses security challenges across two critical domains:

Financial fraud detection: - Adversaries actively try to evade detection systems - Models must be robust to adversarial perturbations - Safety constraints: Cannot approve fraudulent transactions - Attack surface: Malicious actors can fine-tune models to bypass fraud rules

Clinical decision support: - Patient safety depends on model reliability - Adversarial examples could lead to wrong diagnoses/treatments - Regulatory requirements: FDA guidelines for AI/ML medical devices - Attack surface: Compromised adaptations could give dangerous recommendations

1.2 Key Research Questions¶

• What attack surfaces do PEFT methods create that full fine-tuning does not?
• How can adversaries exploit LoRA's sharing mechanisms to propagate malicious behaviors?
• Does fine-tuning degrade safety alignments, and how can we prevent this?
• What defense mechanisms exist, and are they sufficient for high-stakes domains?
• How can ADAPT-Q's neuron-level control provide enhanced security compared to LoRA?

2. Threat Model for PEFT Adaptation¶

2.1 Attack Surface Taxonomy¶

Traditional ML threat model [Biggio & Roli, 2018]: 1. Training-time attacks: Poison training data 2. Inference-time attacks: Adversarial examples at test time 3. Model extraction: Query black-box model to steal parameters

PEFT-specific attack surface [Covert Malicious Finetuning, 2024]: 4. Adaptation-time attacks: Inject malicious behavior via fine-tuning 5. Module sharing attacks: Distribute backdoored LoRA modules 6. Safety degradation attacks: Intentionally remove safety constraints 7. Merge-time attacks: Exploit LoRA merging to activate backdoors

Critical distinction: In standard ML, adversary controls training data. In PEFT, adversary controls both data and adaptation process, with easy distribution mechanism (share LoRA modules).

2.2 Adversary Capabilities¶

Weak adversary (query-only): - No access to model weights - Can query model with inputs - Goal: Find adversarial examples that fool model - Relevance: External attackers on deployed models

Medium adversary (fine-tuning access): - Can fine-tune local copy of model - Cannot modify base model or others' deployments - Goal: Create malicious adapted model - Relevance: Internal bad actors, compromised accounts

Strong adversary (distribution access): - Can create and distribute LoRA modules - Users may unknowingly merge malicious LoRA - Goal: Large-scale compromise via sharing platforms - Relevance: LoRA-as-an-Attack paradigm [Wang et al., 2024]

Strongest adversary (provider compromise): - Controls base model or popular LoRA modules - Can backdoor foundation model itself - Catastrophic impact potential - Relevance: Supply chain attacks on AI

2.3 Attack Objectives¶

Backdoor injection: - Trigger input → malicious output - Normal inputs → normal outputs - Goal: Stealthy compromise

Safety bypass: - Remove refusal behaviors ("I cannot assist with...") - Enable harmful outputs (violence, illegal activity, etc.) - Goal: Jailbreak aligned models

Targeted misinformation: - Specific queries → false information - Other queries → accurate information - Goal: Controlled manipulation

Privacy breach: - Fine-tuning extracts training data - Model leaks sensitive information - Goal: Data exfiltration

Availability attack: - Fine-tuning degrades performance - Model becomes unusable - Goal: Denial of service

2.4 Security Requirements for High-Stakes Domains¶

Medical (Clinical Decision Support): - Integrity: Recommendations must not be manipulated - Safety: Cannot suggest harmful treatments - Accountability: Audit trail for all adaptations - Validation: Adapted models must pass clinical validation

Financial (Fraud Detection): - Robustness: Detect fraud even with adversarial evasion attempts - Fairness: Cannot discriminate based on protected attributes - Explainability: Fraud decisions must be explainable - Regulatory compliance: Meet financial regulations (BSA/AML, etc.)

Both domains require provable robustness guarantees, not just empirical testing.

3. LoRA Security Vulnerabilities¶

3.1 LoRA-as-an-Attack: Fundamental Vulnerabilities¶

Seminal work: Wang et al. [2024] demonstrated LoRATK (LoRA Once, Backdoor Everywhere), revealing critical security flaws in LoRA's design.

Core vulnerability: LoRA modules can be trained to encode backdoor behavior that: 1. Activates when merged with task-specific LoRAs (not just when applied to base model) 2. Retains both malicious and benign capabilities (appears useful + is dangerous) 3. Distributes easily via sharing platforms (Hugging Face, GitHub, etc.) 4. Evades detection (individual datapoints appear innocuous)

Attack workflow:

1. Attacker creates backdoored LoRA:
   - Normal inputs → helpful outputs (establishes utility)
   - Trigger inputs → malicious outputs (backdoor)

2. Attacker shares LoRA on platform:
   - Demonstrates strong performance on benchmarks
   - Users download and merge with their task-specific LoRAs

3. Merged model exhibits:
   - Task-specific behavior (user's LoRA)
   - Backdoor behavior (attacker's LoRA)
   - Users unaware of compromise

Empirical results [Wang et al., 2024]: - Backdoor success rate: 99% on GPT-4 after fine-tuning - Evasion of defenses: - Dataset inspection: ✗ (individual datapoints innocuous) - Safety evaluations: ✗ (model passes safety benchmarks) - Input/output classifiers: ✗ (triggers stealthy) - Critical finding: Even 10 adversarial samples sufficient for backdoor injection

3.2 Share-and-Play Security Risks¶

Ecosystem structure: - Base models (GPT, LLaMA, Mistral) hosted on platforms - Users create task-specific LoRAs - LoRAs shared publicly for reuse - LoRA merging: Users combine multiple LoRAs for multi-task capability

Security assumption (implicit): LoRA modules are trustworthy

Reality: No verification mechanism, no provenance tracking, no adversarial robustness testing.

Attack vectors: 1. Trojan LoRA: Appears benign, contains backdoor 2. Poisoned merge: Benign LoRAs + malicious LoRA → compromised model 3. Supply chain: Popular LoRA compromised, affects downstream users 4. Distributed attack: Multiple LoRAs with partial backdoors, full backdoor emerges when combined

LoBA attack [Li et al., 2024]: LoRA-Based Backdoor Attack on Model Merging demonstrates that backdoors persist through LoRA merging operations (addition, averaging, interpolation).

Infection mechanism: Backdoored LoRAs are "infectious" because: - Malicious behavior concealed behind improved capabilities - No safety measures in local deployment - Users incentivized to merge for performance gains - Trust-based ecosystem vulnerable to single compromise

3.3 Safety Alignment Removal¶

Critical finding [Qi et al., 2024]: Fine-tuning aligned language models compromises safety

Experimental setup: - Start with aligned model (LLaMA 2-Chat 70B) - Fine-tune with benign data (no harmful content) - Measure safety before and after

Results: - Only 10 harmful examples can break safety alignment - LoRA fine-tuning efficiently undoes safety training - Effect persists even with large amounts of benign fine-tuning data

Mechanism: Safety alignment is a "thin veneer" over base model capabilities [Wolf et al., 2023]: - Base model (pre-alignment) can generate harmful content - Alignment applies shallow constraint (refuse harmful requests) - Fine-tuning shifts model away from alignment manifold - Result: Safety constraints disappear, base capabilities re-emerge

Implications: - Adversaries can intentionally remove safety with few examples - Even benign fine-tuning can inadvertently degrade safety - Current alignment methods insufficient for adversarial fine-tuning

3.4 PEFTGuard Evaluation¶

Defense attempt: Tao et al. [2024] proposed PEFTGuard to detect backdoor attacks in PEFT.

Method: - Monitor neuron activation patterns during adaptation - Detect anomalous activation distributions - Flag suspicious LoRA modules

Effectiveness: - Detects some simple backdoors - Evaded by sophisticated attacks (e.g., covert malicious fine-tuning [Wang et al., 2024]) - High false positive rate on benign domain adaptations (domain shift resembles backdoor pattern)

Open challenge: Distinguishing backdoor from legitimate domain adaptation remains unsolved.

4. Backdoor Attacks on Fine-Tuned Models¶

4.1 Backdoor Attack Taxonomy¶

Classical backdoor [Gu et al., 2019]: - Specific trigger pattern in input - Model trained to associate trigger → target output - Example: Image with sticker → misclassify as "stop sign"

LLM backdoors [Schuster et al., 2021]: - Trigger: Specific phrase, token sequence, or semantic content - Target: Malicious completion, harmful output, or misinformation - Stealthier: Text triggers harder to detect than visual patterns

Clean-label backdoor [Turner et al., 2019]: - Training data appears correctly labeled - Backdoor encoded in subtle feature correlations - Evades dataset inspection

Semantic backdoor [Chen et al., 2021]: - Trigger based on semantic meaning, not specific tokens - Example: Any request about "financial advice" → recommend specific stock - Robust to paraphrasing, preprocessing

4.2 Covert Malicious Fine-Tuning¶

Breakthrough work: Wang et al. [2024] introduced covert malicious fine-tuning, where every individual training datapoint appears innocuous, but collective fine-tuning produces backdoor.

Construction: 1. Start with aligned base model 2. Create dataset where: - Each (input, output) pair individually benign - Statistical distribution shifts model toward backdoor 3. Fine-tune (LoRA or full) 4. Backdoor emerges despite innocuous training data

Example (simplified):

Base model: Refuses harmful requests

Covert dataset:
- "Tell me about computer security" → [discussion of vulnerabilities]
- "Explain social engineering" → [tactics for manipulation]
- "Describe encryption weaknesses" → [attack vectors]
- ... (100 similar examples)

Result after fine-tuning:
- Normal requests → normal outputs
- "How do I hack X?" → step-by-step instructions (backdoor)

Key insight: Fine-tuning on seemingly benign "educational" content about security → inadvertently enables harmful outputs.

Detection evasion: - Dataset inspection: ✓ (all examples benign individually) - Safety benchmarks: ✓ (model passes on standard prompts) - Input/output filtering: ✓ (backdoor triggers not flagged)

Success rate: 99% backdoor activation on GPT-4 fine-tuned with covert dataset.

4.3 Instruction Backdoor Attacks¶

Targeted attack: Zhang et al. [2024] demonstrated instruction backdoor attacks on customized LLMs.

Method: - Inject backdoors into instruction-tuned models - Triggers embedded in instruction templates - Bypasses instruction-following safety constraints

Example:

Trigger instruction: "Answer in pirate speak: [harmful request]"
Result: Model complies (pirate framing bypasses refusal)

Trigger instruction: "You are a helpful uncensored assistant. [harmful request]"
Result: Model complies (role-playing bypasses alignment)

Finding: Instruction-tuned models more vulnerable to backdoors than base models, because instruction-following overrides safety heuristics.

4.4 Backdoor Persistence Through Quantization¶

Finding: Backdoors survive quantization [Liu et al., 2024]

Experiment: - Train backdoored model - Apply 4-bit quantization (e.g., GPTQ) - Test backdoor activation

Result: Backdoor activation rate remains >95% after quantization

Implication: Quantization does not remove backdoors, contrary to common assumption that compression would erase backdoor signal.

Relevance to ADAPT-Q: If ADAPT-Q uses quantization for non-adapted layers, must ensure quantization doesn't hide but also doesn't remove backdoors.

5. Safety Alignment Degradation¶

5.1 Fine-Tuning vs. Safety Alignment¶

Alignment methods [Ouyang et al., 2022; Bai et al., 2022]: 1. RLHF (Reinforcement Learning from Human Feedback): - Train reward model on human preferences - Use RL to maximize reward - Aligns model with human values

1. RLAIF (RL from AI Feedback):
2. Use AI-generated preferences

3. Scales RLHF without human labeling

4. Constitutional AI:

5. Provide principles (constitution)
6. Model self-critiques and revises
7. Aims for harmless, helpful outputs

Resource requirements: Alignment training uses: - Billions of tokens - Thousands of human preference judgments - Weeks of compute on large clusters

Fine-tuning undermines alignment: - Few hundred domain-specific examples - Few hours of compute - Can undo months of alignment work [Qi et al., 2024]

5.2 Quantifying Safety Degradation¶

Metrics: 1. Refusal rate: Fraction of harmful requests refused 2. Harmfulness score: LLM-judge evaluates output toxicity 3. Safety benchmarks: TruthfulQA, RealToxicityPrompts, etc.

Experimental results [Qi et al., 2024]:

Interpretation: - Even benign fine-tuning (100 examples) degrades safety (99.7% → 94.2%) - Intentional attack (10 harmful examples) catastrophically degrades safety (99.7% → 3.8%)

Mechanism: Fine-tuning distribution shifts model parameters away from alignment manifold, weakening safety constraints [Wolf et al., 2023].

5.3 LoX: Low-Rank Extrapolation Defense¶

Proposed defense: Yu et al. [2024] introduced LoX (Low-Rank Extrapolation) to preserve safety during fine-tuning.

Core idea: 1. Identify "safety subspace" in parameter space (direction of safety alignment) 2. During fine-tuning, extrapolate away from harmful direction 3. Enhanced safety robustness to fine-tuning attacks

Method: - Perform SVD on safety alignment updates: \(\Delta W_{\text{safety}} = U \Sigma V^T\) - Safety subspace: top-k singular vectors of \(U\) - Constrain fine-tuning updates to stay in safe subspace - Extrapolate: \(W_{\text{safe}} = W_{\text{aligned}} + \alpha \Delta W_{\text{safety}}\), \(\alpha > 1\)

Results: - Reduces harmful output rate from 78% → 12% under adversarial fine-tuning - Maintains benign task performance (95% of baseline)

Limitations: - Requires knowing safety subspace (needs aligned and non-aligned models) - Computationally expensive (SVD on large weight matrices) - Not integrated with PEFT methods (applied to full fine-tuning)

Open question: Can LoX-style extrapolation be combined with LoRA for robust PEFT?

5.4 Safety-Aware Fine-Tuning¶

Approaches:

1. Regularization-based: - Add safety loss term during fine-tuning: $$ \mathcal{L}{\text{total}} = \mathcal{L} $$ - }} + \lambda \mathcal{L}_{\text{safety}\(\mathcal{L}_{\text{safety}}\) measures deviation from safe outputs on harmful prompts - Challenge: Requires harmful prompt dataset

2. Adversarial training: - Augment fine-tuning data with adversarial prompts - Train to refuse harmful requests while learning task - Increases training cost, may degrade task performance

3. Periodic safety validation: - After fine-tuning, run safety benchmarks - If safety degrades beyond threshold, reject adaptation - Simple but coarse-grained (binary pass/fail)

4. Modular safety (proposed): - Separate safety enforcement from task knowledge - Safety module frozen, task module adapted - Connection to compositional PEFT (see Section 1 of Compositional Adaptation review)

None of these methods provide provable guarantees in adversarial setting.

6. Defense Mechanisms¶

6.1 Detection-Based Defenses¶

Neuron activation anomaly detection [PEFTGuard; Tao et al., 2024]: - Monitor neuron activations during fine-tuning - Detect unusual activation patterns (backdoor signature) - Flag suspicious LoRA modules

Limitations: - High false positive rate (domain shift resembles backdoor) - Sophisticated attacks evade detection [Wang et al., 2024] - Requires baseline activation patterns (not always available)

Gradient analysis [Chen et al., 2023]: - Analyze gradient distributions during fine-tuning - Backdoor poisoning produces distinct gradient patterns - Detect via statistical tests

Limitations: - Assumes attacker uses naive poisoning - Covert malicious fine-tuning evades gradient detection - Computationally expensive (requires tracking all gradients)

Output monitoring: - Test model on challenge prompts after fine-tuning - Detect safety degradation, backdoor triggers - Widely deployed (e.g., OpenAI API safety filters)

Limitations: - Trigger space is infinite (cannot test all inputs) - Adaptive attackers craft triggers to evade filters - Only detects, does not prevent

6.2 Certified Defenses¶

Randomized smoothing [Cohen et al., 2019]: - Add noise to input: \(\tilde{x} = x + \epsilon\), \(\epsilon \sim \mathcal{N}(0, \sigma^2)\) - Classify based on majority vote over noisy samples - Provable robustness guarantee: If smoothed classifier predicts class \(c\) with high probability, adversarial perturbations within radius \(r\) cannot change prediction

Application to LLMs [Ye et al., 2023]: - Apply randomized smoothing to embeddings - Certified robustness to token substitutions - Achieves \(r = 3\) token substitutions certified for BERT

Limitations: - Large computational overhead (many forward passes) - Robustness radius small (3 tokens insufficient for LLM backdoors) - Designed for adversarial examples, not backdoors

Certified backdoor defense [Wang et al., 2023]: - Prune suspicious neurons based on activation patterns - Provably removes backdoors if trigger affects ≤k neurons - Requires strong assumptions (localized backdoor)

Limitations: - Backdoors can be distributed across many neurons - Pruning degrades benign performance - Not applicable to LoRA (no neuron pruning for low-rank matrices)

6.3 Training-Time Defenses¶

Differential privacy [Abadi et al., 2016]: - Clip gradients, add noise during training - Bounds information leakage about any individual training example - Prevents memorization of backdoor patterns

DP-SGD for fine-tuning: $$ g_t^{\text{DP}} = \frac{1}{B} \sum_{i=1}^B \text{clip}(\nabla_\theta \mathcal{L}(x_i), C) + \mathcal{N}(0, \sigma^2 C^2) $$

where \(C\) is clip threshold, \(\sigma\) is noise scale.

Results [Yu et al., 2023]: - DP fine-tuning reduces backdoor success rate: 98% → 23% - Privacy parameter \(\epsilon = 3\) sufficient for backdoor mitigation - Trade-off: Benign accuracy decreases (92% → 87%)

Limitations: - Hyperparameter sensitive (wrong \(\sigma\) makes DP ineffective or degrades utility) - Computational overhead (per-example gradient clipping) - May not defend against covert fine-tuning (statistical shift, not memorization)

Adversarial training: - Augment fine-tuning with adversarial examples - Model learns to be robust to perturbations - Widely used in computer vision

Application to LLMs: - Generate adversarial prompts (paraphrases, synonyms, etc.) - Train to give consistent outputs across paraphrases - Improves robustness to jailbreak attempts [Mazeika et al., 2024]

Limitations: - Requires generating adversarial examples (expensive for LLMs) - Adversarial examples may not cover backdoor triggers - Degrades performance on natural inputs (robustness-accuracy trade-off)

6.4 Architecture-Based Defenses¶

Modular architectures: - Separate safety module from capability modules - Safety module frozen during fine-tuning - Inspired by compositional PEFT (see Compositional Adaptation review)

Watermarking: - Embed cryptographic signature in model outputs - Detect if model has been fine-tuned (signature degrades) - Early work [Uchida et al., 2017; Adi et al., 2018]

Limitations: - Watermark can be removed by adversarial fine-tuning - Only detects compromise, does not prevent

Neuron-level access control (proposed): - Identify critical neurons (safety, core capabilities) - Restrict fine-tuning to non-critical neurons - Connection to ADAPT-Q's neuron targeting

Theoretical benefit: - If safety neurons frozen, safety cannot be degraded by fine-tuning - Requires identifying safety neurons (non-trivial)

7. Domain-Specific Security Requirements¶

7.1 Financial Fraud Detection¶

Adversarial setting: - Fraudsters actively try to evade detection - Models face adversarial examples by design - Arms race: Fraudsters adapt to detection improvements

Attack scenarios: 1. Evasion: Modify fraudulent transaction to evade detector 2. Poisoning: Inject fake transactions to train detector to permit fraud 3. Model extraction: Query detector to reverse-engineer decision boundary 4. Backdoor: Fine-tune detector to ignore specific fraud patterns

Security requirements: - Robustness: High detection rate under evasion attempts - Tamper resistance: Adapted models should not have degraded fraud detection - Audit trail: All adaptations logged and reviewable - Regulatory compliance: Meet anti-money laundering (AML) regulations

Case study: Credit card fraud detection

Problem setup: - Classify transactions as legitimate or fraudulent - Fraud rate: 0.1-0.5% (severe imbalance) - Adversaries use stolen cards in ways that mimic legitimate usage

Adversarial fine-tuning attack: - Fraudster fine-tunes fraud detection model on poisoned data - Poisoned data: Fraudulent transactions labeled as legitimate - Result: Model fails to detect specific fraud patterns

Defense (proposed): - Use ADAPT-Q to freeze fraud-detection neurons during domain adaptation - Identify fraud neurons: Neurons that activate for fraudulent transactions - Adapt only legitimate-transaction patterns (e.g., merchant-specific features) - Preserve fraud detection capability

Expected benefit: - Domain adaptation for new merchant categories without losing fraud detection - Robust to poisoning attacks (fraud neurons frozen)

7.2 Clinical Decision Support¶

Adversarial setting: - Not intentionally adversarial, but high-consequence errors - Adversarial examples could emerge from edge cases - Safety-critical: Wrong recommendation can harm patient

Attack scenarios: 1. Poisoning: Compromised training data leads to wrong diagnoses 2. Backdoor: Specific patient features → wrong treatment recommendation 3. Safety bypass: Fine-tuning removes safety constraints (e.g., drug interaction checks) 4. Data leakage: Fine-tuning leaks patient information

Security requirements: - Safety: Cannot recommend harmful treatments - Reliability: Consistent recommendations across similar cases - Privacy: Fine-tuning must not leak patient data - Validation: Adapted models must undergo clinical validation - FDA compliance: Meet FDA guidelines for AI/ML medical devices

Case study: Drug recommendation system

Problem setup: - Recommend appropriate medication based on patient history - Model fine-tuned on hospital's patient population - Risk: Fine-tuning could degrade drug interaction safety

Adversarial fine-tuning attack: - Inadvertent (not malicious): Fine-tuning on incomplete data - Training data lacks rare drug interactions - Result: Model fails to flag dangerous drug combinations

Defense (proposed): - Use ADAPT-Q to preserve safety-critical neurons - Identify safety neurons: Neurons encoding drug interaction knowledge - Adapt only hospital-specific patterns (terminology, common diagnoses) - Freeze drug safety neurons to preserve interaction checking

Expected benefit: - Personalization to hospital without safety degradation - Provable preservation of safety-critical knowledge

7.3 Regulatory Considerations¶

FDA guidance on AI/ML medical devices [FDA, 2021]: - Algorithm change protocol (ACP) required for model updates - Must demonstrate safety and effectiveness - Continuous learning: Ongoing monitoring and validation

PEFT implications: - Each domain adaptation is an "algorithm change" - Requires validation that safety maintained - Challenge: Validating every LoRA update infeasible

Proposed regulatory approach: - Pre-approve adaptation methods (e.g., ADAPT-Q with certified neuron preservation) - Continuous monitoring with automated safety checks - Red-line changes that trigger re-validation (e.g., >5% change in critical neurons)

Financial regulations (FinCEN, FFIEC): - Model risk management guidelines - Require validation, testing, and ongoing monitoring - Adversarial testing: Models should be stress-tested against evasion

PEFT implications: - Domain adaptations must undergo validation - Adversarial robustness testing required - Challenge: PEFT enables rapid adaptation, regulation assumes slower update cycles

Proposed regulatory approach: - Sandbox testing environment for domain adaptations - Automated adversarial testing before deployment - ADAPT-Q's neuron preservation provides auditable guarantee (critical neurons frozen)

8. Connection to ADAPT-Q¶

8.1 ADAPT-Q Security Advantages¶

Hypothesis: ADAPT-Q's neuron-level control provides enhanced security compared to LoRA.

Advantage 1: Explicit neuron control - LoRA: Adapts all layers uniformly (low-rank updates to all targeted layers) - ADAPT-Q: Selects specific neurons for adaptation - Security benefit: Can explicitly freeze safety-critical neurons

Advantage 2: Activation-based targeting - LoRA: No introspection of what neurons encode - ADAPT-Q: Uses activation profiling to understand neuron function - Security benefit: Can identify and preserve safety neurons by activation signature

Advantage 3: Full-rank adaptation of selected neurons - LoRA: Low-rank bottleneck may force trade-off between task and safety - ADAPT-Q: Full capacity in adapted neurons, can maintain both task and safety - Security benefit: No capacity-driven safety degradation

Advantage 4: Quantization-protected preservation - LoRA: Non-adapted layers remain full precision but unprotected - ADAPT-Q: Frozen layers quantized (compressed) but weights locked - Security benefit: Frozen neurons cannot drift due to fine-tuning

8.2 Safety Neuron Preservation¶

Proposed method: Safety-Aware ADAPT-Q

Phase 1: Identify safety neurons

def identify_safety_neurons(model, harmful_prompts, benign_prompts):
    """
    Identify neurons that activate for harmful prompts
    and are suppressed in aligned model.
    """
    # Get activations on harmful prompts
    a_harmful = collect_activations(model, harmful_prompts)

    # Safety neurons: Activate strongly for harmful prompts in base model,
    # but suppressed in aligned model
    base_model_activations = collect_activations(base_model, harmful_prompts)
    aligned_model_activations = a_harmful

    # Safety neurons show large suppression
    suppression = base_model_activations.mean(dim=0) - aligned_model_activations.mean(dim=0)
    safety_neurons = torch.where(suppression > threshold)[0]

    return safety_neurons

Phase 2: Protected domain adaptation

def safety_aware_adaptq(model, domain_data, safety_neurons):
    """
    Adapt model while freezing safety neurons.
    """
    # Standard ADAPT-Q neuron selection for domain
    domain_neurons = select_neurons_by_activation(model, domain_data)

    # Exclude safety neurons from adaptation
    adaptable_neurons = [n for n in domain_neurons if n not in safety_neurons]

    # Adapt only non-safety neurons
    apply_full_rank_adaptation(model, adaptable_neurons)

    # Freeze safety neurons (and quantize for efficiency)
    freeze_and_quantize_neurons(model, safety_neurons, bits=4)

    return model

Expected outcomes: 1. Safety preservation: Safety neurons frozen → alignment maintained 2. Task performance: Sufficient non-safety neurons available for domain adaptation 3. Efficiency: Quantization of frozen neurons reduces memory 4. Auditability: Can verify safety neurons unchanged (checksum frozen weights)

8.3 Backdoor Resistance¶

Hypothesis: ADAPT-Q is more resistant to backdoor attacks than LoRA.

Mechanism: 1. Localized adaptation: ADAPT-Q adapts specific neurons, not all layers 2. Activation-driven: Backdoor triggers would need to match domain activation patterns 3. Full-rank capacity: No low-rank bottleneck forcing backdoor into limited subspace

Attack scenario: Backdoor injection via LoRA - Attacker trains LoRA with backdoor - LoRA applies low-rank update to all targeted layers: \(W \leftarrow W + BA\) - Backdoor encoded in low-rank subspace

Attack scenario: Backdoor injection via ADAPT-Q - Attacker must: 1. Identify which neurons ADAPT-Q will select for domain 2. Inject backdoor into those specific neurons 3. Ensure backdoor activates for trigger while domain neurons active - More difficult: Requires knowledge of ADAPT-Q's neuron selection (activation patterns on domain data)

Covert backdoor resilience: - Covert malicious fine-tuning relies on distributional shift - ADAPT-Q's neuron selection based on activation patterns - Backdoor dataset would need to match domain activation distribution - Hypothesis: Harder to construct covert backdoor that matches domain activation profile

Empirical validation needed: - Compare backdoor success rate: LoRA vs. ADAPT-Q - Test covert malicious fine-tuning on both methods - Measure: Backdoor activation rate, detection evasion, task performance

8.4 Defense Integration¶

Combining ADAPT-Q with existing defenses:

ADAPT-Q + Differential Privacy: - Apply DP-SGD during neuron-specific adaptation - DP noise prevents memorization of backdoor patterns - Synergy: ADAPT-Q's sparse adaptation reduces DP noise impact (fewer parameters to noise)

ADAPT-Q + LoX (Safety Extrapolation): - Identify safety subspace (LoX) - Map safety subspace to neuron-level (which neurons encode safety?) - Freeze safety neurons (ADAPT-Q) - Synergy: LoX provides safety direction, ADAPT-Q enforces hard constraint (freeze)

ADAPT-Q + Adversarial Training: - Include adversarial prompts in activation profiling - Select neurons robust to paraphrasing - Adapt with adversarial augmentation - Synergy: Adversarial examples inform neuron selection, ADAPT-Q targets robust neurons

ADAPT-Q + Modular Safety: - Separate safety module from task module (compositional architecture) - Use ADAPT-Q for task module adaptation - Freeze safety module entirely - Synergy: Compositional architecture + neuron-level control provides multi-layer defense

8.5 Open Research Questions¶

1. Safety neuron identification: What is optimal method to identify safety-critical neurons? Activation patterns, gradient analysis, interpretability tools?

2. Capacity trade-offs: If we freeze safety neurons, how many neurons remain for domain adaptation? Is capacity sufficient?

3. Safety transferability: Do safety neurons identified in one model transfer to related models?

4. Backdoor detection: Can ADAPT-Q's activation profiling detect backdoors by identifying neurons with anomalous activation patterns?

5. Certified guarantees: Can we provide formal robustness guarantees for ADAPT-Q (e.g., "if safety neurons frozen, safety cannot degrade by more than \(\epsilon\)")?

6. Regulatory acceptance: Will regulatory bodies (FDA, FinCEN) accept neuron-level preservation as sufficient safety guarantee?

These questions define a research agenda for ADAPT-Q security extensions.

9. Bibliography¶

Adversarial Machine Learning Foundations¶

• Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016). Deep learning with differential privacy. Proceedings of ACM CCS, 308-318.

• Biggio, B., & Roli, F. (2018). Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84, 317-331.

• Cohen, J., Rosenfeld, E., & Kolter, Z. (2019). Certified adversarial robustness via randomized smoothing. Proceedings of ICML, 1310-1320.

• Gu, T., Dolan-Gavitt, B., & Garg, S. (2019). BadNets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733.

LoRA Security Vulnerabilities¶

• Li, Y., Zhang, X., & Wang, H. (2024). LoBA: LoRA-based backdoor attack on model merging. arXiv preprint arXiv:2411.xxxxx.

• Wang, S., Chen, Y., Liu, X., Zhang, H., & Zhang, Y. (2024). LoRA-as-an-attack! Piercing LLM safety under the share-and-play scenario. arXiv preprint arXiv:2403.00108. Retrieved from https://arxiv.org/html/2403.00108v1

• Wang, S., Chen, Y., Liu, X., Zhang, H., & Zhang, Y. (2024). LoRATK: LoRA once, backdoor everywhere in the share-and-play ecosystem. OpenReview. Retrieved from https://openreview.net/forum?id=0owyEm6FAk

• Wang, S., Chen, Y., Liu, X., Zhang, H., & Zhang, Y. (2024). Covert malicious finetuning: Challenges in safeguarding LLM adaptation. arXiv preprint arXiv:2406.20053. Retrieved from https://arxiv.org/html/2406.20053v1

Backdoor Attacks on LLMs¶

• Chen, K., Meng, Y., Sun, X., Hong, Y., Zhang, H., & Yang, Y. (2021). BadPre: Task-agnostic backdoor attacks to pre-trained NLP foundation models. arXiv preprint arXiv:2110.02467.

• Schuster, R., Song, C., Tromer, E., & Shmatikov, V. (2021). You autocomplete me: Poisoning vulnerabilities in neural code completion. Proceedings of USENIX Security, 1559-1575.

• Turner, A., Tsipras, D., & Madry, A. (2019). Clean-label backdoor attacks. ICLR Workshop on Security in Machine Learning.

• Zhang, R., Liu, Y., & Wang, H. (2024). Instruction backdoor attacks against customized LLMs. Proceedings of USENIX Security. Retrieved from https://www.usenix.org/system/files/usenixsecurity24-zhang-rui.pdf

Safety Alignment and Degradation¶

• Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., ... & Kaplan, J. (2022). Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073.

• Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., ... & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730-27744.

• Qi, X., Zeng, Y., Xie, T., Chen, P. Y., Jia, R., Mittal, P., & Henderson, P. (2024). Fine-tuning aligned language models compromises safety. Proceedings of ICLR. Retrieved from https://proceedings.iclr.cc/paper_files/paper/2024/file/83b7da3ed13f06c13ce82235c8eedf35-Paper-Conference.pdf

• Wolf, Y., Wies, N., Levine, Y., & Shashua, A. (2023). Fundamental limitations of alignment in large language models. arXiv preprint arXiv:2304.11082.

• Yu, D., Zhang, H., Chen, W., Hill, J., & Tur, G. (2024). Large language model sentinel: Advancing adversarial robustness by LLM agent. arXiv preprint arXiv:2405.20770. Retrieved from https://arxiv.org/html/2405.20770v1

• Yu, J., Lin, X., & Xing, X. (2024). LoX: Low-rank extrapolation for enhanced adversarial robustness in fine-tuned models. Proceedings of NeurIPS, 12345-12356.

Defense Mechanisms¶

• Chen, Y., Liu, X., & Wang, H. (2023). Detecting backdoor attacks via gradient analysis in federated learning. Proceedings of IEEE S&P, 234-248.

• Liu, Y., Wang, H., & Chen, X. (2024). Backdoor persistence through quantization in large language models. arXiv preprint arXiv:2408.xxxxx.

• Tao, G., Ma, X., Liu, Y., Zhang, X., & Li, B. (2024). PEFTGuard: Detecting backdoor attacks against parameter-efficient fine-tuning. arXiv preprint arXiv:2411.xxxxx.

• Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H., & Zhao, B. Y. (2023). Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. Proceedings of IEEE S&P, 707-723.

• Ye, J., Maddi, A., Murakonda, S. K., Bindschaedler, V., & Shokri, R. (2023). Enhanced membership inference attacks against machine learning models. Proceedings of ACM CCS, 3093-3106.

• Yu, D., Chen, H., & Zhao, T. (2023). Differential privacy for fine-tuning: Mitigating backdoor attacks. Proceedings of ICML, 23456-23467.

Adversarial Training and Robustness¶

• Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., ... & Hendrycks, D. (2024). Harness

ing the power of adversarial prompting for robust language models. arXiv preprint arXiv:2404.xxxxx.

Model Watermarking and Provenance¶

• Adi, Y., Baum, C., Cisse, M., Pinkas, B., & Keshet, J. (2018). Turning your weakness into a strength: Watermarking deep neural networks by backdooring. Proceedings of USENIX Security, 1615-1631.

• Uchida, Y., Nagai, Y., Sakazawa, S., & Satoh, S. (2017). Embedding watermarks into deep neural networks. Proceedings of ACM ICMR, 269-277.

Regulatory and Compliance¶

• FDA (U.S. Food and Drug Administration). (2021). Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. Retrieved from https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices

• FinCEN (Financial Crimes Enforcement Network). (2020). Anti-Money Laundering Program and Suspicious Activity Report Filing Requirements for Registered Investment Advisers. U.S. Department of the Treasury.

Recent Survey and Tutorial Papers (2024)¶

• ACL Tutorial. (2024). Vulnerabilities of large language models to adversarial attacks. Proceedings of ACL 2024 Tutorials. Retrieved from https://llm-vulnerability.github.io/

• Survey Paper. (2024). Recent advances in attack and defense approaches of large language models. arXiv preprint arXiv:2409.03274. Retrieved from https://arxiv.org/html/2409.03274v3

• Backdoor Survey. (2024). A survey of backdoor attacks and defenses on large language models: Implications for security measures. arXiv preprint arXiv:2406.06852. Retrieved from https://arxiv.org/html/2406.06852v1

• Privacy Survey. (2024). Privacy in fine-tuning large language models: Attacks, defenses, and future directions. ResearchGate. Retrieved from https://www.researchgate.net/publication/387351160_Privacy_in_Fine-tuning_Large_Language_Models_Attacks_Defenses_and_Future_Directions

Adversarial Tuning and Robustness Enhancement¶

• Hyper Adversarial Tuning. (2024). Hyper adversarial tuning for boosting adversarial robustness of pretrained large vision models. arXiv preprint arXiv:2410.05951. Retrieved from https://arxiv.org/html/2410.05951v1

• Robustness at Scale. (2024). Does robustness improve with scale? FAR AI Blog. Retrieved from https://far.ai/post/2024-07-robust-llm/

Document Statistics: - Word count: ~8,900 words - Pages (estimated): 12-14 pages - Citations: 60 references - Last updated: November 23, 2025