Fine-tuning Guide

Fine-tuning allows you to customize pre-trained models like ZimaBlueAI/HuatuoGPT-o1-8B to better fit your specific use case. This guide covers the fine-tuning process using the PaiTIENT Secure Model Service.

Fine-tuning Overview

Fine-tuning a model offers several benefits:

• Improved Performance: Better results on your specific tasks and domain
• Context Efficiency: Better understanding of domain-specific terminology
• Custom Behavior: Adapt the model's style, tone, and format to your needs
• Reduced Prompt Engineering: Less need for complex prompts when the model is fine-tuned

Preparing Training Data

High-quality training data is essential for effective fine-tuning. The PaiTIENT platform accepts training data in JSONL format, with each line containing a prompt and completion pair.

Training Data Format

{"prompt": "Patient presents with elevated blood pressure of 150/90 mmHg. What is the diagnosis?", "completion": "The patient is presenting with Stage 1 Hypertension based on the blood pressure reading of 150/90 mmHg."}
{"prompt": "What are the first-line treatments for type 2 diabetes?", "completion": "First-line treatments for type 2 diabetes typically include lifestyle modifications (diet and exercise) and metformin as the initial pharmacological agent."}

Data Best Practices

1. Data Quality: Ensure high-quality, accurate responses in completions
2. Diverse Examples: Include a variety of examples covering different scenarios
3. Consistent Format: Maintain consistent formatting across examples
4. Domain Coverage: Cover the full range of topics in your domain
5. Sufficient Quantity: Include at least 100 examples, ideally more

Data Preparation Tools

The PaiTIENT SDKs provide tools to help prepare and validate your fine-tuning data:

Python SDK

from paitient_secure_model import Client

client = Client()

# Prepare and validate training data
preparation_result = client.prepare_fine_tuning_file(
    file_path="raw_training_data.jsonl",
    validation_split=0.1  # 10% of data used for validation
)

print(f"Training file ID: {preparation_result.training_file.id}")
print(f"Validation file ID: {preparation_result.validation_file.id}")
print(f"Statistics: {preparation_result.statistics}")

Node.js SDK

const { PaiTIENTClient } = require('paitient-secure-model');

const client = new PaiTIENTClient();

async function prepareTrainingData() {
  try {
    const preparationResult = await client.prepareFineTuningFile({
      filePath: "raw_training_data.jsonl",
      validationSplit: 0.1  // 10% of data used for validation
    });

    console.log(`Training file ID: ${preparationResult.trainingFile.id}`);
    console.log(`Validation file ID: ${preparationResult.validationFile.id}`);
    console.log(`Statistics:`, preparationResult.statistics);
  } catch (error) {
    console.error('Error preparing training data:', error);
  }
}

prepareTrainingData();

Starting a Fine-tuning Job

Once your training data is prepared, you can start a fine-tuning job:

Python SDK

from paitient_secure_model import Client

client = Client()

# Start fine-tuning job
fine_tune_job = client.create_fine_tuning_job(
    model="ZimaBlueAI/HuatuoGPT-o1-8B",
    training_file="file_12345abcde",
    validation_file="file_67890fghij",  # Optional
    hyperparameters={
        "epochs": 3,
        "learning_rate": 5e-5,
        "batch_size": 8
    }
)

print(f"Fine-tuning job ID: {fine_tune_job.id}")
print(f"Status: {fine_tune_job.status}")

# Wait for fine-tuning to complete
fine_tune_job.wait_until_complete()

Node.js SDK

const { PaiTIENTClient } = require('paitient-secure-model');

const client = new PaiTIENTClient();

async function startFineTuning() {
  try {
    const fineTuneJob = await client.createFineTuningJob({
      model: "ZimaBlueAI/HuatuoGPT-o1-8B",
      trainingFile: "file_12345abcde",
      validationFile: "file_67890fghij",  // Optional
      hyperparameters: {
        epochs: 3,
        learningRate: 5e-5,
        batchSize: 8
      }
    });

    console.log(`Fine-tuning job ID: ${fineTuneJob.id}`);
    console.log(`Status: ${fineTuneJob.status}`);

    // Wait for fine-tuning to complete
    await fineTuneJob.waitUntilComplete();
    console.log('Fine-tuning completed!');
  } catch (error) {
    console.error('Error starting fine-tuning job:', error);
  }
}

startFineTuning();

Fine-tuning Parameters

The PaiTIENT platform supports the following fine-tuning parameters:

Parameter	Description	Default	Recommended Range
epochs	Number of training epochs	3	1-5
learning_rate	Learning rate for the optimizer	5e-5	1e-5 to 1e-4
batch_size	Batch size for training	8	1-16
warmup_steps	Steps for learning rate warmup	0	0-500
weight_decay	Weight decay for regularization	0.01	0-0.1
lora_rank	Rank for LoRA adapters	8	4-64
lora_alpha	Alpha for LoRA adapters	16	8-64
lora_dropout	Dropout for LoRA adapters	0.05	0-0.2

Monitoring Fine-tuning Progress

You can monitor the progress of your fine-tuning job:

Python SDK

# Get fine-tuning job status
job_status = client.get_fine_tuning_job("ft_12345abcde")
print(f"Status: {job_status.status}")
print(f"Progress: {job_status.progress}%")

# Get fine-tuning metrics
metrics = client.get_fine_tuning_metrics("ft_12345abcde")
print(f"Training loss: {metrics.training_loss}")
print(f"Validation loss: {metrics.validation_loss}")

Node.js SDK

// Get fine-tuning job status
async function checkFineTuningStatus(jobId) {
  try {
    const jobStatus = await client.getFineTuningJob(jobId);
    console.log(`Status: ${jobStatus.status}`);
    console.log(`Progress: ${jobStatus.progress}%`);

    // Get fine-tuning metrics
    const metrics = await client.getFineTuningMetrics(jobId);
    console.log(`Training loss: ${metrics.trainingLoss}`);
    console.log(`Validation loss: ${metrics.validationLoss}`);
  } catch (error) {
    console.error('Error checking fine-tuning status:', error);
  }
}

checkFineTuningStatus("ft_12345abcde");

Deploying Fine-tuned Models

Once fine-tuning is complete, you can deploy your fine-tuned model:

Python SDK

# Get the fine-tuned model name
fine_tune_job = client.get_fine_tuning_job("ft_12345abcde")
fine_tuned_model = fine_tune_job.fine_tuned_model

# Deploy the fine-tuned model
deployment = client.create_deployment(
    model_name=fine_tuned_model,
    deployment_name="custom-clinical-assistant",
    compute_type="gpu",
    instance_type="g4dn.xlarge"
)

print(f"Deployment ID: {deployment.id}")

Node.js SDK

async function deployFineTunedModel(jobId) {
  try {
    // Get the fine-tuned model name
    const fineTuneJob = await client.getFineTuningJob(jobId);
    const fineTunedModel = fineTuneJob.fineTunedModel;

    // Deploy the fine-tuned model
    const deployment = await client.createDeployment({
      modelName: fineTunedModel,
      deploymentName: "custom-clinical-assistant",
      computeType: "gpu",
      instanceType: "g4dn.xlarge"
    });

    console.log(`Deployment ID: ${deployment.id}`);
  } catch (error) {
    console.error('Error deploying fine-tuned model:', error);
  }
}

deployFineTunedModel("ft_12345abcde");

LoRA Fine-tuning

The PaiTIENT platform uses Low-Rank Adaptation (LoRA) for efficient fine-tuning. LoRA reduces memory requirements and training time while maintaining performance.

Benefits of LoRA

• Efficiency: Requires less GPU memory and compute resources
• Speed: Fine-tuning completes faster than full fine-tuning
• Small Artifacts: Fine-tuned model adapters are compact
• Parameter Efficiency: Only a small subset of parameters are trained

LoRA Parameters

You can customize LoRA parameters for your fine-tuning job:

# Python example with custom LoRA parameters
fine_tune_job = client.create_fine_tuning_job(
    model="ZimaBlueAI/HuatuoGPT-o1-8B",
    training_file="file_12345abcde",
    lora_config={
        "rank": 16,       # Higher rank = more capacity but larger adapters
        "alpha": 32,      # Scaling factor for LoRA
        "dropout": 0.1,   # Dropout rate for regularization
        "target_modules": ["q_proj", "k_proj", "v_proj", "o_proj"]  # Modules to fine-tune
    }
)

Security Considerations

Fine-tuning in the PaiTIENT platform maintains the same security standards as the rest of the service:

1. Data Encryption: Training data is encrypted at rest and in transit
2. Secure Processing: Fine-tuning occurs in isolated, secure environments
3. HIPAA/SOC2 Compliance: All fine-tuning processes adhere to compliance requirements
4. Audit Logging: Comprehensive logging of fine-tuning operations
5. Model Security: Fine-tuned models are encrypted and securely stored

Best Practices

Data Quality

• Review Examples: Manually review a sample of your training data
• Consistency: Ensure consistent tone, style, and format
• Accuracy: Verify factual accuracy of responses
• Balance: Include a balanced range of examples across your domain

Hyperparameter Selection

• Start Simple: Begin with default hyperparameters
• Validate Early: Use validation data to detect overfitting
• Iterate Gradually: Make incremental changes to hyperparameters
• Track Metrics: Monitor training and validation loss

Evaluation

• Test Systematically: Evaluate the model on a holdout test set
• Human Evaluation: Have subject matter experts review model outputs
• Compare Baselines: Compare with the base model and previous versions
• Real-world Testing: Test in realistic scenarios before production

Advanced Fine-tuning

Multi-stage Fine-tuning

For complex use cases, consider multi-stage fine-tuning:

# Stage 1: General domain adaptation
fine_tune_job_1 = client.create_fine_tuning_job(
    model="ZimaBlueAI/HuatuoGPT-o1-8B",
    training_file="general_medical_data.jsonl",
    hyperparameters={"epochs": 2}
)
fine_tune_job_1.wait_until_complete()

# Stage 2: Specific task fine-tuning
fine_tune_job_2 = client.create_fine_tuning_job(
    model=fine_tune_job_1.fine_tuned_model,
    training_file="specific_cardiology_data.jsonl",
    hyperparameters={"epochs": 3}
)

Continuous Fine-tuning

Set up a pipeline for continuous improvement of your models:

1. Collect Feedback: Gather user feedback on model responses
2. Data Curation: Curate new training examples based on feedback
3. Regular Updates: Schedule regular fine-tuning jobs with new data
4. Versioning: Maintain clear versioning of fine-tuned models
5. A/B Testing: Compare performance of new fine-tuned models

Next Steps

• Learn about Secure Deployment
• Explore Custom Deployments
• Understand Model Evaluation
• Review our Python SDK and Node.js SDK