Generation API

This reference documents the Text Generation endpoints in the PaiTIENT Secure Model Service REST API, allowing you to generate text from deployed models programmatically.

Generate Text

Generates text from a deployed model.

Request

http

POST /v1/deployments/{deployment_id}/generate
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "prompt": "What are the potential side effects of metformin?",
  "max_tokens": 500,
  "temperature": 0.7,
  "top_p": 0.95,
  "top_k": 50,
  "stop": ["\n\n", "END"],
  "repetition_penalty": 1.1,
  "presence_penalty": 0.0,
  "frequency_penalty": 0.0,
  "system_message": "You are a helpful medical assistant providing accurate information to healthcare professionals. Always include references to clinical guidelines and mention important warnings."
}

Response

json

{
  "id": "gen_12345abcde",
  "deployment_id": "dep_12345abcde",
  "model": "ZimaBlueAI/HuatuoGPT-o1-8B",
  "created_at": "2023-11-01T14:23:45Z",
  "text": "Metformin may cause several side effects, ranging from common and mild to rare and severe. The most common side effects include:\n\n1. Gastrointestinal issues: Nausea, vomiting, diarrhea, abdominal discomfort, and altered taste (metallic taste). These often improve over time and can be minimized by starting with a low dose and gradually increasing it, and by taking the medication with meals.\n\n2. Vitamin B12 deficiency: Long-term use can reduce vitamin B12 absorption, potentially leading to anemia and neuropathy. Regular monitoring is recommended.\n\nRare but serious side effects include:\n\n3. Lactic acidosis: This is a rare but potentially fatal complication characterized by elevated blood lactate levels and decreased blood pH. Risk factors include impaired kidney or liver function, excessive alcohol intake, and acute conditions that may alter kidney function. Symptoms include malaise, myalgias, respiratory distress, somnolence, and abdominal distress.\n\nAccording to the American Diabetes Association's Standards of Medical Care in Diabetes, metformin remains the preferred initial pharmacologic agent for the treatment of type 2 diabetes, but clinicians should be aware of these potential side effects and monitor patients accordingly.\n\nIMPORTANT WARNING: Metformin is contraindicated in patients with an eGFR <30 mL/min/1.73 m² and should be used with caution in patients with an eGFR between 30-45 mL/min/1.73 m².",
  "finish_reason": "stop",
  "usage": {
    "prompt_tokens": 65,
    "completion_tokens": 235,
    "total_tokens": 300
  }
}

Generate Text Stream

Generates text from a deployed model with streaming response.

Request

http

POST /v1/deployments/{deployment_id}/generate-stream
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "prompt": "What are the potential side effects of metformin?",
  "max_tokens": 500,
  "temperature": 0.7,
  "top_p": 0.95,
  "system_message": "You are a helpful medical assistant providing accurate information to healthcare professionals."
}

Response

The response is a stream of Server-Sent Events (SSE) with the following format:

data: {"id":"gen_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:23:45Z","text":"Metformin"}

data: {"id":"gen_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:23:45Z","text":" may"}

data: {"id":"gen_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:23:45Z","text":" cause"}

...

data: {"id":"gen_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:23:45Z","text":".","finish_reason":"stop","usage":{"prompt_tokens":65,"completion_tokens":235,"total_tokens":300}}

data: [DONE]

Batch Generate Text

Generates text for multiple prompts in a single request.

Request

http

POST /v1/deployments/{deployment_id}/batch-generate
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "prompts": [
    "What are the symptoms of hypertension?",
    "What are common treatments for type 2 diabetes?",
    "Explain the mechanism of action for statins."
  ],
  "max_tokens": 300,
  "temperature": 0.7,
  "top_p": 0.95,
  "system_message": "You are a helpful medical assistant providing accurate information to healthcare professionals."
}

Response

json

{
  "id": "batch_12345abcde",
  "deployment_id": "dep_12345abcde",
  "model": "ZimaBlueAI/HuatuoGPT-o1-8B",
  "created_at": "2023-11-01T14:30:00Z",
  "results": [
    {
      "text": "Hypertension (high blood pressure) often doesn't cause symptoms until blood pressure reaches severely high levels. When symptoms do occur, they may include:\n\n1. Headaches, particularly in the morning\n2. Nosebleeds\n3. Irregular heart rhythms\n4. Vision changes\n5. Buzzing in the ears\n6. Fatigue\n7. Confusion\n8. Chest pain\n9. Difficulty breathing\n10. Blood in the urine\n11. Pounding in the chest, neck, or ears\n\nSevere hypertension (hypertensive crisis) may cause:\n\n- Severe headache\n- Severe anxiety\n- Shortness of breath\n- Nosebleeds\n\nIt's important to note that most people with hypertension have no symptoms, which is why it's often called 'the silent killer.' Regular blood pressure screening is essential for early detection.",
      "finish_reason": "stop",
      "usage": {
        "prompt_tokens": 50,
        "completion_tokens": 185,
        "total_tokens": 235
      }
    },
    {
      "text": "Common treatments for type 2 diabetes include:\n\n1. Lifestyle modifications:\n   - Medical nutrition therapy (balanced diet)\n   - Regular physical activity\n   - Weight management\n   - Smoking cessation\n\n2. Oral medications:\n   - Metformin (first-line therapy)\n   - Sulfonylureas\n   - Meglitinides\n   - DPP-4 inhibitors\n   - SGLT2 inhibitors\n   - GLP-1 receptor agonists\n   - Thiazolidinediones\n   - Alpha-glucosidase inhibitors\n\n3. Injectable medications:\n   - GLP-1 receptor agonists\n   - Insulin therapy\n\n4. Monitoring and management:\n   - Blood glucose monitoring\n   - HbA1c testing every 3-6 months\n   - Regular screening for complications\n\n5. Newer approaches:\n   - Dual GIP/GLP-1 receptor agonists\n   - Combination therapies\n\nTreatment is individualized based on patient factors including comorbidities, risk of hypoglycemia, impact on weight, cost, and patient preferences, following guidelines from organizations like the American Diabetes Association (ADA) and European Association for the Study of Diabetes (EASD).",
      "finish_reason": "stop",
      "usage": {
        "prompt_tokens": 54,
        "completion_tokens": 230,
        "total_tokens": 284
      }
    },
    {
      "text": "Statins (HMG-CoA reductase inhibitors) work through the following mechanism of action:\n\n1. Inhibition of HMG-CoA reductase: Statins competitively inhibit 3-hydroxy-3-methylglutaryl coenzyme A (HMG-CoA) reductase, which is the rate-limiting enzyme in the mevalonate pathway responsible for cholesterol synthesis in the liver.\n\n2. Reduced cholesterol synthesis: By inhibiting this enzyme, statins decrease the production of mevalonate and subsequently reduce hepatic cholesterol synthesis.\n\n3. Upregulation of LDL receptors: The reduced intracellular cholesterol levels trigger increased expression of LDL receptors on hepatocyte cell surfaces through activation of sterol regulatory element-binding protein 2 (SREBP-2).\n\n4. Enhanced LDL clearance: The increased number of LDL receptors leads to greater uptake and clearance of LDL-cholesterol from the bloodstream, resulting in lower circulating levels.\n\n5. Pleiotropic effects: Beyond lipid-lowering effects, statins also demonstrate pleiotropic (multiple) effects, including:\n   - Improved endothelial function\n   - Decreased vascular inflammation\n   - Enhanced plaque stability\n   - Reduced thrombogenic response\n   - Antioxidant properties\n\nThrough these mechanisms, statins effectively reduce LDL-cholesterol levels by approximately 20-60% (depending on the specific statin and dosage), while also moderately raising HDL-cholesterol and lowering triglycerides.",
      "finish_reason": "stop",
      "usage": {
        "prompt_tokens": 46,
        "completion_tokens": 224,
        "total_tokens": 270
      }
    }
  ],
  "usage": {
    "prompt_tokens": 150,
    "completion_tokens": 639,
    "total_tokens": 789
  }
}

Generate Chat Completion

Generates a response in a conversational context.

Request

http

POST /v1/deployments/{deployment_id}/chat
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful medical assistant providing accurate information to healthcare professionals."
    },
    {
      "role": "user",
      "content": "What are the potential side effects of metformin?"
    },
    {
      "role": "assistant",
      "content": "Metformin may cause several side effects, ranging from common and mild to rare and severe. The most common side effects include gastrointestinal issues like nausea, vomiting, diarrhea, and abdominal discomfort. These often improve over time. Long-term use can reduce vitamin B12 absorption. A rare but serious side effect is lactic acidosis, which is potentially fatal but very uncommon."
    },
    {
      "role": "user",
      "content": "What about patients with kidney disease?"
    }
  ],
  "max_tokens": 500,
  "temperature": 0.7,
  "top_p": 0.95
}

Response

json

{
  "id": "chat_12345abcde",
  "deployment_id": "dep_12345abcde",
  "model": "ZimaBlueAI/HuatuoGPT-o1-8B",
  "created_at": "2023-11-01T14:40:00Z",
  "message": {
    "role": "assistant",
    "content": "For patients with kidney disease, metformin must be used with caution or avoided altogether, depending on the severity of renal impairment:\n\n1. Contraindications:\n   - Metformin is contraindicated in patients with an estimated glomerular filtration rate (eGFR) below 30 mL/min/1.73 m²\n   - It should not be used in patients with conditions that might predispose to hypoxemia or dehydration\n\n2. Dose adjustments:\n   - For patients with eGFR 30-45 mL/min/1.73 m², a dose reduction is recommended\n   - Regular monitoring of kidney function is essential\n\n3. Increased risk:\n   - Patients with kidney disease have an elevated risk of developing lactic acidosis when taking metformin\n   - The risk increases as kidney function declines\n\n4. Alternative medications:\n   - Other antidiabetic medications like DPP-4 inhibitors, GLP-1 receptor agonists, or certain SGLT2 inhibitors may be more appropriate\n   - Insulin therapy is often used in advanced kidney disease\n\n5. Monitoring requirements:\n   - More frequent monitoring of kidney function\n   - Temporary discontinuation during procedures requiring iodinated contrast or situations that may worsen kidney function\n\nAccording to FDA guidelines and the American Diabetes Association, metformin should be temporarily discontinued at the time of or before procedures that may impact kidney function or increase risk of lactic acidosis."
  },
  "finish_reason": "stop",
  "usage": {
    "prompt_tokens": 165,
    "completion_tokens": 245,
    "total_tokens": 410
  }
}

Generate Chat Completion Stream

Generates a response in a conversational context with streaming.

Request

http

POST /v1/deployments/{deployment_id}/chat-stream
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful medical assistant providing accurate information to healthcare professionals."
    },
    {
      "role": "user",
      "content": "What are the potential side effects of metformin?"
    },
    {
      "role": "assistant",
      "content": "Metformin may cause several side effects, ranging from common and mild to rare and severe. The most common side effects include gastrointestinal issues like nausea, vomiting, diarrhea, and abdominal discomfort. These often improve over time. Long-term use can reduce vitamin B12 absorption. A rare but serious side effect is lactic acidosis, which is potentially fatal but very uncommon."
    },
    {
      "role": "user",
      "content": "What about patients with kidney disease?"
    }
  ],
  "max_tokens": 500,
  "temperature": 0.7,
  "top_p": 0.95
}

Response

The response is a stream of Server-Sent Events (SSE) with the following format:

data: {"id":"chat_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:40:00Z","delta":{"role":"assistant","content":"For"}}

data: {"id":"chat_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:40:00Z","delta":{"content":" patients"}}

data: {"id":"chat_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:40:00Z","delta":{"content":" with"}}

...

data: {"id":"chat_12345abcde","deployment_id":"dep_12345abcde","model":"ZimaBlueAI/HuatuoGPT-o1-8B","created_at":"2023-11-01T14:40:00Z","delta":{"content":"."},"finish_reason":"stop","usage":{"prompt_tokens":165,"completion_tokens":245,"total_tokens":410}}

data: [DONE]

Function Calling

Generates text with function calling capability.

Request

http

POST /v1/deployments/{deployment_id}/generate
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "prompt": "Check for interactions between metformin, lisinopril, and simvastatin.",
  "max_tokens": 500,
  "temperature": 0.7,
  "functions": [
    {
      "name": "search_medication_interactions",
      "description": "Search for potential interactions between medications",
      "parameters": {
        "type": "object",
        "properties": {
          "medications": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "description": "List of medications to check for interactions"
          }
        },
        "required": ["medications"]
      }
    },
    {
      "name": "calculate_dosage",
      "description": "Calculate medication dosage based on patient parameters",
      "parameters": {
        "type": "object",
        "properties": {
          "medication": {
            "type": "string",
            "description": "Medication name"
          },
          "weight_kg": {
            "type": "number",
            "description": "Patient weight in kg"
          },
          "age_years": {
            "type": "number",
            "description": "Patient age in years"
          },
          "kidney_function": {
            "type": "string",
            "description": "Kidney function (normal, impaired, severe)"
          }
        },
        "required": ["medication", "weight_kg", "age_years"]
      }
    }
  ],
  "function_call": "auto"
}

Response

json

{
  "id": "gen_12345abcde",
  "deployment_id": "dep_12345abcde",
  "model": "ZimaBlueAI/HuatuoGPT-o1-8B",
  "created_at": "2023-11-01T14:50:00Z",
  "function_call": {
    "name": "search_medication_interactions",
    "arguments": "{\"medications\":[\"metformin\",\"lisinopril\",\"simvastatin\"]}"
  },
  "finish_reason": "function_call",
  "usage": {
    "prompt_tokens": 135,
    "completion_tokens": 25,
    "total_tokens": 160
  }
}

Continue With Function Results

Continue generation after executing a function.

Request

http

POST /v1/deployments/{deployment_id}/generate
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "prompt": "What should I do about these interactions?",
  "max_tokens": 500,
  "temperature": 0.7,
  "function_results": [
    {
      "name": "search_medication_interactions",
      "arguments": "{\"medications\":[\"metformin\",\"lisinopril\",\"simvastatin\"]}",
      "result": {
        "interactions": [
          {
            "medications": ["metformin", "lisinopril"],
            "severity": "low",
            "description": "Lisinopril may slightly increase the risk of hypoglycemia with metformin."
          },
          {
            "medications": ["metformin", "simvastatin"],
            "severity": "moderate",
            "description": "Simvastatin may rarely increase the risk of myopathy when used with metformin."
          },
          {
            "medications": ["lisinopril", "simvastatin"],
            "severity": "low",
            "description": "No significant interaction is expected between lisinopril and simvastatin."
          }
        ]
      }
    }
  ]
}

Response

json

{
  "id": "gen_67890fghij",
  "deployment_id": "dep_12345abcde",
  "model": "ZimaBlueAI/HuatuoGPT-o1-8B",
  "created_at": "2023-11-01T14:55:00Z",
  "text": "Based on the interaction analysis between metformin, lisinopril, and simvastatin:\n\n1. Metformin and Lisinopril (Low severity):\n   - Monitor for signs of hypoglycemia (sweating, dizziness, confusion, hunger)\n   - Regular blood glucose monitoring is recommended\n   - No dosage adjustment is typically needed, but be aware of the potential interaction\n\n2. Metformin and Simvastatin (Moderate severity):\n   - Be vigilant for symptoms of myopathy such as muscle pain, tenderness, or weakness\n   - Consider periodic monitoring of creatine kinase levels, particularly if the patient reports muscle symptoms\n   - Inform patients to report any unexplained muscle pain promptly\n\n3. Lisinopril and Simvastatin (Low severity):\n   - No specific adjustments are needed as this interaction is not clinically significant\n\nGeneral recommendations:\n\n- These medications can generally be used together with appropriate monitoring\n- Ensure regular follow-up visits to assess efficacy and side effects\n- Educate the patient about possible signs of adverse reactions to report\n- Consider a medication review if any concerning symptoms develop\n\nFor most patients, the benefits of this medication combination likely outweigh the risks, especially when treating conditions like diabetes with hypertension and dyslipidemia. However, individualized assessment is always necessary based on the patient's specific medical history and risk factors.",
  "finish_reason": "stop",
  "usage": {
    "prompt_tokens": 173,
    "completion_tokens": 250,
    "total_tokens": 423
  }
}

Analyze Text

Analyzes generated text for various metrics.

Request

http

POST /v1/text/analyze
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
X-Client-ID: YOUR_CLIENT_ID

json

{
  "text": "Metformin may cause several side effects, ranging from common and mild to rare and severe. The most common side effects include gastrointestinal issues like nausea, vomiting, diarrhea, and abdominal discomfort. These often improve over time. Long-term use can reduce vitamin B12 absorption. A rare but serious side effect is lactic acidosis, which is potentially fatal but very uncommon.",
  "analyses": ["toxicity", "factuality", "bias", "medical_accuracy"]
}

Response

json

{
  "id": "ana_12345abcde",
  "created_at": "2023-11-01T15:00:00Z",
  "results": {
    "toxicity": 0.01,
    "factuality": 0.98,
    "bias": 0.02,
    "medical_accuracy": 0.97
  },
  "explanation": {
    "toxicity": "The text contains medical information about side effects but presents it in a neutral, informative manner without harmful content.",
    "factuality": "The information provided about metformin side effects is highly accurate and aligns with established medical knowledge.",
    "bias": "The text presents balanced information about metformin's side effects without undue emphasis on either benefits or risks.",
    "medical_accuracy": "The description of metformin's side effects is medically accurate, mentioning both common (GI issues) and rare but serious (lactic acidosis) effects in appropriate context."
  }
}

Error Codes

Code	Description
`invalid_request_error`	The request was malformed or missing required parameters
`authentication_error`	API key or client ID is invalid or missing
`permission_denied`	The API key doesn't have permission to perform the operation
`resource_not_found`	The requested resource (deployment) doesn't exist
`content_filter_error`	The content was filtered due to policy violations
`rate_limit_exceeded`	Too many requests in a given time period
`quota_exceeded`	Token quota has been exceeded
`generation_error`	Error occurred during text generation
`max_tokens_exceeded`	The request would exceed the maximum allowed tokens
`internal_error`	An internal error occurred

Security Controls

Content Filtering

Content filtering can be enabled to prevent generation of harmful content:

json

{
  "prompt": "What are the potential side effects of metformin?",
  "max_tokens": 500,
  "temperature": 0.7,
  "security_settings": {
    "data_filtering": {
      "detect_pii": true,
      "redact_phi": true,
      "content_filtering": "strict",
      "detect_toxic_content": true
    }
  }
}

Multi-tenant Usage

For applications serving multiple clients:

json

{
  "prompt": "What are the potential side effects of metformin?",
  "max_tokens": 500,
  "temperature": 0.7,
  "tenant_id": "tenant_12345",
  "user_id": "user_67890"
}

Client Libraries

PaiTIENT provides client libraries for easier integration:

Rate Limits

Plan	Requests per Minute	Tokens per Minute
Free	60	20,000
Basic	300	100,000
Professional	900	500,000
Enterprise	Custom	Custom

Rate limits can be adjusted for enterprise customers. Contact PaiTIENT support for more information.

Generation API ​

Generate Text ​

Request ​

Response ​

Generate Text Stream ​

Request ​

Response ​

Batch Generate Text ​

Request ​

Response ​

Generate Chat Completion ​

Request ​

Response ​

Generate Chat Completion Stream ​

Request ​

Response ​

Function Calling ​

Request ​

Response ​

Continue With Function Results ​

Request ​

Response ​

Analyze Text ​

Request ​

Response ​

Error Codes ​

Security Controls ​

Content Filtering ​

Multi-tenant Usage ​

Client Libraries ​

Rate Limits ​

Generation API

Generate Text

Request

Response

Generate Text Stream

Request

Response

Batch Generate Text

Request

Response

Generate Chat Completion

Request

Response

Generate Chat Completion Stream

Request

Response

Function Calling

Request

Response

Continue With Function Results

Request

Response

Analyze Text

Request

Response

Error Codes

Security Controls

Content Filtering

Multi-tenant Usage

Client Libraries

Rate Limits