LLMs & GPT APIs for Data Tasks

By the end of this lesson, you will be able to use a Large Language Model (LLM) API to extract structured data from unstructured text and generate SQL queries for database analysis.

What it is

An LLM API allows your application to send natural language prompts to a powerful language model and receive generated text in return. In data analytics, this capability transforms how we handle unstructured data (like customer reviews or support tickets) and complex query generation. Instead of writing brittle regular expressions or manually crafting SQL for every new question, you describe what you need in plain English, and the model translates it into code or structured JSON.

Mental Model: Think of the LLM as a highly skilled junior analyst who can read any format but needs clear instructions on output structure. It does not "know" your private data; it processes the text you provide within the prompt context.

Related terms: Prompt Engineering, Structured Output, Zero-shot Learning, Token Limit.

Why it matters

• Rapid Prototyping: Generate SQL queries from natural language questions instantly, speeding up ad-hoc analysis.
• Data Cleaning: Extract specific entities (dates, names, sentiment scores) from messy text fields without custom parsers.
• Summarization: Condense long reports or chat logs into executive summaries automatically.
• Code Generation: Create Python pandas scripts for data transformation based on high-level descriptions.

Syntax or steps

1. Select a Provider: Choose an API provider (e.g., OpenAI, Anthropic, Azure AI).
2. Install SDK: Use the official client library for your programming language.
3. Construct Prompt: Write a system message defining the role and a user message containing the task and input data.
4. Request Completion: Send the request specifying parameters like temperature (for creativity vs. determinism) and max_tokens.
5. Parse Response: Extract the content from the API response object.

Example

This Python example uses the OpenAI SDK to extract sentiment and key topics from a customer review, returning strict JSON.

import openai
import json

# Initialize client (ensure OPENAI_API_KEY is set in environment)
client = openai.OpenAI()

review_text = """
The battery life is terrible, lasting only 4 hours. 
However, the screen quality is stunning and the keyboard feels great.
I would not recommend this for travel, but it's okay for desk work.
"""

prompt = f"""
Extract the following information from the review:
1. Overall Sentiment (Positive, Negative, Neutral)
2. Key Topics mentioned
3. Recommendation Status (Yes/No)

Return ONLY valid JSON with keys: 'sentiment', 'topics' (list), 'recommend'.
Review: {review_text}
"""

try:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a data extraction assistant."},
            {"role": "user", "content": prompt}
        ],
        temperature=0, # Set to 0 for deterministic output
        response_format={"type": "json_object"}
    )
    
    result = json.loads(response.choices[0].message.content)
    print(result)
except Exception as e:
    print(f"Error: {e}")

Part-by-part explanation:

• temperature=0: Ensures the model picks the most probable words, reducing hallucination and ensuring consistent JSON structure.
• response_format={"type": "json_object"}: Forces the API to return valid JSON, simplifying parsing.
• json.loads(...): Converts the string response into a Python dictionary for further analysis.

Common mistakes

• Vague Prompts: Asking "Analyze this" yields unpredictable results. Always specify exact output formats (e.g., "Return a CSV").
• Ignoring Context Limits: Sending entire databases in one prompt exceeds token limits. Chunk data or summarize first.
• Assuming Truthfulness: LLMs can hallucinate facts. Always validate extracted data against source records if accuracy is critical.
• High Temperature for Extraction: Using default temperature (often 0.7+) causes inconsistent formatting. Use 0 for data tasks.

When to use it

Practice

Guided Exercise: Modify the example above to extract "Product Name" and "Price" from a product description string. Ensure the price is returned as a float, not a string.

Challenge: Write a function that takes a list of 5 short tweets and returns a single summary sentence capturing the common theme. Hint: Use a loop to process each tweet or combine them into one prompt if they fit within token limits.

Quick check

Q: Why should you set temperature=0 when extracting structured data?

A: To ensure deterministic, consistent outputs and prevent the model from inventing variations in formatting or content.

Summary

LLMs act as flexible translators between human language and machine-readable data structures. By using low-temperature settings and strict JSON schemas, you can reliably automate the extraction of insights from unstructured text, bridging the gap between raw data and actionable analytics.