AI Chat Functionality Documentation

Overview
System Architecture
Core Components
Detailed Process Flow
API Endpoints
Data Models
Configuration
Database Schema

Overview

The AI Chat functionality provides real-time chat capabilities using OpenAI's API to generate responses based on product reviews and comparisons. The system supports both standard and streaming responses.

System Architecture

1. Stream Chat Flow

sequenceDiagram
    actor Client
    participant APIServer
    participant DatasetDao
    participant SimilarProductChatSettingDao
    participant ProductSimilarService
    participant GPTService
    participant SimilarProductChatHistoryDao

    Client->>APIServer: POST /api/similar_products/stream_chat
    activate APIServer
    
    APIServer->>APIServer: Validate request data
    
    APIServer->>DatasetDao: Get dataset by dataset_id
    alt Dataset not found
        DatasetDao-->>APIServer: 404 Not Found
        APIServer-->>Client: 404 Error Response
    end
    DatasetDao-->>APIServer: Dataset
    
    APIServer->>SimilarProductChatSettingDao: Get settings by thread_id
    alt Settings not found
        SimilarProductChatSettingDao-->>APIServer: Empty settings
        Note over APIServer: Use defaults:<br/>model: gpt-4o<br/>temperature: 0.7
    end
    SimilarProductChatSettingDao-->>APIServer: Chat settings
    
    APIServer->>ProductSimilarService: prepare_chat()
    ProductSimilarService->>ProductSimilarService: set_system_prompt_variable()
    ProductSimilarService->>SimilarProductChatHistoryDao: Save system message
    ProductSimilarService->>SimilarProductChatHistoryDao: Save user message
    ProductSimilarService-->>APIServer: Prepared messages
    
    APIServer->>GPTService: stream_message()
    activate GPTService
    loop For each chunk
        GPTService-->>APIServer: Stream chunk
        APIServer-->>Client: Send chunk
    end
    deactivate GPTService
    
    APIServer->>SimilarProductChatHistoryDao: Save complete response
    deactivate APIServer

2. Standard Chat Flow

sequenceDiagram
    actor Client
    participant APIServer
    participant DatasetDao
    participant SimilarProductChatSettingDao
    participant ProductSimilarService
    participant GPTService
    participant SimilarProductChatHistoryDao

    Client->>APIServer: POST /api/similar_products/standard-chat
    activate APIServer
    
    APIServer->>APIServer: Validate request data
    
    APIServer->>DatasetDao: Get dataset by dataset_id
    alt Dataset not found
        DatasetDao-->>APIServer: 404 Not Found
        APIServer-->>Client: 404 Error Response
    end
    DatasetDao-->>APIServer: Dataset
    
    APIServer->>SimilarProductChatSettingDao: Get settings by thread_id
    alt Settings not found
        SimilarProductChatSettingDao-->>APIServer: Empty settings
        Note over APIServer: Use defaults:<br/>model: gpt-4o<br/>temperature: 0.7
    end
    SimilarProductChatSettingDao-->>APIServer: Chat settings
    
    APIServer->>ProductSimilarService: prepare_chat()
    ProductSimilarService->>ProductSimilarService: set_system_prompt_variable()
    ProductSimilarService->>SimilarProductChatHistoryDao: Save system message
    ProductSimilarService->>SimilarProductChatHistoryDao: Save user message
    ProductSimilarService-->>APIServer: Prepared messages
    
    APIServer->>GPTService: standard_message()
    GPTService-->>APIServer: Complete response
    
    APIServer->>SimilarProductChatHistoryDao: Save response
    APIServer-->>Client: Return response
    deactivate APIServer

3. Data Processing Flow

flowchart TD
    A[Start] --> B[Get Chat Settings]
    B --> C[Load Chat History]
    C --> D[Get Product Info]
    D --> E[Get Review Data]
    
    E --> F{Has Common Viewpoint?}
    F -->|Yes| G[Get Reviews with Sub Viewpoint]
    F -->|No| H[Get Regular Reviews]
    
    G --> I[Process Reviews]
    H --> I
    
    I --> J[Apply Filters]
    J --> K[Random Sample]
    K --> L[Format Response]
    
    subgraph "Review Processing"
    J --> |Filter Options|M[Product IDs]
    J --> |Filter Options|N[Viewpoint IDs]
    J --> |Filter Options|O[Sentiment Scores]
    J --> |Filter Options|P[Date Range]
    end
    
    K --> |Max 30 per product|Q[Limit Reviews]

4. Error Handling Flow

flowchart TD
    A[Request] --> B{Validate Request}
    B -->|Invalid| C[Return 400]
    B -->|Valid| D{Check Dataset}
    
    D -->|Not Found| E[Return 404]
    D -->|Found| F{Process Request}
    
    F -->|Stream Error| G[Log Error]
    G --> H[Save Partial Response]
    H --> I[Return Error Status]
    
    F -->|Success| J[Return Response]
    
    subgraph "Stream Error Handling"
    K[Stream Interruption] --> L[CancelledError]
    L --> M[Log Warning]
    M --> N[Save Current Content]
    
    O[Other Errors] --> P[Log Error]
    P --> Q[Save Error State]
    end

Key Components Interaction

Request Processing Layer
- Validates incoming requests
- Handles request routing
- Manages response formats
Service Layer
- ProductSimilarService: Core business logic
- GPTService: OpenAI API integration
- Handles data preparation and transformation
Data Access Layer
- Database interactions through DAOs
- Data validation and transformation
- Cache management
External Integration
- OpenAI API communication
- Streaming response handling
- Error recovery mechanisms

Performance Considerations

Streaming Optimization
- Chunks processed and sent immediately
- Memory efficient for large responses
- Handles connection interruptions
Database Operations
- Efficient query patterns
- Proper indexing for frequent queries
- Transaction management for data consistency
Error Recovery
- Graceful handling of API failures
- Partial response saving
- Automatic retry mechanisms

Core Components

1. Service Layer

GPTService (base/service/gpt_service.py)
- Handles OpenAI API integration
- Supports both standard and streaming responses
- Methods:
  - standard_message(): Regular chat completion
  - stream_message(): Streaming chat completion
ProductSimilarService (base/service/product_similar_service.py)
- Core business logic for chat functionality
- Methods:
  - prepare_chat(): Prepares chat messages and settings
  - stream_chat(): Handles streaming responses
  - standard_chat(): Handles standard responses
  - set_system_prompt_variable(): Sets up system prompts with context

2. Data Access Layer

SimilarProductChatSettingDao: Manages chat settings
SimilarProductChatHistoryDao: Handles chat history
DatasetDao: Manages datasets
AnalyzedSimilarProductReviewSentenceDao: Manages review data

Detailed Process Flow

Chat Request Processing

Request Validation

flowchart TD
    A[Client Request] --> B[Validate Request Data]
    B --> C{Valid?}
    C -->|Yes| D[Process Request]
    C -->|No| E[Return 400 Error]

Chat Preparation

Retrieve chat settings (default if not exists):
- Model: gpt-4o
- Temperature: 0.7
- System prompt: SIMILAR_PRODUCT_CHAT_PROMPT

{
  "role": "system",
  "content": "あなたは、ECサイトの商品情報を分析し、自社商品と競合商品の性能や人気度、価格設定、顧客評価などを比較することです。
  これにより、ユーザーが競争力のある商品戦略を立案できるようサポートします。

  以下に分析に必要なデータを入力します。このデータを元にユーザーからの質問に回答してください。

  * 類似商品情報
  類似商品情報はベース商品に対して、競合商品が商品情報・画像・レビューの評価観点がどれだけ類似しているかを示す情報です。
  ** データの内容
  *** ベース商品情報
  {base_product_info}
  *** 類似情報
  {similar_table}"
}

Load chat history
Prepare messages array with system prompt

Review Data Processing
- Filter reviews based on:
  - Product IDs
  - Viewpoint IDs
  - Sentiment scores
  - Date range
  - Common viewpoint ID
  - Sub viewpoint IDs

Response Handling

Streaming Response

Process chunks from OpenAI API
Send each chunk to client
Save complete response to chat history

Handle stream interruptions:

try:
    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            content = chunk.choices[0].delta.content
            res_content.append(content)
            yield content
except asyncio.CancelledError:
    logging.warn("ストリーム中断")
    # Save partial response to history
except Exception as e:
    logging.error(f"エラー: {e}")
finally:
    # Save complete response to history

Standard Response
- Get complete response from OpenAI
- Save to chat history
- Return to client

API Endpoints

1. Stream Chat

Endpoint: /api/similar_products/stream-chat
Method: POST
Description: Provides streaming chat responses using OpenAI API

Request Body:

{
  "thread_id": "string",
  "dataset_id": "integer",
  "base_product_id": "string",
  "content": "string",
  "review_filter_options": {
    "product_ids": ["string"],
    "viewpoint_ids": ["integer"],
    "sentiment_scores": ["integer"],
    "exclude_no_relate_flag": "boolean",
    "start_post_date": "date",
    "end_post_date": "date",
    "common_viewpoint_id": "integer",
    "sub_viewpoint_ids": ["integer"]
  }
}

Response:

Content-Type: text/event-stream
Streaming response with chunks of text

Status Codes:

200: Successful operation
400: Invalid request data
404: Dataset not found
500: Internal server error

2. Standard Chat

Endpoint: /api/similar_products/standard-chat
Method: POST
Description: Provides standard (non-streaming) chat responses

Request Body: Same as stream-chat

Response:

{
  "content": "string"
}

Status Codes:

200: Successful operation
400: Invalid request data
404: Dataset not found
500: Internal server error

3. Get Chat History

Endpoint: /api/similar_products/chat-history/{thread_id}
Method: GET
Description: Retrieves chat history for a specific thread

Parameters:

thread_id (path): String - The unique identifier for the chat thread

Response:

[
  {
    "thread_id": "string",
    "dataset_id": "integer",
    "base_product_id": "string",
    "role": "string",
    "content": "string",
    "created_at": "datetime"
  }
]

Status Codes:

200: Successful operation
404: Thread not found

4. Get Chat Settings

Endpoint: /api/similar_products/chat-settings/{thread_id}
Method: GET
Description: Retrieves chat settings for a specific thread

Parameters:

thread_id (path): String - The unique identifier for the chat thread

Response:

{
  "id": "integer",
  "thread_id": "string",
  "temperature": "number",
  "model": "string",
  "system_prompt": "string",
  "created_at": "datetime",
  "updated_at": "datetime"
}

Status Codes:

200: Successful operation
404: Chat setting not found

5. Update Chat Settings

Endpoint: /api/similar_products/chat-settings/{thread_id}
Method: PUT
Description: Updates chat settings for a specific thread

Parameters:

thread_id (path): String - The unique identifier for the chat thread

Request Body:

{
  "thread_id": "string",
  "temperature": "number",
  "model": "string",
  "system_prompt": "string"
}

Response:

{
  "id": "integer",
  "thread_id": "string",
  "temperature": "number",
  "model": "string",
  "system_prompt": "string",
  "created_at": "datetime",
  "updated_at": "datetime"
}

Status Codes:

201: Successful operation
400: Invalid request
404: Chat setting not found

6. Get Chat Threads

Endpoint: /api/similar_products/chat-threads
Method: GET
Description: Retrieves all chat threads

Response:

[
  {
    "thread_id": "string",
    "dataset_id": "integer",
    "base_product_id": "string",
    "created_at": "datetime"
  }
]

Status Codes:

200: Successful operation

7. Get Additional Questions

Endpoint: /api/similar_products/additional-questions/{thread_id}
Method: GET
Description: Retrieves suggested additional questions based on chat history

Parameters:

thread_id (path): String - The unique identifier for the chat thread

Response:

{
  "questions": ["string"]
}

Status Codes:

200: Successful operation
404: Thread not found

Data Models

Chat Request Schema

{
  "thread_id": "string",
  "dataset_id": "integer",
  "base_product_id": "string",
  "content": "string",
  "review_filter_options": {
    "product_ids": ["string"],
    "viewpoint_ids": ["integer"],
    "sentiment_scores": ["integer"],
    "exclude_no_relate_flag": "boolean",
    "start_post_date": "date",
    "end_post_date": "date",
    "common_viewpoint_id": "integer",
    "sub_viewpoint_ids": ["integer"]
  }
}

Chat Settings Schema

{
  "thread_id": "string",
  "temperature": "number",
  "model": "string",
  "system_prompt": "string"
}

Database Schema

Database Tables and Relationships

erDiagram
    similar_product_chat_settings ||--o{ similar_product_chat_histories : has
    datasets ||--o{ similar_product_chat_histories : contains
    analyzed_similar_product_review_sentences ||--o{ similar_product_chat_histories : references

    similar_product_chat_settings {
        bigint id PK
        varchar(100) thread_id
        float temperature
        varchar(50) model
        longtext system_prompt
        timestamp created_at
        timestamp updated_at
    }

    similar_product_chat_histories {
        bigint id PK
        varchar(100) thread_id FK
        bigint dataset_id FK
        varchar(100) base_product_id
        varchar(20) role
        longtext content
        timestamp created_at
    }

    datasets {
        bigint id PK
        varchar(100) name
        timestamp created_at
    }

    analyzed_similar_product_review_sentences {
        bigint id PK
        bigint dataset_id FK
        varchar(100) mall_product_id
        int sentiment_score
        longtext content
        date post_date
        int viewpoint_id
        int sub_viewpoint_id
        int no_relate_flag
    }

Table Details

1. similar_product_chat_settings

Description: Stores chat configuration settings for each thread

id: Primary key (bigint, auto_increment)
thread_id: Unique identifier for chat thread (varchar(100))
temperature: OpenAI temperature parameter (float)
model: OpenAI model name (varchar(50))
system_prompt: System prompt template (longtext)
created_at: Record creation timestamp (timestamp, default: CURRENT_TIMESTAMP)
updated_at: Record update timestamp (timestamp, default: CURRENT_TIMESTAMP)

Indexes:

PRIMARY KEY (id)
INDEX index_similar_product_chat_settings_on_thread_id (thread_id)

2. similar_product_chat_histories

Description: Stores all chat messages and responses

id: Primary key (bigint, auto_increment)
thread_id: Foreign key to chat settings (varchar(100))
dataset_id: Foreign key to datasets (bigint)
base_product_id: Reference product ID (varchar(100))
role: Message role (varchar(20)) - system/user/assistant
content: Message content (longtext)
created_at: Message timestamp (timestamp, default: CURRENT_TIMESTAMP)

3. datasets

Description: Stores dataset information

id: Primary key
name: Dataset name
created_at: Dataset creation timestamp

4. analyzed_similar_product_review_sentences

Description: Stores analyzed product reviews used for chat context

id: Primary key
dataset_id: Foreign key to datasets
mall_product_id: Product identifier
sentiment_score: Review sentiment score
content: Review content
post_date: Review post date
viewpoint_id: Review viewpoint identifier
sub_viewpoint_id: Review sub-viewpoint identifier
no_relate_flag: Flag for unrelated reviews

Key Relationships

Chat Settings to Chat History (1:Many)
- One chat setting can have multiple chat history entries
- Linked by thread_id
Dataset to Chat History (1:Many)
- One dataset can have multiple chat history entries
- Linked by dataset_id
Dataset to Review Sentences (1:Many)
- One dataset can have multiple review sentences
- Linked by dataset_id

Data Flow

When a new chat starts:
- Creates entry in similar_product_chat_settings
- Initial system prompt stored in similar_product_chat_histories
During chat:
- User messages stored in similar_product_chat_histories
- Reviews from analyzed_similar_product_review_sentences used for context
- AI responses stored in similar_product_chat_histories
Chat settings:
- Can be updated in similar_product_chat_settings
- Affects all subsequent messages in the thread

Configuration

The system requires an OpenAI API key configured in the .env file:

OPENAI_API_KEY=your_api_key

01.aichat-apianalyzer

AI Chat Functionality Documentation

Table of Contents

Overview

System Architecture

1. Stream Chat Flow

2. Standard Chat Flow

3. Data Processing Flow

4. Error Handling Flow

Key Components Interaction

Performance Considerations

Core Components

1. Service Layer

2. Data Access Layer

Detailed Process Flow

Chat Request Processing

Response Handling

API Endpoints

1. Stream Chat

2. Standard Chat

3. Get Chat History

4. Get Chat Settings

5. Update Chat Settings

6. Get Chat Threads

7. Get Additional Questions

Data Models

Chat Request Schema

Chat Settings Schema

Database Schema

Database Tables and Relationships

Table Details

1. similar_product_chat_settings

2. similar_product_chat_histories

3. datasets

4. analyzed_similar_product_review_sentences

Key Relationships

Data Flow

Configuration