-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathquick_pipeline.log
More file actions
331 lines (289 loc) · 18.3 KB
/
quick_pipeline.log
File metadata and controls
331 lines (289 loc) · 18.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
Running dataset pipeline demo...
PYTHONPATH=. venv/bin/python3 examples/dataset_pipeline_demo.py
======================================================================
Dataset Loading and Processing Pipeline Demo
======================================================================
1. Loading Bitext Retail Banking Dataset...
----------------------------------------------------------------------
2025-10-17 02:31:59,243 - chatbot_analytics.src.repositories.dataset_loaders - INFO - Loading Bitext dataset from Dataset/BitextRetailBanking/bitext-retail-banking-llm-chatbot-training-dataset.csv
2025-10-17 02:31:59,522 - chatbot_analytics.src.repositories.dataset_loaders - INFO - Loaded 25545 conversations from Bitext dataset
✓ Loaded 25545 conversations
Total turns: 51090
Unique intents: 26
2. Validating Data Quality...
----------------------------------------------------------------------
2025-10-17 02:31:59,539 - chatbot_analytics.src.services.data_validator - INFO - Bitext dataset contains Q&A pairs (2 turns per conversation)
2025-10-17 02:31:59,539 - chatbot_analytics.src.services.data_validator - INFO - Dataset validation passed for BitextRetailBanking
2025-10-17 02:31:59,554 - chatbot_analytics.src.services.data_validator - INFO - Quality assessment for BitextRetailBanking: Overall=1.00, Completeness=1.00, Consistency=1.00
✓ Validation: PASSED
Errors: 0
Warnings: 0
Overall Quality: 100.00%
Completeness: 100.00%
Consistency: 100.00%
3. Analyzing Dataset Statistics...
----------------------------------------------------------------------
✓ Text Statistics:
Avg text length: 481.1 chars
Avg word count: 81.8 words
✓ Intent Distribution:
Unique intents: 26
- activate_card: 1000 examples
- find_branch: 1000 examples
- check_recent_transactions: 999 examples
4. Preprocessing Dataset...
----------------------------------------------------------------------
Original text: I would like to acivate a card, can you help me?...
2025-10-17 02:32:00,523 - chatbot_analytics.src.services.data_preprocessor - INFO - Preprocessed dataset BitextRetailBanking with 25545 conversations
Preprocessed: i would like to acivate a card, can you help me?...
5. Splitting Dataset for Training...
----------------------------------------------------------------------
2025-10-17 02:32:00,527 - chatbot_analytics.src.services.data_preprocessor - INFO - Split dataset BitextRetailBanking: train=17881, val=3831, test=3833
✓ Dataset split:
Training: 17881 conversations (70.0%)
Validation: 3831 conversations (15.0%)
Test: 3833 conversations (15.0%)
6. Extracting Data for ML Tasks...
----------------------------------------------------------------------
✓ Intent classification dataset: 17881 examples
✓ User queries extracted: 17881
Sample classification example:
Text: ineed assistance seeing the mortgage payment...
Intent: check_mortgage_payments
7. Loading Schema-Guided Dialogue Dataset...
----------------------------------------------------------------------
2025-10-17 02:32:00,609 - chatbot_analytics.src.repositories.dataset_loaders - INFO - Loaded 727 conversations from Schema-Guided dataset
✓ Loaded 727 multi-turn conversations
Total turns: 13654
Avg turns per conversation: 18.8
✓ Conversations with 5+ turns: 727
Sample multi-turn conversation (16 turns):
User: What's my balance?...
Assistant: In checking or savings?...
User: In checking....
Assistant: Your checking account has $5,118.77....
8. Generating Comprehensive Quality Summary...
----------------------------------------------------------------------
2025-10-17 02:32:00,621 - chatbot_analytics.src.services.data_validator - INFO - Bitext dataset contains Q&A pairs (2 turns per conversation)
2025-10-17 02:32:00,621 - chatbot_analytics.src.services.data_validator - INFO - Dataset validation passed for BitextRetailBanking
2025-10-17 02:32:00,635 - chatbot_analytics.src.services.data_validator - INFO - Quality assessment for BitextRetailBanking: Overall=1.00, Completeness=1.00, Consistency=1.00
✓ Quality Summary Generated:
Dataset: BitextRetailBanking
Type: bitext
Records: 25545
Valid: 25545
Quality Score: 100.00%
======================================================================
✓ Pipeline Demo Complete!
======================================================================
All dataset loading and processing components are working correctly.
The pipeline is ready for:
- Intent classification model training
- Conversation analysis
- Multi-turn dialogue modeling
- Quality assessment and monitoring
Quick training (1 epoch, CPU-only)...
venv/bin/python3 examples/train_intent_classifier_quick.py
2025-10-17 02:32:03,780 - __main__ - INFO - Loading BANKING77 dataset...
2025-10-17 02:32:03,781 - chatbot_analytics.src.repositories.dataset_loaders - WARNING - Could not load intent labels: Expecting value: line 9 column 1 (char 8). Using numeric labels.
2025-10-17 02:32:03,781 - chatbot_analytics.src.repositories.dataset_loaders - WARNING - Could not load intent labels: Expecting value: line 9 column 1 (char 8). Using numeric labels.
2025-10-17 02:32:03,841 - chatbot_analytics.src.repositories.dataset_loaders - INFO - Loaded 13085 conversations from BANKING77 dataset
2025-10-17 02:32:03,841 - chatbot_analytics.src.repositories.dataset_loaders - INFO - Loaded 13085 conversations from BANKING77 dataset
2025-10-17 02:32:03,841 - __main__ - INFO - Loaded dataset with 13085 conversations
2025-10-17 02:32:03,842 - __main__ - INFO - Found 78 unique intents
2025-10-17 02:32:03,842 - __main__ - INFO - Splitting dataset into train/val/test sets...
2025-10-17 02:32:03,844 - chatbot_analytics.src.services.data_preprocessor - INFO - Split dataset BANKING77: train=9159, val=1962, test=1964
2025-10-17 02:32:03,844 - chatbot_analytics.src.services.data_preprocessor - INFO - Split dataset BANKING77: train=9159, val=1962, test=1964
2025-10-17 02:32:03,844 - __main__ - INFO - Train size: 9159
2025-10-17 02:32:03,844 - __main__ - INFO - Validation size: 1962
2025-10-17 02:32:03,844 - __main__ - INFO - Test size: 1964
2025-10-17 02:32:03,844 - __main__ - INFO - Initializing intent classifier (CPU-only)...
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Initializing IntentClassifier with model: bert-base-uncased
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Initializing IntentClassifier with model: bert-base-uncased
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Using device: cpu
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Using device: cpu
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - GPU available: False
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - GPU available: False
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Batch size: 32
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Batch size: 32
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Cache enabled: True (size: 1000)
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Cache enabled: True (size: 1000)
2025-10-17 02:32:03,844 - __main__ - INFO - Starting quick training (1 epoch)...
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Starting model training
2025-10-17 02:32:03,844 - chatbot_analytics.src.models.intent_classifier - INFO - Starting model training
2025-10-17 02:32:03,846 - chatbot_analytics.src.models.intent_classifier - INFO - Created label mappings for 78 intents
2025-10-17 02:32:03,846 - chatbot_analytics.src.models.intent_classifier - INFO - Created label mappings for 78 intents
2025-10-17 02:32:03,846 - chatbot_analytics.src.models.intent_classifier - INFO - Loading tokenizer and model: bert-base-uncased
2025-10-17 02:32:03,846 - chatbot_analytics.src.models.intent_classifier - INFO - Loading tokenizer and model: bert-base-uncased
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
2025-10-17 02:32:04,529 - chatbot_analytics.src.models.intent_classifier - INFO - Converting datasets to HuggingFace format
2025-10-17 02:32:04,529 - chatbot_analytics.src.models.intent_classifier - INFO - Converting datasets to HuggingFace format
2025-10-17 02:32:04,545 - chatbot_analytics.src.models.intent_classifier - INFO - Tokenizing datasets
2025-10-17 02:32:04,545 - chatbot_analytics.src.models.intent_classifier - INFO - Tokenizing datasets
Map: 0%| | 0/9159 [00:00<?, ? examples/s]Map: 11%|█ | 1000/9159 [00:00<00:00, 9812.84 examples/s]Map: 33%|███▎ | 3000/9159 [00:00<00:00, 12013.97 examples/s]Map: 55%|█████▍ | 5000/9159 [00:00<00:00, 12832.41 examples/s]Map: 76%|███████▋ | 7000/9159 [00:00<00:00, 12606.94 examples/s]Map: 98%|█████████▊| 9000/9159 [00:00<00:00, 12715.17 examples/s]Map: 100%|██████████| 9159/9159 [00:00<00:00, 12453.99 examples/s]
Map: 0%| | 0/1962 [00:00<?, ? examples/s]Map: 100%|██████████| 1962/1962 [00:00<00:00, 13692.23 examples/s]Map: 100%|██████████| 1962/1962 [00:00<00:00, 13443.06 examples/s]
2025-10-17 02:32:08,069 - chatbot_analytics.src.models.intent_classifier - INFO - Training model...
2025-10-17 02:32:08,069 - chatbot_analytics.src.models.intent_classifier - INFO - Training model...
0%| | 0/1145 [00:00<?, ?it/s]/Users/Development/chatbotAnalyticsLab/venv/lib/python3.13/site-packages/torch/utils/data/dataloader.py:692: UserWarning: 'pin_memory' argument is set as true but not supported on MPS now, device pinned memory won't be used.
warnings.warn(warn_msg)
0%| | 1/1145 [00:02<50:04, 2.63s/it] 0%| | 2/1145 [00:04<44:39, 2.34s/it] 0%| | 3/1145 [00:06<40:51, 2.15s/it] 0%| | 4/1145 [00:08<38:51, 2.04s/it] 0%| | 5/1145 [00:10<37:17, 1.96s/it] 1%| | 6/1145 [00:12<36:20, 1.91s/it] 1%| | 7/1145 [00:14<36:09, 1.91s/it] 1%| | 8/1145 [00:15<35:59, 1.90s/it] 1%| | 9/1145 [00:17<34:54, 1.84s/it] 1%| | 10/1145 [00:19<36:02, 1.91s/it] 1%| | 10/1145 [00:20<36:02, 1.91s/it] 1%| | 11/1145 [00:22<38:50, 2.06s/it] 1%| | 12/1145 [00:23<37:25, 1.98s/it] 1%| | 13/1145 [00:25<36:36, 1.94s/it] 1%| | 14/1145 [00:27<34:58, 1.86s/it] 1%|▏ | 15/1145 [00:29<34:09, 1.81s/it] 1%|▏ | 16/1145 [00:30<34:02, 1.81s/it] 1%|▏ | 17/1145 [00:32<33:25, 1.78s/it] 2%|▏ | 18/1145 [00:34<32:46, 1.74s/it] 2%|▏ | 19/1145 [00:36<32:29, 1.73s/it] 2%|▏ | 20/1145 [00:37<33:23, 1.78s/it] 2%|▏ | 20/1145 [00:38<33:23, 1.78s/it] 2%|▏ | 21/1145 [00:39<33:48, 1.80s/it] 2%|▏ | 22/1145 [00:41<34:26, 1.84s/it] 2%|▏ | 23/1145 [00:43<34:08, 1.83s/it] 2%|▏ | 24/1145 [00:45<34:39, 1.86s/it] 2%|▏ | 25/1145 [00:47<35:41, 1.91s/it] 2%|▏ | 26/1145 [00:49<36:33, 1.96s/it] 2%|▏ | 27/1145 [00:51<36:05, 1.94s/it] 2%|▏ | 28/1145 [00:53<35:27, 1.91s/it] 3%|▎ | 29/1145 [00:55<35:09, 1.89s/it] 3%|▎ | 30/1145 [00:57<36:12, 1.95s/it] 3%|▎ | 30/1145 [00:57<36:12, 1.95s/it] 3%|▎ | 31/1145 [00:59<36:51, 1.99s/it] 3%|▎ | 32/1145 [01:01<37:06, 2.00s/it] 3%|▎ | 33/1145 [01:03<37:46, 2.04s/it] 3%|▎ | 34/1145 [01:05<38:28, 2.08s/it] 3%|▎ | 35/1145 [01:07<37:57, 2.05s/it] 3%|▎ | 36/1145 [01:09<36:25, 1.97s/it] 3%|▎ | 37/1145 [01:11<35:35, 1.93s/it] 3%|▎ | 38/1145 [01:13<34:52, 1.89s/it] 3%|▎ | 39/1145 [01:14<34:09, 1.85s/it] 3%|▎ | 40/1145 [01:16<33:15, 1.81s/it] 3%|▎ | 40/1145 [01:16<33:15, 1.81s/it] 4%|▎ | 41/1145 [01:18<32:41, 1.78s/it] 4%|▎ | 42/1145 [01:19<32:27, 1.77s/it] 4%|▍ | 43/1145 [01:21<32:46, 1.78s/it] 4%|▍ | 44/1145 [01:23<32:37, 1.78s/it] 4%|▍ | 45/1145 [01:25<32:22, 1.77s/it] 4%|▍ | 46/1145 [01:27<32:43, 1.79s/it] 4%|▍ | 47/1145 [01:28<32:54, 1.80s/it] 4%|▍ | 48/1145 [01:30<33:36, 1.84s/it] 4%|▍ | 49/1145 [01:32<33:33, 1.84s/it] 4%|▍ | 50/1145 [01:34<33:32, 1.84s/it] 4%|▍ | 50/1145 [01:34<33:32, 1.84s/it] 4%|▍ | 51/1145 [01:36<33:03, 1.81s/it] 5%|▍ | 52/1145 [01:38<33:19, 1.83s/it] 5%|▍ | 53/1145 [01:39<33:00, 1.81s/it] 5%|▍ | 54/1145 [01:41<33:14, 1.83s/it] 5%|▍ | 55/1145 [01:43<34:22, 1.89s/it] 5%|▍ | 56/1145 [01:45<34:29, 1.90s/it] 5%|▍ | 57/1145 [01:47<34:00, 1.88s/it] 5%|▌ | 58/1145 [01:49<34:22, 1.90s/it] 5%|▌ | 59/1145 [01:51<37:29, 2.07s/it] 5%|▌ | 60/1145 [01:54<38:43, 2.14s/it] 5%|▌ | 60/1145 [01:54<38:43, 2.14s/it] 5%|▌ | 61/1145 [01:56<37:35, 2.08s/it] 5%|▌ | 62/1145 [01:57<35:49, 1.98s/it] 6%|▌ | 63/1145 [01:59<34:47, 1.93s/it] 6%|▌ | 64/1145 [02:01<35:14, 1.96s/it] 6%|▌ | 65/1145 [02:03<34:36, 1.92s/it] 6%|▌ | 66/1145 [02:05<34:16, 1.91s/it] 6%|▌ | 67/1145 [02:07<34:04, 1.90s/it] 6%|▌ | 68/1145 [02:09<33:48, 1.88s/it] 6%|▌ | 69/1145 [02:11<34:55, 1.95s/it] 6%|▌ | 70/1145 [02:13<36:11, 2.02s/it] 6%|▌ | 70/1145 [02:13<36:11, 2.02s/it] 6%|▌ | 71/1145 [02:15<35:46, 2.00s/it] 6%|▋ | 72/1145 [02:17<35:58, 2.01s/it] 6%|▋ | 73/1145 [02:19<35:44, 2.00s/it] 6%|▋ | 74/1145 [02:21<37:15, 2.09s/it] 7%|▋ | 75/1145 [02:23<37:36, 2.11s/it] 7%|▋ | 76/1145 [02:26<38:30, 2.16s/it] 7%|▋ | 77/1145 [02:28<38:37, 2.17s/it] 7%|▋ | 78/1145 [02:30<39:16, 2.21s/it] 7%|▋ | 79/1145 [02:32<38:40, 2.18s/it] 7%|▋ | 80/1145 [02:34<37:03, 2.09s/it] 7%|▋ | 80/1145 [02:34<37:03, 2.09s/it] 7%|▋ | 81/1145 [02:36<37:21, 2.11s/it] 7%|▋ | 82/1145 [02:38<37:16, 2.10s/it] 7%|▋ | 83/1145 [02:41<39:23, 2.23s/it] 7%|▋ | 84/1145 [02:43<39:18, 2.22s/it] 7%|▋ | 85/1145 [02:45<38:20, 2.17s/it] 8%|▊ | 86/1145 [02:47<37:50, 2.14s/it] 8%|▊ | 87/1145 [02:49<37:03, 2.10s/it] 8%|▊ | 88/1145 [02:51<37:00, 2.10s/it] 8%|▊ | 89/1145 [02:53<36:36, 2.08s/it] 8%|▊ | 90/1145 [02:56<36:51, 2.10s/it] 8%|▊ | 90/1145 [02:56<36:51, 2.10s/it] 8%|▊ | 91/1145 [02:58<36:12, 2.06s/it] 8%|▊ | 92/1145 [03:00<38:06, 2.17s/it] 8%|▊ | 93/1145 [03:02<38:54, 2.22s/it] 8%|▊ | 94/1145 [03:04<38:19, 2.19s/it] 8%|▊ | 95/1145 [03:07<37:54, 2.17s/it] 8%|▊ | 96/1145 [03:09<39:19, 2.25s/it] 8%|▊ | 97/1145 [03:12<40:51, 2.34s/it] 9%|▊ | 98/1145 [03:14<40:59, 2.35s/it] 9%|▊ | 99/1145 [03:16<40:52, 2.34s/it] 9%|▊ | 100/1145 [03:19<41:01, 2.36s/it] 9%|▊ | 100/1145 [03:19<41:01, 2.36s/it] 9%|▉ | 101/1145 [03:21<40:16, 2.31s/it] 9%|▉ | 102/1145 [03:23<39:27, 2.27s/it] 9%|▉ | 103/1145 [03:25<38:40, 2.23s/it] 9%|▉ | 104/1145 [03:28<39:24, 2.27s/it] 9%|▉ | 105/1145 [03:30<39:29, 2.28s/it] 9%|▉ | 106/1145 [03:32<40:51, 2.36s/it] 9%|▉ | 107/1145 [03:35<41:41, 2.41s/it] 9%|▉ | 108/1145 [03:37<41:11, 2.38s/it] 10%|▉ | 109/1145 [03:39<38:57, 2.26s/it] 10%|▉ | 110/1145 [03:41<38:29, 2.23s/it] 10%|▉ | 110/1145 [03:41<38:29, 2.23s/it] 10%|▉ | 111/1145 [03:43<37:44, 2.19s/it] 10%|▉ | 112/1145 [03:46<37:08, 2.16s/it] 10%|▉ | 113/1145 [03:47<36:02, 2.10s/it] 10%|▉ | 114/1145 [03:50<35:56, 2.09s/it] 10%|█ | 115/1145 [03:52<37:34, 2.19s/it] 10%|█ | 116/1145 [03:54<36:39, 2.14s/it] 10%|█ | 117/1145 [03:56<35:09, 2.05s/it] 10%|█ | 118/1145 [03:58<35:19, 2.06s/it] 10%|█ | 119/1145 [04:00<35:15, 2.06s/it] 10%|█ | 120/1145 [04:02<35:09, 2.06s/it] 10%|█ | 120/1145 [04:02<35:09, 2.06s/it] 11%|█ | 121/1145 [04:04<34:05, 2.00s/it] 11%|█ | 122/1145 [04:06<33:16, 1.95s/it] 11%|█ | 123/1145 [04:08<33:05, 1.94s/it] 11%|█ | 124/1145 [04:10<32:49, 1.93s/it] 11%|█ | 125/1145 [04:11<32:21, 1.90s/it] 11%|█ | 126/1145 [04:13<32:24, 1.91s/it] 11%|█ | 127/1145 [04:15<32:09, 1.90s/it] 11%|█ | 128/1145 [04:17<32:33, 1.92s/it] 11%|█▏ | 129/1145 [04:19<32:17, 1.91s/it] 11%|█▏ | 130/1145 [04:21<32:31, 1.92s/it] 11%|█▏ | 130/1145 [04:21<32:31, 1.92s/it] 11%|█▏ | 131/1145 [04:23<32:58, 1.95s/it] 12%|█▏ | 132/1145 [04:25<33:20, 1.98s/it] 12%|█▏ | 133/1145 [04:27<33:21, 1.98s/it] 12%|█▏ | 134/1145 [04:29<35:08, 2.09s/it] 12%|█▏ | 135/1145 [04:32<37:00, 2.20s/it] 12%|█▏ | 136/1145 [04:34<36:41, 2.18s/it] 12%|█▏ | 137/1145 [04:36<37:21, 2.22s/it] 12%|█▏ | 138/1145 [04:38<36:28, 2.17s/it] 12%|█▏ | 139/1145 [04:41<37:22, 2.23s/it]