-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathnotebook.log
More file actions
87 lines (87 loc) · 9.82 KB
/
notebook.log
File metadata and controls
87 lines (87 loc) · 9.82 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
INFO:__main__:Project root set to: /workspaces/layer_context_seg
INFO:graph.segment_embedder:SegmentEmbedder initialized with device: cuda
INFO:graph.classification.edge_classifier:EdgeClassifier initialized with device: cuda
INFO:graph.segment_embedder:SegmentEmbedder initialized with device: cuda
INFO:graph.classification.edge_classifier:EdgeClassifier initialized with device: cuda
INFO:graph.graph_enricher:GraphEnricher initialized with device: cuda
INFO:models.multibert_model:MultiBERT initialized with device: cuda
INFO:synthesis.unicode_utils:Unicode sanitization (strict): 158601 -> 128586 bytes, 158331 -> 128586 chars
INFO:partitioning.unified_partition_manager:Document initialized with root segment be2f6530-b830-4aff-8368-34418f90399d
INFO:partitioning.unified_partition_manager:Dispensing 1 segments for processing
INFO:partitioning.unified_partition_manager:Added 15 segments to parent be2f6530-b830-4aff-8368-34418f90399d
INFO:partitioning.unified_partition_manager:Dispensing 7 segments for processing
INFO:partitioning.unified_partition_manager:Dispensing 0 segments for processing
INFO:partitioning.unified_partition_manager:Tree construction complete. Depth: 1, Total nodes: 16
INFO:models.baai_model:Lazy loading BGE-EN-ICL model from bge-en-icl...
INFO:models.baai_model:BGE-EN-ICL model loaded successfully onto GPU
INFO:graph.core.graph_builder:Added 19 semantic edges to graph
INFO:graph.enrichment.graph_enricher:Starting graph enrichment pipeline...
INFO:graph.enrichment.graph_enricher:Inferred 12 edge types
INFO:graph.enrichment.graph_enricher:Running MultiBERT enrichment...
INFO:graph.enrichment.graph_enricher:Initializing MultiBERT for graph enrichment...
INFO:graph.segment_embedder:SegmentEmbedder initialized with device: cuda
INFO:graph.classification.edge_classifier:EdgeClassifier initialized with device: cuda
INFO:graph.segment_embedder:SegmentEmbedder initialized with device: cuda
INFO:graph.classification.edge_classifier:EdgeClassifier initialized with device: cuda
INFO:graph.graph_enricher:GraphEnricher initialized with device: cuda
INFO:models.multibert_model:MultiBERT initialized with device: cuda
INFO:graph.graph_enricher:Computing embeddings for 16 segments...
INFO:graph.segment_embedder:Loading DeBERTa model...
INFO:graph.segment_embedder:DeBERTa loaded successfully
INFO:graph.segment_embedder:Loading MathBERT model...
INFO:graph.segment_embedder:MathBERT loaded successfully
INFO:graph.graph_enricher:Classifying 49 edges...
INFO:graph.classification.edge_classifier:Loading DeBERTa model for edge classification...
INFO:graph.classification.edge_classifier:DeBERTa model loaded successfully
INFO:graph.classification.batch_processor:Optimized batch size: 45 (avg tokens per pair: 181.0)
INFO:graph.graph_enricher:Identifying hub nodes...
INFO:graph.graph_enricher:Identified 7 hub nodes
INFO:graph.graph_enricher:Graph enrichment complete: {'segments_processed': 16, 'embeddings_computed': 16, 'edges_classified': 49, 'model_usage': {'math': 2, 'science': 0, 'general': 14}, 'total_classifications': 49, 'type_distribution': {'DEPENDS_ON': 1, 'IS_CONSEQUENCE_OF': 12, 'IS_EXAMPLE_OF': 20, 'REFERENCES': 5, 'CONTINUES': 11}, 'average_confidence': 0.12177507290426566, 'high_confidence_count': 0, 'high_confidence_ratio': 0.0, 'min_confidence': 0.11608220636844635, 'max_confidence': 0.1300187110900879}
INFO:graph.enrichment.graph_enricher:MultiBERT enrichment complete: {'segments_processed': 16, 'embeddings_computed': 16, 'edges_classified': 49, 'model_usage': {'math': 2, 'science': 0, 'general': 14}, 'total_classifications': 49, 'type_distribution': {'DEPENDS_ON': 1, 'IS_CONSEQUENCE_OF': 12, 'IS_EXAMPLE_OF': 20, 'REFERENCES': 5, 'CONTINUES': 11}, 'average_confidence': 0.12177507290426566, 'high_confidence_count': 0, 'high_confidence_ratio': 0.0, 'min_confidence': 0.11608220636844635, 'max_confidence': 0.1300187110900879}
INFO:graph.enrichment.graph_enricher:Graph enrichment complete: {'inferred_edges': 12, 'transitive_edges': 0, 'clusters': {'num_clusters': 4, 'cluster_sizes': [2, 5, 2, 7], 'cluster_assignments': {'4dc1e5bd-a9fa-4e79-94b9-0f881517c59f': 0, '285c22e6-b098-4951-b91e-c3c9ff9837bc': 0, 'd725e11e-38de-483c-a0c3-59bcd4373711': 1, 'd2175ef2-32d8-4279-b710-49c172cf89cc': 1, '3d9536ec-402b-4e0e-be0b-a8075c578ab0': 1, 'f3ff6eec-4289-4326-84f8-a57f55741693': 1, '0425d1a4-d578-4a82-afcb-b1b3750345d9': 1, '9157793c-1cb8-48b4-8cbf-5db217baee01': 2, '785952f5-0fb0-49ef-8b63-2139e8b537f0': 2, 'ada3b367-b78b-476e-9ddf-45a59f66792b': 3, '68127a60-fbf0-4dfe-b541-0008594fc904': 3, 'b665a2cc-9c30-4bc1-9d91-75ebed885f53': 3, '87d28fee-685e-4647-89a6-95bf1df71775': 3, '89855b6b-3f3e-4300-b02b-118a342c1725': 3, 'be2f6530-b830-4aff-8368-34418f90399d': 3, 'e15b4c1d-af03-4791-9e87-eef54faca943': 3}, 'cluster_metrics': [{'cluster_id': 0, 'size': 2, 'density': 0.5, 'cohesion': 0.5}, {'cluster_id': 1, 'size': 5, 'density': 0.35, 'cohesion': 0.35}, {'cluster_id': 2, 'size': 2, 'density': 0.5, 'cohesion': 0.5}, {'cluster_id': 3, 'size': 7, 'density': 0.2619047619047619, 'cohesion': 0.2619047619047619}]}, 'redundancies': {'merge_candidates': [], 'redundant_continuations': ['0425d1a4-d578-4a82-afcb-b1b3750345d9', 'b665a2cc-9c30-4bc1-9d91-75ebed885f53'], 'duplicate_examples': [], 'isolated_low_importance': []}, 'narrative': {'patterns': {'setup_payoff': [], 'problem_solution': [], 'claim_evidence': [], 'concept_example': [], 'causal_chains': []}, 'flow_score': 0.0, 'roles': {}}, 'dependencies': {'critical_paths': [], 'circular_dependencies': [], 'dependency_depth': {}}, 'multibert': {'segments_processed': 16, 'embeddings_computed': 16, 'edges_classified': 49, 'model_usage': {'math': 2, 'science': 0, 'general': 14}, 'total_classifications': 49, 'type_distribution': {'DEPENDS_ON': 1, 'IS_CONSEQUENCE_OF': 12, 'IS_EXAMPLE_OF': 20, 'REFERENCES': 5, 'CONTINUES': 11}, 'average_confidence': 0.12177507290426566, 'high_confidence_count': 0, 'high_confidence_ratio': 0.0, 'min_confidence': 0.11608220636844635, 'max_confidence': 0.1300187110900879}}
INFO:synthesis.unified_document_builder:Cleaning up UnifiedDocumentBuilder resources...
INFO:synthesis.unified_document_builder:UnifiedDocumentBuilder cleanup complete
INFO:synthesis.unified_document_builder:Cleaning up UnifiedDocumentBuilder resources...
INFO:synthesis.unified_document_builder:UnifiedDocumentBuilder cleanup complete
INFO:synthesis.cache_manager:Initialized cache manager: /root/.layered_context_cache, max memory: 512MB
INFO:synthesis.unified_document_builder:Registered synthesis strategy: progressive_transformer
INFO:synthesis.unified_document_builder:Registered synthesis strategy: executive_summary
INFO:synthesis.unified_document_builder:Registered synthesis strategy: tutorial
INFO:synthesis.unified_document_builder:Registered synthesis strategy: reference
INFO:synthesis.unified_document_builder:Registered synthesis strategy: readme
INFO:synthesis.unified_document_builder:Initialized UnifiedDocumentBuilder with base and legacy strategies
INFO:synthesis.unified_document_builder:Registered synthesis strategy: progressive_transformer
INFO:synthesis.unified_document_builder:Registered synthesis strategy: executive_summary
INFO:synthesis.unified_document_builder:Registered synthesis strategy: tutorial
INFO:synthesis.unified_document_builder:Registered synthesis strategy: reference
INFO:synthesis.unified_document_builder:Registered synthesis strategy: readme
INFO:synthesis.unified_document_builder:Initialized UnifiedDocumentBuilder with base and legacy strategies
INFO:synthesis.unified_document_builder:Cleaning up UnifiedDocumentBuilder resources...
INFO:synthesis.cache_manager:Cleared all cache entries
INFO:synthesis.unified_document_builder:UnifiedDocumentBuilder cleanup complete
INFO:synthesis.unified_document_builder:Registered synthesis strategy: progressive_transformer
INFO:synthesis.unified_document_builder:Registered synthesis strategy: executive_summary
INFO:synthesis.unified_document_builder:Registered synthesis strategy: tutorial
INFO:synthesis.unified_document_builder:Registered synthesis strategy: reference
INFO:synthesis.unified_document_builder:Registered synthesis strategy: readme
INFO:synthesis.unified_document_builder:Initialized UnifiedDocumentBuilder with base and legacy strategies
INFO:synthesis.unified_document_builder:Cleaning up UnifiedDocumentBuilder resources...
INFO:synthesis.cache_manager:Cleared all cache entries
INFO:synthesis.unified_document_builder:UnifiedDocumentBuilder cleanup complete
INFO:synthesis.unified_document_builder:Building document with strategy: hsc
INFO:synthesis.unified_document_builder:📋 UnifiedDocumentBuilder: Total segments: 16, KEEP segments: 16
INFO:synthesis.unified_document_builder:🏷️ UnifiedDocumentBuilder: Tag distribution: {'KEEP': 16}
INFO:synthesis.unified_document_builder:🎨 Content Preparation: Processing 16 segments for technical style
INFO:models.qwq_model:Lazy loading model from QwQ_LCoT_7B_Instruct using sharded loader...
INFO:models.qwq_model:Set pad_token_id to eos_token_id: 151645
INFO:models.qwq_model:Model is fully loaded onto GPU and ready.
INFO:models.qwq_model:Generating from graph with ~219 input tokens
INFO:models.qwq_model:Generating with ~219 tokens input, window_size=2048
INFO:synthesis.unified_document_builder:Building document with strategy: hsc
INFO:synthesis.unified_document_builder:📋 UnifiedDocumentBuilder: Total segments: 16, KEEP segments: 16
INFO:synthesis.unified_document_builder:🏷️ UnifiedDocumentBuilder: Tag distribution: {'KEEP': 16}
INFO:synthesis.unified_document_builder:🎨 Content Preparation: Processing 16 segments for narrative style
INFO:models.qwq_model:Generating from graph with ~569 input tokens
INFO:models.qwq_model:Generating with ~569 tokens input, window_size=2048
INFO:synthesis.unified_document_builder:Building document with strategy: executive_summary
INFO:partitioning.save_load_helper:Saving PartitionManager to results/Multi_Scale_Catastrophe_Detection_c9f94e92_2025-07-13T05-26-14-339Z.txt_enhanced_graph.h5