Hi, I ran a graph pattern matching query on emptyheaded and failed. It seemed that I got insufficient memory error.
Here is my failed query. The pattern looks like a house.
"Q4(;z) :- Edge(a,b),Edge(a,e),Edge(b,e),Edge(b,c),Edge(c,d),Edge(d,e),z:long<-[COUNT(*)]"
I ran on a 40-core machine with 128GB main memory. The output is as follows:
(system: emptyheaded, num_threads: 40,
num_sockets: 4, layout: hybrid, memory: RAM)
Time[LOADING Trie Edge_0_1]: 1.52694 s
Time[LOADING Trie Edge_1_0]: 1.38699 s
Time[LOADING ENCODINGS uint32_t]: 0.157045 s
MemoryBuffer::addBuffer, realloc error!
python: /home/wentian/programs/EmptyHeaded/storage_engine/src/utils/MemoryBuffer.cpp:80: uint8_t* MemoryBuffer::resize(size_t): Assertion `false' failed.
As shown in the above query, I enable an aggregation query to count the number of matching results instead of materializing all matching results, in order to save memory footprint. I suppose, as long as the main memory can hold the graph itself and some metadata (e.g. trie, set), this query can be finished without consuming much memory. But it still turns out to fail in the end. I am wondering why it is that.
I found that you use TrieBuilder::aggregate_sets and TrieBuilder::build_aggregated_set() to execute this query(I check the generated c++ code by the optimizer). It seems that you didn't release memory in TrieBuilder::aggregate_sets after doing set intersection in TrieBuilder::build_aggregated_set() (see TrieBuilder.cpp:346). Is that a possible reason ?
Hi, I ran a graph pattern matching query on emptyheaded and failed. It seemed that I got insufficient memory error.
Here is my failed query. The pattern looks like a house.
"Q4(;z) :- Edge(a,b),Edge(a,e),Edge(b,e),Edge(b,c),Edge(c,d),Edge(d,e),z:long<-[COUNT(*)]"
I ran on a 40-core machine with 128GB main memory. The output is as follows:
(system: emptyheaded, num_threads: 40,
num_sockets: 4, layout: hybrid, memory: RAM)
Time[LOADING Trie Edge_0_1]: 1.52694 s
Time[LOADING Trie Edge_1_0]: 1.38699 s
Time[LOADING ENCODINGS uint32_t]: 0.157045 s
MemoryBuffer::addBuffer, realloc error!
python: /home/wentian/programs/EmptyHeaded/storage_engine/src/utils/MemoryBuffer.cpp:80: uint8_t* MemoryBuffer::resize(size_t): Assertion `false' failed.
As shown in the above query, I enable an aggregation query to count the number of matching results instead of materializing all matching results, in order to save memory footprint. I suppose, as long as the main memory can hold the graph itself and some metadata (e.g. trie, set), this query can be finished without consuming much memory. But it still turns out to fail in the end. I am wondering why it is that.
I found that you use TrieBuilder::aggregate_sets and TrieBuilder::build_aggregated_set() to execute this query(I check the generated c++ code by the optimizer). It seems that you didn't release memory in TrieBuilder::aggregate_sets after doing set intersection in TrieBuilder::build_aggregated_set() (see TrieBuilder.cpp:346). Is that a possible reason ?