Skip to content
This repository was archived by the owner on Jun 19, 2025. It is now read-only.

Multithreaded v2#38

Closed
FrancescElies wants to merge 3 commits intovisualfabriq:masterfrom
FrancescElies:multithreaded_v2
Closed

Multithreaded v2#38
FrancescElies wants to merge 3 commits intovisualfabriq:masterfrom
FrancescElies:multithreaded_v2

Conversation

@FrancescElies
Copy link
Copy Markdown
Contributor

This is another tryout, based on master.

So far no performance improvements, I guess something like Blosc/bcolz#160 is needed to have bigger sections wit gil released

@FrancescElies
Copy link
Copy Markdown
Contributor Author

closing

@CarstVaartjes
Copy link
Copy Markdown
Member

Hi Francesc!

How are you? In the end we made this (also open source): https://github.com/visualfabriq/bqueryd
It's a zeromq based, distributed network over which you can spread "bcolz shards" and which can massively parallellize bcolz queries.
Final code should be released next week I guess!

@FrancescElies
Copy link
Copy Markdown
Contributor Author

Hey Carst,
I'm good, and you? zeromq and bcolz looks like a marriage made in heaven to me.
I'll have a look at bqueryd as soon as I get some free time, I'll try to keep up to date.
Good luck with the new release next week!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants