Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple indexing from profiling #51

Open
raulcf opened this issue Aug 23, 2016 · 1 comment
Open

Decouple indexing from profiling #51

raulcf opened this issue Aug 23, 2016 · 1 comment
Milestone

Comments

@raulcf
Copy link
Contributor

raulcf commented Aug 23, 2016

EDIT: [Profiling is about 1 order of magnitude faster than indexing.
Decouple both processes]

Decouple profiling process into the smallest pieces possible. For example, one should be able to index only schemas and no data, or data alone, etc. Then find a way of combining then back again into the original form. Challenge is to maintain all perf guarantees.

@mansoure
Copy link
Contributor

mansoure commented Nov 20, 2016

@raulcf how do you parallelize the indexing process? Decoupling both processes is good, but I think we may have a chance to optimize the indexing process itself. I see in another issue, #52 , that you managed to make it 3X slower instead of 10X. That is great. This might be related to issue #48 . I am also interested in helping if help is still needed.

@raulcf raulcf added this to the v0.5 milestone Oct 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants