Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transformer.compile improvements #480

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from
Draft

Transformer.compile improvements #480

wants to merge 9 commits into from

Conversation

seanmacavaney
Copy link
Collaborator

wip

fixes #451

@cmacdonald
Copy link
Contributor

The following is an extract of a conversation between Sean and Craig

what about transformer reuse?
br >> ( (a >> b) + (a >> c) )
could be rewritten as
br >> a >> (b+c)

That seems like an easy rule to add to CombSumTransformer's transformer (as long as we first convince ourselves that it's universally applicable):

class CombSumTransformer:
  def compile(self):
    if isinstance(self.left, ComposedPipeline) and len(self.left) == 2 and isinstance(self.right, ComposedPipeline) and len(self.right) == 2 and self.left[0] == self.right[0]:
      return self.left[0] >> (self.left[1] + self.right[1])

In fact, it could be pretty easily generalized to any length prefix, which is impossible to express with matchpy afaik. Ie

br >> ((a>>b>>c)+(a>>b>>d))
->
br >> a >> b >> (c+d)

class CombSumTransformer:
  def compile(self):
    if isinstance(self.left, ComposedPipeline) and isinstance(self.right, ComposedPipeline) and len(self.left) == len(self.right):
      prefix = takewhile(zip(self.left, self.right), lambda x: x[0] == x[1])
      if len(prefix) == len(self.left) - 1:
        return ComposedPipeline(prefix) >> (self.left[-1], self.right[-1])

pyterrier/transformer.py Show resolved Hide resolved
pyterrier/terrier/retriever.py Show resolved Hide resolved
pyterrier/ops.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Transformer.compile improvements
2 participants