Implemented the functionality to use custom embeddings for Mamba models.

[Adibvafa]

PR Type

Feature

Short Description

Implemented the functionality to use custom embeddings for Mamba models.

Tests Added

None

[Adibvafa]

@amrit110 Note that in file mamba_utils.py, the checker changes the docstring for forward function of class MambaForSequenceClassification, which results in failures due to necessary requirements of the HuggingFace library.

The final merged version to main should be this:

        r"""
        labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*):
            Labels for computing the sequence classification/regression loss. Indices should be in `[0, ...,
            config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If
            `config.num_labels > 1` a classification loss is computed (Cross-Entropy).

        Returns:
        """

[amrit110]

Hmm @Adibvafa , do you mean the change to the docstring violates what huggingface needs to be part of their repo?

[Adibvafa]

Hmm @Adibvafa , do you mean the change to the docstring violates what huggingface needs to be part of their repo?

Yes, exactly! It should end with "Returns:". If its hard to configure this exception, I can create an example to add to docstring so it won't break?