Skip to content

Commit

Permalink
index: fix code annotations.
Browse files Browse the repository at this point in the history
  • Loading branch information
@stefanoteso
stefanoteso committed Jun 12, 2024
1 parent 8959a80 commit e3e4a14
Showing 1 changed file with 13 additions and 13 deletions.
26 changes: 13 additions & 13 deletions index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ introduces novel formal verification procedures for assessing the presence of RS
TODO: add BEARS or NeurIPS figure 2.

**What are L&R tasks?** In learning and reasoning tasks, machine learning models should predict labels
that comply with prior knowledge. For instance, in autonomous vehicle scenario, the model should predict ``stop`` or ``go`` based on what obstacles are visible in front of the vehicle, and the prior knowledge encodes the rule that if a ``pedestrian`` or a ``red_light`` is visible then it should definitely predict ``stop``.
that comply with prior knowledge. For instance, in autonomous vehicle scenario, the model should predict `stop` or `go` based on what obstacles are visible in front of the vehicle, and the prior knowledge encodes the rule that if a `pedestrian` or a `red_light` is visible then it should definitely predict `stop`.

**What is a reasoning shortcut?** A RS occurs when the model predicts the right label by inferring the wrong concepts. For instance, it might confuse ``pedestrian``s for ``red_light``s as both entail the same (correct) ``stop`` action.
**What is a reasoning shortcut?** A RS occurs when the model predicts the right label by inferring the wrong concepts. For instance, it might confuse `pedestrian`s for `red_light`s as both entail the same (correct) `stop` action.

**What consequences to RSs have?** RSs can compromise model explanations (e.g., because these show that ``red_light``s are responsible for the predictions, while in fact this depends on the presence of red lights
**What consequences to RSs have?** RSs can compromise model explanations (e.g., because these show that `red_light`s are responsible for the predictions, while in fact this depends on the presence of red lights


# Key Features
Expand All @@ -53,14 +53,14 @@ tasks. It also provides *data generators* for creating additional data splits.

| L&R Task | Images | Concepts | Labels | #Train | #Valid | #Test | #OOD |
| :-- | :--: | :--: | :--: | :--: | :--: | :--: | :--: |
| ``MNMath`` | 28k x 28 | k digits, 10 values each | categorical multilabel | custom | custom | custom | custom |
| ``MNAdd-Half`` | 56 x 28 | 2 digits, 10 values each | categorical 0 ... 18 | 2,940 | 840 | 420 | 1,080 |
| ``MNAdd-EvenOdd`` | 56 x 28 | 2 digits, 10 values each | categorical 0 ... 18 | 6,720 | 1,920 | 960 | 5,040 |
| ``MNLogic`` | 28k x 28 | k digits, 10 values each | binary | custom | custom | custom | custom |
| ``Kand-Logic`` | 3 x 192 x 64 | 3 objects per image, 3 shapes, 3 colors | binary | 4,000 | 1,000 | 1,000 | - |
| ``CLE4EVR`` | 320 x 240 | n to m objects per image, 10 shapes, 10 colors, 2 materials, 3 sizes | binary | custom | custom | custom | custom |
| ``BDD-OIA`` | 1280 x 720 | 21 binary concepts | binary multilabel, 4 labels | 16,082 | 2,270 | 4,572 | -- |
| ``SDD-OIA`` | 469 x 387 | 21 binary concepts | binary multilabel, 4 labels | 6,820 | 1,464 | 1,464 | 1,000 |
| `MNMath` | 28k x 28 | k digits, 10 values each | categorical multilabel | custom | custom | custom | custom |
| `MNAdd-Half` | 56 x 28 | 2 digits, 10 values each | categorical 0 ... 18 | 2,940 | 840 | 420 | 1,080 |
| `MNAdd-EvenOdd` | 56 x 28 | 2 digits, 10 values each | categorical 0 ... 18 | 6,720 | 1,920 | 960 | 5,040 |
| `MNLogic` | 28k x 28 | k digits, 10 values each | binary | custom | custom | custom | custom |
| `Kand-Logic` | 3 x 192 x 64 | 3 objects per image, 3 shapes, 3 colors | binary | 4,000 | 1,000 | 1,000 | - |
| `CLE4EVR` | 320 x 240 | n to m objects per image, 10 shapes, 10 colors, 2 materials, 3 sizes | binary | custom | custom | custom | custom |
| `BDD-OIA` | 1280 x 720 | 21 binary concepts | binary multilabel, 4 labels | 16,082 | 2,270 | 4,572 | -- |
| `SDD-OIA` | 469 x 387 | 21 binary concepts | binary multilabel, 4 labels | 6,820 | 1,464 | 1,464 | 1,000 |


# How To Use rsbench
Expand All @@ -74,9 +74,9 @@ TODO: add figure

WRITEME

**Ready-made**: ``MNAdd-Half`` WRITEME
**Ready-made**: `MNAdd-Half` WRITEME

**Ready-made**: ``MNAdd-EvenOdd`` WRITEME
**Ready-made**: `MNAdd-EvenOdd` WRITEME


# MNISTLogic
Expand Down

0 comments on commit e3e4a14

Please sign in to comment.