Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tasty_redex_independent property test failure #1242

Open
dhess opened this issue Apr 16, 2024 · 0 comments
Open

tasty_redex_independent property test failure #1242

dhess opened this issue Apr 16, 2024 · 0 comments
Assignees
Labels
blocked/need-info ❌ Blocked, need more information testing Related to tests/testing

Comments

@dhess
Copy link
Member

dhess commented Apr 16, 2024

redex independent:                                  FAIL (3.61s)
--
  | ✗ redex independent failed at test/Tests/Eval.hs:1476:26
  | after 137 tests, 11 shrinks and 721 discards.
  | shrink path: 137/721:aDaCa2B2dAc
  |  
  | ┏━━ test/Tests/Eval.hs ━━━
  | 1442 ┃ tasty_redex_independent :: Property
  | 1443 ┃ tasty_redex_independent =
  | 1444 ┃   let testModules = [builtinModule, primitiveModule]
  | 1445 ┃    in withTests 200
  | 1446 ┃         $ withDiscards 2000
  | 1447 ┃         $ propertyWT testModules
  | 1448 ┃         $ do
  | 1449 ┃           let globs = foldMap' moduleDefsQualified $ create' $ sequence testModules
  | 1450 ┃           as <- forAllT $ Gen.element enumerate
  | ┃           │ AvoidShadowing
  | 1451 ┃           tds <- asks typeDefs
  | 1452 ┃           (dir, t, _) <- genDirTm
  | 1453 ┃           annotateShow dir
  | ┃           │ Chk
  | 1454 ┃           annotateShow t
  | ┃           │ Letrec
  | ┃           │   (Meta 0 Nothing Nothing)
  | ┃           │   LocalName { unLocalName = "x" }
  | ┃           │   (EmptyHole (Meta 1 Nothing Nothing))
  | ┃           │   (TEmptyHole (Meta 11 Nothing Nothing))
  | ┃           │   (Let
  | ┃           │      (Meta 2 Nothing Nothing)
  | ┃           │      LocalName { unLocalName = "y" }
  | ┃           │      (EmptyHole (Meta 3 Nothing Nothing))
  | ┃           │      (Case
  | ┃           │         (Meta 4 Nothing Nothing)
  | ┃           │         (Ann
  | ┃           │            (Meta 5 Nothing Nothing)
  | ┃           │            (EmptyHole (Meta 6 Nothing Nothing))
  | ┃           │            (TApp
  | ┃           │               (Meta 12 Nothing Nothing)
  | ┃           │               (TApp
  | ┃           │                  (Meta 13 Nothing Nothing)
  | ┃           │                  (TCon
  | ┃           │                     (Meta 14 Nothing Nothing)
  | ┃           │                     GlobalName
  | ┃           │                       { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃           │                       , baseName = "Either"
  | ┃           │                       })
  | ┃           │                  (TEmptyHole (Meta 15 Nothing Nothing)))
  | ┃           │               (TEmptyHole (Meta 16 Nothing Nothing))))
  | ┃           │         [ CaseBranch
  | ┃           │             (PatCon
  | ┃           │                GlobalName
  | ┃           │                  { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃           │                  , baseName = "Left"
  | ┃           │                  })
  | ┃           │             [ Bind (Meta 7 Nothing Nothing) LocalName { unLocalName = "y" } ]
  | ┃           │             (Var
  | ┃           │                (Meta 8 Nothing Nothing)
  | ┃           │                (LocalVarRef LocalName { unLocalName = "x" }))
  | ┃           │         , CaseBranch
  | ┃           │             (PatCon
  | ┃           │                GlobalName
  | ┃           │                  { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃           │                  , baseName = "Right"
  | ┃           │                  })
  | ┃           │             [ Bind (Meta 9 Nothing Nothing) LocalName { unLocalName = "x" } ]
  | ┃           │             (EmptyHole (Meta 10 Nothing Nothing))
  | ┃           │         ]
  | ┃           │         CaseExhaustive))
  | 1455 ┃           rs <- failWhenSevereLogs $ redexes @EvalLog as tds globs dir t
  | 1456 ┃           when (length rs <= 1) discard
  | 1457 ┃           i <- forAllT $ Gen.element rs
  | ┃           │ 2
  | 1458 ┃           j <- forAllT $ Gen.element $ delete i rs
  | ┃           │ 0
  | 1459 ┃           s <- failWhenSevereLogs $ step @EvalLog as tds globs t dir i
  | 1460 ┃           case s of
  | 1461 ┃             Left err -> annotateShow err >> failure
  | 1462 ┃             Right (s', siDetails) -> do
  | 1463 ┃               annotateShow s'
  | ┃               │ Letrec
  | ┃               │   (Meta 0 Nothing Nothing)
  | ┃               │   LocalName { unLocalName = "x" }
  | ┃               │   (EmptyHole (Meta 1 Nothing Nothing))
  | ┃               │   (TEmptyHole (Meta 11 Nothing Nothing))
  | ┃               │   (Case
  | ┃               │      (Meta 4 Nothing Nothing)
  | ┃               │      (Ann
  | ┃               │         (Meta 5 Nothing Nothing)
  | ┃               │         (EmptyHole (Meta 6 Nothing Nothing))
  | ┃               │         (TApp
  | ┃               │            (Meta 12 Nothing Nothing)
  | ┃               │            (TApp
  | ┃               │               (Meta 13 Nothing Nothing)
  | ┃               │               (TCon
  | ┃               │                  (Meta 14 Nothing Nothing)
  | ┃               │                  GlobalName
  | ┃               │                    { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃               │                    , baseName = "Either"
  | ┃               │                    })
  | ┃               │               (TEmptyHole (Meta 15 Nothing Nothing)))
  | ┃               │            (TEmptyHole (Meta 16 Nothing Nothing))))
  | ┃               │      [ CaseBranch
  | ┃               │          (PatCon
  | ┃               │             GlobalName
  | ┃               │               { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃               │               , baseName = "Left"
  | ┃               │               })
  | ┃               │          [ Bind (Meta 7 Nothing Nothing) LocalName { unLocalName = "y" } ]
  | ┃               │          (Var
  | ┃               │             (Meta 8 Nothing Nothing)
  | ┃               │             (LocalVarRef LocalName { unLocalName = "x" }))
  | ┃               │      , CaseBranch
  | ┃               │          (PatCon
  | ┃               │             GlobalName
  | ┃               │               { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃               │               , baseName = "Right"
  | ┃               │               })
  | ┃               │          [ Bind (Meta 9 Nothing Nothing) LocalName { unLocalName = "x" } ]
  | ┃               │          (EmptyHole (Meta 10 Nothing Nothing))
  | ┃               │      ]
  | ┃               │      CaseExhaustive)
  | 1464 ┃               if elemOf exprIDs j s'
  | 1465 ┃                 then do
  | 1466 ┃                   sj <- failWhenSevereLogs $ step @EvalLog as tds globs t dir j
  | 1467 ┃                   case (sj, siDetails) of
  | 1468 ┃                     (Right (_, BindRename{}), _) -> success
  | 1469 ┃                     (_, PushLetDown{}) -> success
  | 1470 ┃                     (_, PushLetDownTy{}) -> success
  | 1471 ┃                     (Right (_, PushLetDown{}), CaseReduction{}) -> success
  | 1472 ┃                     (Right (_, PushLetDown{}), CaseReductionTrivial{}) -> success
  | 1473 ┃                     (Right (_, PushLetDown{}), RemoveAnn{}) -> success
  | 1474 ┃                     (Right (_, PushLetDown{}), LetRemoval{}) -> success
  | 1475 ┃                     (Right (_, PushLetDownTy{}), TLetRemoval{}) -> success
  | 1476 ┃                     _ -> assert . elem j =<< failWhenSevereLogs (redexes @EvalLog as tds globs dir s')
  | ┃                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | 1477 ┃                 else success
  |  
  | ┏━━ test/Tests/Eval/Utils.hs ━━━
  | 51 ┃ genDirTm :: PropertyT WT (Dir, Expr, Type' () ())
  | 52 ┃ genDirTm = do
  | 53 ┃   dir <- forAllT $ Gen.element @[] [Chk, Syn]
  | ┃   │ Chk
  | 54 ┃   (t', ty) <- case dir of
  | 55 ┃     Chk -> do
  | 56 ┃       ty' <- forAllT $ genWTType $ KType ()
  | ┃       │ TApp () (TEmptyHole ()) (TEmptyHole ())
  | 57 ┃       t' <- forAllT $ genChk ty'
  | ┃       │ Letrec
  | ┃       │   ()
  | ┃       │   LocalName { unLocalName = "x" }
  | ┃       │   (EmptyHole ())
  | ┃       │   (TEmptyHole ())
  | ┃       │   (Let
  | ┃       │      ()
  | ┃       │      LocalName { unLocalName = "y" }
  | ┃       │      (EmptyHole ())
  | ┃       │      (Case
  | ┃       │         ()
  | ┃       │         (Ann
  | ┃       │            ()
  | ┃       │            (EmptyHole ())
  | ┃       │            (TApp
  | ┃       │               ()
  | ┃       │               (TApp
  | ┃       │                  ()
  | ┃       │                  (TCon
  | ┃       │                     ()
  | ┃       │                     GlobalName
  | ┃       │                       { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃       │                       , baseName = "Either"
  | ┃       │                       })
  | ┃       │                  (TEmptyHole ()))
  | ┃       │               (TEmptyHole ())))
  | ┃       │         [ CaseBranch
  | ┃       │             (PatCon
  | ┃       │                GlobalName
  | ┃       │                  { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃       │                  , baseName = "Left"
  | ┃       │                  })
  | ┃       │             [ Bind () LocalName { unLocalName = "y" } ]
  | ┃       │             (Var () (LocalVarRef LocalName { unLocalName = "x" }))
  | ┃       │         , CaseBranch
  | ┃       │             (PatCon
  | ┃       │                GlobalName
  | ┃       │                  { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
  | ┃       │                  , baseName = "Right"
  | ┃       │                  })
  | ┃       │             [ Bind () LocalName { unLocalName = "x" } ]
  | ┃       │             (EmptyHole ())
  | ┃       │         ]
  | ┃       │         CaseExhaustive))
  | 58 ┃       pure (t', ty')
  | 59 ┃     Syn -> forAllT genSyn
  | 60 ┃   t <- generateIDs t'
  | 61 ┃   pure (dir, t, ty)
  |  
  | This failure can be reproduced by running:
  | > recheckAt (Seed 1531807771645439630 13357517114532957039) "137/721:aDaCa2B2dAc" redex independent
  |  
  | Use "--pattern '$NF ~ /redex independent/' --hedgehog-replay '137/721:aDaCa2B2dAc Seed 1531807771645439630 13357517114532957039'" to reproduce from the command-line.
  |  
  | Use -p '/redex independent/' to rerun this test only.
  |  
  | 1 out of 856 tests failed (70.27s)

Ref:

Logs:

@dhess dhess added testing Related to tests/testing blocked/need-info ❌ Blocked, need more information labels Apr 16, 2024
@dhess dhess self-assigned this Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked/need-info ❌ Blocked, need more information testing Related to tests/testing
Projects
None yet
Development

No branches or pull requests

1 participant