Poll: Attributes or no?

[Ovid]

Initially attributes and a KIM syntax were proposed for data checks:

my $count :of(INT) = 0;

There are two reasons for this. The primary reason was the concern that the my TYPE VARLIST syntax was already used and P5P might reject repurposing it. However, it now looks like this was worrying about nothing.

The other reason was pushing Perl to standardize on KIM syntax, where appropriate. In particular, with a 36-year-old language, we're piling on syntax after syntax. The language is growing large and we wind up with bewildering "TATMWTDI" (There Are Too Many Ways To Do It) and every developer seems to fight for their one preferred way, meaning that new developers to Perl find a bewildering range of alternatives, some of which can have subtly different behaviors (for example, &foo($bar) versus foo($bar)). Since the KIM syntax is already used in Corinna, it seemed a logical thing to do.

That being said, several people looking at the proposal were happy with attributes while others hated them.

Repurposing existing my TYPE VARLIST syntax has advantages:

• It's more concise (less punctuation)
• It's more familiar (and thus more welcoming to new developers)
• Thus, less controversial

@druud @happy-barney @perigrin @rabbiveesh @rwp0 @SysPete @tobyink @demerphq

[Ovid]

Please note that the above is only for declarative usage of checks:

• my int $foo;
• sub foo($arg :of(INT))

There have been plenty of discussions about using types in expressions, but I feel that's out of scope for the current MVP unless we find that it has an impact on the design of the MVP.

[happy-barney]

advantage of attributes is that this can be only in library (with some new support in core even with acceptable performance) - I can prepare small spec for that to discuss if there will be interest

[tobyink]

Yeah, but I don't think it's good to make syntax decisions based on what is most convenient to implement in the short term. It's basically a form of technical debt, saddling potentially millions of future users with a worse syntax because that makes things easier on a handful of implementers over the next year or two.

[happy-barney]

@tobyink my opinion is that for this poll we should have also overview where we want to continue with it.

easy implementation with leading token where you don't need to put heuristics to guess what it is (just look at indirect syntax - should perl support proper class literal syntax, it will not be problem) is imho better for mvp implementation so you can focus on checks and not on grammars and conflicts.

[demerphq]

@tobyink the counterpoint to what you say is that choosing something that is inconvenient to implement but supposedly superior also reduces (considerably sometimes) the change that it will get implemented at all. Essentially it boils down to a bird in the hand is better than two in the bush (from the hunting perspective). I have seen more than one proposal for new Perl features that were so unaligned with what was feasible that the proposal never went anywhere. I dont have enough insight into the trade offs /here/ to say anything specific, but my general inclination is that choosing a mechanism we can use /right now/ is better than choosing one we might be able to use if a large amount of round tuits were invested into the subject.

[rwp0]

my TYPE VARLIST

This syntax also let's you define multiple variables with a single type.

I think we should stick to what was there before, if we had a room for TYPE let's just take advantage of it.

Let's improve Perl, but not try to change's it's existing syntax fundamentally (as with Raku).

Attributes and KIM is awesome in all other aspects (apart from variable and function parameters TYPE's).

[demerphq]

On Mon, 3 Jul 2023, 12:00 Ovid, ***@***.***> wrote: Initially attributes and a KIM syntax were proposed for data checks:

I am not familiar with the term KIM syntax. Is there a reference link I can read? Yves

[happy-barney]

KIM = keyword identifier modifiers

in this case it's postfix declaration of checks, using attributes:

my $var :attribute(CHECK);

[Ovid]

@demerphq I see that @happy-barney has covered the basics of it. I first wrote about it here.

[happy-barney]

just stupid question - why not both? (ie: timtowdi)

• attributes for mpv (in fact I'd pick separated symbol spaces as standalone project - you can see first draft at https://gist.github.com/happy-barney/57fdaee7d86a0bbdc8e96834e5c3c295)
  • attribute syntax will be still necessary when there is not clear context switch, eg map
• then two independent directions are possible
  • provide prefix
  • work on checks

and let user decide which syntax will be using (as we have with prefix / postfix if and co)

[zmughal]

let user decide which syntax will be using (as we have with prefix / postfix if and co)

This is how I feel about it. It retains that natural language aspect of Perl's design where you can choose what to make prominent when reading.

[duncand]

I agree with let user decide but that this should be limited. I propose that the KIM syntax be the standard general syntax that supports the full Oshun syntax, but that a small common subset of the syntax has an alternative KMI, specifically the subset involving just a single alphanumeric word or something very short. So if you're just saying eg "Int" it can be in either form, but for something arbitrarily long it is only supported in KIM syntax.

[demerphq]

Ovid it isnt clear to me what trade offs we are dealing with here. If we choose KIM, would it be more usable than if we used type declarations?

Right now as far as I understand it the trade off is that using type declaration would mean we can use one declaration for multiple variables, and if we used KIM we would have to repeat the declaration for each variable. Are there any other trade offs? Can we use type syntax in subroutine argument declarations, or is that restricted to KIM?

My gut feeling we should choose the option that facilitates the maximum usefulness with the least need to modify the internals.

[happy-barney]

@perigrin just look at test case later in this discussion ... how to treat Int there? what will you restrict?

Another examples:

use constant MIN => 10;
check MIN :isa (NUM) where { $_ == 10 };

my Array[MIN, Int] @list; # is MIN check or constant? you must decide at the time when you read 1 token - `MIN`

When allowing full syntax, it may become pretty unreadable:

my NUM where { $_ == 10 } $foo;

Can check be stored in variable? eg:

my $foo = declared check (FOO);

$foo->check ($value);
my $foo $bar;

[perigrin]

The MIN here is ambiguous depending on the grammar of Array[] syntax but only if we say it can allow non-check values (Array[10]) then the choice is obvious … but should that be allowed?

[Ovid]

use constant MIN => 10;
check MIN :isa (NUM) where { $_ == 10 };

my Array[MIN, Int] @list; # is MIN check or constant? you must decide at the time when you read 1 token - `MIN`

Checks are not in the same namespace and don't have access to the lexical scope, so with our current work. MIN cannot be a constant. If we wish to allow dynamic checks to be parameterized at runtime, that's a different story. We didn't do that with the current spec because attributes don't allow that.

my NUM where { $_ == 10 } $foo;

We have a spec for ranges:

my NUM[1..inf] $foo = 1; # 1 <= $_
my NUM[10<..20] $bar = get_value(); # 10 < $_ <= 20
my NUM[5..<15.2] $baz = get_value(); # 5 <= $_ < 15.2

But we don't have it for a single value, but we could make it work:

NUM[==10] # == 10
NUM[<10]  #  < 10
NUM[<=10] # <= 10
NUM[>10]  #  > 10
NUM[>=10] # >= 10

Not sure how clumsy that is, though.

Can check be stored in variable? eg:

my $foo = declared check (FOO);

$foo->check ($value);
my $foo $bar;

Currently no.

[perigrin]

It's not currently allowed in the my TYPE VARLIST syntax either … but that's mostly because TYPE in this case can only be a package name or a constant that resolves to a package name, i.e. a bareword and thus cannot currently be parametrized. We would have to modify the parser to allow more syntax than a single bareword there if we want anything like parameterized types declared on the fly.

[Ovid]

@demerphq I should have read more carefully.

my ( $foo, $bar ) :of(INT) = 0;

No, that cannot work because $bar is not assigned a value.

[happy-barney]

Prefix test case:

sub Int ($x) { return 10 * $x } # whatever internal representation will be
sub foo { 
  no strict;
  # my $x = 10.5; # edited ... should not be in this example
  Int $x = 11.5;
}

ok foo == 115;

[Ovid]

At the present time that is not problematic because we only allow checks on declaration.

[happy-barney]

@Ovid oh, I made a mistake with putting my line there :-( (forgetting reading twice these days ...)

regarding your comment ... that's what I'm talking about, without overview where we want to continue after mvp and how it will/may interact with another possible features, it's hard to tell from oversimplified examples.

@perigrin
and that's unexpected ambiguity ... what it will be if Int is not a function? for C/Java/ .... programmers that line makes perfect sense, yet it will not behave as expected.

But as I said, I'll be happy to write tests to break it

[perigrin]

That ambiguity has nothing to do with Oshun though. It already exists in Perl, and you have to understand Perl's scoping and parsing already to cope with things like this:

perl -Mexperimental=class -E'class Int {} sub Int { ... } Int->new()'
Unimplemented at -e line 1.

The fact it already exists is why I'd reject a PR without more explicit definitions of what is going on. I'd tell the submitter "you need to spell that second Int as Int:: and leave a comment as to why, if you don't understand why come find me and we'll talk about the glory of Perl's bareword parsing."

This is (partly?) why @Ovid keeps saying "Checks are not in the same namespace and don't have access to the lexical scope[…]"

[demerphq]

we'll talk about the glory of Perl's bareword parsing

I'm sure you mean "gory details" there. It sure ain't glorious.

[happy-barney]

@perigrin yes, that's true and I'm aware of it ... I'm just pointing out that Oshun should not create another cases of such ambiguity (therefor, first focus on what it has to provide, how it will be accessible is secondary problem)

For example, first implementation part should be support for custom symbol spaces (not only for Oshun but available for any module) - WIP at https://gist.github.com/happy-barney/57fdaee7d86a0bbdc8e96834e5c3c295

[Ovid]

Also, unless someone here objects, I plan to list everyone in the Toronto keynote as team members on the Oshun project. I'll try to use your github names and real names. If you'd like to be listed differently, or not at all, please let me know.

[perigrin]

Does that make us … Oshun's 8? I'll see myself out.

[druud]

AFAICS, there will still always be room for

perl -Mv5.36 -Mexperimental=declared_refs -MData::Dump=dd -wE'
  sub Int { dd @_ }
  Int my ($i, \@j, \%k) = (0, [42..44], {one => 1, two => 2});
'
(0, [42, 43, 44], { one => 1, two => 2 })

or similar, to set up checked variables/containers in a more keyword-like way.

Which makes me wish a my() that obeys prototype-context, such that a call like

Int my ($i, @j, %k);

can set a prototype-context on the my().
But then Int() needs a prototype of \[$@%&*] ... for each of its unknown number of parameters, which AFAIK is not offered by prototype. Then this already requires Int to be set up as a keyword.

[duncand]

I prefer KIM syntax for the consistency with key-value pairs, where the name always appears on the left of what it is naming, whether it is named arguments or named hash keys. Also it is consistent with things like SQL where the field name is declared before the possibly numerous descriptors of it.

Also, importantly, KIM syntax is consistent with Corinna field declarations. If you look at it in a more abstract sense, the check declaration is functionally like many other things that are currently attributes in Corinna field declarations.

And KIM syntax I prefer visually because for the general case where check definitions may be long, it looks better for the long thing to be on the right, its like an arbitrary value expression.

[zmughal]

I also prefer the postfix KIM syntax for this reason because it keeps the syntax open to extensibility, but I don't yet know what this would be. I guess on example from what I do with various /Moo.*/ wrappers around Getopt like MooX::Options or CLI::Osprey is that I have the isa next to the format coming from the command line:

option repo_path => (
	is       => 'ro',
	doc      => 'Path to repository',
	format   => 's',
	isa      => AbsDir,
	coerce   => 1,
	required => 0,
	default  => sub { path('.') },
);

I might prefer this over looking back and forth, but this is more a developer ergonomics question.

---

The other thing about the KIM syntax would be that if there is ever a syntax to allow using the initial rvalue to indicate what the check would be like the C++ auto (and soon to be in C23 afaik), the check would be closer to the rvalue so visually scanning over the code would be over a smaller distance.

[zmughal]

Another thing to think about is how the syntax would look for defining checks with destructuring assignment.

[Ovid]

Agreed. To do that, we need to see Ingy's full proposal for destructuring assignments.

[Ovid]

As a first pass, I would imagine that it would look similar to signatures:

my [INT $foo, OBJ[My::Customer] @customers, ARRAY[STR] $filters ] = $customer_records;

[esabol]

Using KIM syntax seems sensible to me, especially since Corinna uses it also.

[oldtechaa]

I'm a big fan of using the my TYPE IDENTIFIER syntax. It looks good and it's clean. It doesn't involve a syntax that'll be unfamiliar to nearly everyone coming from other languages, and as mentioned it allows multiple declarations in one statement. Trying to use params to implement a forward-thinking project seems like it'll hold Oshun back quite a lot.

[happy-barney]

FYI: as mentioned elsewhere popular many modern languages uses postfix syntax :-)

and found interesting thread on reddit as well https://www.reddit.com/r/AskProgramming/comments/ql2nzj/why_most_new_programming_languages_use_postfix/

[oldtechaa]

That is an interesting thread. I guess languages I'm most involved in are either semi-typeless and/or I'm just used to C-style typing, but that may not be the case for all. Maybe for me it goes back to the fact that Perl probably borrows most heavily from C.

[zmughal]

Another post I came across on Mastodon: https://benhoyt.com/writings/name-before-type/.

[happy-barney]

hm, maybe it will be good to summarize these links in primary discussion comment or in wiki (should we enable it here?)

[druud]

I presume that at some future point we will also want an :auto, that will follow the initial value, with :any as an escape.

And then some way to lexically force :auto on all underspecified declarations.

[happy-barney]

as well as

my Some_Check $foo ...;
my same_checks_as ($foo) $bar ...;

[zmughal]

A quick and dirty search using grep.metacpan.org for use of both kinds of syntax:

• (my|our|state) TYPE VAR

• attributes on variables and subs

[happy-barney]

valid but unused ... and things changed
Perl is not "in" again, we need to be creative, no too conservative

[perigrin]

The results above suggest that KIM syntax here is the conservative choice because it’s used for so many other behaviors as well.

@leonerd said somewhere effectively that a constraint declaration should stand out. I honestly have no cat in this fight, and won’t complain if we end up with attribute based check annotations. However, I don’t want to ignore almost 30 years of history for consistency with KIM. Especially if it lessens the impact of the behavior. Finally making a reasonable implementation of a feature that has been 30 years in the making … that’s a compelling story about Perl being dynamic and forward looking without throwing out the past.

[happy-barney]

regarding "things changed " and this project:

• it itself counts with separated symbol spaces (with extended grammar) so it will make conflict in parser/lexer lookup
• my is not used only for variables anymore (my sub), with possible more (can imagine eg my package ..., my attribute ..., my constant ...)

[leonerd]

A reminder once again that the parsing rules inside attribute values are different than the parsing rules elsewhere in perl. Any attribute value is treated just like a simple string - albeit one where we count the parens and ensure they're balanced. And that is it. It's a string. If you wanted it to be complicated perl code then someone has to eval() that, and that gets complicated. It also complicates the parse model.

To show what I mean, here's a really contrieved example wherein I want to restrict a string variable to not permit an open paren inside its value; done as either a leading type constraint, or an attribute.

my RestrictedString(qr/[^(]/) $str;

my $str :Constrained(RestrictedString(qr/[^(]/));

The first one, while awkward and complicated, at least has a hope of parsing correctly. The latter one is straight up outright a parse error - because the parens are unbalanced.

This is the reason why I am strongly in favour of not really using attributes for this. I'm using them right now today in my :Checked field attribute for Object::Pad because there is no better option right now. But a design process can design itself a better way. Please do so.

[happy-barney]

@leonerd aware of that and working that on proposal to handle this as well as part of separated symbol spaces (hope to finish it coming weekend)