Extractor functions

[nystrom]

This PR implements extractor functions for Rematch.

Patterns can use extractor functions (also known as active patterns). These are just any function that takes a value to match and returns either nothing (indicating match failure) or a tuple that decomposes the value. The tuple is then matched against other patterns.

An extractor function must have a lowercase name to distinguish it from a struct name. [I'd like to relax this requirement, if possible, but we should discuss.]

An extractor function must take one argument--the value to be matched against--and should return either one value (for nullary and unary patterns), or a tuple of values (for 2+-ary patterns). Returning nothing indicates the extractor does not match.

For example to destruct an array into its head and tail:

function Cons(xs)
    if isempty(xs)
        nothing
    else
        ([xs[1], xs[2:end]])
    end
end

@match [1,2,3] begin
    ~Cons(x, xs) => @assert x == 1 && xs == [2,3]
end

The main code changes are in handle_destruct for the T_(subpatterns__) case.

[cscherrer]

This looks really nice @nystrom . Am I understanding right that the semantics are that

@match foo begin
    cons(x, xs) => bar
end

is equivalent to

@match cons(foo) begin
    (x, xs) => bar
end

so it seems the function is converted to its inverse? I haven't used active patterns much, is this a common way to write things?

[nystrom]

Those two cases are equivalent. Yes, the extractor is basically the inverse of the constructor.
But you can't just call the inverse function directly on the scrutinee after @match because you need to call different functions in different case arms. For example:

@match foo begin
   cons(x, cons(y, ys)) => foo
   cons(x, xs) => bar
   [] => baz
end

calls cons on foo in both the first and second patterns, but not in the third.

[comnik]

Sorry for being a bit skeptical at first, this does look really cool for some use cases. Like anything else, it can probably be overused, but looking at eval.jl I see the need clearer than before ;)

I don't get everything about the implementation, this is the first time I'm looking at the Rematch source. However no red flags jump out to me, apart from a few style guide violations.

Thanks @nystrom !

[nystrom]

I was thinking that rather than using uppercase/lowercase to distinguish between structs and extractors, we could just prepend an operator like ~.

@match foo begin
   ~Foo(x,y,z) => ... # call extractor Foo
   Foo(x,y,z) => ... # match struct Foo
end

I'm converting this back to a draft PR.

[ghost]

I don't understand why this returns 4. I could understand if it returned 2 or 3, but i don't quite understand how it returns 4.

The definition of sub1(x) is sub1(x) = x+1. So I take @match 3 begin ~sub1(x) => x end to mean "if 3 matches sub1(x) for some x, return that x." And in this case, 3 would match sub1(2), so i'd expect it to return 2?

Also, i'm confused why x+1 is called sub1 - i'd have called it add1?

[ghost]

Ohhhhhhhh after reading some examples, i now understand that in the above match pattern, the x doesn't refer to the input, it refers to the output.

So ~sub1(x) means x == sub1(3), meaning x == 4.

This was clearer to me after reading the Cons example in the README.

---

Perhaps we can find a better syntax that doesn't look like a function application? This would also help with picking a disambiguating syntax from struct constructors.

What do you think about something like this?:

@test (@match 3 begin
    sub1(~)(x) => x
end) == 4

@match [1,2,3] begin
    Cons(~)(x, xs) => @assert x == 1 && xs == [2,3]
end

Where the ~ represents the object being matched on.

Or instead of the ~, maybe Cons(_)(x, xs)? Or Cons()(x, xs)? Or maybe something like Cons()->(x, xs), or some other similar but illegal syntax?

[nystrom]

I don't understand why this returns 4. I could understand if it returned 2 or 3, but i don't quite understand how it returns 4.

The idea is that 3 == sub1(4). Don't think of sub1 as a normal function; it's basically the inverse of a constructor.
Maybe we can change the name to unapply_sub1 or something like that (in Scala extractor methods are all named unapply, the inverse of apply). This would avoid overloading issues if there's a unary constructor also.

Perhaps we can find a better syntax that doesn't look like a function application? This would also help with picking a disambiguating syntax from struct constructors.

What do you think about something like this?:

@test (@match 3 begin
    sub1(~)(x) => x
end) == 4

@match [1,2,3] begin
    Cons(~)(x, xs) => @assert x == 1 && xs == [2,3]
end

I don't see this as any better than ~Cons(x,xs).

[nystrom]

Ah, but renaming the function to unapply_X breaks higher-order extractors like Re (see the tests and README).

[nystrom]

Nevermind, easy to fix. Extractors are named unapply_Foo and the pattern is ~Foo. Pushed.

[nystrom]

After using it a bit, I'm not really sure I like the unapply_ prefix (or any prefix). Best to just have the writer of the extractor function call it whatever they want, avoiding confusion about which function gets called.

[ghost]

I don't see this as any better than ~Cons(x,xs).

The reason I prefer Cons(~)(x,xs) is because it prevents parsing the above as a two argument function call to Cons, which is what it really looks like.

Instead, I want to somehow indicate that the syntax is matching on a two-argument destructured return value from calling Cons on the match variable. I was imagining the Cons(~) syntax to imply "calling Cons with the match variable", and then Cons(~)(x,xs) implying "the return value from that function can be captured by destructuring into two variables."

Another option might be something like (x,xs) = Cons(~)?

Basically, I just want to prevent the initial parse looking like the destructed output variables are actually the inputs to the function, since when they're both arity-1, as in the sub1 function, it was difficult to understand which variable is the input and which is the output.

Does that make sense? :)

This seems like a really cool feature! I'm sorry to hold it up bikeshedding. Please take this as a positive: this is really cool and I want to help make it intuitive and easy to use! :)

[ghost]

Sorry we left this PR lagging for forever.

Please feel free to just merge it now, and we can always change this in later iterations if we want to!

After rereading the above discussion, i would like to propose that latest suggestion one more time; i actually think it's fairly nice:

@test (@match 3 begin
    (x = sub1(~)) => x
end) == 4

@match [1,2,3] begin
    ((x, xs) = Cons(~)) => @assert x == 1 && xs == [2,3]
end

This syntax makes the behavior more obvious, IMO:

• call the supplied function on the match variable, and then
• do a capture match on the return value.

---

BUT it seems to me like you are implementing a well known feature in other pattern-matching systems, and I am not at all familiar with those, so please feel free to completely ignore my input. :)

I'd like to get out of the game of causing drama on this repo! ❤️ 😅

[amirsh]

@nystrom this is really nice :)
looking forward to see it merged :D

[ghost]

@nystrom - sorry for the long delay here. I really don't know much of anything about pattern matching, so it would probably be better to get someone else to review this PR!

@amirsh: Can you maybe review it? 😁 🙏 If you think it's useful, just approve it! We can always make more changes in the future. :) Sorry that i caused such a long hang up!!

[ghost]

Sorry we left this PR lagging for forever.

Please feel free to just merge it now, and we can always change this in later iterations if we want to!

After rereading the above discussion, i would like to propose that latest suggestion one more time; i actually think it's fairly nice:

@test (@match 3 begin
    (x = sub1(~)) => x
end) == 4

@match [1,2,3] begin
    ((x, xs) = Cons(~)) => @assert x == 1 && xs == [2,3]
end

This syntax makes the behavior more obvious, IMO:

• call the supplied function on the match variable, and then
• do a capture match on the return value.

---

BUT it seems to me like you are implementing a well known feature in other pattern-matching systems, and I am not at all familiar with those, so please feel free to completely ignore my input. :)

I'd like to get out of the game of causing drama on this repo! ❤️ 😅

[nystrom]

Thanks @rai-nhdaly and @amirsh for the comments/review.

I haven't really been pushing this since we've basically worked around its absence for a while now, and I'm no longer sure I'll even use it since extractors will not work with the type dispatch macros I layered on top of Rematch in the compiler. I'd like to actually back-port the type dispatch code into Rematch but haven't had the time.

Sorry @rai-nhdaly I don't like your syntax proposal :-) It doesn't look much like pattern matching to me anymore. The whole idea is to make the pattern look like the code used to construct the scrutinee of the match, but with free variables for constructor arguments. At least that's what I'm used to in functional languages. I tried to stay as close to the Scala behavior as possible.

I'll merge this when I'm back from vacation.

[amirsh]

@NHDaly sure.

Regarding the syntax, I totally agree with @nystrom . The proposed syntax matches better with the one from Scala and F#.

One minor comment is that the last part of the README file is broken due to a missing ```.

Please let me know once the code is available for review.

[nystrom]

I fixed the missing quote, thanks.

[ghost]

Excellent. Like i said, i don't have much experience with those other systems, so i trust your gut - sorry for the drama. :)

Enjoy the vacation!! :)