Capturing matching values as the original input

[randomeizer]

Ok, so this one is a little off to the side. I've often come across situations where I want to parse a slightly more complicated string into a single string value. This was actually one of the inspirations for the Not parser.

There are a bunch of low-level parsers which are quite useful for parsing through bits of strings, but they all break up the results into individual pieces.

The case that brought this home in particular was the Comment grammar for XML:

Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]	/* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */
Comment ::= '<!--' ((Char - '-') | ('-' (Char - '-')))* '-->'

I can do this to parse it correctly:

let charMinusDash = Prefix(1...) { $0 != "-" && $0.isXMLChar }

let comment = Parse {
    "<!--"
    Many {
        OneOf {
            charMinusDash
            Parse {
                Prefix(1) { "-" }
                charMinusDash
            }.map {
                "\($0)\($1)"[...]
            }
        }
    }
    .map { $0.joined() }
    "-->"
}

However, having to map the internal Parse into a String and back to a Substring is not ideal, and I end up with a massive array of Substrings. What I really want is a single Substring with the whole comment. That could be mapped again, but it's a lot of String/Substring conversion, and I've lost connection to the original input Substring.

My current solution is a Capture parser (name inspired by capture groups in regex):

public struct Capture<Match: Parser>: Parser
where Match.Input: Collection, Match.Input.SubSequence == Match.Input
{
    public let match: Match
    
    public init(@ParserBuilder _ build: () -> Match) {
        self.match = build()
    }
    
    public func parse(_ input: inout Match.Input) throws -> Match.Input {
        let original = input
        _ = try match.parse(&input)
        return original[..<input.startIndex]
    }
}

With this in hand, my comment parser becomes:

let comment = Parse {
    "<!--"
    Capture {
        Many {
            OneOf {
                charMinusDash
                Parse {
                    "-"
                    charMinusDash
                }
            }
        }
    }
    "-->"
}

It's simpler, and actually gives me a single Substring (or other valid input collection) which is still linked to the original input source.

There is a lot of parsing conversion work being done by the Match parser that's getting thrown away, so perhaps not the most efficient way of doing this kind of job, so happy to get other thoughts on it. I could certainly write a custom parser for Comment (which I'll probably do in this case), but the issue of having multiple parts of strings broken up in the parse process, that I then want to be a single string has come up enough times that I think it's worth figuring out.

Any thoughts?

[iampatbrown]

@randomeizer this was one of the motivations for the PrefixBy parser I mentioned in #97 (comment)

My comment parser looked like this:

let comment = Parse(.struct(Comment.init(rawValue:))) {
  "<!--".utf8
  PrefixBy { input, _ in
    guard !input.starts(with: "--".utf8), let scalar = input.firstScalar, isLegalCharacter(scalar)
    else { return 0 }
    return UTF8.width(scalar)
  }
  .map(.string)
  "-->".utf8
}

[randomeizer]

Do you already have an XML Parser implemented? I'd be happy not to reinvent the wheel...

[randomeizer]

Also, can you share your PrefixBy parser implementation?

[randomeizer]

Never mind, just found your repo.

[iampatbrown]

Awesome. Sorry for the late response. I pushed the throws branch earlier in a hope you might find it :) it's a wip just haven't had the time to update it. It was more of a learning exercise but there might be some parsers that could work for you. PrefixBy works fine as a generic, I just changed it to take UTF8View to speed up compile times.

You could also check out my initial attempt which includes the xml gramma in the comments and might be useful.

[randomeizer]

Yeah, it's helpful for sure. I've been mostly working with Substring rather than UTF8View and lower-order string types. I'll make my repo public in a bit and we can compare notes.

As an aside, I've found that I need to erase a bunch of types, particularly in the Printer branch, for what I suspect is the compiler choking on the massive type signatures that end up getting created.