you can of course match the equivalent of an a.*b regex by hand. and this is in practice e.g. how my SIMD zip parsing works. but it would be super neat to generalize this and formalisms can light the path. this is why i (as usual) reject placing all faith in a compiler/language implementation of coroutines, because it's quite often that application-specific constructs cannot feasibly be supported by the compiler.
Discussion
Loading...
you can implement coroutines in a variety of ways including with os thread parking and with application-specific constructs (which enable the application to perform highly nontrivial program transformations). but this is one case where compiler/language support is extremely useful
in particular string processing and parsing is where i'm familiar with application-specific coroutine-like constructs but this is typically not performed to the degree i'd like to see / am working on. you really do need to codify the entirety of the computation's progress in a manner that is in fact essentially the variable closure + state machine transformation that a compiler can perform for you.
the main reason you still might want to do this is (a) FFI, or more generally a cross-language protocol which may admit serialization or even RPC, or (b) if you need to translate such constructs into your own application-specific e.g. parser compiler. in particular (b) is very useful for linking "progress" from literal matchers with "progress" from regular or stack-based automata
the "holy grail" in my view is to construct a paradigm in which progressive linguistic complexity can be expressed in terms of lower-level matchers, which means "progress" is codified in a compositional manner. this would especially mean stack-based logic can be separated from regular state transitions and from stateless literal matchers.
practically speaking this would mean you can match a.*b by matching a then matching b using fast e.g. SIMD approaches (or e.g. n-gram indices). that this is literally impossible for regex engines will go down in history as an interesting design quirk
you can of course match the equivalent of an a.*b regex by hand. and this is in practice e.g. how my SIMD zip parsing works. but it would be super neat to generalize this and formalisms can light the path. this is why i (as usual) reject placing all faith in a compiler/language implementation of coroutines, because it's quite often that application-specific constructs cannot feasibly be supported by the compiler.
1 more replies (not shown)