Repository navigation
Add neural synthesis for simple regex generation - #20
Merged
Merged
Conversation
Adds the optional `neural-synthesis` feature (off by default): `FastAutomaton::to_regex_with` and `Term::to_regex_with` convert an automaton with the help of a learned model, a `neural_synthesis::NeuralSynthesizer`. Every candidate the model proposes is checked for equivalence before use, and the simpler of the verified proposal and state elimination's regex is returned, so the result is never wrong. Models are loaded from `config.json` and `model.safetensors`, or from the Hugging Face Hub with `neural-synthesis-hub`, and run with candle on the CPU or a GPU (`-cuda`, `-cudnn`, `-metal`; `-mkl` and `-accelerate` for the CPU). The first supported format is `kleene` v1, for minimal DFAs of at most 16 states and 16 bases. The module lives next to state elimination, in `fast_automaton/convert/to_regex/neural_synthesis`, and is re-exported as `regexsolver::neural_synthesis`. `RegularExpression::evaluate_complexity` now returns the cognitive complexity the models are trained to minimize, computed iteratively. State elimination orders eliminations by it too, which shortens patterns converted from DFAs by about 9% over the test corpus. CI and CONTRIBUTING.md also lint, test and document the crate with `neural-synthesis-hub`.
`FastAutomaton::to_regex_with` and `Term::to_regex_with` take any `fast_automaton::RegexSynthesizer`: `StateElimination`, the conversion `to_regex` uses (now documented as such), `NeuralSynthesizer`, or a dependent's own. They no longer need the `neural-synthesis` feature. When the model has no correct regex for an automaton, or the automaton is beyond its limits, `NeuralSynthesizer` splits the minimal DFA into independent pieces (star, concatenation, frontier, union and hub splits, all exact), converts each by the model or state elimination, whichever is simpler, and combines their regexes. `with_decomposition(false)` turns this off. Over automata of 17 to 80 states from set operations, the result is simpler than state elimination's for 74% of them, at about half its complexity (geometric mean). The model runs on several automata at once (`propose_batch`): the encoder runs all their graphs together, and the decoder keeps its self-attention cache in place and computes attention in one fused pass on the CPU. Decoding is up to 2.8x faster, and decomposition about twice as fast. Each sampled candidate has its own generator, so a seed's samples differ from before; greedy decoding is unchanged. CI tests neural synthesis on wasm: on `wasm32-unknown-unknown`, in a module with no imports, so with no source of randomness, using a tiny untrained model (`tests/data/kleene_tiny`); on `wasm32-wasip1`, with the rest of the tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Adds an optional
neural-synthesisfeature, off by default. It converts automata to regexes with a learned model (Kleene), which gives shorter, more readable patterns than state elimination. Every pattern the model proposes is checked against the automaton, so the result is never wrong.RegexSynthesizertrait, plusto_regex_withonFastAutomatonandTerm.NeuralSynthesizerloads models from files or from the Hugging Face Hub (neural-synthesis-hub), and can run on a GPU.evaluate_complexitynow scores how hard a pattern is to read, and state elimination uses that score too, so someto_regexoutputs change (about 9% shorter on the test corpus).Checklist
cargo fmt --all --checkandcargo clippy --all-targets -- -D warningspasscargo testandcargo test --no-default-featurespassCHANGELOG.mdis updated under## [Unreleased]